ETL from 250+ disparate customer host systems
Page 1 of 1
ETL from 250+ disparate customer host systems
My task is to produce an ETL process that pulls data daily from our customer systems (which can be most any database/OS). The data will be loaded into our central system, an analysis will be run and the aggregate data solution sent back to all customers.
Any recommendations for tools to achieve this scenario? I'd like to be able to connect to each customer's host (via whatever protocol) and execute the ETL from my central location. I will be able to install a client on the customer's system, in most cases. I've had some frameworks/tools recommended to me but I don't have enough experience yet to decide the best course of action.
Here are my current ideas, very open to suggestion:
Any insight is greatly appreciated,
Bill
Any recommendations for tools to achieve this scenario? I'd like to be able to connect to each customer's host (via whatever protocol) and execute the ETL from my central location. I will be able to install a client on the customer's system, in most cases. I've had some frameworks/tools recommended to me but I don't have enough experience yet to decide the best course of action.
Here are my current ideas, very open to suggestion:
- Communication/messaging to heterogeneous systems (Apache Service Mix)
- ETL process & data quality/cleansing/reporting (Apache Camel, Pentaho, Clover etc.) to pull data in
- Automated analysis and processing (ANT, ETL tool, legacy apps) to generate solutions
- Package/distribution of results (Apache Service Mix, FTP)
Any insight is greatly appreciated,
Bill
bg4- Posts : 1
Join date : 2010-05-20
Similar topics
» Merging customer data from disparate sources to create a master customer dimension
» Customer Dimension from multiple systems
» De-normalizing Customer Information to create a Customer Dimension
» Designing Single set of dimensions for disparate source data
» Physical Deletes in Source Systems?
» Customer Dimension from multiple systems
» De-normalizing Customer Information to create a Customer Dimension
» Designing Single set of dimensions for disparate source data
» Physical Deletes in Source Systems?
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum