ETL Execution & Scheduling
3 posters
Page 1 of 1
ETL Execution & Scheduling
I had a question about the ETL design. Given that ETL would involve tools/scripts/programs written in different programming languages and platforms, what platforms/tools are used to manage the execution and scheduling of these? How do we make sure that these tools/scripts/programs run at the same time and pull data into the data warehouse?
Thanks,
Amit
Thanks,
Amit
ETLDev- Posts : 1
Join date : 2016-02-03
Re: ETL Execution & Scheduling
I would use a dedicated ETL tool (Informatica, SSIS, Talend, Datastage, ODI, etc.) otherwise, as you say, you have to hand code not only the actual data transformations but also the control framework to make sure everything runs with the correct precedents, errors are handled appropriately, etc.
Except for the simplest of processes I think you would have to be a bit of a masochist these days (or very confident in you abilities) to hand code your ETL system
Except for the simplest of processes I think you would have to be a bit of a masochist these days (or very confident in you abilities) to hand code your ETL system
nick_white- Posts : 364
Join date : 2014-01-06
Location : London
Re: ETL Execution & Scheduling
Some etl tools have built in scheduling and execution capabilities, but generally, they are pretty limited in functionality. Most sites use an enterprise scheduling tool and execute etl processes using scripts calling the etl tool's CLI (command line interface). The CLI will usually return a status code that the scheduling tool would use to check progress of the load.
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum