Kimball Forum
Would you like to react to this message? Create an account in a few clicks or log in to continue.

ETL Logging vs Audit Dimension

Go down

ETL Logging vs Audit Dimension Empty ETL Logging vs Audit Dimension

Post  arowshan Tue Mar 06, 2012 4:41 pm

I have been reading and playing around with the sample SSIS packages in the Microsoft Data Warehouse Toolkit and was hoping to get some other opinions on this:

The Audit Dimension that they propose in the book contains the package logging as well which means they are adding logging information such as package start and end time in the Audit Dimension both for Extract and Load pacakges. I was thinking ETL execution and error logging should be in the back room not part of the model.

arowshan

Posts : 23
Join date : 2011-10-18
Location : Vancouver, Canada

Back to top Go down

ETL Logging vs Audit Dimension Empty Re: ETL Logging vs Audit Dimension

Post  ngalemmo Tue Mar 06, 2012 5:15 pm

Well, it has to be somewhere. It really all depends on wither there are requirements for users to see it. You can go either way with it.
ngalemmo
ngalemmo

Posts : 3000
Join date : 2009-05-15
Location : Los Angeles

http://aginity.com

Back to top Go down

ETL Logging vs Audit Dimension Empty Re: ETL Logging vs Audit Dimension

Post  bciampa Sat Mar 10, 2012 5:23 pm

Arowshan,

I have not read the book that you reference but I suppose that package logging (if I understand how you are using the term) may have a different purpose from error logging. In the environment in which I work, we have a table in our presentation area that logs the last successful refresh time of an ETL job. When reports are written from that star, the report writer can display that date and time so that users will know that they are looking at salary data (for example) that is current as of that date and time. If an ETL job "bombed" at some point the fact that the ETL job ended in an error (but not the reason) is also logged. Logic is written into the report to alert the user that the report is unavailable and we are working on the problem. The reason for the error, however, is not made available to the user. That is stored in the back room for the ETL developers to see.

Thanks,
Brian

bciampa

Posts : 8
Join date : 2012-02-24

http://valuabledata.blogspot.com

Back to top Go down

ETL Logging vs Audit Dimension Empty Re: ETL Logging vs Audit Dimension

Post  Mike Honey Tue Mar 13, 2012 12:06 am

Hi arowshan,

I look at the Kimball Audit Dimension as primarily performing an Audit function, not Logging. It offers a much more convenient way to review your overall package execution history than trying to query Logging data.

IMO there's very little overlap between the two. Both are useful.

Good luck!
Mike

Mike Honey
Mike Honey

Posts : 185
Join date : 2010-08-04
Location : Melbourne, Australia

http://www.mangasolutions.com

Back to top Go down

ETL Logging vs Audit Dimension Empty Re: ETL Logging vs Audit Dimension

Post  Sponsored content


Sponsored content


Back to top Go down

Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum