Overnight Load Failure Frequency - Benchmark?
4 posters
Page 1 of 1
Overnight Load Failure Frequency - Benchmark?
I'm wondering if there are any benchmarks as to the percentage of time that the overnight load process for a data warehouse is successful (on-time, untouched by developers or support staff) vs late or encountering a problem requiring some sort of intervention. Obviously there's no "one size fits all" answer; there are many factors including the number and types of source systems, data volume, data type, system/data maturity, etc. But are there rough benchmarks, such as problems once a month is "normal" vs. "above/below avg," etc.
djschiff- Posts : 2
Join date : 2010-01-18
Re: Overnight Load Failure Frequency - Benchmark?
If you are dealing with a well defined source with decent data management, such as an ERP system, load failures (those not caused by external issues such as hardware or network failure) should be near zero. Even uncontrolled sources should have sufficient processes in place to separate bad data from good to prevent a full failure.
In other words, if you regularly encounter load issues, your process is broken.
In other words, if you regularly encounter load issues, your process is broken.
Re: Overnight Load Failure Frequency - Benchmark?
Well "external issues," I guess that's the rub, isn't it? A network issue here, a database problem or server glitch there. Suddenly there are three mornings that appear only as "data warehouse load issues" to the users, but there's little the data warehouse staff can do.
djschiff- Posts : 2
Join date : 2010-01-18
Re: Overnight Load Failure Frequency - Benchmark?
That's the nature of the beast. There are a lot of moving parts and all IT has to do their bit to keep things running. The best you can do is grin an bear it, and keep documentation.
Re: Overnight Load Failure Frequency - Benchmark?
If another system has caused the problem then don't report them as "data warehouse load issues"
nick_white- Posts : 364
Join date : 2014-01-06
Location : London
Re: Overnight Load Failure Frequency - Benchmark?
Nick, you must never have been a DBA (Default Blame Acceptor), otherwise you'd know that any system problem is automatically the fault of the biggest database in the chain, until incontrovertibly proven otherwise by the over-worked, under-valued database team
![-](https://2img.net/i/empty.gif)
» Data Warehouse Failure Statistics
» Loading Fact table
» Load Dimensions
» ETL for Fact load
» Question asked about ETL (little explanation needed)
» Loading Fact table
» Load Dimensions
» ETL for Fact load
» Question asked about ETL (little explanation needed)
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum
|
|