Help required to reload data in star schema
2 posters
Page 1 of 1
Help required to reload data in star schema
Hi,
I am new to data warehouse technologies. Looking for exports help to understand how to load data for the scenario described below.
In my DW system, I will be getting files on daily basis which contains sales information, and I will be loading these files to star schema.
Let’s says, I have received to files on in the past 10 days and loaded all to files to the data warehouse. I am performing SCD type 3 load with flag variable.
Now, on Day 11, I have to undo all the changes done by loading data on day 6 and reload the data with a new file( got revised file for day 6).
Please can you help me to understand how should I approach this scenario
Thanks,
Gokul
I am new to data warehouse technologies. Looking for exports help to understand how to load data for the scenario described below.
In my DW system, I will be getting files on daily basis which contains sales information, and I will be loading these files to star schema.
Let’s says, I have received to files on in the past 10 days and loaded all to files to the data warehouse. I am performing SCD type 3 load with flag variable.
Now, on Day 11, I have to undo all the changes done by loading data on day 6 and reload the data with a new file( got revised file for day 6).
Please can you help me to understand how should I approach this scenario
Thanks,
Gokul
gokul_ifs- Posts : 1
Join date : 2015-07-10
Re: Help required to reload data in star schema
Two options, both very similar...
1. Roll back all changes made from the load being changed and everything subsequent to it. Process the updated file and then all the subsequent files.
Simple to implement but obviously a lot of data to process.
2. Rollback all changes related to amended records in the new version of the file. Reload the amendments and all records in the subsequent files that relate to the amended records.
More complicated to implement but may be the only solution if you have a tight load window and only a small number of records in the amended file have changed.
Alternatively, you may be able to treat the changes as genuine changes rather than corrections of errors. You'd need to work with your users to determine the correct logic for processing these changes and then implement it in your ETL
1. Roll back all changes made from the load being changed and everything subsequent to it. Process the updated file and then all the subsequent files.
Simple to implement but obviously a lot of data to process.
2. Rollback all changes related to amended records in the new version of the file. Reload the amendments and all records in the subsequent files that relate to the amended records.
More complicated to implement but may be the only solution if you have a tight load window and only a small number of records in the amended file have changed.
Alternatively, you may be able to treat the changes as genuine changes rather than corrections of errors. You'd need to work with your users to determine the correct logic for processing these changes and then implement it in your ETL
nick_white- Posts : 364
Join date : 2014-01-06
Location : London
Similar topics
» Star schema for a data warehouse
» Only way to pull data from star/snowflake schema is by using facts?
» How to Handle Updates to Dimension Attributes (SCD 1) that have a Cardinality > 1 (Context is Data Warehouse Star Schema)
» Data Modeling Question (Bridge Tables?) for Star Schema for Proposals/Awards for university
» How to convert required OLTP schema into dimensional model
» Only way to pull data from star/snowflake schema is by using facts?
» How to Handle Updates to Dimension Attributes (SCD 1) that have a Cardinality > 1 (Context is Data Warehouse Star Schema)
» Data Modeling Question (Bridge Tables?) for Star Schema for Proposals/Awards for university
» How to convert required OLTP schema into dimensional model
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum