Archiving older data
2 posters
Page 1 of 1
Archiving older data
Suppose you archive detailed data which is older than three years on slower media. Decision is made. Which problems do you face then? Ok, it takes some time to load the data again if necessary. But that is no problem, as long as the user is aware of it.
What I am worried about is the fact that in time the layout of dimensions and fact-tables might change. Attributes added, which have had no meaning in the past. If then you want to reload the old facts in the existing fact table, you have to do something manually. You have to write a job to fill in the meaning of new attributes when reloading the past. The best solution would NOT to have two tables: one with the new layout and one with the layout of the past. Even worse, if more and more attributes are added in time, you get more different table layouts
How is this solved, doing it automatically, if possible in a DB2-environment?
Thanks
Ron
What I am worried about is the fact that in time the layout of dimensions and fact-tables might change. Attributes added, which have had no meaning in the past. If then you want to reload the old facts in the existing fact table, you have to do something manually. You have to write a job to fill in the meaning of new attributes when reloading the past. The best solution would NOT to have two tables: one with the new layout and one with the layout of the past. Even worse, if more and more attributes are added in time, you get more different table layouts
How is this solved, doing it automatically, if possible in a DB2-environment?
Thanks
Ron
revdpoel- Posts : 24
Join date : 2010-06-11
Re: Archiving older data
This is normally handled with partitions. By partioning fact tables based on transaction dates it is a fairly simple matter to move older partitions to less expensive (but still on-line) media. Any schema changes would be reflected in the partitions as they are still part of the active database.
If your intent is to take the data offline, you should avoid using internal DB formats unless you are absolutely certain you will always be using the same DBMS (and future versions of the DBMS would be able to read the archive). A more generic approach would be to extract the data to a delimited text file and include a copy of the table's DDL with the file. Restoring would be a manual effort.
If your intent is to take the data offline, you should avoid using internal DB formats unless you are absolutely certain you will always be using the same DBMS (and future versions of the DBMS would be able to read the archive). A more generic approach would be to extract the data to a delimited text file and include a copy of the table's DDL with the file. Restoring would be a manual effort.
Similar topics
» Reporting table data repository vs. Dimensional data store
» Is it a best practice that Data warehouse follows the source system data type?
» clickstream fact data coming in with different levels of dimensional geography data
» difference between data mart and data warehouse at logical/physical level
» Using the Dimensional Data Warehouse as source data for the OLTP process
» Is it a best practice that Data warehouse follows the source system data type?
» clickstream fact data coming in with different levels of dimensional geography data
» difference between data mart and data warehouse at logical/physical level
» Using the Dimensional Data Warehouse as source data for the OLTP process
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum
|
|