Does a SCD Type 1 Change Response Always Update All Historical Records?
4 posters
Page 1 of 1
Does a SCD Type 1 Change Response Always Update All Historical Records?
Often when implementing slowly changing dimensions, a particular dimension will have only certain columns that are interesting to view changes over time (Type 2) while the remainder simply need to be updated (Type 1). Given this combination of Type 2 and Type 1 change responses how does one handle historical records? More simply put, anytime a Type 1 change is made, even one that occurs simultaneously with any other SCD Type, are all historical rows updated or is only the current row updated?
Change Scenarios for a particular row in an ETL load:
1. Row has both Type 2 and Type 1 changes
2. Row has only Type 2 changes
3. Row has only Type 1 changes
Example:
Scenarios:
1. Sally – got married and changed her name from Roberts to Jones while also providing a corrected birth date
2. Sally – changed positions from Admin to Sr Admin
3. Doug/Dug – corrected first name
4. Doug/Dug – changed position from Tech to Manager
When processing changes for Sally in scenario 1 (Keys 2 and 3), should the original incorrect birth date be changed?
Does the decision depend only on the fact that a Type 1 change is made or should additional business rules apply?
In other words, circling back to the original question, do Type 1 changes, made alone or with other changes, require updating all of history?
If Type 1 should always update history, are you aware of business scenarios where even if incorrect original values should be retained and thus Type 1 would only impact the current row?
Your always insightful ideas appreciated.
THANKS!
Change Scenarios for a particular row in an ETL load:
1. Row has both Type 2 and Type 1 changes
2. Row has only Type 2 changes
3. Row has only Type 1 changes
Example:
Key | BusKey | FirstName(Type1) | LastName(Type2) | Position(Type2) | Birthday(Type1) | Current |
5 | 482 | Doug | Smith | Manager | 06/18/1962 | Y |
4 | 482 | Doug | Smith | Tech | 06/18/1962 | N |
1 | 482 | Dug | Smith | Tech | 06/18/1962 | N |
6 | 862 | Sally | Jones | Sr Admin | 02/26/1968 | Y |
3 | 862 | Sally | Jones | Admin | 02/26/1968 | N |
2 | 862 | Sally | Roberts | Admin | 02/26/1969 | N |
Scenarios:
1. Sally – got married and changed her name from Roberts to Jones while also providing a corrected birth date
2. Sally – changed positions from Admin to Sr Admin
3. Doug/Dug – corrected first name
4. Doug/Dug – changed position from Tech to Manager
When processing changes for Sally in scenario 1 (Keys 2 and 3), should the original incorrect birth date be changed?
Does the decision depend only on the fact that a Type 1 change is made or should additional business rules apply?
In other words, circling back to the original question, do Type 1 changes, made alone or with other changes, require updating all of history?
If Type 1 should always update history, are you aware of business scenarios where even if incorrect original values should be retained and thus Type 1 would only impact the current row?
Your always insightful ideas appreciated.
THANKS!
JoeSalvatore- Posts : 4
Join date : 2009-06-19
Re: Does a SCD Type 1 Change Response Always Update All Historical Records?
Yes, if you have a mix of type 1 and type 2 columns in a type 2 table, a change to a type 1 column is applied to all rows for a particular business key.
Re: Does a SCD Type 1 Change Response Always Update All Historical Records?
To answer your other questions, if there is a need to sometimes retain history for those columns, they are no longer type 1 columns... right? They either become type 2 or 3. In the case of type 3, you would still update all rows for a given business key.
But, if you are in a situation where maybe they want to see history and other times maybe they don't, the best solution is to maintain both a type 1 and type 2 table for the dimension and carry both foreign keys in all facts.
But, if you are in a situation where maybe they want to see history and other times maybe they don't, the best solution is to maintain both a type 1 and type 2 table for the dimension and carry both foreign keys in all facts.
Re: Does a SCD Type 1 Change Response Always Update All Historical Records?
Hi,
I can see you have mentioned the following:
"Yes, if you have a mix of type 1 and type 2 columns in a type 2 table, a change to a type 1 column is applied to all rows for a particular business key."
We are currently applying this logic - ie updating all SCD1 columns - my concern here is performance with millions of rows and as the data builds up, what is the best practice?
thanks
I can see you have mentioned the following:
"Yes, if you have a mix of type 1 and type 2 columns in a type 2 table, a change to a type 1 column is applied to all rows for a particular business key."
We are currently applying this logic - ie updating all SCD1 columns - my concern here is performance with millions of rows and as the data builds up, what is the best practice?
thanks
amrit- Posts : 1
Join date : 2010-02-11
Updating millions of rows
One way to handle updating millions of rows is to rename the dimension table, recreate the dimension under the old name, and insert from the old table, joining to the source of the updates. Some databases handle Type 2 updates better than others.
Jeff Smith- Posts : 471
Join date : 2009-02-03
Similar topics
» What to do when facts change? Update the fact table with update and deletes? or use SCD type 2?
» Late arriving dim *change* - how to update the fact?
» Transactional Fact and update of records
» Correction of Type-2 SCD Fields' Historical Values Results in Effective Date Range Splits with no Changes
» SCD2 Type Change Question
» Late arriving dim *change* - how to update the fact?
» Transactional Fact and update of records
» Correction of Type-2 SCD Fields' Historical Values Results in Effective Date Range Splits with no Changes
» SCD2 Type Change Question
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum