Is it possible to design a DW in parallel with its source systems?
3 posters
Page 1 of 1
Is it possible to design a DW in parallel with its source systems?
Hi,
I have come across the following scenario a few times in my career and was wondering what people's views are on how to approach it:
Scenario
Your company/client has a greenfield project and/or is undertaking a platform replacement program; so new CRM/Transactional/Finance/ERP systems are being designed and built, and business processes will change due to the enhanced capabilities of the new systems - and the company also wants a brand new data warehouse/reporting platform.
The program managers, who are not experienced data warehouse people (and you wouldn't expect them to be), plan for all systems to go live on the same day and therefore plan for all systems to go into UAT, Sys Test, etc. on the same day.
From a DW perspective, when you start on day 1 the only requirements the users/SMEs can give you are in terms of the existing platform/processes and so may not be relevant on the new platform, and none of the source systems exist - so no physical data model to analyse and no data available to profile.
While you can obviously do some initial investigations and get your processes in place and agreed, I don't believe it is possible to develop a DW in parallel with its source systems being developed; and if you try you are liable to have to do a significant amount of re-work at a later date.
Do people agree?
If they do, how far behind source system build should the DW build lag? Start Dev when source systems move to Test, or UAT, or Prod, or 3 months after go-live?
If DW development lags behind source system development, is it reasonable for the program managers to expect DW to catch-up at some point and still deliver to Prod on the same date as the source systems? If so, at what point in the development cycle would you have expected the DW to have caught up?
I have come across the following scenario a few times in my career and was wondering what people's views are on how to approach it:
Scenario
Your company/client has a greenfield project and/or is undertaking a platform replacement program; so new CRM/Transactional/Finance/ERP systems are being designed and built, and business processes will change due to the enhanced capabilities of the new systems - and the company also wants a brand new data warehouse/reporting platform.
The program managers, who are not experienced data warehouse people (and you wouldn't expect them to be), plan for all systems to go live on the same day and therefore plan for all systems to go into UAT, Sys Test, etc. on the same day.
From a DW perspective, when you start on day 1 the only requirements the users/SMEs can give you are in terms of the existing platform/processes and so may not be relevant on the new platform, and none of the source systems exist - so no physical data model to analyse and no data available to profile.
While you can obviously do some initial investigations and get your processes in place and agreed, I don't believe it is possible to develop a DW in parallel with its source systems being developed; and if you try you are liable to have to do a significant amount of re-work at a later date.
Do people agree?
If they do, how far behind source system build should the DW build lag? Start Dev when source systems move to Test, or UAT, or Prod, or 3 months after go-live?
If DW development lags behind source system development, is it reasonable for the program managers to expect DW to catch-up at some point and still deliver to Prod on the same date as the source systems? If so, at what point in the development cycle would you have expected the DW to have caught up?
nick_white- Posts : 364
Join date : 2014-01-06
Location : London
Re: Is it possible to design a DW in parallel with its source systems?
I first got into a situation like that in parallel with an SAP implementation. At the time I told myself I never want to get into a situation like that again. But, in the 15 years or so after that, I keep running into similar situations. It's a real pain. You basically struggle with incomplete and vague information for about 70% of the project and things don't start crystalizing until about 90% of the way through. Chaos ensues and somehow everything gets done.
It's not the best way to do it. There is a lot of revisions, and revisions to the revisions. Frankly, I would rather come in after the operational system design is solid, but it never seems to work out that way.
It's not the best way to do it. There is a lot of revisions, and revisions to the revisions. Frankly, I would rather come in after the operational system design is solid, but it never seems to work out that way.
Re: Is it possible to design a DW in parallel with its source systems?
I've been on projects like this as well. You do the best you can do given the fluctuating source database design. The biggest pain is taking all the extra time to document why the reporting model is late for the PHB's.
BoxesAndLines- Posts : 1212
Join date : 2009-02-03
Location : USA
Similar topics
» Physical Deletes in Source Systems?
» Linking or Merging Similar dimensions from Different Source Systems
» Multiple source systems, some with many Business Key namespaces, and little organisation engagement
» How to model 4 parallel views/hierarchies of a dimension?
» When to combine facts from different systems...
» Linking or Merging Similar dimensions from Different Source Systems
» Multiple source systems, some with many Business Key namespaces, and little organisation engagement
» How to model 4 parallel views/hierarchies of a dimension?
» When to combine facts from different systems...
Page 1 of 1
Permissions in this forum:
You cannot reply to topics in this forum