Thoughts about SAP BW? I could use some fresh viewpoints...

I'm a BI consultant and data architect in New England and I've been designing and implementing dimensionally modeled data warehouses for a long time. I am a full believer in this stuff and understand the theory quite well. However, in light of one of last year's big ERP/BI company mergers, we are involved in a lot more business with companies using SAP. I've implemented two data warehouses against SAP (and other sources) and the resulting BI solutions have been well received by the business users. Being a Kimball purist, I have generally dismissed BW as a DW solution in the past. Admittedly I don't know a lot about the product but what research I have done, implementations I've seen and comparison's I've made convince me it is not a better solution than a dimensionally modeled DW. In the real life examples I've seen, the solution is comprised of an ODS (usually akin to a data dumping ground) and a bunch of disconnected cubes. I've also come to understand that the development time for incremental BW based solutions are typically triple that of a dimensional model. My experience and observations come to the same conclusion. Despite this, there is always tremendous political pressure to justify a dimensional approach to DWing vs the "Canned" BW solution. I can see value in using BW integrated change detection capabilities so assist the staging process, but I've yet to see comprehensive data model in BW that even comes close to the benefits of a good dimensional model. But again, political pressures are huge from senior management that have no technical understanding nor desire to understand the differences. Its too easy to jump on the wagon that waves the "Integrated Solution" flag as best practice.

I'd be interested to hear what others have to say on this subject. I am planning on attending BW (BI7) training very soon so that my opinions and conclusions have a more informed basis. I'd like to know if anyone here has had a successful experience with BW. I'm not closed minded to the possibility that a good Kimball based architecture can be deployed through BW...because I just don't know. Its possible that most BW implementations are just done poorly with little guiding strategy and architecture that are based on business related needs. For the moment, I'm still a dimensional purist and advocate.

Thanks for your opinions!
Bob

Hi Bob,
from my perspective there are indeed advantages if you have a complete and integrated BI stack from one vendor that matches your requirements. And that's the point "if it really matches your requirements" and "is it really integrated"?

First of all i would question. Is SAPs offering today really an integrated BI stack and clearly currently it is not as most of the bought vendors solutions are today not integrated into the SAP landscape as it should be. You often have to use what they call integration kits (e.g. SAP Strategy Management for Balacend Scorecard) to integrate and then only partly integrate or they deliver freshly build applications like SAP Business Planning and Consolidation formerly Outlooksoft where in the upcoming NetWeaver version some strong features (e.g. Business Process
Flows) are left out due to whatever reasons. Clearly they are strong when it comes to extract business content from e.g. ERP into BW but what's it all worth it when your front end applications maybe lack functionality and/or integration?

For me it's always key to look at the complete solution as BI isn't only about delivering a database that i can query but even more about how can we as a company bring it to the end users/information workers so that they really can take advantage of the solution. It's all about disseminating the information into the organization.

In terms of the raw data modelling i'm sure that you'll nowhere find a proper dimensional modell being applied on a BW solution as the core concept of BW is a "snowflaked" dimensional modell. They not only have the dimension table but a hell lot of additional attribute tables (happy joining) to support e.g. multilanguage for each dimension which really blows up tablestructures in BW systems. Another thing is table names and name spaces and things that make it even more complex and not easier to understand. From my perspective they try to cover each and every circumstance that can happen in their data modell concept but for me this only leads to unnecessary complexity.

My conclusion is that if you need BI applications like budgeting, consolidation, balanced scorecard one can use SAPs offerings if they cover business users requirements. Data modelling is delivered by those applications and what's behind the scenes doesn't play this much of a role (e.g. low data volumes). If you think of pure reporting and analytics i clearly opt for self made systems. Integrating both worlds is the key to make BI successful in an organization.

Cheers
Mario

Thanks Mario,

We have had very little question on the application side, as our customers are predominantly using or changing to Business Objects. Even the companies that have been die hard BW shops recognize why SAP bought BO. BO has better BW connectivity now too so most of the old performance issues are gone. Its the data architecture side that is constantly coming into question, as these companies have spent a lot of time and money on BW. In my experience very few BW developers and architects understand dimensional modeling and all the concepts and advantages. They will all say they do because they know what a star schema is, but that's about as far as it goes. Unfortunately, discussions on BW vs. dimensional model are rarely discussions as much as subjective arguments since one rarely understands fully the other technology. I realize that every company's decision is based on their own set of needs and circumstances. For companies with heavy BW investment that have created systems that meet business expectations and needs, there may be little motivation or value in changing. For companies with a younger SAP history, however, that may not have quite so many years and dollars invested, is the integrated data solution worth more than total cost of ownership and a more robust, scalable architecture?

Bob

From the modelling perspective as i said the problem is that BWs data modelling concept is set into stone. They have one way how to modell your EDW and it's not what one would define as dimensional modelling and that's it.

"is the integrated data solution worth more than total cost of ownership and a more robust, scalable architecture?"

Quite interesting question and quite contrary one too. Isn't usually one advantage of integrated solutions the lower TCO?
From what i see it comes down to what is the overall application architecture strategy? Often the companies just say our strategy is SAP but then they forget a good part of their business. SAP is "only" business processes. This was their story the last 15? years. But what is with business productivity? Isn't the benefit for a company to bring together processes and productivity which surely should reflect in the overall application architecture which BI is a part of? I think this is the point where you can "get" your customers. They need to see that it's not all about SAP even if they think so in first place but that they have to need a broader view of what runs their business. OK hard to communicate if you are a consultant and want to keep the project but from my perspective only technical reasons won't convince any CEO/CFO.

Mario

Your reference to TCO being lower for an integrated is exactly the issue at hand. Yes, it would seem unquestionably obvious that integrated would be cheaper...so obvious that nobody even bothers to measure it, they just accept as a given. If you don't bother to consider the level of staffing, the difference in development durations, inability to analyze the business across processes without involving IT having to create yet another Multi-Cube, and the inability to get atomic level detail without spending 100's of thousands (possibly up to 1M) on BI Accelerator hardware for adequate performance, then yes CO is going to be lower...though I doubt very Total.

Your point about technical reasons not convincing a CFO/CFO are very true, and its very difficult to paint the picture of the resulting business impact when the reasons start at the technical level. As soon as you leave the technical level is when it becomes subjective without having done things both ways and can produce tangible comparisons to identical projects...but that approach obviously isn't timely or cost effective. I am curious though if anyone done such a study.

Bob

I have not done such a study, but in my previous company engagement, our internal architect team was having the same problem. Convincing the management that adoption of SAP BW as the EDW is not the correct direction as to our proposed approach of extraction of data from SAP to build our very own EDW. In the end, management did not buy our view and believe that SAP has a truly integrated solution. How wrong are they...

Neverthelss, I have to say that any company that embark on a SAP solution will be very much sold to the solution. I have to admit that as much as we try to advocate extraction of data from SAP as the right decision, it is still a daunting task trying to understand the table design and structures from SAP ERP. Unless the design and data dictionary is easily available (which I doubt so), this shall poise a big detractor for most management (including IT heads)!

In my previous job which I left 2 and a half years ago I was working on the data warehouse team. We had a data warehouse on an Oracle database. For reporting, we used Cognos. A POC for SAP BW had already been done before I started working there but the product was then considered "immature".
As more and more sources became SAP, the pressure to go to SAP BW became bigger. The biggest part of the informations systems department were "SAP people" (a lot of them knowing only SAP). Frequent remarks were made: "Why don't you use SAP BW ? This is much better". We replied "Who says BW is better ?". "Oh, the people of SAP tell us this". Like a Mercedes vendor is going to recommend Audi...
Finally we were "forced" to do a POC of BW. After a while, the POC was changed to a study of the impact on the data warehouse if switching to BW. The switch to BW was already decided.
We then attended a course and after several months I started to begin to understand BW. Terrible interface (like SAP developers never read a book on GUI), an even more terrible reporting interface, users complaining, our own SAP people beginning to question why "we" had chosen BW...
I left that company not because of BW but I am very glad that we don't use SAP here...but we use BusinessObjects which has been bought by SAP...
However, there is a positive side: knowledge of SAP BW looks very good on your CV.

Since posting this originally, I've attended some SAP BW training in an effort to become more knowledgeable. It was very apparent that the product was developed by an ERP company...Its development environment and approach have the same nuances, overhead and complexities. There are some strengths you can't take away from the product, like its extractors, change detection and a very multi-developer friendly infrastructure. My problem with it is its delivery capabilities...what it provides to the end user experience.

As we all know, database architecture has a huge impact on overall BI success potential. You want the user experience to be pleasant, meaning they can get all the information they need, easily, accurately, detailed, across any combination of business processes and with great system performance. The user should not have to work with IT in order to look at existing data in a way that IT people had not considered...A solid architecture provides that inherently. In my opinion there is no way the existing BW delivery architecture can ever meet these requirements. With Infocubes being their delivery core, they will always exhibit the same limitations as solely OLAP DW solutions...namely stand-alone structures with predetermined content. When I asked how many Infocubes a typical mid-sized company might develop, I was told maybe 25 to 100. Are you kidding me??? How can a user ever know where to find what they need when there may be so many places to look. Just like OLAP, if the developers didn't include an attribute you need in the Infocube (but does exist in a dimension) then you are out of luck until it gets added in the next development cycle...Never mind that to maintain performance cubes can not be permitted to include everything under the kitchen sink just in case a user needs it. Same is true if you need cross business process data between two Infocubes...If it was never pre-considered, you're out of luck. To me, that is NOT an example of BI meeting business needs. To me that's old school IT data center mentality. I was also surprised to learn that much of the cool functionality like being able to automatically access detail data (DataStore Objects) outside the cube only works with NetWeaver products. The really eye-opening thing to me is that an InfoCube really isn't a cube at all, its a snowflake schema relational structure of sorts, with an OLAP engine on the front end. In my opinion THAT is the very reason performance has the reputation it does and drove the need for the BI Accelerator.

So...after all this my current opinion is that there is great BW functionality to leverage on the extraction and staging end of things, but on the delivery side its a very poor architecture to support strong BI by today's growing standards. I say all this not to bash BW, because I've come to realize/accept that regardless of any arguments, BW will continue to be used by a great many companies. I also don't wish to hurt any relationships we have with SAP, for obvious reasons. Recognizing that BW will be around for a long time and for a number of reasons (remember VHS vs Beta?), I hope they will use the strengths and capabilities that a good dimensionally modeled DW provides as the standard to which they measure what they bring to market, adjusting course as necessary. In the end I want BI initiatives that I'm involved with to reach their full potential, and not be throttled, governed or otherwise limited unnecessarily. Too idealistic?...Maybe, but at least I've said my piece. Cheers!

"there is great BW functionality to leverage on the extraction and staging end".. ok, but you don't need to actually implement BW to take advantage of the extractors...

What I have seen at companies that have implemented BW are as follows:

1. HUGE support staff (mostly contractors)
2. Very frustrated end-users trying to do simple analysis.

In general, due to overly complex processes involved in getting anything done, most departments engage in renegade efforts to do their reporting. Usually they execute 'list edits' out of the OLTP system into spreadsheets which they then export to Access, or they would run a BEX query, again into a spreadsheet, which is again put into Access so they can work with the data. It's real ugly. But, if you were to talk to the SAP team, most would deny any such thing going on...

Business Objects has a long way to go to make this any better. Problem is BW is a very closed system. Those familiar with BOBJ know that a Universe can only connect to one source. In BW, a 'source' is one InfoCube or one MultiProvider or one DSO and so on... And, it gets worse... under the current implementation, SAP recommends that a BEX query be used as the source... otherwise you may run into problems with security and hierarchies. Oh, yes, and if the BEX query changes, the Universe is invalid and needs to be rebuilt...

Putting BOBJ aside (as there are a host of other BI tools that work with BW), the biggest problem with BW and SAP in general is the total disregard for providing a SOX compliant environment. The only mechanism SAP provides to do analysis is through a spreadsheet. Once data gets into a spreadsheet, audit and compliance goes out the window.

One of the reasons that BW has a real hard time delivering, is that it's not a database driven warehouse. Rather it's an application stack that provides warehousing capabilities. So the development process is a completely proprietary effort. If you were to look at the database that BW is loaded on directly, it would appear nonsensical. Why? Because SAP wants you to use the BW application layer to acquire the data. Consequently, the development resources are expensive and the development process is quite cumbersome. Additionally, BW requires you to precalculate results in what they call a cube. (I won't go there today) So this means that true Ad Hoc, ie building queries on the fly, isn't something you can do out of the box. Today there are ways of simulating that, but it's messy. This was suppose to be addressed this year, but I've heard it's now pushed off till 2011. The best thing a BW client can do today is push their stuff to the BW Accelerators, and use Explorer.

SAP has a much cleaner solution which comes from their BOBJ acquisition, but they sell it very passively, as it doesn't provide the same level of vendor reliance. It's called SAP Rapid Marts. While this solution does require you to use Data Integrator (the ETL tool from Business Objects), the target is a standard SQL Server or Oracle database. The target model is Kimball based and the ETL is fully modifiable through the BOBJ Data Integrator ETL tool. It still takes time to implement but the end results are far more open and flexible.

Heh, heh....

Last year I was working in a SAP shop and had the same sort of discussions about Rapid Marts... the answer I got depended on who I talked to. The SAP/BOBJ side of the house was pushing it while the SAP/SAP side was recommending against it. I don't know if it is still the case with this year's model...

Part of the reason SAP/BOBJ is pushing it is because, quite frankly, BOBJ doesn't work very well against BW. BW's restrictive interface basically cripples the functionality a Universe is designed to provide. They only way today to get full Universe functionality is to place a data federation layer between BOBJ and BW, which makes the whole thing a bigger pig than it is. And, even then, unless you are very careful what you expose through BW, you may not be able to integrate data very well.

Bottom line, the 'cleaner solution' (i.e. Rapid Marts) is simply saying: Don't use BW at all...

Yup, that's the answer... Avoid the pig altogether, and put the lipstick on something you can control.

Hi guys. The company I'm with are currently implementing SAP ERP using SAP BW, BEx and BOBJ to do the reporting. My position is a BI Developer.

I found the above posts a very good read. I know the last post was June 2010 but can I ask if any of you (anyone) have any further comments to make now we find ourselves two years on? Having read the comments it was good to see comparison between what SAP BW had to offer and a self-designed DW. There were some negative comments posted. Can I ask if the negative comments are still valid or has the software been found to be better than anticipated, or the software been updated to cater for the demanding needs?

Any further updates on the above will be most greatly appreciated! Thanks in advance.