I’m amazed at how easy it is to find information these days.  Today I decided to do a follow-up post based on my most recent WebIntelligence demo post.  After a little searching around and I found a number of demos that had been posted by David Collins Dixon onto Youtube.  David has written a number of books about the SAP Business Information Warehouse (or BW for short).

SAP Integration Demos

Here are the list of demos:
http://www.youtube.com/user/davidcollinsdixon

Crystal Reports Demo leveraging SAP Hierarchies (No AUDIO)

David does an especially nice job with the Crystal Reports demo and shows you how to use the Hierarchy Node so Crystal Reports can properly leverage your SAP Hierarchy definition.

Xcelsius Demos

There are also a couple of demos from the guys at Everything Xcelsius:
http://www.youtube.com/user/XcelsiusGurus

If you find any SAP BusinessObjects videos that you think others would benefit from, just add links to them in a posted comment.

«Good BI»

After attempting to move my blog off wordpress.com on Friday, I decided it was going to be more involved and I had a bit more testing to do before I was willing to make the final switch… but that’s another story.

So what do I have for you this week?  Well, I ran across a great demo by Ingo Hilgefort.  Ingo is a great resource.  He was one of the original folks that helped Crystal Decisions develop the SAP Integration Kit way back in the day and you can read more about him on his blog.  He’s what you would call the resident expert.  Last week a saw a demo that Ingo put together and I really liked it and wanted to share it here.

Watch the Demonstration

Excellent Overview of SAP BusinessObjects and WebIntelligence Integration:

You’ll see that he demonstrates some of the latest fold/unfold capabilities that come with BusinessObjects XI 3.1 SP2. Do recognize that is is NOT a replacement for the upcoming Pioneer release. I recently spoke with a customer who didn’t like the fact that the WebIntelligence document had to be in a specific layout/format in order to leverage fold/unfold. REMEMBER! WebIntelligence is an adhoc reporting solution that allows you to easily create formatted reports. It’s not targeted for power analysts. Pioneer is coming… in the meantime you’ll still need to use BEx Web Analyzer for that that power analyst group.

SAP BusinessObjects Roadmap for WebIntelligence

NOTE: for the latest roadmap updates on the SAP BusinessObjects Product Integration go to http://sdn.sap.com

Download the PPT

Here is a download of his powerpoint presentation in PDF format:
http://trustedbi.com/files/sapwebi_webinar.pdf

Note the slides that explain how the items for the BEx Query panel are translated to objects within the Universe.  It’s important to understand what objects will be exposed to the adhoc user as the build the report and which objects, e.g. characteristic restrictions, won’t be.

Make sure as well you get your hands on the SAP BusinessObjects Best Practices Guide as well.

«Good BI»

I haven’t seen a lot of information available that talks about what’s new in the release of the latest version, BusinessObjects XI 3.1, so I thought I would point out some of the highlights I’ve come across.  You may also want read the complete What’s New in BusinessObjects XI 3.1 guide from the website.

Release Highlights

BusinessObjects XI 3.1 is a release that is focused on bringing parity to our platform support for both Java and .NET.  In BusinessObjects XI 3.0 we did not provide a .NET version of our standard BI portal, InfoView.  In BusinessObjects XI 3.1 we have released a native .NET portal which has near parity with the Java version.  I say near parity, because there are a few things missing, namely:

• Encyclopedia
• Dashboard Builder
• Voyager
• BI Mobile
• Polestar

For companies who find comfort in an all Microsoft World, this release will be welcomed.  BusinessObjects XI 3.1 has added support for Windows Server 2008, SQL Server 2008 and Sharepoint Portal Server 2007.  (There is also Microsoft Active Directory Forest Support as well!)  If you are using the SAP Integration Kit, this is also now available on the .NET platform.

Web Application Container Service

One of the way in which BusinessObjects has been able to balance both rich feature sets and cross platform compatibility is through the introduction of a new server to the BusinessObjects Platform.  This service is called the WACS, Web Application Container Service, and is a java container.  What this allows BusinessObjects to do is to run existing java applications through the BusinessObjects framework.

Let me explain how this will work in practice.  The BusinessObjects Administration Console, the CMC, is written in Java.  Instead of rewritting this application in .NET, BusinessObjects simply added the WACS to the BusinessObjects service bus and will execute application via this service.  The beauty of this solution is that the .NET administrator never has to worry about it.  BusinessObjects manages everything.

What other Applications can use the WACS?

Besides the CMC, there are a number of other applications that could within the WACS.  This includes:

• Interactive DHTML Viewer for WebIntelligence
• Query as a Web Service (for supporting Xcelsius)
• LiveOffice

There is an important document that covers what is supported and what is not supported within the WACS framework.  In theory all native java-based aspects of the BusinessObjects application suite could be run through the WACS but the question is whether or not it is officially supported.  This document is called:  Web Application Container Server (WACS): Supported and Unsupported Features for BusinessObjects Enterprise XI 3.1and it explains the details.  You can download it here.

Additional Platform Enhancements

BusinessObjects XI 3.1 supports IPv6 100% across the product suite (except for Desktop Intelligence).  They have also added their first native 64-bit architecture.  Since BusinessObjects XIr2 SP3, the platform has been able to run on 64-bit operating systems however the platform still runs as a 32-bit application.  The first native 64-bit release is an the HP-UX IA-64 Itanium hardware.

Life Cycle Manager

Life Cycle Manager, LCM for short, s a web-based utility that allows the administrator to:

• Promote BI content from one system to another (without affecting the dependencies of the content)
• Manage dependencies of BI content
• Manage different versions of BI content
• Roll back (yahoo!) promoted BI content

This tool does not install with the platform, but is a separate install.

“So isn’t that what the Import Wizard is for?”, you may be asking?  Well, certainly the import wizard has been used to provide this type of functionality in the past – especially with the introduction of BIAR files in BusinessObjects XI; however I think you’ll see from the list of features in LCM, it is better to have a specialized tool for the LCM process.

Life Cycle Manager provides a number of features over the Import Wizard (besides rollback) including:

• Granular control over the objects selected (yes, we trust you)
• Scheduling Promotions Jobs (a job is a collection of BI content or InfoObjects to be precise)
• Integrated Version Control (via Subversion)
• Database connection overrides and mapping (Tired of changing database connection strings?)
• Auditing
• The ability to “preview” to changes before committing them
• Rollback, Did I mention rollback?

There is a new LCM Job server which has been added to the platform to manage the promotion process.

Have you ever created a BIAR file and realized you missed a file?  Now with LCM you can save your Promotion job and use it to generate a BIAR file.  If you find that something is missing, you can simply open up the promotion job , select the missing objects and regenerate the BIAR file.

BIAR ≠ BIAR

When is a BIAR file not a BIAR file?  Well, with the LCM tool, the BIAR files which are generated by the LCM tool are LCMBIAR files.  As the name indicates, the LCMBIAR files are NOT compatible with the BIAR files used with the Import Wizard.  Bummer.

Voyager

Voyager was the other biggie.  A lot of work was put into providing interface enhancements and increased performance.  This is probably the last release before the eagerly anticipated release of “Pioneer” in 2009, which will combine Voyager’s intuitive user interface with the powerful OLAP capabilities of the BEx OLAP tools.

Voyager now has a number of new visualizations including Box-Plot, Scatter, Bubble and Radar charts.    There are also a number of features that were in OLAP Intelligence, which were missing in earlier versions of Voyager, namely, Exception highlighting, URL Linking to other BI Content and Favorite Groups.

Have you ever made a mistake when building a Voyager workspace?  Well, now you have full undo/redo capabilities.  After being so used to the undo/redo capabilities in WebIntelligence, I always found the missing feature extremely annoying and I’m sure other did to.

Fly Over Of Additional Features

Rather than talk about additional features in detail, I thought I would briefly mention some of them here and perhaps in the future I can blog about them in detail if there is interest.

Semantic Layer -Support for BEGIN_SQL, Prompt support for codes AND descriptions.

Crystal Reports – Dual Monitor support, flash printing to PDF.

WebIntelligence -Support for Extension Points, Auto-Save, RelativeValue(), Multi-Pass Bursting

SAP Integration – SNC for BusinessObjects Universes (Prior to XI 3.1, this was only available for Crystal Reports), Improve Prompt Support for Hierarchy and Hierarchy Node dependencies.

Dashboard Builder – Printing, Interportlet Communication

Summary

There are some good things that have come out in this latest release.  This release is also the latest ‘reset point’ for BusinessObjects XI 3.X maintenance, so you can expect fix packs for XI 3.0 to no longer be released.

Hope you found this information useful.  I’ve tried to keep it short and sweet.

«Good BI»

So what is Multipass SQL?

Multipass SQL. To calculate comparisons or to correctly calculate non-additive measures in report break rows, the query tool must break the report down into a number of simple queries that are processed separately by the DBMS. The query tool then automatically combines the results of the separate queries in an intelligent way. Breaking a single complex request into several small requests is called multipass SQL. Multipass SQL also allows drilling across several conformed data marts in different databases, in which the processing of a single galactic SQL statement would otherwise be impossible. Finally, multipass SQL gives the aggregate navigator a chance to speed up the report, because each atomic SQL request is simple and easily analyzed by the aggregate navigator. (from Ralph Kimball)

Welcome Multipass 2.0 with BusinessObjects!

Okay – it’s not new, but I dubbed it 2.0 because in my opinion BusinessObjects takes a superior approach to the Multipass SQL problem.

BusinessObjects employs a number of techniques to solve traditional multipass problems in a unique way.  Users of BusinessObjects can get the power of Multipass SQL without the traditional contraints and overheads of inserting records into temporary tables and requiring the report creator to engage in a number of complex steps to get the desired results.  (Anyone ever run out of temp space?)

Highlighted here are some of the most common scenarios in which BusinessObjects employs Multipass 2.0 techniques.

Single dimension across multiple fact tables – The IT organization can configure the Universe (BusinessObjects Semantic Layer) to generate multiple SQL statements when measures exist in from multiple fact tables.

Using the grains of measurement in the same query -  Sometimes you want to be able to do create a query that shows Previous YTD, YTD, QTD, MTD for the same measure, e.g. Revenue.

Requiring the end results to be used in a calculation – Displaying values such as % contribution, ratios (revenue per employee), etc.

Semi-additive measures – My personal favorite.  These measures are traditionally measures that aggregate over all dimensions EXCEPT time.  Some examples are stock level, account balance.  The stock level in the database display the number that are on-hand as of the end of the day.  The account balance displays how much money is in my account and the end of the day.  If I look at these figures across geographies, then they need to be summed up; however if I want to look at the values for the current month, I should only use the value as of today.

Analyzing a subset of data – Sometimes I want to be able to analyze a subset of data to another subset of data.  For example, get all the latest customer transactions and compare them to the historical transactions.

In every single one of these cases, BusinessObjects delivers the results that the user needs without creating temporary tables and inserting thousands of records.  Instead BusinessObjects uses a number of techniques including:

• Derived Tables
• Multi-SQL Universe Options
• Universe Contexts
• Query-Drill
• Report Variables

How To Whitepaper Available

If I’ve whet your appetite for more information, then you will want to download the whitepaper from the SAP SDN site.  This whitepaper explains in detail exactly how to accomplish each of these scenarios using BusinessObjects.  I’ve had some problems with these documents from time to time, so I’ve also made them available from my blog directly.  You can download the whitepaper here and the powerpoint here.

The bottom line is that modern high performance databases are experts at handling extremely complex SQL.  Their optimization engines mean that it is no longer necessary to break queries down into managable pieces and perform multiple passes in the traditional sense.

What Do You Think?

If  you have a complex query scenario that can’t be solved by one of the techniques used in this whitepaper, I’d love to hear from you.

Enjoy.

«Good BI»

Now that SAP has clearly and unambiguously stated that BusinessObjects is the new face of BI for SAP, we have many customers who want more information about connectivity with SAP and are surprised when we recommend that they connect to an existing BW Query instead of to the entire Infocube.

Why is connecting to SAP BI/BW Queries a best practice?

There are several reasons for connecting via SAP BW Queries:

• Performance – The BW OLAP Engine performs its processing against the query cube. In the event, you connect an external analysis tool directly to an InfoCube, a query cube consisting of all the characteristics and key figures in that InfoCube is generated on the fly in order to handle the request processing. This process adds significantly to the processing time of the request made by the end-user when compared to reporting against a pre-defined BW Query.
• Capabilities – There are several capabilities exposed via the interface to BW Queries that are not exposed when connecting directly to the cube such as calculated and restricted key figures to name a few.
• Flexibility – The BW Query Designer offers an extension to the data modeling environment in which changes are relatively easy to make as compared to the effort required to change an InfoCube.

Is this required across all the reporting tools? (WebI, Voyager, Crystal Reports)

Its mostly applicable to WebI and Voyager. There’s more flexibility with Crystal since it has more connectivity interfaces to SAP and it is not designed to be an ad-hoc reporting and analysis tool. Xcelsius will typically be built against scheduled content since most dashboard data is usually only updated every 24 hours.

What are the advantages and disadvantages of this different approaches to SAP Connectivity?

Advantages of BW Query – The BW Query can take advantage of Query capabilities such as CKF and RKF. The BI Team can also provide better control over how much data user can request.

Advantages of InfoCube – The BI Team can expose entire dataset to user without additional work from BI team

Disadvantages of BW Query – Using BW Queries requires additional work from BI team to setup queries specifically around areas of the business, e.g. marketing, sales, etc. It limits amount of information available to user for a given query. That being said, the adhoc report designer can combine multiple queries within a single report. In addition a well defined BW Query can serve as the data sources for multiple reports. (whereas traditionally in SAP BW, we saw 1 query = 1 report)

Disadvantages of InfoCube -Users can get lost in the large amount of data available to them. There is a lack of CKF and RKF, which will lead to users creating more filters and calculations at the report level. These efforts will likely be duplicated across multiple reports. Duplication leads to duplicate work and the potential for different answers based on different formulas (i.e., no “single source of the truth” for those formulas)

Performance of BW Query – Better. Because of the constraint on information available to the end-user, the query design can test a subset of data and guarantee a certain level of performance.

Performance of InfoCube – Vulnerable. Because the entire cube is visible, there is no what to know what combinations of information the user might pick. Without specific controls over the type and amount of data returned, it is impossible to determine who long the query might run.

Flexibility of BW Query – The BW Query is more flexibility in regards to exposing “processed” data (CKF and RKF). It also provides flexibility when it comes to making changes to underlying data because not every change to the InfoCube will impact a BW Query.

Flexibility of InfoCube – The Infocube is the most flexible with regards to the data exposed.

What data structures in SAP are supported within BusinessObjects?

WebIntelligence/Voyager supports: Standard and Transactional InfoCubes, Remote InfoCube (not recommended due to performance), MultiCube, MultiProvider.

Crystal Reports supports: Standard and Transactional InfoCubes, Remote InfoCube (not recommended due to performance), MultiCube, MultiProvider, ODS, R/3 Infosets, ABAP Queries, ABAP Functions. ABAP Data Clusters, Transparent Tables, Pool Tables, Cluster Tables, zTables, InfoViews

Conclusion

It’s been exciting to see how quickly SAP customers have been able to unlock the data that they have in their Business Warehouse to a whole new groups of users.  Users can now create their own reports from scratch using the WebIntelligence interface.  Executive users and front-line contributors can have their personalized data delivered directly to the desktop.

I can’t wait to see what will unfold within our next generation of tools.

So how we do do this exactly? How can I use BusinessObjects to create custom groups on-the-fly? How can I get beyond my requirements for multi-pass SQL and speed up my reports? The answer is Sets.

Set Data

All the information regarding sets is located is the tables called sets_*. The most important tables are:

• set_set – contains the set name, description, etc.
• set_set_detail – contains all the set member information including member key, join date, leave date.

Here is a universe I created that you can download and use with the sample database provided in my earlier blog post. We need to add the set_set and set_set_detail tables to our WebIntelligence Universe so that the set member information they contain can be used within our query.

Here you can see that we created a join to the DW_CRM_CONTACT table. We also created a self-join on the set_set_detail table. The self join make sure that we reference the edate value of 12/31/2999, which is the default edate for current members of any set. In the case of a dynamic set, If a member had left the set then the edate would not be equal to 12/31/2999.

In addition to this, we also joined the set_set table to the set_set_detail table using the setversion_id key. (It may appear that the setversion_id is always the same as the set_id, but it is not so only join the tables via the setversion_id)

Here is a look at the Custom Product Groups

BEST PRACTICE: Whenever you may need to alias a table. If you alias a table, then all references to that table should be aliased. You can see that because I needed to reference the set_set_detail and set_set tables twice, once for products and once for contacts, I aliased both.

User Prompts

After joining the tables, we needed to create the filters for our custom groupings to make it easy for users to select which groups they would like to use on the report.

I specifically defined the filters a multi, which will allow the user to select more than one filter. This is important because it provides even more power for your end-users. Now they can do direct comparisons of one group to another, even if a member (e.g. contact) exists in more than one group. The only downside is that it means that a member can be counted twice, this is why I recommended that when you define your sets you should use naming conventions that identify the sets as being mutually exclusive.

Here is what the prompt will look like in action:

In my universe I did not use cascading prompts to first limit the sets by folder, but that is certainly an option since folder information is located in the set_project table.

Getting Results

After configurating the universe and setting up the prompts this is an example of the results you will see. I have run a report and selected multiple custom groups which are mutually exclusive. These groups on based on the current periods sales amounts:

Here you can see the report is broken down by my custom groups which were defined and processed by the Sets Engine.

Summary

I’ve really enjoyed taking you on a quick tour of Set Analysis and showing you some of the ways that you might get more value out of your existing data assets. Sets offers so many solutions around data analysis:

• Custom Groupings – which allows the organization to more nimble as changes such as mgmt structures, mergers and acquisitions take place.
• Classic Segmentation – Joiners, Leavers, Stayers (which in turn are perfect for analysis by a predictive tool so that you can see why are people joining or leaving? What are the influencers?)
• Advanced Analysis – “Give me all quad-pack customers who have never purchased phone accessories from a company owned retail store.”

The ability to divide customers up into categories of: highly profitable down through :unprofitable, and using that to drive the business because the bottom line of any business is the customer and your organization’s ability to make a profit.

Although dashboards are currently in vogue and everyone wants one, I often wonder how organizations are analyzing the information beyond the dashboards. I have nothing against dashboards, but I would encourage you to ask yourself this. Once we see that sales are falling, what am we going to do? How am we going to do root cause analysis? Were all customers affected? What about products? Segmentation might well hold the answer to these yet unsolved mysteries.

I recently had a customer who was in the process of migrating from a legacy version of Microstrategy to BusinessObjects. They had been able to migrate over most of their core reports, but there were several reports that required a number of custom groups.

Microstrategy’s approach was to allow the customer to define any number of groups and filters from with the Microstrategy Desktop tool. These global filters and group definitions can then be leveraged by the report editor for building the report. Microstrategy’s SQL generation engine uses multi-pass SQL when creating the reports. Multi-pass SQL means that a single SQL statement cannot deliver back the correct results, so the report data is written into temporary tables. These temporary tables are then combined with additional queries to obtain the final results.

In the case of BusinessObjects, it wasn’t going to be possible to take the same approach. BusinessObjects does not generate multi-pass SQL. Instead, BusinessObjects focuses on creating extremely complex SQL in a single SQL statement. In most cases, this works just as well, but this was an exception. This sales report used a combination of multiple filters at multiple levels in the heirarchy. The Heirarchy was also defined within the custom groupings and not within the database.

The right answer is: Modify the Data Warehouse and ETL processes to make the underlying data better fit your reporting requirements. You see, the problem was that these custom groups should be defined within the data warehouse natively. This would allow for simplified SQL that can be run in a single pass. Translation? Reports the use custom groups will run a heck of a lot faster.

In my situation, the customer wasn’t going to be able to make these changes, but still wanted to migrate the report to BusinessObjects.

The best answer is: Use Sets. Sets gives you the ability to generate those custom grouping within the data warehouse. Unlike Microstrategy which will run the same multi-pass SQL over and over again, sets will allow us to “cache” the previous passes using sets and run single-pass SQL. We can then tie the set tables into the data warehouse and add the custom groupings to our universe.

The advantage of sets over ETL is that it can be much more dynamic. Sets can allow an analyst to create a new set in a matter of minutes and then see a report broken down by the new custom group.

In my case it was just what the doctor ordered. By introducing sets and adding the set tables to the BusinessObjects semantic layer, we now have access to the sets, which in this case, the customer called custom groups. It was perfect.

Now, not only will the customer be able to migrated this complex report from Microstrategy to BusinessObjects, but they will also be able to run the report much faster than ever before because the report will be generated through a single-pass process.

Tomorrow, we’ll take a look at how to set this up using our simplified data.

Can I Get Row-level Security with WebIntelligence and Still Avoid a Document Refresh?

Sure!

As a follow-up to my post yesterday I wanted to step back and say that normally you do want to handle row-level security from within the semantic layer. The problem is that you can’t get the benefit of the speed of data that comes from historical instances without requiring each user to run the report for themselves.

I did discover that you can provide the same type of security within WebIntelligence. WebIntelligence also has a function which also returns the name of the current user. The function is CurrentUser.

1. The first thing I need to do is build the semantic layer to include my security table.

2. Next, create a new variable which returns a 1 or 0 based on whether or not the user should see the data or not.

=If(CurrentUser()=”Administrator” Or CurrentUser() = [Username];1;0)

3. Create the report and apply the filter to the entire report or to the appropriate data block.

4. Here is what Ron’s sees, when he looks at an historical instance of the WebIntelligence report.

Notice that the Data Summary on the left hand side shows me that there were 71 records but even so, the restricted user ron is only allowed to 3 of the 71.

The beauty of this is that when I look at this data via BusinessObjects LiveOffice, I will see exactly the same restrictions enforced.

REMEMBER: Normally if you need row-level security, then just apply it to the universe. IF, however, you need row-level security on historical instances without requiring them to be refreshed, then use this great trick.