Now that SAP has clearly and unambiguously stated that BusinessObjects is the new face of BI for SAP, we have many customers who want more information about connectivity was SAP and are surprised when we recommend that they connect to an existing BW Query instead of to the entire Infocube.

Why is connecting to SAP BI/BW Queries a best practice?

There are several reasons for connecting via SAP BW Queries:

• Performance - The BW OLAP Engine performs its processing against the query cube. In the event, you connect an external analysis tool directly to an InfoCube, a query cube consisting of all the characteristics and key figures in that InfoCube is generated on the fly in order to handle the request processing. This process adds significantly to the processing time of the request made by the end-user when compared to reporting against a pre-defined BW Query.
• Capabilities - There are several capabilities exposed via the interface to BW Queries that are not exposed when connecting directly to the cube such as calculated and restricted key figures to name a few.
• Flexbility - The BW Query Designer offers an extension to the data modeling environment in which changes are relatively easy to make as compared to the effort required to change an InfoCube.

Is this required across all the reporting tools? (WebI, Voyager, Crystal Reports)

Its mostly applicable to WebI and Voyager. There’s more flexibility with Crystal since it has more connectivity interfaces to SAP and it is not designed to be an ad-hoc reporting and analysis tool. Xcelsius will typically be built against scheduled content since most dashboard data is usually only updated every 24 hours.

What are the advantages and disadvantages of this different approaches to SAP Connectivity?

Advantages of BW Query - The BW Query can take advantage of Query capabilities such as CKF and RKF. The BI Team can also provide better control over how much data user can request.

Advantages of InfoCube - The BI Team can expose entire dataset to user without additional work from BI team

Disadvantages of BW Query - Using BW Queries requires additional work from BI team to setup queries specifically around areas of the business, e.g. marketing, sales, etc. It limits amount of information available to user for a given query. That being said, the adhoc report designer can combine multiple queries within a single report. In addition a well defined BW Query can serve as the data sources for multiple reports. (whereas traditionally in SAP BW, we saw 1 query = 1 report)

Disadvantages of InfoCube -Users can get lost in the large amount of data available to them. There is a lack of CKF and RKF, which will lead to users creating more filters and calculations at the report level. These efforts will likely be duplicated across multiple reports. Duplication leads to duplicate work and the potential for different answers based on different formulas (i.e., no “single source of the truth” for those formulas)

Performance of BW Query - Better. Because of the constraint on information available to the end-user, the query design can test a subset of data and guarantee a certain level of performance.

Performance of InfoCube - Vulnerable. Because the entire cube is visible, there is no what to know what combinations of information the user might pick. Without specific controls over the type and amount of data returned, it is impossible to determine who long the query might run.

Flexibility of BW Query - The BW Query is more flexibility in regards to exposing “processed” data (CKF and RKF). It also provides flexibility when it comes to making changes to underlying data because not every change to the InfoCube will impact a BW Query.

Flexibility of InfoCube - The Infocube is the most flexible with regards to the data exposed.

What data structures in SAP are supported within BusinessObjects?

WebIntelligence/Voyager supports: Standard and Transactional InfoCubes, Remote InfoCube (not recommended due to performance), MultiCube, MultiProvider.

Crystal Reports supports: Standard and Transactional InfoCubes, Remote InfoCube (not recommended due to performance), MultiCube, MultiProvider, ODS, R/3 Infosets, ABAP Queries, ABAP Functions. ABAP Data Clusters, Transparent Tables, Pool Tables, Cluster Tables, zTables, InfoViews

Conclusion

It’s been exciting to see how quickly SAP customers have been able to unlock the data that they have in there Business Warehouse to a whole new groups of users.  Users can now create their own reports from scratch using the WebIntelligence interface.  Executive users and front-line contributors can have their personalized data delivered directly to the desktop.

I can’t wait to see what will unfold within our next generation of tools.

I think I may have found what you are looking for. This is an excellent guide that you can use to bring best practices to all your Xcelsius work. This 16 page guide is an well thought out guide for using standard around colors, data organization, testing, etc. Implementing these best practices will undoubtedly improve your ability to create world-class Xcelsius models that are easy for others to maintain after the fact.

This guide was written by Matt Lloyd with contributing materials from Ryan Goodman and Richard Reynolds.

Download it today!! You’ll be glad you did.

Remember the Mouse revolution? I’ll never forget the first time I saw a Macintosh and used the mouse to move a pointer across the screen. I could start programs, paint pictures and do all kinds of cool things without learning a new interface.

Well, Jeff Han recently introduced the world to an entirely new generation of user interface during the annual TED conference. It is a Minority Report moment.

Its a new multi-sensor touchscreen which allows the user interface to completely disappear. You have to see it to believe it. It is truly incredible.

http://www.ted.com/index.php/talks/jeff_han_demos_his_breakthrough_touchscreen.html

So how we do do this exactly? How can I use BusinessObjects to create custom groups on-the-fly? How can I get beyond my requirements for multi-pass SQL and speed up my reports? The answer is Sets.

Set Data

All the information regarding sets is located is the tables called sets_*. The most important tables are:

• set_set - contains the set name, description, etc.
• set_set_detail - contains all the set member information including member key, join date, leave date.

Here is a universe I created that you can download and use with the sample database provided in my earlier blog post. We need to add the set_set and set_set_detail tables to our WebIntelligence Universe so that the set member information they contain can be used within our query.

Here you can see that we created a join to the DW_CRM_CONTACT table. We also created a self-join on the set_set_detail table. The self join make sure that we reference the edate value of 12/31/2999, which is the default edate for current members of any set. In the case of a dynamic set, If a member had left the set then the edate would not be equal to 12/31/2999.

In addition to this, we also joined the set_set table to the set_set_detail table using the setversion_id key. (It may appear that the setversion_id is always the same as the set_id, but it is not so only join the tables via the setversion_id)

Here is a look at the Custom Product Groups

BEST PRACTICE: Whenever you may need to alias a table. If you alias a table, then all references to that table should be aliased. You can see that because I needed to reference the set_set_detail and set_set tables twice, once for products and once for contacts, I aliased both.

User Prompts

After joining the tables, we needed to create the filters for our custom groupings to make it easy for users to select which groups they would like to use on the report.

I specifically defined the filters a multi, which will allow the user to select more than one filter. This is important because it provides even more power for your end-users. Now they can do direct comparisons of one group to another, even if a member (e.g. contact) exists in more than one group. The only downside is that it means that a member can be counted twice, this is why I recommended that when you define your sets you should use naming conventions that identify the sets as being mutually exclusive.

Here is what the prompt will look like in action:

In my universe I did not use cascading prompts to first limit the sets by folder, but that is certainly an option since folder information is located in the set_project table.

Getting Results

After configurating the universe and setting up the prompts this is an example of the results you will see. I have run a report and selected multiple custom groups which are mutually exclusive. These groups on based on the current periods sales amounts:

Here you can see the report is broken down by my custom groups which were defined and processed by the Sets Engine.

Summary

I’ve really enjoyed taking you on a quick tour of Set Analysis and showing you some of the ways that you might get more value out of your existing data assets. Sets offers so many solutions around data analysis:

• Custom Groupings – which allows the organization to more nimble as changes such as mgmt structures, mergers and acquisitions take place.
• Classic Segmentation – Joiners, Leavers, Stayers (which in turn are perfect for analysis by a predictive tool so that you can see why are people joining or leaving? What are the influencers?)
• Advanced Analysis – “Give me all quad-pack customers who have never purchased phone accessories from a company owned retail store.”

The ability to divide customers up into categories of: highly profitable down through :unprofitable, and using that to drive the business because the bottom line of any business is the customer and your organization’s ability to make a profit.

Although dashboards are currently in vogue and everyone wants one, I often wonder how organizations are analyzing the information beyond the dashboards. I have nothing against dashboards, but I would encourage you to ask yourself this. Once we see that sales are falling, what am we going to do? How am we going to do root cause analysis? Were all customers affected? What about products? Segmentation might well hold the answer to these yet unsolved mysteries.

I recently had a customer who was in the process of migrating from a legacy version of Microstrategy to BusinessObjects. They had been able to migrate over most of their core reports, but there were several reports that required a number of custom groups.

Microstrategy’s approach was to allow the customer to define any number of groups and filters from with the Microstrategy Desktop tool. These global filters and group definitions can then be leveraged by the report editor for building the report. Microstrategy’s SQL generation engine uses multi-pass SQL when creating the reports. Multi-pass SQL means that a single SQL statement cannot deliver back the correct results, so the report data is written into temporary tables. These temporary tables are then combined with additional queries to obtain the final results.

In the case of BusinessObjects, it wasn’t going to be possible to take the same approach. BusinessObjects does not generate multi-pass SQL. Instead, BusinessObjects focuses on creating extremely complex SQL in a single SQL statement. In most cases, this works just as well, but this was an exception. This sales report used a combination of multiple filters at multiple levels in the heirarchy. The Heirarchy was also defined within the custom groupings and not within the database.

The right answer is: Modify the Data Warehouse and ETL processes to make the underlying data better fit your reporting requirements. You see, the problem was that these custom groups should be defined within the data warehouse natively. This would allow for simplified SQL that can be run in a single pass. Translation? Reports the use custom groups will run a heck of a lot faster.

In my situation, the customer wasn’t going to be able to make these changes, but still wanted to migrate the report to BusinessObjects.

The best answer is: Use Sets. Sets gives you the ability to generate those custom grouping within the data warehouse. Unlike Microstrategy which will run the same multi-pass SQL over and over again, sets will allow us to “cache” the previous passes using sets and run single-pass SQL. We can then tie the set tables into the data warehouse and add the custom groupings to our universe.

The advantage of sets over ETL is that it can be much more dynamic. Sets can allow an analyst to create a new set in a matter of minutes and then see a report broken down by the new custom group.

In my case it was just what the doctor ordered. By introducing sets and adding the set tables to the BusinessObjects semantic layer, we now have access to the sets, which in this case, the customer called custom groups. It was perfect.

Now, not only will the customer be able to migrated this complex report from Microstrategy to BusinessObjects, but they will also be able to run the report much faster than ever before because the report will be generated through a single-pass process.

Tomorrow, we’ll take a look at how to set this up using our simplified data.

Since the merger with SAP, we’ve begun to adopt most of the SAP policies. One policy is related to vacation. SAP strongly encourages employees to take their vacation each year and not allow it to pile up. In fact, the maximum amount of vacation you are allowed to roll over each year in 5 days. This has lead some individuals to get serious when it comes to using up their vacation.

Here is an email reply I received from a colleague recently:

Now that’s what I call a vacation!!

A less common way to create sets is to create sets is through free-hand SQL. Normally you will want to leverage to power of the set engine to design the SQL for you to return the correct results, but in rare cases you might find it helpful to create your own SQL Statements.

Free Hand SQL

The only item that needs to exist in the SELECT clause is ID. If an ID is specified and the SQL is valid, then the records will be imported into the set tables, if not, you will not be able to chose “Parse” to validate your SQL.

For our example, we can create a set called “All Customers” and use the following FreeHand SQL.

select CONTACT_ID as ID from dbo.DW_CRM_CONTACT

The SQL may be as complex as you require. As long as the SQL is valid, you should be able to insert it here and use it with Set Analysis.

I am not aware of a limit for the Free Hand SQL.

Import

Set Analysis also provides the ability to import individual records into the set tables. This is valuable if users have exported information into Excel and manually developed a list of records they wish to add to a set.

You can either store the text file in the BusinessObjects Repository or on the physical filesystem. Unfortunately the schema definition is pretty limited, so I recommend you use one of the sample files below as a guide. As long as the file contains at least 1 column and the key is the FIRST column, the import will work fine.

Click on the Import button and pick your file. Do not type in the name manually. If necessary, you may need to define a file format. I recommend that you use comma delimited files with a double-quote text qualifier (”).

Here is a sample import file of DM Contacts.

NOTE: In order to use the Upload feature, a c:\tmp directory must be defined on your system.

NOTE: If you store the file on a filesystem, you may need to make sure that the Sets Query Service (AAQueryMgr.exe) has rights to access the file across the network. By default all BusinessObjects services run as LocalSystem which means by default they will not have access to UNC file paths
.

I’ve decided it’s probably a good idea to break down these tutorials a bit. Although I love to share all this information, it’s probably getting a little long for a single post.

Visual Sets

When creating sets in Set Analysis, Visual Sets are extemely helpful. Visual Sets allow you to quick look at the union, intersection, exclusion of multiple sets simultaneously. Let’s say for example I want to see all Gold Customers who are also High Tech Customers but have NOT purchased products from our computer department. Let’s assume I already have the following three sets:

• Gold Customers
• High Tech Customers
• All Products from the Computer Department

Let’s create a new static Set using DM Contacts, called Gold Customer - No Computer. Next we will add a Visual Set. We will chose 3 sets. One set for each of the three Collections.

• Collection 1(blue) - add Gold Customers (All Gold Customers)
• Collection 2 (red) - add High Tech (All High Tech Customers)
• Collection 3 (yellow) - add Computer (All Products in the Computer Department)

In our case we want all Gold Customers who have NOT purchase Products in the Computer department, therefore we do not want to select anything in the yellow circle.

I can click on Counts at any time and the system will show many how many records are in the selected set.

From the selected venn diagram regions, I can see from the displayed count that there are 78 customers who fall into this category. Click Next>>, specify the description and OK to save the set.

Remember, just like before we must build the set before it will contain any objects. Once you build the set you will see there are 78 records retrieved.

Isn’t that easy? How cool is that?

NOTE: Remember how we specified Converters in Set Analyzer Architect? This converter allows us to chose information from DM Products, but have it converted products to contacts. In our example, it converted “products in the computer department”, to contacts who have purchased products from the computer department.

w that we’ve spent some time with sets, it’s time to begin to see the technology in action. Business Objects has made sets easy to build and maintain. There is virtually no limit to the ways you can now choose to look at your customers and look at how those customers are contributing to revenue, profit and margins.

Using Sets to Maximize Revenue

Click HERE is a quick online demo that we’ve built to explain how some of our telecommunications customers use sets to maximize revenue, cross sell and up sell opportunities.

Creating Sets

The beauty of sets is the fact that you don’t have to involve IT in making changes to the underlying data structures to segmentation. With the set tables that come with Set Analysis, end users can create their own custom collections of products, customers, etc. BusinessObjects users will then be able to use those sets together with the BusinessObjects Suite to provide in depth reporting of key metrics and kpis with respect to these custom collections of customers. Whether you want to track standardized silver, gold and platinum customers or time, or simply do some adhoc customer analysis, sets gives you what you need.

In XI 3.0 you can create sets using our Performance Manager dashboard interface. A set can contain data for only one subject. The subject is normally identified as a primary key in a dimension table. In the case of the Customer subject, the Customer Id key will be the key we are tracking.

Create Calendar

The single remaining configuration component that still remains is to create a calendar. Calendars are used by BusinessObjects to allow administrators to automate task such as refreshing sets and to allow us to track the changes of sets over time. We need to tell BusinessObjects how often track the changes in our sets.

Based on continuing our demo from the my previous post, lets create a calendar monthly. To create calendar, navigate to the Dashboard & Analytics Setup. Next, choose the Time Config > Calendar submenu. Here is where you can see a list of all available calendars. Let’s choose New Calendar

Based on the same data provided in my previous post, we will create a calendar:

1. Name: DM_Monthly
2. Interval: Monthly
3. Format: First Day of Period, 9/21/04
4. Range: Jul-00 to Mar-03

After you have created the calendar the screen should look like this:

NOTE: Once a calendar has been created, you cannot change values such as interval.

Now that the setup is complete, we can move to Dashboards & Analytics. Chose the Set Analysis > Sets submenu.

The first thing we need to do is create a folder so that we can categorize our sets. Choose New Folder and let’s name our folder: Customer Types.

Let’s also create a second folder called: Customer Segments.

Now let’s create our first set.

Static Sets

The first set we will create will be a static set. A static set is a collection of objects that either:

• Doesn’t change over time or
• if it changes, we are not concerned about tracking those changes.

Some examples of static customer sets are:

1. Customers Living in Montreal
2. Homeowners with more than one child
3. Key accounts younger than 30 years old
4. Customers in the UK with sales over 10,000 in 2001

We will be able to create a set of customers using any of the customer table reference that we setup previously. Let’s create a set of all High Tech GenX Customers, that is: All Customers with Internet and in the 30-40, 40 - 50 Age Bands.

First we need to name our static set and set some attributes for our set. Our set will be called:

1. High Tech GenX
2. Folder: Customer Types
3. Description: This set will contain a list of all customers who are considered Highly Technical, Generation Xers.
4. Scope: Public
5. Set Type: Static

Click Apply. Now we will specify the content of the set. In our case, we will use Visual Data.

NOTE: The interface can be a little tricky. You will see Add/Copy/Remove buttons next to Sets and also find Add/Edit/Remove/Empty button within the Set Building Sets section. The buttons on the left operate on the entire set itself. The buttons on the right only affect the selected building step.

There are 5 was to add information into a set. They are:

• Visual Data - Allows you to look at all the fields associated with your subject (e.g. customers) and visually see how many records will be included in the set. (Most Popular)
• Visual Sets - Allow you to reuse existing sets to create new sets. For example you can visually see the intersection between High Tech GenX Customers and Gold Customers for example.
• Free Hand SQL - Allows the users to specify their own SQL against the database. (Not Recommended)
• Event Analysis - Allows you to create new sets based on a records movement within a dynamic set for a specific time period. For example, all customers who were leavers between Oct-02 and Dec-02.
• Import - Allows you to manually import records (e.g. contact_id values) from a csv or txt file.

We will chose Add > Visual Data and be taken to the window, where we can select the attributes of our set members.

We will chose:

• Has Internet - Y
• Age Range - 30-40 and 40-50

We will select the attribute on the left and choose the arrow to move the attribute to the right. (If prompted, chose Exact Counts). The system will tell you that 1135 contacts have Internet. Choose the Y checkbox.

Next select the Age Band attribute on the left and expand the section. You will see that of those with Internet, 321 are aged 40-50 and 302 are aged 30-40. Select both of these checkboxes. This means that when we build the set, we will expect a total of 623 members.

Now choose Next and fill in the Description: Internet and Aged 30-50

Now we can choose OK and we can see the resulting set.

We could continue to add additional steps, which would allow us to modify this set further (allowing us to add or subtract members), but our example will only have one step. Now we need to build the set.

Building Sets

Building Sets is the process of the Set Analysis engine using the definition of the set to generate the appropriate SQL statement(s) that brings back the correct records. It will then insert those records into our set tables for future reference.

In the screenshot above, you can see the cursor next to the Build… icon. Click build now and watch as the set engine identifies 623 members for the set. Until you build the set, your set will have zero members. You an always see how many members a step and the last date/time the set was rebuilt.

If at any point you edit the building sets of a static set, the set will be automatically emptied (all set members will be deleted) and you will need to rebuild the set.

Dynamic Sets

The process of creating dynamic sets is very similar to that of static sets, except the set will be processed for specific time periods. Let’s create a dynamic set of Gold Customers. Gold Customers are customers who have made volume purchases and purchased more than $3000 of products from us in each period. We will again use the Visual Data to build our new set.

First we need to name our static set and set some attributes for our set. Our set will be called:

1. 1 - Gold Customers
2. Folder: Customer Segments
3. Description: Our Worldwide Gold Customers
4. Scope: Public
5. Set Type:Dynamic
6. Calendar: DM_Monthly
7. Membership History: Temporal (allows entire history of set membership to be kept)
8. Allow Auto-Fill: Yes

Click Apply. Now we need to build our first step. Let’s choose Add > Visual Data.

First, when specifying dynamic, temporal sets, it is important to specify the time element as the first component of the set detail. We want to make sure we are only including customers who have made a purchase is the current period. Because the sales date is part of the Orders Subject, we need to choose DM Orders.

Use the arrow to move the Start Date to the right hand side. You will be prompted to specify the Added Field Condition. We will need to specify, Start Date, between, with the values of Relative, Fixed, start of processing period and Relative, Fixed, end of processing period. The Yes/No condition should be selected.

In our case we will see that 18107 records are No. By default No may be selected. Make sure and change it to Yes.

TIP: Usually you will want Yes/No conditions to be Yes, so under Options (above the first Yes/No Condition line), you can chose the setting: Default Yes/No conditions to Yes.

Next, let’s look for Contacts who have purchase at least 12 items from us in a single monthly period. So chose Sum of Volume. You need to be careful here. Notice how we chose Sum of Volume. This is critically important, otherwise it would only choose customers with sales of one transaction with a Volume of 12. That’s not what we want. We want to look at total volume for the entire period.

Finally we want Contacts who have purchased more than $3000 of items in the same period. Let’s chose Sum of Actual Sales greater than 3000.

Now chose Next>> and enter a description: High Volume, High $$ and click OK.

After creating the set, BusinessObjects gives you the option to Autofill the set. The Autofill process uses the set engine to populate the historical values for the given set to a predetermined number of periods. In our case, we will autofill all 17 monthly periods from Jun-00 to Sep-03.

Click on AutoFill/Rollback…, Complete (radio button) and OK.

Now the request to process the set historically will be sent to the Set Analysis engine and after it processes all 32 periods, you can click on the Membership view and see the an historical view of the set.

Here you can see each period and the number of joiners, members and leavers.

Create two more sets for Silver and Bronze customers with volume purchases. I recommend Silver as Volume: 12 or more, Sales: 1000 - 3000. I recommend Bronze as Volume: 12 or more, Sales: <1000.

NOTE: If you want to get more members for each period, you can drop the volume requirements and only look at Amount of Sales.

Tips For Organizing Sets

It is important to note that you cannot create subfolders, so I recommend you create development standards when it comes to sets. Because all folders and sets are organized in alphabetical order, I recommend that you use the colon “:” to differentiate folder depth.

For example:

Level1
Level1:Level1a
Level2
Level2:Level2a
Level2:Level2b

In the same way, use colons, numbers and names to organize your sets (1, 2, 3 or A, B, C will help create order.) For example, if I am going to create three sets which are mutually exclusive, such as Gold, Silver, Bronze, I would name them:1-Gold, 2-Silver, 3-Bronze. Now they will appear in the correct order. If however I am going to put more than 1 set in the folder, I might called it ProjA:1 - Gold, ProjA:2 - Silver and ProjA:3 - Bronze. That way I can call the next ones ProjB, etc (or whatever project name made sense in my situation).

Try to keep set names to less than 25 characters.

Consider using other symbols such as asterisk(*), plus(+), etc at the end of the set name to indicate addition important information (such as the fact that this set depends on another set).

If you don’t organize your sets, they will become cluttered. Another advantage of these types of naming conventions is that all this set information is stored in a relational database, so we can create BusinessObjects reports which can help us keep our sets organized. We can parse out colons an other special characters which may have special meaning to the organization.

TECHNICAL DETAILS

set_set - contains the information about the specific sets

set_set_details- contains the information about the set members. We use set_set_details to determine the joiners, stayers, leavers for a specific time period (typically monthly)

I have found that setting up the Metadata around Sets was one of the most confusing things about learning sets. The problem was two-fold. The interface of Set Architect is old and not terribly intuitive. It is little more than a database table browser with a few rules built in.

I typically learn by example, so I put together what I felt was the best combination of screenshots to show you the core of what you will likely run into when you need to configure Set Architect.

To download the sample database that we used for this example, please click here.

You may find trouble reading through the Set Architect manual before install the Set Analyzer client because the document isn’t available on our website. The document is contained within the installer program but I thought it might be a good idea to make it available here in case you need it.

• Set Analyzer Administrator Guide - Administration and Setup Guide, which primarily uses Set Architect (required for using Sets)
• Set Analyzer User Guide - Legacy documentation on how to use the Set Analyzer client tool. This tool was made obsolete because the primary interface for sets has been integrated into the Business Objects Performance Manager dashboard interface.

Metadata Configuration within SetArchitect

There are four core components of the metadata that need to be configured from within Set Architect. They are the:

• Tables/Objects - the database tables and the field definitions we will use when generating SQL to create our sets.
• Subjects - traditionally the dimension tables. Subjects are types of data that we might want to create a set of. For example Customers, Products, Suppliers, Contacts
• Lookups - tables that are used to lookup decodes, e.g. customer status, type, category.
• Converters - advanced metadata that tells Set Analysis how to convert a query from one subject to another. Converters are used for the most powerful types of sets.

Let’s look first at the database tables and database fields metadata, which is called Tables and Objects respectively.

Configuring Tables/Objects Metadata

We need to define the metadata for four tables in our example: CONTACTS, SECTORS, PRODUCTS and FACT.

CONTACT TABLE AND FIELDS

The first table we needed to set up is customer. The unique key for this table is Contact Id. Because the customer table is not a lookup table, we indicate that this is indeed a source table.

Most of this metadata can be imported automatically when you use the Load Tables and Load Fields button at the top of the interface.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only a few modifications to the default metadata that was imported using the import tables button. I added:

• to contact_id, the linked table DM Contacts
• to sector_id, the linked table DM Contact Sector
• a new field called Annual Income Band with the following formula (SQL Server Syntax):
  case when <a>.annual_income = 0 then ‘Unknown’ when <a>.annual_income < 30000 then ‘<30K’ when <a>.annual_income < 50000 then ‘30-50K’ when <a>.annual_income < 75000 then ‘50K-75K’ when <a>.annual_income < 125000 then ‘75K-125K’ else ‘125K+’ end
• a new field called Age Band with the following formula (SQL Server Syntax):
  case when <a>.age < 30 then ‘<30′ when <a>.age < 40 then ‘30-40′ when <a>.age < 50 then ‘40-50′ else ‘50+’ end

The new fields could have been defined within a database view (which is what I would recommend), but I wanted to demonstrate how they can also be added directly from within the metadata of Set Analysis. The syntax will be database specific. (There is much more information about this in the Set Architect Manual)

SECTOR TABLE AND FIELDS METADATA

The second table we needed to set up is sector. The unique key for this table is Sector Id. Because the sector table is a lookup table for customer, we indicate that this is indeed a Lookup Object and NOT a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only one modifications to the default metadata that was imported using the import tables button. I added:

• to sector_id, the linked table DM Contact Sectors

As I’ve mentioned before you can define lookup tables from within Set Architect, but why? Why not create a new view and join the Contacts and Sectors tables into a common view on the sector_id? By pushing this to the database, you can simplify your configuration and you don’t have to worry about a lookup tables with lookups (e.g. state -> country -> continent).

PRODUCT METADATA

Next is the product table. The unique key for this table is Product Id. Because the product table is not a lookup table, we indicate that this is indeed a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only one modifications to the default metadata that was imported using the import tables button. I added:

• to product_id, the linked table DM Products

FACT METADATA

The fourth and final table we needed to set up is fact table. There is no unique key for this fact table, but that doesn’t matter because we won’t be creating sets on top of orders. (If we needed to, then we would need an order_key which could be inserted into the set_ tables for linking). Because this fact table is not a lookup table, we indicate that this is a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only a few modifications to the default metadata that was imported using the import tables button. I added:

• to contact_id, I added the linked table DM Contacts
• to product_id, I added the linked table DM Products
• a new field called Quarter with the formula (SQL Server Syntax):
  case when <a>.week between 1 and 13 then ‘Q1′ when <a>.week between 14 and 26 then ‘Q2′ when
  <a>.week between 27and 39 then ‘Q3′ else ‘Q4′ end

Keep in simple. In real life, if you wanted to add additional calculations like quarter or profit (sales_actual - cost), then do it in the database or through a database view. This will make your life easier when it comes to long term maintenance.

Once all the tables have been defined, we can move to Subjects.

DEFINING SUBJECTS

Subjects of a set are the central theme of the set. it will be associated with the table that most uniquely describes the subject for which sets are created. In our example, we are using contacts and products. We want to be able to create sets of both contacts and sets of products, so we will define two subjects.

Contact Subject

Here is how we will define the Contact Subject. (to enlarge click on picture)

Product Subject

Here is how we will define the Product Subject. (to enlarge click on picture)

Next, we need to check our lookups. This information should already be defined based on the changes we made when defining the sector table and defining it as a lookup for contacts.

DEFINING LOOKUPS

Here is how we will define the metadata for the sector lookup. (to enlarge click on picture)

DEFINING CONVERTERS

Converters are just plain cool. When I went through training on sets, I never really grasp what it is they do or their role in defining sets. Converters are used to allow you to convert from one subject to another. Clear as mud? Well, actually they allow you to define sets based on the “related” subject. Here are a couple of great examples:

• The set of customers that buy these products
• The set of products that have been purchased by these customers
• The set of other products sold with these products

So now you can track the set of Gold Customers who have purchased Premium Products during the period.

Converting Contacts to Products

The SQL we used for our converter was:
select distinct PRODUCT_ID as ID
from DW_CRM_FACT_ORDERS
where CONTACT_ID in ( <sub_query> )

Converting Products to Customers

The SQL we used for our converter was:
select distinct CONTACT_ID as ID
from DW_CRM_FACT_ORDERS
where PRODUCT_ID in ( <sub_query> )

The help text is completely optional and is not exposed to Set Analysis within the Performance Manager dashboard.

CONCLUSION

We’ve had an opportunity to walk through an example that covers the most common scenarios that I’ve seen in the field. If you have additional questions or comments, please list them below. I will use the database and metadata associated with this article to demonstrate the power of sets.

Technical Footnote

The information is stored in the following tables:

• set_table - contains Table metadata
• set_field - contains Field/Object metadata
• set_transform - contains the Converter metadata

Additional metadata we didn’t cover above:

• set_calendar & set_period - contains the calendar information that is generated by the Performance Manager engine from within the dashboard interface.
• set_user - contains the user login information
• set_user_group - contains the user group information
• Other tables of interest around security include set_profile, set_grp_member, set_grp_source, set_grp_subj.

When I was growing up I always enjoyed putting puzzles together. I loved the challenge of putting all those pieces together so that I could see the whole picture. Sometimes I would borrow a puzzle from a friend and as I was close to finishing the puzzle, I would discover 1 or 2 puzzle pieces were missing. The puzzle was incomplete.

From a BI perspective, APOS has been a long partner of BusinessObjects. My first opportunity to work with APOS came while I was working at Crystal Decisions. Our next generation Crystal Enterprise product was new and although it provided a great framework for the future, there were a few pieces missing from our complete BI picture. Thankfully, APOS developed a number of powerful solutions to fill those gaps so that our customers didn’t have to wait until tomorrow’s release for the functionality they needed today.

Do you feel you are missing a piece from BusinessObjects XI 3.0?

Business Objects has many great partners, but probably non-better than APOS. In January of this year, APOS was awarded the Business Objects Technology Partner of the Year Award for 2007. They have used the BusinessObjects SDK more extensively than any other BusinessObjects partner. They don’t just one solution. Their tools provide dozens of potential missing pieces that might fit your BI puzzle.

My Favorite APOS Tools

So what’s so great about APOS? Well, here are my top 3 list favorite solutions:

1. InfoScheduler - mass scheduling - dead simple. Scheduling has always been one of those areas were BusinessObjects added a few new features with ever release. Unfortunately however it’s not an area that tends to get a lot of attention. It was only in XI 3.0 that we introduced native bursting within the platform. InfoScheduler provides an Excel-based report scheduling tool that allows mass scheduling of reports from an Excel spreadsheet. How easy is that? Can you say zero training? Users can copy paste, import, export data into Excel to provide scheduling for 10, 100 or 1,000s of reports.
2. Object Manager - Although the CMC is a comprehensive administration console, administrators of large reporting environment often struggle with managing this volume of information. In XI 3.0, BusinessObjects introduced a number of new enhancements which made working with multiple objects much easier; however Object Manager takes it to the next level. You can manage users, groups, reports, security and even the report life cycle process. Object Manager provides power, ease-of-use and flexibility.
3. KPI - One of the most common requests I hear from customers is the ability to do reporting off their CMS. BusinessObjects XI provides an auditing database to allow customers to report off of user activity, but how do administrators report off the objects within the CMS? With KPI, APOS allows you to extract all the XML metadata that is stored within the CMS to an external relational database. Now you can create reports that allow you to:

• Analyze Business View and Universe configuration information
• Examine Crystal and Desktop Intelligence report structure
• See How many reports use the database field profit
• Show CMS Statistics
• Discover how many reports are scheduled to run tomorrow.

You can even set an automatic schedule to refresh the external metadata database on a regular basis. It’s your own mini-datamart for reporting off your BI system. You have to see it to believe it.

Southwest is using all three of these tools and several more!

Can’t Find Your Missing Piece?

If you’ve look through the APOS website and you can’t find the type of functionality that you feel is missing from BusinessObjects, call APOS anyway. Necessity is the mother of invention. Each of the solutions that APOS provides today was developed from the real-world needs of customers just like you. APOS would love to talk to you about extending BusinessObjects in a way that they may not have thought of.

Puzzle Complete

As a kid I was rarely able to find the missing puzzle piece, but in the area of extending BusinessObjects, APOS continues to deliver. Their depth of experience means they understand how BusinessObjects works today and how to make it work better for you tomorrow.

So How Exactly Does Set Analysis Work?

Set Analysis works together with the tables that are already part of your Data Warehouse. For example, in your Data Warehouse today you have customers, products and orders, etc. Therefore all the tables that below to Set Analysis must reside in the same database as your datasource. (From this perspective, I like was SQL Server allows you to do. In SQL Server, as long as you specify the database.owner.table, you can have the database tables in physically different database instances. It works because they can share the same connection.)

Installing SetAnalyzer

The first thing you might have problems with is finding the Set Analyzer software. Although the engines for using Set Analyzer are built into the BusinessObjects Platform, you still need to install the SetArchitect legacy client-server software to do some minimal configuration.

Traditionally BusinessObjects provided SetArchitect as a seperate download; however starting in XI 3.0, it is located with your BusinessObjects XI 3.0 installation media under the Add-Ons directory. The actual file path is: D:\BOEXI30_Install_Files\win32_x86\Collaterals\Add-Ons\SetArchitect.

You may have trouble finding the documentation for the legacy Set Architect client application. The documentation will be installed when you install the Set Architect Add-on from the BusinessObjects XI 3.0 installation media. However if you want to see the documentation prior to the installation you can download it here:

• Set Architect Documentation
• Set Analyzer Documentation

NOTE: Traditionally Set Analyzer existed as a client-server application, but once it was integrated into the BusinessObjects Platform, it no longer became necessary to use Set Analyzer. Business Objects has moved all but some very basic configuration steps to the web-based Performance Manager interface.

Set Tables

The tables which belong to the Set Analysis application all belong with set_. There are just over 100 tables in all. These tables definitions get created one of two ways. The set_ tables are created when:

1. The Business Objects administrator creates the repository from the Dashboard and Analytics Setup Screen. From within System Setup\Repository, the administrator defines the repository connection and “Creates” the Performance Manager tables. When using this method, not only are the set tables created, but the tables for all the BusinessObjects Performance Manager Engines are created. (Standard Method)
   
2. The Set Analysis Administrator uses Set Analyzer Architect to “Setup” the repository. This method will only create the set tables for the Set Analyzer application. (Legacy Method - Not Recommended)

Once the Set Analysis tables have been created, you will be able to begin configuring your Set Analysis Data Sources. In a typical BusinessObjects Performance Manager Repository, this is what you would see:

Defining Your Data Sources

When setting up Set Analysis for the first time, there are only two tasks that need to be performed from within the legacy Set Architect application.

1. You will need to define a connection, so that Set Architect understands how to connect to the database which contains your data warehouse data and the set analyzer tables.
2. Next, you will need to set the metadata for your data warehouse tables so that set analyzer understands how to query the underlying database and create the sets.

Optimizing Your Data Sources and Set Analyzer

I highly recommend that if possible, you create database views on top of all the data sources that you plan to use for creating sets. For example if you have a customer table with 5 or 10 associated look-up tables for things such as geography, demographics, customer details, etc; then join these tables together and create a master view. Be aware that it is very important that your view NOT contain duplicate customer keys.

in Set Analyzer you need to define subjects. These are non-lookup tables associated with the fact table. In traditional data warehousing venacular they would be called dimension tables. These dimension tables need to be defined as Subjects in Set Architect.

If there is a many-to-many relationship between two tables, then I would need to create two subjects for each of the two tables. For example, if products have many suppliers and suppliers supply multiple products, (e.g. a many to many relationship exists), then I will need to define suppliers and products as Subjects. This will allow me to create complex sets such as:

• All suppliers who supply less than 10 products.
• All products which are supplied by suppliers in Georgia, who supply at least 5 products.

In my next post, I’ll provide screenshots of a sample metadata configuration as well as an excel spreadsheet for you to download and use yourself. All you will need to do is import the data into your favorite database.

Do you have an old leather jacket? You know the kind. The jacket that reminds you of the good old days. Each time to try it on, it fits so perfectly, you wonder why you don’t get it out more often. It’s tough, it’s cool, it’s unique, and everyone should have one. When I bring it out and show the kids - they love it.

Well, from a software perspective, my “old leather jacket” is Set Analysis. I recently had a situation where a customer needed the capabilities of Set Analysis in their organization to solve some specific reporting needs. I had to dust off the manuals and fire it back up again, but when I did, I wondered why we’d been apart so long.

Set Analysis is a product that was created by Next Action and was acquired by Business Objects way back in October of 1999. Business Objects then launched a company called Ithena, which created analytic application and use set analysis extensively. I provide this background to let you know that the Set Analysis product has been around for a long time and it still brings value today.

Set Analysis provides ability to categorize information. The fancy term for this is called segmentation and is typically market or customers segmentation (although it also can relate to products, employee, etc.) This capability is critical to help organizations understand how the customers and products are affecting their business. We use collections, or sets, to make lists of items to communicate and to manage.

In this Venn diagram you can see there are three sets:

• Last year’s gold customers
• Current silver customers
• Customers at risk

Segmentation products let Set Analysis make it easy to pick where you want to focus. For example the light green area represents the collection of: Last year’s gold customers who are at risk but are not currently silver.

Here is another example related to mortgage providers:

1. Customers who have a good credit rating
2. Customers who are paying an above average interest rate
3. Customers who have not refinanced in the last 24 months.
4. Customers who have never made a late payment.

Customers who exist in a set are called members.

Next, a mortgage company might take all four collections and market those customers who are members of all four sets with the goal of offering them a more competitive interest rate before they shop their mortgage with a competitor.

So what’s the big deal?

I hear what you are saying. What is the big deal? That’s just a normal query and you are right it is. What is facinating is the ability within a sets engine like Set Analysis to see the trends in these sets over time. Although it’s possible to get a snapshot of these collection members through a standard database query, what you don’t see is changes to the sets over time. For example you need to know:

1. How many Platinum Customers do I have? 1,000.
2. If I had 900 last month, did I have 100 join or did I have 200 new customers join and 100 leave?
3. Where did they go? Did they become Gold? Silver? Are they off the radar?

It’s fascinating to me to see how organizations are so focused on what they sell and yet there is so much value in looking at what you didn’t sell… and who you didn’t sell it to!

In a down economy like we see today, it seems even more important than ever before to understand what your customers are doing.

In the next few posts, I’ll take a little bit of a deeper dive into the product and share a few tricks and tips.

If you need to troubleshoot Xcelsius 2008 runtime messages here is another link you might want to check out. I recently ran across an error message “2170″ but was not able to find it on my original list of error messages, but then I finally found it.

Click Here for the description of the runtime message.

UPDATE! The solution to the 2170 message when using Xcelsius was provided via YOUR posted comments to this blog entry. Thanks to Chris Chen for providing the solution:

http://technicalsupport.businessobjects.com/cs/forums/thread/22717.aspx

If you’ve had a chance to use the new BusinessObjects XI 3.0, one of the first things you will notice is that all your services are now managed by a local process called a Server Intelligence Agent. This agent is responsible for starting and stopping all the services that it manages.

After installing BusinessObjects XI 3.0, the first thing I did was start the CCM (Central Configuration Manager). I noticed I only had two services running.

Where are my services?

They are now managed based on how the Server Intelligence Agent. I don’t want to get into the details of what the Service Intelligent Agent does. This is clearly explained in the documentation. Instead I would like recommend to you a new best practice with the SIA.

But the CMS is King right?

In the XI 3.0 system there is a bit of a catch 22. The CMS is still king. Nothing works without the CMS. When the SIA starts up, it checks to see if it has a local CMS or not. If it does, it will start the CMS. After the CMS has started, it will then read the list of servers from the CMS database that it manages. Finally, it will start all those services. If there is no local CMS for a SIA, the SIA will wait for a remote CMS to start so it can read it’s list of managed services from the CMS and start them up.

But what if the CMS won’t start? What if I only want to start the CMS?

We need to manage this potential occurence.

Recommended Best Practice

Create a new NODE with a CMS to be used for maintenance only.

I strongly recommend that you create a second NODE for each server which will have a local CMS. There may be a situation when you want to bring the system up in maintenance mode and do some work without all the other services starting.

This also gives you the opportunity to start another CMS should you have a problem with your primary CMS. Here are the steps for creating a new NODE with only a CMS.

From within the CCM choose the Add Service Intelligence Agent from the menu. The SIA, like the CMS, needs a specific port to be assigned to it. It cannot pick a random available port. For the name, use the same name as your primary NODE but add the “_MAINT” to the end. Next pick an available port. In my case I used 6501.

Choose Next and fill out the CMS login information so that the configuration information for your new NODE can be written to the CMS database and choose Finish.

Although services can be added from the web based Central Management Console (CMC) interface, let’s start the SIA now. If you look at the properties of this SIA, you will see that it is associated with a remote CMS. We want to add a new “local” CMS to this SIA. (This means the SIA will have it’s own CMS to manage).

Go to the web based CMC and navigate to the Servers screen and see the new NODE we just created. Now we are going to add a new CMS server.

There are a number of ways to add a new server. Here I have highlighted three.

You can select the new NODE and choose the icon, drop-down menu or the new right-click menu. I love this new Web 2.0 that makes the interface work like a traditional windows environment. It provides a huge productivity gain.

Next, you are prompted to choose the server to add. We want to choose Core Services and Central Management Services. Next you will have an opportunity to add optional, additional services. The CMS supports an additional single sign-on service. I recommend you add that as well.

Now you need to tell which NODE to associate this service too. We want to chose our new _MAINT NODE.

The message is warning us that without the correct database configuration and setup, the CMS will not start. In our case, we have never set up a CMS on this NODE, so this is a warning we should heed. We will not be able to successfully start the CMS without additional setup.

Choose Create.

Unfortunately when setting up this service we don’t get a chance to change our final settings before the service is created, so after we create the service, we need to got back into the properties of the service and adjust two important settings.

Open up the properties of the new CMS and modify the listening port and the automatic start-up settings. The default port for the CMS is 6400, however my default CMS might already be running on this port, to temporarily change this to port 6401 - ONLY for testing. I will change it back to 6400 as soon as testing is complete.

NOTE: if you are ONLY going to use this SIA when the other SIA has been stopped, then you may use 6400 but understand that when you attempt to start this CMS, if another CMS is already running on port 6400, then this CMS will not be able to start. If you will want to use two CMSes at the same time, then you will need to use two different ports.

In addition, make sure and automatically start this server when the Server Intelligence Agent starts. Remember, when I manually start my new SIA, (MYXI30_MAINT) I will need a CMS to start as well.

Configuring the Database Connection for the CMS

Now there are TWO WAYS to properly configure the database connection. The fast and automatic way or the slow and manual way.

Option 1 - Fast and Automatic

The fastest way to setup the connection information for your CMS is to find an existing .dbinfo file for a working CMS and NODE combination and copy the file and rename it to the name of your new CMS.

In our case, the name of my working NODE with an associated CMS was MYXI30_ALL so the name of the associated .dbinfo file was found under: C:\Program Files\BusinessObjects 12.0\win32_x86 and is called _boe_MYXI30_ALL.dbinfo.

Copy this file and rename the file to the name of your new NODE with the _MAINT. In my case the node will be called _boe_MYXI30_MAINT.dbinfo.

Now when you start the CMS it will have the data necessary to connect to the CMS. Next time your new NODE starts, so will this CMS.

Option 2 - Slow and Manual (and ultimately the “right” way)

In order to manually configure the connection information for the CMS so that it will know how to connect to the CMS, you need to use the CCM. The CCM is the only tool that will allow you to configure the connection information. This cannot be done from within the CMC. From the CCM on the server, make sure the new SIA is NOT running.

Next, choose the Specify CMS Database icon so that we can define the CMS connection information for our new NODE.

You may receive an error message. This is normal.

Just choose OK to continue. Eventually you will get the prompt to configure the connection for the CMS database. We do NOT want to create or recreate our CMS, we simply want to update the data sourse settings, so choose the first option and choose OK.

You will be prompted to choose the associated database driver for your CMS and to enter the credentials.

Once this as been completed, a .new file will be created in the C:\Program Files\BusinessObjects 12.0\win32_x86. The file will be called _boe_NODENAME.dbinfo. In my case it is called _boe_MYXI30_MAINT.dbinfo.

If you are using an auditing database you may also wish to configure this by choosing the auditing configuration and following similar steps for that database.

When you are finished start the SIA and the CMS will start as well.

Conclusion

Remember, if the CMS does not start automatically, check the settings in the CMC for the service and make sure that the new CMS service is configured to start automatically when the SIA starts. This is a parameter that can be set for each service.

Hopefully this has been helpful in explaining some of the new best practices around XI 3.0 Administration. In addition I hope this will give you a quick introduction to the SIA and it’s relationship to the CMS. Your new “CMS-only” node will be great when you need to just bring up the CMS so you can access the CMC.

Enjoy.