In the real world, the migration between two OBIEE environments is really take time .The last time we did this for a customer migrating from 10g to 11g, we had to spend roughly 2 weeks just running through the dashboards clicking on all objects to do the 1st pass of the upgrade. When a customer asked us to migrate from 11g to 12c we were bracing ourselves for a similar ordeal. But the folks at Oracle have really done a great job this time with the Baseline Validation Tool (BVT). It allowed us to very quickly do a 1st pass through all the catalog objects and do a quick sanity check of the catalog and RPD after the upgrade.
The challenge is you need to make sure the content (analysis, catalog object, charts…) in two environments is same after the migration. BVT is FREE tool that enables the creation of a baseline set of data that can be compared with data from another system. While we used it for the upgrade it can be used in any situation where we need to run through 2 environments
BVT as a tool is quit well thought thru and allows for several checks to be done in an automated manner.
Data Validation: Allows to check the numbers in the reports between the baseline and new environment.
BVT even downloads the data from the report as excel or CSV and it can be used to analyze the differences later.
Visual Verification: The purpose of this is to make sure that visually the 2 environments look the same. During the launch they spoke about using some sophisticated analytics to do the comparison, but whatever they have done it does pretty good job to compare the outputs. BVT generates reports for both analyses and dashboards
The tool checks all the reports and dashboards in the catalog and produces a report which can even be shared with the customer.
The output is exported and can be manually checked later of passed to the customer as evidence.
BI Catalog Verification: Compares the metadata of catalog objects (rather than objects themselves)
Logical Query validation: Used to validate the logical query between the 2 environments. Used in conjuction with the data and visual validation gives a complete picture of the 2 environments.
The logical query query is captured as evidence and for the failed ones can be used to determine the reason for the failure. Using the logical query in conjuction with the data and visual verification gives us complete confidence on the outcome of the testing.
Requirements
The following are the steps we followed to get it all working in our environment
First, you have to download OBI Developer Client Tool (12.2.1.0.0) if you don’t have one. The tool zip file is located in [Installation Folder]\bi\components\oracle.bi.bvt
After unzip the tool, you need to fill in 2 OBIEE instance detail in testconfig.xml file such as
The testconfig.xml controls the behaviour of the BVT and is present in the root folder of the tool.
You can tweak the various tests to be done by tweaking the settings in the config file.
Catalog plugin config: Use this to setup the folders to be used for comparison
UI plugin config: Use this to tweak the algorithm for the visual verification algorithm. The main setting to be done on this is the Threshold score with a value between 0 and 1, where a value of '1' indicates a 100% pixel-by-pixel match, and a value of '0' indicates a complete mismatch.
Repots plugin config: Set a limit on the maximum tolerance for differences in floating point numbers in report data. Ex:"0" for no tolerance. "0.1, "0.001". It’s very useful for percentage measures.
Logical query plugin config: Set the folder to be used for the comparison
Dashboard plugin config: You can set the prompt combinations to be used for the BVT testing.
You need to run DVT three times. First & second for 2 OBIEE instances. The third for compare them & get the comparison result.
Examples of running :
bin\obibvt.bat -config testconfig.xml -deployment BI11g
bin\obibvt.bat -config testconfig.xml -deployment BI12c
bin\obibvt.bat -compareresults Results\PrePatch Results\PostPatch -config testconfig.xml
After running 3 test, you will get the Results folder, which contain information of each OBIEE instance and Comparisons folder, which contains comparisons results along with all the detailed evidence.
The goal of this post was to document our experience with the new regression testing tool shipping with OBIEE 12c; Setting up the configuration and the running comparisons is fairly straightforward; the results are detailed and handy. There is no doubt that this tool will be useful for OBIEE regression testing. In conclusion, the Oracle BI development team is making big strides in addressing pain points that have made working with OBIEE and migrating and maintaining different Oracle BI environments a challenge.
Click below to engage us learn more about 12c upgrade and its benefits.