Wednesday, 30 April 2014

Tutorial: Migrating OPA projects to Cloud

In this tutorial we will be going through the steps on how to migrate Oracle Policy Automation projects from version 10.4 (On Premise OPA) to 11 (Cloud based OPA).

Why we need to migrate manually our OPA projects

The main driver behind the manual steps highlighted below is that at this stage there is no automated migration tool that allows users of OPM 10.4 to use their rulebases in OPA 11 (Cloud version). Attempting to open a 10.4 rulebase with OPA 11 will spawn an instance of the OPM 10.4 software making it impossible to upload the rulebase to the Oracle Hub.

What OPA files do we need to amend

As of the 29/04/2014 the files to be amended so to allow OPM Cloud version to accept our 10.4 rulebase are:
  1. the .prj - the rulebase project file
  2. the Interview.xint - default interview definition look up for OPA Cloud
  3. all the rule files stored in doc will have to be converted to docx - OPM Cloud appears not to support binary based word documents

Manual amendments to the OPA project file (.prj)

The first line of the OPA XML project definition file declares the project version, we need to amend it so to be compatible with the OPM cloud version.

The project file (line 1 excerpt) before:
<rulebase-project product-version="10.4.2.18" schema-version="1"> 
The project file (line 1 excerpt) after the amendment:
<rulebase-project schema-version="1" product-version="11.2.0.171">
By changing the product-version we are able to open the project in OPM Cloud.

Convert the doc and xsl OPA rules to docx and xslx

OPM Cloud edition does not support binary based doc formats I created a small perl script that uses the word converter and excel converters distributed by Microsoft so to convert all rules at once. Create a file with the content:
use strict;
use warnings;
use 5.10.1;
my $dir = ".";
my @doc_files = glob "$dir\\*.doc";
foreach (@doc_files){
my $doc_file_name = $_;
my $docx_file_name = $doc_file_name . 'x';
my $command = '"C:\Program Files (x86)\Microsoft Office\Office12\Wordconv.exe" -oice -nme '. $doc_file_name . '" "' . $docx_file_name . '"';
system($command);
}
my @xls_files = glob "$dir\\*.xls";
foreach (@xls_files ){
my $xls_file_name = $_;
my $xlsx_file_name = $xls_file_name . 'x';
my $command = '"C:\Program Files (x86)\Microsoft Office\Office12\excelcnv.exe" -oice "'. $xls_file_name . '" "' . $xlsx_file_name . '"';
print $command;
system($command);
}
and save as 'convertBinaryToXML.pl'.

Note: the script above has been successfully tested on Strawberry Perl on a Windows 7 operating system only. Should you want to run it on any *nix operating system amend the path definitions.

Drop in a bog standard OPA Interview definition (Interview.xint)

After the amendment to the project file we will be able to open the project. OPM Cloud will be looking for a Interview definition file in the root directory . If unable to find the file OPM will present us with an error. Create a file with the content:
<?xml version="1.0" encoding="utf-8"?>
<interview product-version="11.2.0.171">
  <documents />
  <screens />
  <screen-order ID="" />
</interview>
 and save it in the Project root directory with the name 'Interview.xint'.
Doing this will allow us to successfully open the rulebase and if no errors are encountered upload it to the Hub. If you are presented with errors we potentially have to follow two more steps.

Errors in the data model

OPM Cloud attempts to automatically infer attribute types. If the attribute name or its usage does not give enough clues to the engine to successfully recognise the type it will throw an error. To solve this errors we need to navigate to the 'Data' tab and (in the top section of the client) we will be displayed with the offending attribute. At this point double clicking on the attribute and selecting the attribute manually will fix the issue. Just to provide an idea of how often this issues may occur in the example rulebase we used for the FATCA automation efforts only six attributes weren't correctly recognised over 2534 total attributes.

Errors in the rules

OPM Cloud seems to be slightly more picky in the way it interprets rules. During the conversion of FATCA from OPM 10.4 to OPM Cloud version I was presented with two kind of errors:
  1. Some space separated inline attribute declarations were converted into Tabs. I believe this is an issue in the Word and Excel converters. Nonetheless, a search and replace fixes the problem.
  2. Some of the rules had no explicit 'or' and 'and' at the end of every filtering clause. OPM Cloud complained about these and they had to be amended manually also*.
Note: This amendments should be conducted by somebody that has great knowledge of the rules structure and the policy they are modelling.

Conclusion

In this tutorial we have gone through the steps that allow the migration of OPM rulebases from the version 10.4 to 11 Cloud Edition. With a few very simple steps we are able to open, test and upload the rulebase to the Oracle Cloud enabling clients to have a smooth transition and upgrade path.
Migrating from OPA On Premise to OPA On Cloud may come with some behavioural caveats. After such a migration has been conducted a full regression has to be run on the rulebase to make sure behaviour has not been modified. In a rulebase with a full suite of Unit Tests the regression should be as simple as re-running the tests.

Friday, 10 January 2014

Batch Processor in Oracle Policy Automation (OPA)

In this article, we will be exploring the Batch Processor in Oracle Policy Modelling. The Batch Processor allows a large number of cases to be processed in batch. It can take input from comma separated files or from a database and can output to the same.
Let me give an example of where the batch processor may come in handy. Imagine a Telecommunications organisation that recently implemented OPA for its customers to check what mobile phone packages they can apply for. However, that organization already has data about its existing customers in a database and will also want to know which packages are applicable to its existing client base. The best way to do this will be by running this data in a batch rather than conducting an interview for each existing customer.
The Batch Processor sees its usefulness in other areas like conducting what-if analysis and generating test scripts from existing Excel data. We will explore input from and output to CSV files. This article assumes you have working knowledge of OPA; I will make it as practical as possible so let us begin by modelling a simple rulebase.
Sample Policy 
  • The customer is eligible to apply for the Gold Plan if the customer has been a registered customer for at least 3 years and the total amount spent by the customer is above £3,000.
  • The customer is eligible to apply for the Silver Plan if the customer has been a registered customer for at least 2 years and the total amount spent by the customer is above £2,000.
  • Any registered customer is eligible to apply for the Basic Plan.

The OPA rules for this policy are as shown below:                                  


The data model is as shown below:

Batch Processor Zero-configuration
If you are using CSV files as the input to the Batch Processor, much of the mapping from CSV data to OPA data can be done automatically through the Zero-configuration, by following a few simple guidelines which we will discuss in this section.
Entities
The CSV file name should be the public name of the entity. In our example, we will use two CSV files: "global" and "customer". "customer" is the public name of 'the customer' entity as shown below:




The CSV files are as shown below:
Input Attributes
The columns on the first row in the CSV files should be named with the attribute public names.

Identifier
By default, a column with the heading "#" is assumed to be a unique identifier for that entity.

Output attributes
To represent output attributes, enclose such public names in brackets.

One-to-Many Relationships
One-to-many relationships can also be represented in CSV files by using the public name of the relationship as the column title. For example, the relationship between global and 'the customer' entity is a one-to-many relationship which I can represent  in the customer.csv as shown below:

In the diagram above, customer instances #1 to #4 are instances under global instance #1 while customer instances #5 to #7 are instances under global instance #2.
The full content of the CSV files are shown below. The global.csv file has two instances of 'global'. You can think of this like two separate interviews (or 2 test cases). The customer.csv file has 7 instances representing 7 customers.


Running the Batch Processor
The batch processor is a standalone application that is run from the command line using the parameters listed below:
  •  java -jar determinations-batch.jar <command line parameters> (JAVA)
  • Determinations.Batch.exe <command line parameters> (.NET)

To make running the Batch Processor easier especially when using Zero-configuration, I recommend placing all your files and rulebase in one folder. For example, take the rulebase ZIP file (found in the output folder in the Development folder of your OPA project) and place it in a separate folder. In that same folder, create another sub-folder to hold your CSV input files (name it "csv" so you don't have to specify the name of the folder in the CLI parameters).

Then, you can open a command prompt, navigate to this folder and run the batch processor from this location.

Note: If you are not running the Batch Processor from its default location, you must specify the full path e.g. "C:\Program Files (x86)\Oracle\Policy Modeling\bin\determinations-batch.jar"
Batch Processor Output
When the Batch process is complete, a folder (called "csv.out") is automatically created. This folder used to be named "output" in earlier versions of OPA (I'm using version 10.4.4). This folder contains the same CSV files in the input folder but with the generated output values.

Let's view the content of these CSV files. First, the customer.csv:

Notice that the output field – out_cust_years-reg – has been filled out. This is just the number of years between the current date (January 08, 2014) and the dates the customer registered.
Let's also view the global.csv file, where the output should count the number of customers that are eligible for each plan.

Summary
In this article, we have seen how to use CSV as input to the Batch Processor. We have also seen how to ensure that our csv files are properly formatted so that zero-configuration can take place. There are move advanced topics like writing a configuration file to be used for running the batch process and also database connections but this gives a good overview of the Batch processor.

I hope you have found this article insightful.

Saturday, 23 November 2013

Setting up Eclipse for OPA Java Development

This article describes my favoured Eclipse set up to work with OPA web applications (ODS and OWD) in Java. I will describe a set up for OWD, but this can easily be adapted for ODS.

Create a new Eclipse dynamic web project


Select a target runtime, I am using Tomcat 7. If you don’t have one set up already then follow the usual Eclipse method of adding or installing a web runtime. And use 2.4 for the web module version.


Locate your OPA Java runtime (downloaded from Oracle). Unzip the archive somewhere and locate the "web-determinations" directory. Expand the contained web-determinations.war using an unzip tool.


I now re-use this expanded WAR across several projects that share the same OPA version.


The next step is to link this directory into you web project.



 So your Eclipse project contents should look like this.


 Now we need to tell Eclipse to use the OWD war directory. Go to the project properties (alt-enter or right click on project and select properties). Go to the Deployment Assembly section and select Add and select the Folder directive.

 Select the expanded WAR directory this is now accessible via the folder linked into the Eclipse project


  
The deployment assembly should now look like


Depending on what you want to do you may decide to remove the Eclipse generated "WebContent" directory completely. However if you will be making any changes to the web context then it is useful to have a second web content directory as it acts as an overlay allowing you to override files in the out of the box web application without changing the original. However you will have to remove the additional web.xml at this point to allow things to work - so delete or rename the WebContent/WEB-INF/web.xml file.


You can now run your Eclipse hosted OWD. Right click on the project and select Run As -> Run on Server. The console should start to show the Tomcat (in my case) start up logging and a browser window will open on the OWD home url.


Now for some rulebases. If you are working on a specific rulebase then link it in whatever way suits you best. For this article I am going to expose all of  the compiled rulebases that are supplied in the OPA Java Runtime.

Go back to the project deployment dialog and add a new folder as below.

 Using the standard out of the box OWD configuration this directory should be mapped to the WEB-INF/classes/rulebases directory.
 Now restart the server and refresh the OWD url.



You are now ready to go developing Java extensions, playing around with configuration files or Velocity templates. Some things to note.
  • Any resources that are present in the out of the box WEB-INF/classes directory can be overriden without changing the originals by adding them into the project src directory, this includes Velocity templates and properties files found in OWD.
  • You can override web.xml settings by placing a copy appropriately in your project WebContent directory, and this can be used to add other public web resources also.

Thursday, 16 December 2010

OPA Release 10.2.0 - birds eye view

I have just been playing around with the new 10.2 release of Oracle Policy Automation (OPA) and I have to say this is quite a major step forward for a minor point release. For those who have not seen the release note, the following is extracted from the Oracle 10.2 release overview

  • Translation Support: Excel spreadsheets that contain all the translatable parts of a rulebase, which are updated with the new strings to be translated when you modify rules or screens,etc. 
  • Entity Containment: New hierarchical data model that simplifies data collection, debugging, and rule authoring when using relationships 
  • Pluggable Determinations Server: New extensibility model for Determinations server that
    allows you to do pre- or post-processing of incoming data, or do as needed retrieval of data from an external data source, for example 
  • Interview Web Service: Create fully customized stateful interview experiences in your GUI technology of choice 
  • Interview Screen Preview: Preview interview screens without having to go through the entire interview 
  • Flexible Interviews: Entity-level goals on the summary screen, more control over labels used for entity collection, and more 
  • Working out if a question has been asked: New "Not Currently Known" operator. 
  • New functions for text strings: Contains, StartsWith, EndsWith, IsNumber, Length. 
  • Entity Equality: Ability to write rules that compare one entity instance with all the other
    instances in a collection 
  • New Word/Excel 2007/2010 look and feel 
  • New Parsers and Improved Right-to-Left support: Arabic, Hebrew, Turkish, German, Finnish,Chinese (Traditional) 
  • Native Subversion Support: Check files in and out directly from subversion without a MSSCCAPI connector, and see differences using Word's track changes feature 
  • Custom Function Enhancements: Support for temporal values and generally cleaned up custom functions 
  • Support for Looping Rules: Ability to write self-referential rules, e.g. the person's tax payable = the person's tax payable for the current tax year + for (the prior tax year, the person's tax payable) 
  • Platform Support: Windows 7 x64, Office 2010, .NET Framework 4.0, Oracle VM, MS Visual Studio Team Server 2010

Nice to see subversion support finally making it in there, although having just moved from subversion to mercurial myself I still think it is a bit of a shame there is not a plugin environment to allow support for any version control system of choice, in particular in OPA's case the ability to support storing of Word and Excel files in Sharepoint.

For me personally it was a relief to see the Windows 7 x64 bit and the Office 2010 support. Running a VM on my laptop with Vista and Office 2007 in order to use OPA was becoming teeth (and disk) grindingly tedious.

The determinations server extensibility options look promising. I haven't had a chance to play with it in detail yet but it certainly tidies up the whole data injection process and provides enough hooks for a decent amount of flexibility and cusomisation. Perhaps a decent runtime auditing and performance monitoring add-on can now be created.

My personal favourite however is the new Interview Engine component. This is a mid-level native Java/.NET libarary that can be used to drive interviews through logical controls on logical screen instances, leaving the sourcing of the data for the screens completely to the application, including rendering of the screens using any appropriate technology. This promises to open the door for some decent rich clients or web 2.0 clients to enhance the interview experience. This will be my first port of call when I get some decent  dev time to play with the product.

Luke Studley is a consultant with Monad Solutions