Introduction to Decision Model and Notation in Oracle Process Cloud Service

Introduction and Feature Preview

With the latest release of the Process Cloud Service (17.1.3), Oracle has released a preview version of its much awaited Decision Model and Notation (DMN) engine. The DMN standard provides a powerful meta-model, notation and semantics for modeling of operational decision and business rules. It is a critical addition in the process, case and decision modeling framework providing an open standard for modeling the data and logic associated within a process.

The notation standard aims at defining requirements and decision logic for manual and automated decision-making by representing a complete, executable model which can either be used independently or in conjunction with a business process. Similar to BPMN, it aims to support designing of decision models, providing guided operational decision and potentially automation of operational decisions. A combination of process and decision modeling simplifies business processes by eliminating and replacing sections of a process model with a decision model. The decision logic of the process model is captured as a separate yet linked model. By focusing on decisions and processes independently the requirements process is more focused. A decision-driven process acts on the outcome of the evaluation of decision logic in several possible ways, including:

  • Changing the sequence of activities that are taken after a decision, including what the next activity or process that is required to meet the directive of the process.
  • Selecting between the paths on the diverging or the splitting side of a gateway.
  • Deciding who or what participant should perform the needed activity.
  • Creating data values that will be consumed later in the process.

A decision  can be thought of as a meta gateway with merging inputs (events and data) and splitting outputs that direct the process in an overarching way (i.e., participants, tasks, and gateways).

Integrating business process modeling with decision modeling, as well as event processing, can create comprehensive, agile solutions that are easy to change and enhance. Without decision modeling, process modeling in BPMN can both overcomplicate a model with logic and miss critical design details. As decisions are modeled, things will be discovered that the process must do to accommodate the results of those decisions.

By associating a decision model with an activity in a business process model, it becomes an executable specification for the decision-making to be carried out in that activity. If the decision model is encapsulated in a decision service, the business process can execute the decision-making automatically at the correct point in the process. The following diagram provides a high level overview of how process and decision model can be used together.

image

This blog post intends to provide an introduction to the basic concepts of DMN, its constructs, illustration of a real like use case and how to get started with DMN in Oracle Process Cloud Service.

First Things First: DMN Fundamentals

The objective of a decision model is to create a shared understanding about logic, data and process that are fully traceable from business requirements to IT implementations. The Decision Modeling and Notation standard from the Object Management Group (OMG) is a notation that accomplishes these goals. The true value proposition of DMN lies in providing a common language that bridges the gap between across business, IT and analytics department within organisations, thereby improving collaboration, reuse and ability to implement accurate and consistent decision management solutions. The DMN semantics are based on the following principles:

  • Bridge the gap between requirements and technical implementation by mapping graphical notation to the underlying constructs of a business rule execution environment.
  • Specify standard and easy to understand decision requirements diagrams that communicates processing information.
  • Link business rules and analytics to business objects in order to share and tune decision information across process implementers.
  • Provide a format to share decisions and copy decision models between process improvement tools.

In the context of DMN, a Decision is an act of determining an output value, based on a number of input values, using logic defining how the output is determined from the inputs. A Decision Model is a formal model of an area of decision-making, expressed as decision requirements and decision logic. A decision model is defined in two levels, the Decision Requirements Level and the Decision Logic Level which is explained in a bit of detail here. At the decision requirements level, the notation is simple enough to make the structure of the model immediately apparent; yet, the decision logic level provides a specification of the decision-making which is precise enough to allow full automatic validation and execution.

Decision Requirements Level

The Decision Requirements Level is an abstract level of modeling comprising of a Decision Requirements Graph (DRG) represented in one or more Decision Requirements Diagrams (DRD). It defines the decisions to be made, the interrelationships, and the required components for the decision logic. The DMN decision requirements diagram is a notation depicting important elements of decision-making and their dependencies. The following table outlines the DMN decision diagram constructs, their symbols and meaning.

DRG Construct

DMN Notation

Description

Elements
Decisions image A decision element corresponds to the business concept of an operational decision. It is the act of determining an output value from a number of input values, using some decision logic. The inputs to a decision may be input data elements or the outputs of other decisions. The decision logic may include the invocation of one or more business knowledge models.
Input Data image An input data element corresponds to the business concept of data. It is a data structure whose component values describe the case about which decisions are to be made. Input data elements typically model high-level concepts of busi11111ness significance, e.g., “Application form”, “Claims history” or “Invoices.”Information used as an input by one or more decisions.
Knowledge Source image A knowledge source (with a wavy edge) defines an authority for decisions or business knowledge models — for example, a manager responsible for a decision, a policy manual, or a piece of legislation with which a set of rules must comply.
Business Knowledge Model image A business knowledge model (with clipped corners) corresponds to business concepts such as “expertise,” “know-how” or “policy.” It is a function that encapsulates an area of business knowledge as executable decision logic, possibly expressed as business rules, an analytic model, or an algorithm. The business knowledge model is parameterized, and is therefore a reusable component that may be called from multiple decisions, or from other business knowledge model.A function encapsulating business knowledge, in the form of business rules, decision table or analytic model.
Requirements
Information Requirement image Represents an input data or decision output required for a decision.
Knowledge Requirement image Represents an invocation of a business knowledge model.
Authority Requirement image Denotes the dependence of a decision requirement diagram element with another element that acts as a knowledge source or guidance.
Artifacts
Text Annotation image Consists of a square bracket to capture explanatory text or comment.
Association image It is a connector that links a text annotation to the decision requirement diagram element.

Decision Logic Level

A DMN diagram does not depict the decision logic of a particular decision. DMN decision logic level is used to specify a complete expression of logic, potentially sufficient for automation. Each decision in a decision model has a value expression that describes how a decision outcome is derived from its required input, possibly invoking a business knowledge model. In addition a decision logic level allows boxed expressions to be associated with elements in the decision requirements level. Boxed expressions allow the decision logic to be decomposed into small pieces that can be notated in a standard way and associated with elements at the decision requirements level. The type of DMN boxed expressions supported in Oracle PCS are:

  • Function Construct – Allows creating parameterized logic that can apply multiple times to a set of input with different parameter values.
  • If Then Else Construct – Decision function with three logic parts: If (Boolean test), Then (result if the test is true), and Else (result if the test is not true).
  • Context Construct – Provides a mechanism to create a list of named logic elements, where logic lower in the list may reference logic results higher in the list by name.
  • List Construct  – This construct provides a list of independent logic elements whose output contains outputs of all logic elements.
  • Relation Construct  – Allows creating a relational table where each cell contains a logic element. For example, a cell in a relation construct could be a function, decision table, expression or if-then-else.
  • Friendly Enough Expression Language (FEEL) – Create logic that evaluates to a single value by entering readable text.
  • Decision Table – A decision table comprises of rules with tests matching input on one side and output on the other side.

A lot of the information provided above can be summarized in an easy to visualize mind map that I have created. The mind map shows a relationship between a number of DMN components across both decision requirement diagram, decision logic and decision services.

DECISION MODEL

A Simple Use Case : Loan Straight Through Approval Decisioning

Imagine you are working at a bank in the personal lending fulfilment department. A customer wishes to make a request to determine his lending eligibility for a personal loan. Can you accept his request, determine and outcome and communicate to him the result? You have to make a business decision. What are the criteria for the correct decision and what information is your choice based on. The bank has standard rules and guidelines for handling the request, some of which are:

  • Customers need to provide collateral for personal lending above a certain amount.
  • The identity of the customer has to be properly verified
  • Moreover, which type of document is allowed depends on the customer’s nationality and lending value
  • Does the customer already have existing loans with the bank or other banks?

Decisions in DRD are represented by rectangles and input data by rounded rectangles. In the example DRD illustration shown below “Determine Lending Approval” is the top-level decision. “Determine Lending Eligibility” and “Determine Lending Limit” are sub decisions, the outcome of which is considered by top-level decision. They are also sub decisions because the questions they pose must be answered before the main question, “Determine Lending Approval”, is answered. The arrows depict a flow of the decisions into the top-level decision. There can also be more than one top-level decision. Third element types, called knowledge sources help us depict dependencies on policies or regulatory requirements. In this case, the “Determine Lending Limit” sub decision depends on “APRA #571.1 Revisions to Large Exposures”, a particular retail banking regulation by a financial regulatory authority. DMN diagrams provide a high-level overview of how a decision depends on other decisions, policies or regulation and input data.

 

DMN-DRG

As stated before a DMN diagram does not depict the decision logic of a particular decision. Decision logic specifies the details of one decision, so that it can be easily and unambiguously understood by a human. In the rest of this blog post, I will provide detailed steps through which the above decision model can be translated into a decision logic executable as a DMN decision model in Oracle PCS and subsequently exposed as a decision service to be consumed by a business process.

DMN as a first class Citizen in Oracle Process Cloud Service

If your PCS instance has been updated to 17.1.3, you can enable DMN design and runtime by navigating to Administration > UI Customization > and check Enable DMN in Composer.

image

Decision Model in DMN

Once the DMN editor is available in composer, you should be able to create a new Decision Model in any available project space in PCS. The decision model that I created and which will be covered here is called Lending Approval. A decision model in DMN can have more than one decisions that are based on boxed expressions with each decision taking in an input type and resulting in its own output. A decision model can also be service enabled by defining a decision service and associating one or more decisions to it. The interface of the decision service is determined by the input association and output is determined by the outputs associated with the decisions. The following illustration provides an anatomy of a DMN decision model in Oracle PCS.

 

image

 

Decision Tables in DMN

Let’s assume that the retail lending application use case being discussed has an overarching process model (will discuss this shortly) that allows lending approvals without any manual approvals for certain scenarios. In order to determine this, a decision model has to evaluate lending eligibility as well as calculate an approximate lending limit that can be approved for the applicant. An applicant’s personal lending request is deemed eligible if:

  • It it below a threshold determined by the financial institution’s risk taking appetite for approving lending without any securities (in our example, let us assume it is 5G)
  • For any lending request that is above this threshold, the applicant must have:
    • A valid nationality
    • A valid identification document which is verified

Finally, the decision logic behind these checks should ensure that all the above conditions are met for the lending request to be deemed eligible. If at least one condition is not met, then the lending request is denied. Once an eligibility has been determined, the decision function then also needs to calculate the maximum lending limit based on the credit information provided by the applicant.

There are a few ways in which the decision logic for the above scenario can be implemented, the most common being using a decision table. A decision table can have one or more rules where each rule contains input and output entries. The input entries are the condition and the output entries the conclusion of the rule. If each input entry (condition) is satisfied then the rule is satisfied and the decision result contains the output entries (conclusion) of this rule. A decision table also has a hit policy that specifies what the results of the evaluation of a decision table consist of. Each row in the decision table represents a rule where as the columns represent conditions. Conditions are based on input elements or expressions/functions and can be restricted to data types, list of values or ranges.  A detailed visual of a decision table in DMN along with its key components is shown below with some explanatory text. If you carefully read the table it does the following:

  • Sets the lending eligibility as true if the sum of current and any existing lending (denoted as aggregated sought exposure) is less than 5k.
  • For any lending above the 5k threshold, the applicant needs to be Australian, have a valid identification status (Valid) and identification document (Passport, Election Card or National Identity Card) to be eligible.

image

When a decision is created in a decision model, it requires an input and output definition. For our example, I created a complex LendingRequest element with a few simple elements which would be used in the decision table. You can also use an existing business object for an input variable.

image

Creating Bucketsets in a Decision in DMN

Input expressions defined in the decision table can be bucketed as list of values or a value range to restrict the condition evaluation to a value within the bucket. This is very useful in long decision tables as it restricts users from entering invalid values that would otherwise result in an exception.

image

Decision Table Hit Policy

Decision tables in DMN has another very interesting feature called hit policy. A hit policy specifies how many rules of a decision table can be satisfied and which of the satisfied rules are included in the decision table result. The hit policies supported in Oracle PCS are Unique, Any and First and Collect which are described in more details in the table below.

Hit Policy Description Usage Scenarios
Unique (Single) Only one rule is allowed to fire for any one combination of inputs. In this context, all inputs are assumed to be independently from each other, i.e. that any combination is actually possible. Immigration rules to determine if a person should be allowed a visa or be declined.
Any (Single) Multiple rules cover the same combination of input values. However, this overlap is only allowed if the overlapping rules have the same output. A diagnosis of a medical condition might need an Any policy if various ranges of similar diagnostic test identify the same condition.
First (Single) Rules might overlap. But only one rule fires: The First hit policy simply assumes an ordering of the rules – they are evaluated from top to bottom. As soon as one rule fires, the output of that rule is used as result of the decision. Once one rule hits, no other rule is checked. A hotel booking reservation where the first match room is allocated based on a customer’s preference.
Collect (Multiple) Multiple rules can fire without an explicit order. Multi hit tables either return a set or aggregate the final results of the decisions. DMN’s default is provide the set of output values as result. Multi-rule table can apply aggregation functions such as min, max, avg to derive a single value from a set of output values Create a score card to assign points to an inspected based on multiple checks and with points off for various defects.

If you now look at the DetermineLendingEligibility decision table again, you would notice that the first cell of the table is denoted with a capital F. This is setting the hit policy of the table to First which means the decision table will stop evaluation as soon as the first condition is true.

Combining Multiple Decisions in a DMN Decision Model

A single decision model can have one to many decision services and likewise one to many decisions. Decisions can be associated with decision services which provides a service interface to invoke a decision logic construct. As originally explained in our use case, we also wanted to calculate the automatically derived lending limit from a business rule. Whilst determining such a limit in real world would be based on very complex rules, a simplified version is shown below. The lending limit is based on the difference between the total valuation of a customer’s security against a sum of his/her outstanding loans. The result of the expression can also be evaluated against a number range to determine the decision output as shown in the decision table below. Note that [ denote that the number in the range is inclusive while ( denotes it is exclusive.

image

Testing a DMN Decision Service

The DMN composer in PCS has very strict design time validations for decision tables and models. Once the rule components are validated, they can be deployed as a standalone application in the same way as a PCS application. Once a DMN application is deployed, it will provide a rest endpoint to post a request and get a response. Here is a sample JSON request to the Rest endpoint and I use in Postman to invoke the service. The output is an decision interpretation object returning the result of each executed decision table in the decision service.

{
“LendingRequest”:
{
“customerName”: “Arun”,
“lendingNumber”: 123123,
“nationality”: “Australian”,
“securityInOrderValuation”: 3000,
“aggregatedSoughtExposure”: 4900,
“identificationStatus”: “Invalid”,
“identityDocumentType”: “Passport”
}
}

image

Using the New Decision Activity in a Process Model

The final piece to the puzzle is to use a DMN decision component in the process model. Needless to say there is a new Decision Activity available in the System palette in PCS composer to associate a decision service to the activity. The data association wizard in PCS can then be used to pass and retrieve information from a decision function. The following process model shows how the decision model discussed in this example can be used in an overarching process for straight through approval of customer retail lending. The result from the decision function are used to determine the process flow:

  • If the lending limit is less than 5000 and the eligibility criteria are met, the lending is automatically approved without any manual intervention.
  • If the lending limit is greater than 5000 and the eligibility criteria are met, the lending request is sent to a Credit Approver to view and either approve or reject the request.
  • If the lending eligibility criteria are not met, the lending is rejected and the process terminates.

image

Application Links and Testing the Overall Solution

The PCS process and DMN business rule application discussed in this blog can be downloaded from here. When these applications are deployed to the runtime, they can be tested from end to end. The Manage Lending Submission process model has an event start (initiate lending submission) node which means that the process can be invoked as a SOAP endpoint. The section below provides a few test scenarios and how process map behaves based on the output determined by the DMN decision model.

Scenario 1 – Straight Through Approval – Sub 5k Loan

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:man="http://xmlns.oracle.com/bpmn/bpmnCloudProcess/MLS/ManageLendingSubmission" 
xmlns:len="http://xmlns.oracle.com/bpm/bpmobject/MLS/LendingRequestMetadata"> 
<soapenv:Header/> 
<soapenv:Body> 
<man:initiateLendingSubmission> 
<len:LendingRequestMetadata> 
<len:customerName>Arun Pareek</len:customerName> 
<len:lendingNumber>123123213</len:lendingNumber> 
<len:nationality>Australian</len:nationality> 
<len:securityInOrderVal>3000</len:securityInOrderVal> 
<len:aggregatedSoughtExposure>4900</len:aggregatedSoughtExposure> 
<len:identificationStatus>Valid</len:identificationStatus> 
<len:identityDocumentType>Passport</len:identityDocumentType> 
<len:approver>BBCManager</len:approver> 
<len:lendingEligibility>false</len:lendingEligibility> 
<len:lendingLimit>0</len:lendingLimit> 
<len:customerEmail>arun.pareek@rubiconred.com</len:customerEmail> 
</len:LendingRequestMetadata> 
</man:initiateLendingSubmission> 
</soapenv:Body> 
</soapenv:Envelope>

image

Scenario 2 – Lending Requires Manual Approval – Loan > 5k with Valid Nationality, Identification Status/Document

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:man="http://xmlns.oracle.com/bpmn/bpmnCloudProcess/MLS/ManageLendingSubmission" 
xmlns:len="http://xmlns.oracle.com/bpm/bpmobject/MLS/LendingRequestMetadata"> 
<soapenv:Header/> 
<soapenv:Body> 
<man:initiateLendingSubmission> 
<len:LendingRequestMetadata> 
<len:customerName>Arun Pareek</len:customerName> 
<len:lendingNumber>123123213</len:lendingNumber> 
<len:nationality>Other</len:nationality> 
<len:securityInOrderVal>10000</len:securityInOrderVal> 
<len:aggregatedSoughtExposure>13000</len:aggregatedSoughtExposure> 
<len:identificationStatus>Valid</len:identificationStatus> 
<len:identityDocumentType>Passport</len:identityDocumentType> 
<len:approver>BBCManager</len:approver> 
<len:lendingEligibility>false</len:lendingEligibility> 
<len:lendingLimit>0</len:lendingLimit> 
<len:customerEmail>arun.pareek@rubiconred.com</len:customerEmail> 
</len:LendingRequestMetadata> 
</man:initiateLendingSubmission> 
</soapenv:Body> 
</soapenv:Envelope>

image

Scenario 2 – Lending Rejected – Invalid Identification Status

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:man="http://xmlns.oracle.com/bpmn/bpmnCloudProcess/MLS/ManageLendingSubmission" 
xmlns:len="http://xmlns.oracle.com/bpm/bpmobject/MLS/LendingRequestMetadata"> 
<soapenv:Header/> 
<soapenv:Body> 
<man:initiateLendingSubmission> 
<len:LendingRequestMetadata> 
<len:customerName>Arun Pareek</len:customerName> 
<len:lendingNumber>123123213</len:lendingNumber> 
<len:nationality>Other</len:nationality> 
<len:securityInOrderVal>15000</len:securityInOrderVal> 
<len:aggregatedSoughtExposure>12000</len:aggregatedSoughtExposure> 
<len:identificationStatus>Invalid</len:identificationStatus> 
<len:identityDocumentType>Passport</len:identityDocumentType> 
<len:approver>BBCManager</len:approver> 
<len:lendingEligibility>false</len:lendingEligibility> 
<len:lendingLimit>0</len:lendingLimit> 
<len:customerEmail>arun.pareek@rubiconred.com</len:customerEmail> 
</len:LendingRequestMetadata> 
</man:initiateLendingSubmission> 
</soapenv:Body> 
</soapenv:Envelope>

image

Summary and Closing Thoughts

The Decision Model and Notation not only adds clarity to business logic, it sharpens and simplifies business processes. A process model should represent the procedural flow of tasks. A decision model should represent the declarative conditions leading to the conclusions of business decisions. In the past, due to lack of alternatives, this was mixed up in process modeling notations leading to complex and convoluted process models and eventually maintenance headache. Decision models are relatively new to most organizations and the support for the DMN standard has just released in Process Cloud Service. This blog post should hopefully provide you with the necessary knowledge, tool and best practices for implementing decision models in business process and stay ahead of times. to start No one reading this column should feel behind the times. DMN and BPMN are very powerful together and using them wisely together could provide organizations with many advantages such as:

  • Maintain all operational and analytical business logic (of business rules) together in one place (rather than scatter it across wrong places)
  • Reduce process models to simplest form so that they do no consume the length of board room walls
  • Increases speed to value and agility of the solution as decision models can be updated agnostic of the process models and vice versa.
  • Provides reuse of business decisions across multiple processes
  • Allow creation of customized views of the same decision logic whereby a process model calls a different view of a decision model as appropriate
  • Deploy advanced and sophisticated technology in an optimum manner:
  • Give more control to business people to govern and change the logic behind business decisions.

The Curious Case of Missing Port Type in Oracle PCS

I was recently working on a simple process in PCS for a license approval flow. Given the purists that I am, I began by defining definitions for the various message based activities used in the process flow. The process was an asynchronous process with a few intermediate events. A simplified snippet is shown here for visualization.

process

In order to implement the above process, I created a service definition (WSDL) with the following schematic. As you can notice, there is a portType for accepting requests into the process (fc.myst.bp.TestDrive) and a callback portType for sending messages out of the process (fc.myst.bp.TestDrive.CallBack). Each of the portTypes have operations for catch and throw messages respectively.

WSLD schematic

Creating a service definition is considered a best practice as it will not lead to multiple definitions created by the process when generating interfaces from message based implementations. So instead of using “Define Interface” we tend to use “Use Interface“.

Use Interface

If we navigate through the configuration wizard for creating a reference based on an existing service previous uploaded into PCS, we will hit a dead end. Both the portTypes configured in the WSDL will appear under the Callback Port Type drop-down whereas the drop-down under the Port Type combo box will be empty. This will also inhibit creation of the service reference based on the service definition. This is shown in the chain of images below.

blankPortType

In order to resolve this problem, I had a closer look at the interface definition WSDL. Turned out that i had created an abstract WSDL which is perfectly fine given that the composite assembly would just use it to implement an interface and generate a service endpoint from the deployed composite. Tried to re-import the WSDL a few times to see if the issue could be addressed but it persisted.

At last, I changed the WSDL to a concrete definition by adding arbitrary service endpoints to the bindings and re-imported into PCS.

Concrete WSDL

Voila! I was able to create a new reference this time by selecting Use Interface from the start message event with the reimported WSDL.

port

The source code for the PCS project discussed in this blog can be downloaded from here.

Accelerating Innovation through Device, Human and Process Centric Integration

Introduction – A step towards a Digital Journey through the Cloud

In an increasingly digitized world, technology enabled transition is increasingly becoming a strategic ally for organisations and how they deliver services and products more effectively. Organisations need to orient their business along capabilities and commercially viable ‘proof points’ that can scale quickly. A key to this approach is to avoid large multi-year road maps and refrain from multi-million dollar investment projects to be locked into an uncertain future. The concept of providing a differentiated customer experience is the only way to create a sustainable advantage. A recent Oracle Partner event (Sydney PaaS Hackathon) was a perfect stage for partners in APAC to demonstrate the power and capabilities of the Oracle Cloud Platform to accelerate a customer’s digital transformation journey. Now that the dust has settled on the amazing Sydney Hackathon where we participated and won the second prize for our prototype using Oracle PaaS products, it is time to share what we actually built and how. In this opening blog, I will provide a glimpse of the end to end solution that we put together using nine different Oracle Cloud Services in the 28 hour marathon. Along the way, I would also share the best practices and lessons learnt of integrating the different Cloud Service and an approach to building solutions around user centric design principles. The core prototype and use case was around building a real time remote patient well being monitoring system. In particular, the solution centered around patients suffering with blood pressure related disease which is a major cause of health related deaths in Australia.

Before describing the end to end solution, it is appropriate to describe the real problems that patients with blood pressure related disease suffer. This would provide some key drivers behind the solution that we have implemented. Let’s have a quick look at some of the frustrations sufferers have to deal with while pro-actively managing their blood pressure:

  • Forgetfulness – Lifestyle changes are very important in managing high blood pressure. But lifestyle changes along do not help. Patients have to regularly take their blood pressure readings and prescribed drugs to subdue prolifiration of their condition. Forgetting to take timely readings or medication is a leading cause of sudden anomalies in blood pressure readings.
  • Routine Health Checks – With the busy lives most people have today it can be really hard to fit in regular visits to the pharmacy for routine blood pressure checks.
  • Uncertainty – People with high blood pressure are at risk of suffering from untimely stroke and heart attacks. They often visit their medical practitioners in reaction to an event or during a scheduled appointment as opposed to being looked after and being proactively advised.

Use Case – Remote Patient Monitoring and Connected Care

We built a Patient Monitoring System to manage personal health (specifically blood pressure) that remotely connected patients with their community pharmacists and medical practitioners allowing proactive management of their well being. This will be quite a read but if you manage to stay till the end, the blog will provide you with a method to identify and innovate on real world problems that have been ignored for long. Upon exploring the use case further, we identified 4 key personas beginning with Bruce Wayne (our patient).

  • Bruce suffers from high blood pressure and struggles to visit his physician and pharmacist for regular blood pressure check ups. He also forgets his pills and is uncertain how this will effect his health and well being in an unforeseen event.
  • Jenny is Bruce’s local community pharmacist. She collects and review’s Bruce’s blood pressure data and makes her assessment for the next course of action.
  • Jenny like to consult with a  qualified medical practitioner such as a physician or a cardiologist from time to time when there is a critical reading or when she spots an anomaly and needs a second opinion.
  • Lisa who is a medical practitioner believes that with the right technology solution, she can make more informed and proactive decisions for her patients without meeting them in person.
  • Finally, our fourth persona is the State Health Department that has always expressed a desire to collect well being information anonymously to identify key geographical clusters associated with higher risks of heart attacks.
Role/Persona Background/Job Function Important Needs

Patient
(Bruce Wayne)

Patient

  • Works as a Project Manager
  • Often has to work under stressful circumstances
  • Has a family history of heart related disorders
  • Suffers from Hypertension (High Blood Pressure)
Bruce would benefit from being able to take remote  blood pressure reading which is automatically sent to his pharmacist to minimize routine travel. He uses technology heavily and would not mind notifications to remind him when he misses to take his reading and prescriptions. He is also happy to be proactive consulted and advised from his health practitioner and pharmacist when there is a problem.

Pharmacist
(Jenny Andrews)
Pharmacist

  • Works in a Pharmacy as an Community Pharmacist
  • Collects and records Blood Pressure Readings for routine patients
  • Prepares medications by reviewing and interpreting physician orders
  • Controls medications by monitoring blood pressure and advising interventions
As a pharmacist, Jenny would like to automatically detect and manage information regarding when any of her community patient has a critical blood pressure reading so that she can take the next proactive action.

Medical Practitioner
(Lisa Brown)

Medical Practitioner

  • Diagnosis, monitoring and treatment of high blood pressure
  • Physical Examination of Routine Patients
  • Inform patients about their blood pressure and how they can make lifestyle changes to prevent or improve hypertension
As a medical practitioner, Lisa wants to be able to analyse her patients well being records over a period of time so that she can detect patterns, anomalies and exceptions.

State Health Department
logo-gov-dep-health

  • Targeted prevention activities for disadvantaged populations
The state department of health is interested in the analytics side of things. It would be great if they can correlate high blood pressure to a geographical region, weather condition or time of the year.

The Remote Health Monitoring prototype discussed in the rest of this blog aims to address the core challenges discussed earlier.  We put our thoughts in action by integrating close to 10 Cloud Services in 28 hours for our use case. Oracle IoTCs, ICS, MCS, Application Container Cloud Service (BICS), APICS, Business Intelligence Cloud Service (BICS),  and PCS were the core Cloud services that were the heart of our solution.

Solution at a Glance : myHealth

For the Hackathon, we created a fully functional and cloud integrated application called myHealth. The key objective of myHealth was to shift patient healthcare from cure to prevention. The overarching solution emphasized on building an infrastructure to continuously oversee the vital parameters of a patient and provide necessary alerts and interventions at near real time by leveraging IoT, services and a digital platform in a cost effective way.

Oracle Cloud Services in Action

As we wanted to remotely measure and stream health data, process it, and analyse it in real time to detect and react to an anomaly, Oracle Internet of Things Cloud Service was the ideal choice to be at the heart of the solution. Of the numerous blood pressure monitors available today, we used iHealth Sense Wrist wireless devices. Once a patient would take a reading from a this device, it would instantly push that data onto the device cloud. We accessed the iHealth APIs to create a web-hook to publish this stream of data into the IoT Cloud using Application Container Cloud Service and Integration Cloud Service. This is where interesting things would happen:

  • Using IoT Cloud Service, we built explorations which were visual representation of real time data coming from remotely registered devices/gateways for insightful data interrogation. The explorations were built to refine and sanitize incoming real time data stream using a few out of the box patterns such as eliminating duplicates and detecting fluctuations and define criteria to filter critical events from the normal.
  • Once explorations were published, it becomes available as one of the message formats to choose from when configuring an IoT Application’s integration with external services, such as Oracle Business Integration (BI) Cloud Service, Oracle Mobile Cloud Service or an enterprise application based on HTTP endpoints.
  • After isolating critical events from the data stream that required manual intervention, a process flow is triggered in Oracle PCS where a pharmacist or a medical practitioner can assess the anomaly and take further action such as provide phone consultation, schedule and appointment, renew/change a prescription, issue referrals, order meds, etc.
  • We also created a myHealth mobile app in Oracle MCS that would allow for push notifications to a.) Patients when they have forgot to take their readings or leave a perimeter (area of residence and b.) Pharmacists and doctors when a process is triggered due to a critical reading.
  • Finally we created a range of predictive and analytical reporting dashboards leveraging Oracle Database Cloud Service (APEX), Business Intelligence Cloud Service and Data Visualization Cloud service.

Here is an overall connected architecture for the use case. The entry point and heart of the solution is the Internet of Things Cloud Service (IoTCS). Patients can self measure their blood pressure readings remotely through tele-monitoring devices that collect and send data to either an IoT server directly or through a gateway.

Oracle Cloud Connected Architecture

This blog aims to explains how we specifically used Oracle IoTCS that fascinated us and is where we built quite a lot of our components and was at the heart of our use case. Hopefully the use case will present by itself.

Oracle Internet of Things Cloud Service (IOTCS)

The IoT Information Value Loop

The Information Value Loop mode shown below can serve as the cornerstone for an approach to IoT solution development for potential use cases. This model provides discrete but connected stages. An action in the real world allows creation of information about that action, which is then communicated and aggregated across time and space, allowing analysis of the data in the service of modifying future acts.

IOT_IVL

A Quick Primer to Oracle IoT Cloud Service

To realize the use cases for myHealth to the information value loop model depicted above, we were able to utilize Oracle IoT Cloud Service to:

  • Collect proximity information about a patient’s movement i.e. to check if the remote patient is leaving his house.
  • Collect blood pressure and pulse measurement about a patient.
  • Register directly connected devices (Beacons and Blood Pressure Monitors) or Gateways so that information can be collected from these devices remotely in real time.
  • Perform real time pattern matching on data streams to sanitize information by eliminating duplicates, detecting missing events, etc.
  • Perform data aggregation, filtering and correlation on multiple streams to generate business events.
  • Raise message alerts based on business events.
  • Define external integration touch points to invoke systems, services and process to act on critical events.
  • Remotely monitoring devices and assets.

Among the various components of Oracle IoTCS, here is a quick diagram explaining the core components and the relationship between them.

IoT-DataModel

In addition to the above server side artifacts, Oracle IoTCS provides a ranges of client APIs for platforms such as C, Java, JavaScript, and Android for devices to activate, register and send messages to an IoT server. Devices can either connect directly to the IoTCS server or they can connect to a Gateway which in turn communicates to the IoT server. Once connected, a device can begin to send messages to the IoT server based on a pre-defined Device Model definition (analogous to a message contract).

Oracle IoT CS Device Models – Registration and Activation

For our myHealth IoT application we created two device models to collect patient health readings and proximity readings respectively.

  • iHealthSense – This device model (urn:demo:health) was created to define a message format for the blood pressure reading received in the IoT server through any devices that capture this information. In addition to the predefined System Attributes, a device model allows creating Custom Attributes through the console. This allows specifying a message format that can be published from any device that uses this device model.

 iHealtSense

  • estimoteProximity – This device model (urn:demo:proximity) defines a message format that any registered proximity devices such as Beacons use to send messages to the IoT server. Beacons can relay geographical coordinates as well as proximity of a device/person/instrument from them. In addition, certain Beacons can also send temperature, humidity as well as images.

estimoteProximityDM

The next step for us was to create, register and activate Devices that can use these device models and communicate with the IoT server. Creating a Device in the IoT server is a two part process.

  • Registering a Device – Registration of a device is done in the IoT CS console. The following image shows how a single device is being registered by entering appropriate meta data level information about the device.

IoT-Device-Metadata

Once the device registration details are saved and successful, Oracle IoT CS server will create a unique Device ID based which can then be used by the actual physical device APIs to send and receive messages from IoTCS server. A provisioning file is also created on the web console that stores an encrypted connection credentials required to connect to this device and activate it on the IoT server through the IoT APIs. Any client side API’s would then require this provisioning file to interact with the activated device. The provisioning file is a configuration file that contains a hashed password, the URL of the IoT CS server and the device activation id registered on the IoT server. When this provisioning file is downloaded, a file protection password needs to be specified which will be used to fetch the connection details in the hashed file.

IoT-Device-Metadata-Activation

Application Container Cloud Service

When we began looking into the best ways to push data in IoT, we realized that the NodeJS IoTCS APIs are 100 times faster than the Rest APIs. We implemented the client APIs used to communicate with the Oracle IOT CS server using Node.js. Both Oracle MCS and Oracle Application Container Cloud Service provides containers to run and execute Node code.

Setting up Node.js and Node Package Manager

Node.js can be downloaded from here and the IoT CS client libraries can be downloaded from here. Depending upon your preferences, you can either use a standard text editor to write the Node.js code or use an IDE that supports it. Next install the client libraries as global libraries through Node Package Manager (node-pm) using the following command.

npm install forever -g

  • Activating a Device – When a remote device sends a message to the registered device in the IoT CS, it is automatically activated. The following is a code snippet in Node.js using the Oracle IoTCS Node.js libraries that connects to a remote server as a standalone client using the provisioning file and activates a device. Once a device is activated, it can send and receive messages.
dcl = require("device-library.node");
dcl = dcl({debug: true});
//This is the code to load the provisioning file for the device downloaded from the IoT Server
dcl.oracle.iot.tam.store = ("iHealthSense-WA-UU112313-provisioning-file.conf");
//Password specified when downloading the provisioning file
dcl.oracle.iot.tam.storePassword = ("#Musicrules12");
var myModel;
var virtualDev;
// Create a directly connected device and activate it if not already activated
var dcd = new dcl.device.DirectlyConnectedDevice();
if (dcd.isActivated()) {
 dcd.close();
} else {
 dcd.activate(['urn:demo:health'], function (device) {
 dcd = device;
 console.log(dcd.isActivated());
 if (dcd.isActivated()) {
 dcd.close();
 }
 });
}

Save the above code snippet as DeviceActivatation.js, go to a command prompt (assuming you are using windows) and type

node DeviceActivation.js

If we check the state of the device in IoT CS server, it would be Activated. The device activation also associates the selected device model with the device. This means that the device can begin accepting messages in the format described in the device model.

iHealthWristRegistration

  • Sending Messages to a Device – After a device has been activated, the following Node.js code fragment, acting as a remove directly connected device, can be used to publish a message to the IoT CS server.

dcl = require("device-library.node");
dcl = dcl({debug: true});
dcl.oracle.iot.tam.store = ("iHealthSense-WA-UU112313-provisioning-file.conf");
dcl.oracle.iot.tam.storePassword = ("#Musicrules12");
var myModel;
var virtualDev;
// Create a directly connected device and activate it if not already activated
var dcd = new dcl.device.DirectlyConnectedDevice();
if (dcd.isActivated()) {
 getHWModel(dcd);
} else {
 dcd.activate(['urn:demo:health'], function (device) {
 dcd = device;
 console.log(dcd.isActivated());
 if (dcd.isActivated()) {
 getHWModel(dcd);
 }
 });
}
// Display the device model
function getHWModel(device){
 device.getDeviceModel('urn:demo:health', function (response) {
 console.log('-----------------MY DEVICE MODEL----------------------------');
 console.log(JSON.stringify(response,null,4));
 console.log('------------------------------------------------------------');
 myModel = response;
 startVirtualHWDevice(device, device.getEndpointId());
 });
}
//Send Message to the Device based on the Device Model Attributes
function startVirtualHWDevice(device, id) {
 var virtualDev = device.createVirtualDevice(id, myModel);
 var bpHigh = Math.floor(Math.random() * (180 - 100)) + 100;
 var bpLow = Math.floor(Math.random() * (120 - 60)) + 60;
 console.log('High BP'+ bpHigh);
 var newValues = {
 bloodPressureHigh: bpHigh,
 bloodPressureLow: bpLow,
 patientName: "Arun Pareek",
 patientAge: 32,
 patientEmail: "<a>arun.pareek@rubiconred.com</a>",
 patientContactNumber: "0424978874",
 patientGender: "Male",
 pulse: 60,
 ora_latitude: -37.817000,
 ora_longitude: 144.946000
 //timeStamp: new Date().toDateString()
 };
 virtualDev.update(newValues);
 virtualDev.close();
};

Once a message is received for a device, it can be audited in the IoT CS service console. Pretty simple and powerful.  The Node.js code can also be tweaked to send a bulk message to the device.
IncomingDeviceMessage
Aggregation, Analytics and Actions using Oracle IoT CS

Data collected from multiple sensing devices come in different formats and at different sampling rates—that is, the frequency at which data are collected. Oracle IoT CS allows creation of data exploration sources that can collect structured raw data and present it as a continuous stream.

Exploration Sources

An exploration source can be created in an IoTCS application based on a device model. This exploration source will read and store the raw data that is received in the IoT server from the remotely connected device. The exploration source message format is based on a pre-defined device model defined in the IoT application.

Data and Explorations

Explorations

A message exploration can be used to analyze and pattern match data from different exploration message to provide a real time stream of messages. Explorations in Oracle IoT Cloud Service are very powerful as they support message aggregation and analytics through a number of pre-defined patterns (A pattern is a template that already has the business logic built into it) such as duplicate elimination, missing event detection, fluctuation detection, etc. An exploration is created based on an exploration source.

Data and Explorations-2

One an exploration is created, it is in an unpublished state. Explorations are visual representation of real time disparate event data flows for insightful data interrogation from IoT devices. Explorations provide a number of feature such as correlating, filtering and grouping for managing data streams in real time. These data functions can be aggregated on a time or event based window and used in creating a summary view of data. Business rules can also be defined in explorations to review incoming events and apply business logic to generate resulting data fields that can be displayed in tabular or graph forms. Once an exploration is edited and finalized, it has to be published for the changes to take effect. For our use case, we created a standard exploration that would stream the blood pressure and pulse reading of patients in real time.

Editing an Exploration

Duplication Elimination and Anomaly Detection

Explorations in an Oracle IoT Cloud Service application can be nested, which means, we can create another exploration based on an existing one. As a best practice, it is always recommended to not add any business logic in the base exploration but to create a derived exploration from it. The pipeline shown below provides the necessary abstraction for the core explorations used for analytics from the source explorations.

DuplicateEliminationDesign

The below exploration is merely built on the previous exploration but is based on the duplicate elimination pattern. This exploration will only stream data events that has a unique msgId. The window specifies the time period or interval until which the pattern will be active for each incoming message.

EliminateDuplicateBP

Adding Business Logic to Explorations

Finally, the last nested exploration if the CriticalBP stream that is built up on the EliminateDuplicateBP as the exploration source. Filters can be applied on exploration so that the stream only generates data events of particular importance. For instance, blood pressure readings where the systolic pressure is greater than 140mm or less than 90mm is considered critical and could lead to hyper or hypotension. These blood pressure readings should raise an alert that can be viewed by and acted upon by a healthcare professional.

ExplorationBusinessLogic

The output exploration, in its published state, shows a real time critical blood pressure events for different patients along with their geographical coordinates (source of the reading) and contact information. Data from remote devices is pushed to this live output stream such that it would constantly refresh when a new event arrives rather than manually refreshing the view.

IOT-Exp-CriticalBP-Message

Getting a Proximity Alert

As I indicated before, there are a number of additional use cases that can be implemented with the core use case of remote patient monitoring. When we were at the Hackathon, we interviewed a few people who had problems with blood pressure to know their key pain points. The biggest one across many people was forgetfulness, i.e. forgetfulness to take readings on time and take prescribed drugs/renew prescription. We needed a solution to remind a patient to take their readings on time. But sometimes reminder alone for a blood pressure reading is not sufficient as the patients need to have their reading device on them. One way to solve this is by using beacons to track the proximity of a patient from his home.

IoT-Exploration-Proximity

Visualize the high level message flow diagram below. if a beacon is attached to a patient’s house door or exit, it keep sending proximity information to the Oracle IoT CS server. When the proximity is breached (i.e. when a patient has left is house), an IoT exploration stream can then check if a reading has been taken for the day. In case of a missed reading, it can generate an alert leading to a notification on the patient’s phone. Off course, this integration would not be possible without using Oracle Mobile Cloud Service but developing this feature is a breeze.

EventAlertDesign

Raising Alerts from Explorations

Streaming data into IoT explorations is very useful for real time data analysis for predictive intelligence and reactive actions. To react to an anomaly or an event(s) of interest, create a message format (CriticalBloodPressure) from an exploration to generate an alert of suitable severity. The event message type is based on the attributes defined or created as part of the exploration. IoT exploration attributes need not be the same as the attributes defined in the device model as they can be renamed and logical ones added.

ExplorationMessageFormat

Integrating IoT Application with External Systems

After an alert has been raised in an Oracle IoT CS application, it can forward the message to an external application in the form of a push request. As such a number of publish subscribe or one way messaging pattern can be built using IoT Cloud Service as central point for generating business events.  Ensure to decouple any external integration from IoT CS through a proper service abstraction layer. Oracle API Manager Cloud Service can be very useful here to provide a gateway for outbound integration that can consume business events from IoT CS and publish them to other cloud services or SaaS applications as described in the integration pattern below.

IoTExternalIntegration

Apart from providing an out of the box integration with Business Intelligence CS and Mobile CS, it can pretty much publish a message to any external application that is capable of receiving messages over the HTTP channel. Once a HTTP pipe is established with an enterprise application, Oracle IoT CS provides an ability to confirm a connectivity test with the application. Once a connection is established, the Streams tab allows selecting a message format available in the IoT application (typically an alert) to be sent to the HTTP endpoint.

Iot-Integration

 

ExternalIntegrationTest

Oracle Process Cloud Service (PCS)

In the information value loop for Internet of Things explained earlier, critical business events often require human intervention. Oracle Process Cloud Service can close this loop as it can ACT on information of interest by bringing people, process and information together. Once a process consumes an event of interest, it can notify a healthcare pharmacist to analyse his or her critical reading.

ProcessNotification

Manage Health Alert Process

One of the main challenges in device to human interfaces is that, although users have the smart devices, they minimally follow the suggested course of action and, eventually, the devices end up serving as shelfware. The Manage Health Alert Oracle Process Cloud Service process, shown below, provides an ability to integrate processes that involve devices, applications and require human intervention. The PCS process outlines a map that facilitates patient-pharmacist-medical practitioner interaction. The process allows:

  • Review of critical and historical readings for detecting any existing patterns to rule out health related red-herrings.
  • Pharmacists to reach out to the patient or their emergency contacts to inquire about their well being or possibly do a tele-health check.
  • Automatically schedule consultation and appointments with a physician detailed check.
  • Reviewing or renewing prescriptions for the patient and delivering it to them.

These are just a few among many potential human to human interaction use cases made easy through IoT and PCS integration.

PCSProcess

Just like its on premise sibling (Oracle BPM Suite 12c), Process Cloud Service provides an instance tracking console to view and manage live instances. The console provides a detailed graphical and activity audit trail along with an ability to act on touch points that requires human intervention.

PCSTaskFlow

Oracle PCS provides basic HTML5 based web forms which can define the user interface allowing end users to interact with business processes. The form composer provides an editor for creating basic forms that can present information based on the data elements defined in the process. We created the following form for pharmacists to review a critical reading for a patient in order to determine the next course of action. The data captured from an end user in the form acts as a decision point that can trigger alternative process paths.

PCSForm

 

Reporting and Dashboards – Utilizing Oracle Database CS, Business Intelligence CS and Data Visualization CS

Exploratory Data Analysis is the process of sifting through the data to get a sense of whats going on. Extracting insight from data requires analysis, the fourth stage in the Information Value Loop. A derived insight is a consumable piece of information that can be used to make decisions based on current events or make future predictions. A simplified reference architecture pattern to derive analytical reporting from IoT Cloud Service is presented below:

  • Structured data can be stored in transactional or historical data marts.
  • Unstructured data can be stored in distributed file based databases storage platforms.
  • From a reporting layer perspective, there are multiple solution components that can be used such as Apex reports, Oracle BI and Data Visualization dashboards, business activity monitoring dashboards, etc.

IoTReportingPattern

Of the many different reports that can be built to compliment the above solution, we built a few to aid the medical practitioners to dig into a patient’s health reading over a period of time to assess the abnormality over a time range. The following report created in Oracle Database Cloud Service (APEX) tabulates the reading of a patient over a month. This can be accessed as a link and by providing arguments from a PCS form link.

ApexReport

There can be various renditions of the above information the most common of which is a bar chart of systolic and diastolic pressure and pulse rate average each day over month. The histograms can visually highlight up or down trends in blood pressure over a period of time which can be useful to detect spikes/troughs over a baseline.

IoT-Apex-Chart

Anonymizing Patient Data

The last solution that we built to support our use case was to sanitize the patient health readings by removing personal and confidential data and presenting the information in a statistical way that could benefit national/state government health record keepers. How do we do this? Comes Oracle Data Visualization Cloud Service (DVCS) to our rescue. With the blood pressure reading streamed and sent to Oracle BICS from IoTCS, we built a Custom Points Map to the coordinates from where the readings were taken onto a geographical map. This resulted in a heat map showing regions/places with susceptibility to critical blood pressure patients. This sort of anonymized data is immensely useful for government health agencies to plan for aid work and improve the ability to respond faster to critical conditions.

DVCSChart

To Sum it All Up

Transferring healthcare monitoring from hospitals to the home is disrupting traditional models of healthcare delivery. The positive implications of an IoT ecosystem in healthcare are numerous—from remote patient monitoring to predictive analytics—and can result in improved population health management. While this may seem like a futuristic daydream to those on the sidelines, leaders in the healthcare industry are already integrating IoT devices to improve healthcare. The speed to value at which use cases like the one covered in this detailed blog can be realized with little to no infrastructure setup cost using a combination of Oracle PaaS services is unprecedented.

There are a number of other potential use cases such as traffic incident monitoring, pest management, smart homes, smart parking management and other industrial applications that can be realized using a similar architectural pattern. The speed to value at which these solutions can be prototyped and productized using the Oracle PaaS Cloud service is remarkable and the Hackathon was great example of how complicated end to end solutions can be developer with ease.

If you would like to know more about the use case or technical details of the cloud services that we used, please feel free to leave your feedback or comments.

Suppress Approval Controls in Oracle BPM – Hidden Workspace Feature

My good friend and Oracle ACE Director Antonis Antoniou blogged has about how approval controls can be disabled in the Oracle BPM Suite 12c workspace.

http://antonis-antoniou.blogspot.com.au/2015/08/suppress-approval-controls-from-bpm.html

This article is not an attempt to splog what Antonis has already shared. In his blog Antonis provides a mechanism to trick the BPM 12c human workflow engine into disabling the approval controls for a task, visible in the BPM workspace, by manually modifying the .task configuration file. While this is a great workaround, it would become cumbersome if this was to be done for all tasks. If this is a critical functionality, there will be an additional manual step to verify that the approval controls do not appear for every task that is accessed from the workspace. Also as this configuration is embedded in the .task file which is part of the composite assembly, any change in requirement to enable approval flow in the BPM workspace again, will require a redeployment of the composite.

Ironically, there is another feature available from the BPM workspace, albeit still HIDDEN, to disable the task approval controls at the workspace level.

Login to the BPM Workspace as an administrator and navigate to Administration –> Application Preferences. Among the many workspace preferences that can be set or changed from here, the one relevant to disabling approval control in the BPM workspace is the Worklist Action Menu option. Choosing between Show or Hide will either enable or disable the feature at runtime.

image

By default the value for this option is set to Show. The Action menu will show a list of pre defined workflow actions that are available for the selected task in the workspace.

image

When the Worklist Action Menu control is set to Hide and saved through the Application Preferences setting, the Action button will be disabled on the workspace.

image

This is a true runtime change meaning, administrators can flick the values to choose their desired behaviour. Once this change is applied, it also effects all tasks that are accessed from this workspace. 

This also available in Oracle BPM Suite 11g but only if you are using 11.1.1.7 (PS6) from the Application Preferences console for workspace administrators.

Restrict Reassignments in Oracle BPM

Introduction

The human workflow engine in Oracle SOA and BPM suite manages the assignment and lifecycle of human tasks deployed to the runtime infrastructure. In this post, I will share an interesting use case around restricting task reassignments to only certain group of users. This article will provide a comprehensive documentation (solution, what happens in the background, limitations and behaviours across different versions of Oracle BPM ) of how to implement this requirement using a demo application.

Before that let me provide a mind map of how task assignments work in the Oracle human workflow engine.

image

As you can gather from the above image, there are many strategies for task assignment that an organization can adopt to suit their specific needs. I would definitely recommend reading the following blogs to go over each of them in more details.

https://blogs.oracle.com/ateamsoab2b/entry/bpm_11g_dynamic_task_assignment

https://technology.amis.nl/2014/02/13/humantask-assignment-in-oracle-bpm-suite-and-soa-suite/

The use case discussed and demonstrated in this post is different from the ones discussed in the amazing blogs above.

Swimlane roles are only indicative of the default task assignment mechanism used by the human workflow engine which can be easily overridden by any of the strategies discussed above. Once a task has been assigned, either to a pool of users or an individual users, it may undergo many state transitions before being completed. The most common state transition is Task Reassignment. Although the actual state of a task is still Assigned, a task may be transferred between different participants from its original assignee. When a task is to be reassigned to another user, the BPM workspace queries the underlying directory for all available members. Task can then be reassigned to a member selected from the returned list. There is often a requirement to restrict reassignments to a custom group of users with certain privileges or access (such as users only belonging to certain roles). 

Prerequisites

The post demonstrates the requirement and use case through a sample BPM application. In order to view, deploy the execute the application, a list of prerequisites is provided here.

Softwares

Runtime

  • Oracle BPM Suite 11.1.1.7.8
  • Oracle BPM Suite 12.2.1.0 [Optional]

Downloads

Problem Statement

The best way to illustrate the use case is through a simple expense management process illustrated below.

image

The process has three different swim lane roles viz. Employee, EmployeeManager and FinanceManager, one each for the type of task and their performers in the process. These swim lane roles can be mapped to users, groups or application roles from either the Fusion Middleware Enterprise Manager console or the BPM workspace so that only users with designated access can view these tasks.

Once an Administer Payment task is created, it will be assigned to either a user, group or role based on the assignment strategy.

Now imagine that a user who has an access to this task wishes to reassign this task to only peer i.e users within his working group. As indicated earlier, task reassignments is a very common feature of workflow based systems. Tasks can be reassigned for many reasons such as to transfer ownership, during user vacations, for peer review to name a few. Follow the steps provided below after you have deployed the sample application to see what it the default task reassignment behaviour of the BPM workspace.

  • Login to WebLogic Administration console as an administrator and create three dummy users viz. employee, manager and finmanager.
  • Login to the Oracle BPM Workspace as a workflow admin user and access the Role administration console.
  • Once a process is deployed, the swim lane roles will be created automatically and can be viewed via Organization > Roles in the BPM workspace.
  • Edit these roles to add/associated the users that were created before. The swim lane role to user association is simple. (Employee > employee, EmployeeManager > manager and FinanceManager > finmanager)

image

  • Now login to the BPM workspace as an employee user to to initiate an instance of the process (An initiator task allows creating a BPM process instance through the BPM workspace)
  • Under the Applications menu, you would notice a link titled Approve Employee Expenses (same name as the BPM process name).
  • Click the link to open a pop-up window of the task form, enter some expense information and submit the form. For the purpose of this use case, ensure that the total of all expenses is less than 1000.

image

  • Once the form is submitted, the process sequence will execute the business rule to check if the expenses needs a manager approval. If the total of these expenses is below 1000, then a manager approval is not required and it will go directly to the Administer Payment task.

image

  • Log back in to the BPM workspace, this time as a user in the FinanceManager role (finmanager). A Review Expense Application task will appear in the inbox.
  • Try to reassign this task by clicking on the Action > Reassign button from within the taskflow. A Reassign Task popup will appear allowing you to search for users, groups or application roles that this task can be reassigned to.
  • Perform a blank search and you would see that it lists all available users, groups and roles currently configured in WebLogic as potential candidates for this reassignment.

image

The BPM workspace is integrated with the active directory and as such the number of users, groups or roles that will be available depends upon the base AD query filter. More often than not this is a problem as users want to restrict this search and only be able to reassign tasks among their group.

Solving the Puzzle

Reassignments can be restricted in Oracle BPM Suite 11g using what is known as Potential Assignee. This feature allows restricting the task assignment to only users or group of users assigned to an application role. It is a design time configuration available from the Organization.xml file in the BPM project.  In order to demonstrate how this works, we would need a new WebLogic group, a new user, change associations of two users to a group and map this group to the FinanceManager role.

  • Login to WebLogic Administration console as an administrator and create three dummy users viz. financereviewer.
  • Also create a group and name it as finance (or choose a well suited name for a group)
  • Now select the user finmanager and financereviewer one at a time and from the Groups tab assign them to the newly created finance group.
  • Login to the Oracle BPM Workspace as a workflow admin user, access the Role administration console and modify the FinanceManager role mapping.
  • Remove the previously associated finmanager user from it and add the finance group to this role.

image

Now go to the composite project in JDeveloper in BPM Project Navigator view and open Organization. All the swimlane roles used in the project/processes will be part of this file. Select the FinanceManager role from the list of available roles. You would notice that there is an input field in this palette against Potential Assignee. The magnifier icon allows searching for application roles from within the Organization.xml file or from a configured identity provider (here AD provider configured in WebLogic).

Restricting reassignment could be as simple as selecting the same swimlane role against this field. This will limit the users available for reassignment of any task(s) in the FinanceManager role to the role configured against the Potential Assignee field. In this example the FinanceManager role is mapped to the finance group which in turn has two users associated to it (finmanager and financereviewer). The mechanics of how this works can be verified by deploying the project to the server again with the Update existing object on deploy option checked.

When the project is redeployed with this flag checked, the role associations for the swimlane roles would be lost and have to be recreated through the BPM workspace.

image

Once the process is redeployed, create another expense record as an employee and submit it so that it reaches the FinanceManager queue. Open the Review Expense Application task from the inbox view of the BPM workspace and from the Actions menu click on Reassign. In the Reassign Task popup, if we do a blank search, we would see only the swimlane role, the finance group and the users belonging to the group instead of all directory users, groups and roles.

image

The scenario depicted here simply mapped the same swimlane role to the Potential Assignee field which is a simple use case. For complex restrictions on reassignments, create an application role under the BPMRolesApp strip to map selected users/groups that a task is allowed to reassigned to and use this role instead.

What Happens Under the Hood

It is also interesting to know what happens under the hood. When a BPM project is deployed with a Potential Assignee set, it creates an entry in the BPM_EXT_USER_PROPERTY_GLOBAL table (in the <APP>_SOAINFRA). The PROPERTY_VALUE_COLUMN_NAM contains a value of STR_COL<N>. The BPM_EXT_USER_PROPERTY_VALUE table in the same schema has the STR_COL<N> column where the value of the potential assignee for the application role in the deployed project is stored.

image

When a search is performed from the BPM workspace for users from the Reassign Task pop-up, the getPermittedAssignees operation in invoked on the human workflow engine. This operation returns a boolean value indicated if the reassignment should be restricted along with a list of permitted assignees.

image

You can read more about the Human Workflow web service APIs in the following blog post.

https://beatechnologies.wordpress.com/2011/08/22/oracle-human-workflow-web-service-apis/

Important Note

Potential Assignee only effects reassignments. It does not restrict assignment of the task into the application role configured for the Potential Assignee field.

Known Issues and Limitations

Unique Constraint Violation when Potential Assignee is Set for Multiple Application Roles

If a BPM project has multiple swimlane roles with Potential Assignee is deployed to the infrastructure, it will try to add this entry multiple times in the BPM_EXT_USER_PROPERTY_GLOBAL creating a unique constraint violation in the database.

 There was an error deploying the composite on server: Deployment Failed: oracle.toplink.exceptions.DatabaseException: java.sql.SQLIntegrityConstraintViolationException: ORA-00001: unique constraint (DEVBPM_SOAINFRA.PROP_NAME_PROP_VALUE_UNIQUE) violated Call: INSERT INTO BPM_EXT_USER_PROPERTY_GLOBAL (GUID, LAST_UPDATED_BY_IDCTX, LAST_UPDATED_DATE, CREATED_BY_IDCTX, MULTI_VALUED_PROPERTY, PROPERTY_DATA_TYPE, CREATED_BY, PROPERTY_NAME, LAST_UPDATED_BY, PROPERTY_STRING_VALUE, CREATED_DATE, PROP_VALUE_COLUMN_NAME) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) bind => [4a195fc8-083c-4af0-81d5-942f0deacf0a, jazn.com, 2016-03-10 11:58:40.877, jazn.com, 0, FREEFORM_STRING, workflowsystem, POTENTIAL_ASSIGNEE, workflowsystem, null, 2016-03-10 11:58:40.877, STR_COL5] 

This is a know product bug and can be fixed by applying patch 22779160 available from Oracle support site. This patch is only available for 11.1.1.7.8 version of Oracle BPM presently but if you are running a different version, a back port patch can be requested.

Potential Assignee is mapped to a Swimlane and not Task

Remember that potential assignee can only be mapped to the swimlane and not directly to the task. If you have two tasks in the same swimlane and each needs a different restricted reassignment then this will not work. There are two options to overcome this limitation.

  • Either split the swimlane into two, each having its own task which is a quick and cheap option but interferes with the process modelling best practice.
  • Create a custom Java Class which is associated to a .task file to handle restricted assignment and reassignments.

Reassignments in Oracle BPM Suite 12c

Reassignments work completely opposite in BPM Suite 12c to how they work in 11g. By default, reassignments are restricted to the members configured for the swimlane role. I deployed the same process in the 12c runtime, created an instance of the expense management application and submitted it. The expense was created to exceed the auto approval threshold and the process flow created a approval task for the manager to approve the expenses. Potential Assignee was not set for the EmployeeManager role but a blank search in the Reassign Task popup from the BPM Process Workspace only returned users associated to the swimlane role instead of all directory members. It is remarkable to see how a certain functionality is completely juxtaposed between two product versions.

image

The usage of Potential Assignee is still significant in BPM processes developed in 12c albeit to cater to task reassignments if they are to be to be allowed for a wider list of members than those associated to the swimlane role.

Conclusion

Reassignment of tasks is a core function of any BPM based workflow. This article provides a deep dive into how this feature works across two Oracle BPM suite versions and what is provided in the tool to accommodate restricted reassignment.

Please feel free to share opinions, feedbacks and suggestions and I will be glad to provide any help.

Oracle SOA Suite 12c Administrator’s Guide Released

Your blogger, for the past few years have piled up more things that he can chew.  As a result the blog posts have dried up a little as blogging involves both time and novelty and I have been a little short on the first lately. Of that may activities that consumed me, none was more time taking, challenging and difficult as writing the Oracle SOA Suite 12c Administrator’s Guide . The book was announced a month ago and is finally out and available. Here is a sneak peak a the cover.

0860EN_2614_Oracle SOA Suite 12c Administrator's Guide_0


Oracle SOA Suite 12c is the most comprehensive and integrated infrastructure on the market today that is used for building applications based on service-oriented architecture. With the vast number of features and capabilities that Oracle SOA Suite 12c has to offer comes numerous complexities and challenges for administration. Oracle SOA Suite 12c Administrator’s Guide covers all the core areas of administration needed for you to effectively manage and monitor the Oracle SOA Suite environment and its transactions, from deployments, to monitoring, to performance tuning, and much, much more.

Manage, monitor, and troubleshoot SOA composites and OSB services from a single product set. Understand core administrative activities such as deployments, purging, startup and shutdown, configuration, backup, and recovery. Also learn about new features such as Oracle Enterprise Scheduler, lazy loading, work manager groups, high availability, and more.

With topic areas ranging from the simple to the complex, this book is intended for novice, mid-level, and experienced administrators of the Oracle SOA Suite 12c platform as well as Oracle WebLogic Server and Oracle Database administrators interested in diving into the product.

I would finally try to make a brave and best attempt to summarize in brief the key learning’s from the book:

  • Navigate Oracle Enterprise Manager Fusion Middleware Control
  • Monitor and manage the Oracle SOA Suite 12 c infrastructure
  • Deploy and promote code
  • Monitor and manage services
  • Configure and administer the environment
  • Manage the dehydration store and enterprise scheduler service
  • Troubleshoot Oracle SOA Suite 12c infrastructure
  • Set up backups, recovery, and high availability

For those who are keen to see what the book has to offer, its style and approach, a sample chapter can be accessed from the following link

 Sample Chapter

Any feedback, comments and inquiries are most welcome!

Arun Pareek

Functional Testing Business Processes In Oracle BPM Suite 11g

Over the last couple of projects, I have been trying to establish a set of methodologies around initiation, development, deployment and management of business processes using Oracle BPM Suite 11g. In a series of posts, I will be taking about a lot of problem areas that can be effectively tackled in order to deliver a highly successful BPM project using this technology. The areas I intend to cover are around design and modelling best practices, team development, delivery methodologies, testing, troubleshooting, automation, performance tuning and operational management to name a few. However in terms of priority and the nature of pain experienced in each of these aforementioned areas, I am of the opinion that a great chunk is attributed to testing and maintaining code quality. Business processes are in constant need to be truly dynamic more than ever so that planned as well as ad-hoc changes can be promoted seamlessly. Also with Oracle BPM Suite 11g promising greater involvement of business users in terms of making deployable changes to the processes, ensuring that functionality and quality of the build/changes is maintained is very crucial. In the previous blog post, I had elaborated on unit testing strategies for business processes and methods of achieving them from the Oracle BPM Suite 11g studio. Proper unit testing is important in terms of validating that the core logical outcomes of a business processes are executed as expected. This ensures that all sequence flows and paths in a business process are tested and conformant. The post can be accessed here:

https://beatechnologies.wordpress.com/2013/05/05/unit-testing-business-processes-in-oracle-bpm-suite-11g/

However, apart from testing the basic process paths a lot of other things demand testing too. Business processes can populate data in downstream systems, integrate with services, create events, send notifications, assign workflow tasks which may have their own lifecycle, etc. From a quality perspective all of these steps must be verified too. The overall functionality of the business processes must be thoroughly tested to determine their reliability before being deployed to the actual IT infrastructures. Organizations do invest in quality measures to evaluate features of business processes by employing a comprehensive testing strategy, involving analysts, developers, functional testers and business users. However they begin to invest in test automation very late in the project lifecycle. This poses a significant threat and concern, particularly when the business processes are continuously changing and evolving. An increasing pace of business process changes as well as applications growing more complex means that many functional testing teams are reaching the breaking point. Change management becomes difficult as there is always a risk of introducing new bugs down the way. As much as we all would like to have agility, the process is defeated by having improper developer operations. Having a regression functional testing suite very early in BPM project development cycle is, by my opinion a must have thing for any successful and timely implementation. More often than not, it is not the choice of technology that is responsible for this but rather the lack of some important know-how’s.

Typical Problem Areas and how Automating Functional Testing can help

In my interactions with different project teams, here are some typical questions they have, especially with respect to testing business processes in Oracle BPM Suite 11g.

  • Is it possible to really approach functional testing early in the project cycle considering the nature of composite applications involved (business processes, services, rules, etc.).
  • Human tasks components in business processes need to have user interfaces (which may have their own development lifecycle). How can functional testing be achieved before a fully operational UI?
  • Are their any developer tools, approaches or frameworks that can be utilized to automate functional testing?
  • What kind of a functional testing strategy is suitable, cost efficient and good to contain a wide variety of problems in the project lifecycle?

If we begin to investigate the possibilities, they may be endless. I, am particularly aware of many custom products which promises to take care of all of these. However hey tend to be expensive, have a steep learning curve and may not always fit the bill. This act as a big deterrent. However, in this article, I am keen on presenting an approach that is quick, easy, developer friendly and can be easily catered in the development phase of the project. The best way to begin with functional testing is to base it on the business process design. Model driven functional testing ensures that critical processes and their paths are covered and that any variation of those tests can easily be amended and rerun without adding to significant time to the test cycle. Functional testing can start earlier and better quality tests are produced and maintained in a test repository. This can then be part of the continuous build and integration iterations to ensure sufficient test coverage in accelerated delivery cycles. A part of model driven testing strategy was described in details with respect to ascertaining the cyclomatic complexity factor of the business processes and deriving test cases based on that. This approach is a very good starting point, especially because this can be used to build unit tests too. Once the basic process paths are determined, the functional test suites can incorporate a lot of test steps to cover the functional aspects such as validate business logic and rules, verify data across systems, check events and notifications, etc..

With all this being said, however, automation of functional testing will only be successful if an organization’s underlying quality fundamentals are solid and everyone clearly understands how testing can continuously support process iterations. Another big advantage to automate early is potential time savings. Considering that multiple stakeholders will need to repeat their testing task, every time something changes, doing them manually becomes a burden sooner or later. Have a look at the sheet I’ve compiled based on various time and frequency of performing functional tests and the effort involved in each case.

image

The columns marked in green in the above sheet reflects the most likely scenarios in terms of the amount of time spent on testing in various Oracle BPM Suite 11g projects. The effort involved in manual testing can range from 20 days to 4 months for a very simple and less frequently changing business process to a complex and often changing business process. This effort does not accounts for test planning but simply reflect the time spent in test execution. The columns in red represents scenarios that can be applicable for certain project types too and in those cases the amount of time spent in manual testing can be even 8 months. This offcourse is under the assumption that all quality standards of software functional testing are maintained. 

Approach and Tooling

As indicated before, there are many players in this field providing varied approaches and option. This article will not debate their merits here. In the course of my involvement with BPM projects, I seems to incline towards SOAPUI. This is particularly due to the fact that it is pretty inexpensive (there is also a freeware if ~$350 license cost seems significant) and does most of the basic functional testing that can be considered acceptable from a quality perspective. It is fairly straight forward too and the learning curve is minimal. Chances are also high that in most projects there will be developers with considerable amount of experience using SOAPUI, if they have been involved in a SOA project in the past.

Having said that, this article will talk about how to plan and implement a comprehensive and automated functional testing suite for business processes developed using Oracle BPM Suite 11g.

The Employee Expense Approval Business Process

I will use the same employee expense approval business process that was discussed in my previous blog. This process has primarily all the ingredients that any complex business process has i.e it is fairly unstructured, have business rules, human workflow components, arbitrary cycles, gateways, events and writes to external applications.  As a refresher, the process map looks like below in the JDeveloper studio.

image

Technology Prerequisite(s)

The modelling workspace and the version of Oracle BPM Suite used in this demonstration is 11gR1PS5 i.e 11.1.1.6. The version of SOAP UI being used is 4.5.0 (excuse my backwardness in terms of working with the latest versions of these tools). I use the open source version of SOAPUI but for better productivity and ease of use you may consider using the Pro version too. A connection to the database is also necessary to be able to create certain database objects that the project requires. The version of the database, though, can be any generic with the only exception that it has to be Oracle database.The concepts discussed in this blog are however generic and applicable to any version of the above products.

Project Setup

In order to import the implemented expense approval project in JDeveloper, obtain the composite (ExpenseApprovalComposite.zip), unzip it and open the application (.jws) file in the studio. As can be noticed from the BPMN mode, the process populates some database tables with expense and payment records at various stages. To be able to deploy and execute the project, a database schema and certain tables needs to be created. On the Weblogic server side, there is a need to create a data source and add a new connection factory to the database resource adapter.

Doing this set-up is pretty simple. I have created some SQL and ANT scripts that can do all of this. Follow the instructions below to execute these scripts to create the required resources.

Create the Database Artefacts

  • Download and unzip the DatabaseSchema.zip file in a local directory.
  • Open the edit the create_expense_structure.bat and simply substitute the sys password with the one for the database you are using. This script assumes that a local XE flavour of the database is running. If this is any different then modify the bat file accordingly.
  • Open a command prompt and cd to the directory containing the bat file. Simply run the batch file in the prompt.
  • This will create a database user “EXPENSE” and two tables in it called EMPLOYEE_EXPENSE_PAYMENT and EMPLOYEE_EXPENSE_RECORD.
  • Verify that the scripts ran okay and the database artefacts were created.

Create the Weblogic Server Artefacts

  • Unzip the ResourceAdapter.zip in a local directory.
  • Open the build.properties file and substitute the middleware home location with the one being used. Also modify the values of the database server, weblogic server, domain, connection credentials etc. Properties that needs to be changed are marked in the build.properties file through comments.
  • Open a command prompt and cd to the local directory where the contents of the .zip file are placed.
  • Type ant makeDataSource to create the required datasource in the weblogic server to connect to the EXPENSE schema created in the step before.
  • Once the build is successful type ant createResourceAdapterEntries in the same command prompt.
  • This will create the database adapter connection factory and update it with the datasource created before. It also updates the DbAdapter deployment plan to commit all these changes onto the server.
  • Verify that the script executes successfully. If it doesn’t then there may be a problem in substituting values in the build.properties file.
  • As a last resort, you may configure these resources manually on the server or alternatively send me your configuration files along with the error(s).

Creating the Functional Test Suite

Once the project has been set-up and the required artefacts are created, it is time to look into creating a functional testing suite for the expense approval processes. Once we create the functional tests, they can be used for regression every time there are changes made to the business processes. A good practice is to always accept any changes when it is accompanied by a functional test case and when all existing test cases pass the regression. However the dynamics here may be dependent upon the level of business user’s involvement in creating ad-hoc changes to the business process after it is deployed. If business users are empowered to make dynamic changes, then it cannot be expected of them to create/modify the functional test cases supporting their changes. This situation demands a change control process in place. I am of the personal opinion that to ensure the quality of business processes running on the production environment, changes must always be supported with regression. Whether business analysts and process analysts create these tests or whether they get it done from developers is a matter of having a governance around the change control.

Nevertheless it is pertinent to have an initial set of functional tests. If you managed to read the previous blog post on unit testing business processes in Oracle BPM Suite 11g, you must have noticed how we employed cyclomatic complexity to determine the number of unique test cases required to provide a cent percent test coverage. From the functional testing perspective, we will still create the same number of test cases but also have a series of test steps to validate that the business process is indeed doing all the right things.

As determined before the following test cases shows the various routes a process can take from all the conditional nodes. To ensure the utmost quality, we will have as many functional test cases too.

Test Case 1: Submit, Auto Approve, Finance Approve, Approve
Test Case 2: Submit, Auto Approve, Finance Reject, Refer
Test Case 3: Submit, Refer, Manager Approve, Finance Approve, Approve
Test Case 4: Submit, Refer, Manager Approve, Finance Reject, Refer
Test Case 5: Submit, Refer, Manager Reject, Reject

The following section will demonstrate a series of test steps that will be created to fully functionally test one of the test cases (here Test Case 3: Submit, Refer, Manager Approve, Finance Approve, Approve). Other test cases can be created in the similar lines.

  • Deploy the ExpenseApprovalComposite to a BPM server and grab its wsdl  from the enterprise manager (since the business process has a message start event it can be invoked as a web service)
  • Launch soapUI and create a new soapUI project. Put ExpenseApprovalTest as the project name and copy the wsdl localtion in the Initial WSDL/WADL field. Also check the option to create sample requests for all operations.

image

  • This will create a soapUI project in the existing workspace. Now right click on the project and click on New TestSuite to create a new test suite inside the project. Name it as Expense Approval Functional Tests.
  • A test suite can have multiple test cases that can be run either individually or all together as part of the test suite (either sequentially or in parallel). Once the test suite is created, right click on it to create a New TestCase inside it. Name the test case as T3_Refer_ManagerApprove_FinanceApprove_Approve.

image

  • A test case is constituted of a series of test steps that are executed in sequence one after another. Testing a business process will involve initiating the business process through the message based event, capturing data in the human tasks, simulating user behaviour on the workflow screens and verifying that once the process is completed, it is in the right state, populates the right statuses in the end systems and follow the expected process path.
  • A test case in its due steps may need to refer some constant values or store values intermittently to be used in later steps.  soapUI allows creation of global, test suite level as well as test case level properties. While properties at the test suite level are more appropriate to hold any constants (or pass them between multiple test cases), properties in the test cases are used primarily for passing information between multiple steps. Create the following properties at each level. You will notice that the test suite properties have been set to some constant values.

image

  • The properties can then be accessed as variable using the below construct.
${#TestSuite#TASKSTATE}
${#TestCase#CONVERSATION ID}
  • It is commonly believed that a business process with human tasks will need a functional user interface in order to test it. However it is not true. Oracle BPM Suite 11g offers both Web Services as well as JAVA based API’s to interact with the workflow tables. The operations in these API’s can be used to search and query for tasks, update task payloads and outcome, attach comments and attachments etc. Access the following posts to be able to get a glimpse of the human workflow API’s.

https://beatechnologies.wordpress.com/2011/08/24/using-java-apis-for-oracle-human-workflows

https://beatechnologies.wordpress.com/2011/08/22/oracle-human-workflow-web-service-apis

  • In this article I will be using the Web Service API’s to simulate the human tasks when building the functional test steps. The first step in the test case is to create a unique identifier (random number) that can be used to set the conversation id while invoking the business process. Setting up a conversation id is important as the composite/component instances ID’s can be determined by querying the dehydration store against it. In soapUI a Groovy Script based test step can be used to generate a random number. Name this step as Create Conversation Identifier and paste the following groovy code snippet in the code editor.
testRunner.testCase.setPropertyValue("CONVERSATION ID", "${java.util.UUID.randomUUID()}")
log.info(java.util.UUID.randomUUID())

image

  • The groovy script dynamically populates the CONVERSATION ID property in the test case with the uniquely generated number.
  • Add a new SOAP Test Request and name it as Submit Employee Expenses – submit expense application. When needing to specify the operation to invoke select ApproveEmployeeExpenseBinding –> submitExpenseApplication from the available options. As a matter of best practices, I typically append the name of the process model object that is being interacted with in the name of the test step. This is important to maintain a visibility of which event in the process is invoked or which human task is updated/actioned.

image

  • Copy the following payload to trigger the business process. The combination of business unit and expense total will force the business rule activity in the business process to send this instance to a manager review.
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:app="http://www.rubiconred.com/ApproveEmployeeExpense" xmlns:empl="http://www.rubiconred.com/bpmtest/empl">
<soapenv:Header/>
<soapenv:Body>
<app:submitExpenseApplication>
<empl:Expense>
<empl:Employee>
<empl:EmployeeID>988977698</empl:EmployeeID>
<empl:FirstName>Arrun</empl:FirstName>
<empl:LastName>Pareek</empl:LastName>
<empl:Email>arun.pareek@rubiconred.com</empl:Email>
<empl:PhoneNumber>0424987673</empl:PhoneNumber>
<empl:ManagerID>6312313</empl:ManagerID>
<empl:DepartmentID>132313</empl:DepartmentID>
<empl:BusinessUnit>3</empl:BusinessUnit>
</empl:Employee>
<!--1 or more repetitions:-->
<empl:ExpenseList>
<empl:Amount>80000</empl:Amount>
<empl:Type>Travel</empl:Type>
<empl:Date>2013-02-06</empl:Date>
<empl:Justification>Travel on Business</empl:Justification>
<empl:Authoriser>Matt</empl:Authoriser>
</empl:ExpenseList>
<empl:ExpenseList>
<empl:Amount>40000</empl:Amount>
<empl:Type>Travel</empl:Type>
<empl:Date>2013-02-07</empl:Date>
<empl:Justification>Travel on Business</empl:Justification>
<empl:Authoriser>John</empl:Authoriser>
</empl:ExpenseList>
</empl:Expense>
</app:submitExpenseApplication>
</soapenv:Body>
</soapenv:Envelope>
  • It is also very important to note here that the business process, by the virtue of message based start and end events, is implemented as an asynchronous web service. When the process completes it will send an asynchronous call-back to the initiator at a call-back address. Basically Oracle BPM Suite 11g supports WS-Addressing. The conversation id generated in the previous step can be used as the unique correlation identifier when invoking the submitExpenseApplication operation. Also the initiator can define a call-back address (Reply To in the addressing header) where the engine can send the call-back response.
  • In soapUI WS-Addressing related headers can be added by clicking on the WS-A tab in the test step. The WS-A addressing enabled option has to be selected and the must understand flag is to be set to TRUE. The MessageID field is populated with the CONVERSATION ID test case property. Also provide a reply to address like below to let the BPMN engine register the the call-back address for this instance. This will be used in a future step.
http://rubiconred-pc:2222/ApproveEmployeeExpensePortTypeCallBackBinding
  • The first test to be performed is to be able to successfully determine whether the soap request was successfully delivered to the end point. Since it is an asynchronous request, the response is not received in the same thread. Hence the only way to determine a success is to verify that the HTTP status code received is correct. To do that, add a status assertion for a Valid HTTP Status Code. Enter 202 (Accepted) as the expected status code. This status code reflects that the request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed or fault out when processing actually takes place. The 202 response does not requires an initiator’s connection to the server persist until the process is completed.

image

  • The above test step will create an instance of the expense approval process which can be seen and verified through the enterprise manager.  The next test step to add is to retrieve the composite instance, ecid and the composite state from the CUBE_INSTANCE and COMPOSITE_INSTANCE tables in the dehydration store against the conversation id. The conversation id supplied by the initiator is stored in the CUBE_INSTANCE table. This will correctly grab the instance metadata for the instance that was created in the test steps above.
  • Add a new JDBC Request test step and name it as Retrieve Composite Instance ID. A JDBC test step will require a JDBC driver and connection string and an SQL query to be specified. But before this can event work, you will need to copy the ojdbc5.jar under the $soapUIHome/bin/ext folder. This jar facilitates the connection to an oracle database from soapUI. The following values can be specified for the different properties in the JDBC request test step. Be careful to substitute the database connection details with the ones reflecting your environment.
Driver : oracle.jdbc.driver.OracleDriver
Connection String : jdbc:oracle:thin:ps5_soainfra/welcome123@localhost:1521:xe
SQL Query :SELECT COMPOSITE_INSTANCE.ECID, COMPOSITE_INSTANCE.ID, CUBE_INSTANCE.COMPOSITE_NAME, CUBE_INSTANCE.COMPOSITE_REVISION, COMPOSITE_INSTANCE.STATE FROM COMPOSITE_INSTANCE,
CUBE_INSTANCE WHERE COMPOSITE_INSTANCE.CONVERSATION_ID =CUBE_INSTANCE.CONVERSATION_ID AND CUBE_INSTANCE.CONVERSATION_ID= '${#TestCase#CONVERSATION ID}'

image

  • The SQL query returns important metadata with respect to the composite instance that is created. soapUI allows adding assertions to each test steps in a test case. Assertions are a very good way to add functional validation at each test steps. The following screenshot shows the four different assertions that are added to this test step. The composite name and version are validated against the values of these properties specified in the functional test. The state value of 0 indicates that the process instance has been initiated.

image

  • The instance might not be created immediately and available to be queried against the dehydration store. It may take a couple of seconds for the instance to be initiated. It may be required to add a Delay test step in between the SOAP and JDBC request test steps.
  • The values retrieved from the JDBC request test step and the original payload passed in the SOAP test request can be saved to the test case properties so that they can be used in the next test steps. A Property Transfer test case can be added for this purpose. Name the step as Capture Process Properties and add four test step properties viz. ECID, INSTANCE ID, PAYLOAD and EMPID. For each of the properties the appropriate values can be captured by specifying the source test step, the variable (request, response or raw request, etc.) and the target test step property. When the property transfer test step is executed the test suite properties will have the most current values depending upon the payload entered and the results retrieved by querying the database schemas.

image

  • Once the process is triggered/initiated it creates an entry in the EMPLOYEE_EXPENSE_RECORD table with the relevant employee and expense details. The EXPENSE_APPROVAL_STATUS column at this stage should be marked with a SUBMITTED status, assuming it is used for tracking the state of expenses. From a functionality standpoint it should be verified whether this is indeed the status. To be able to do that is soapUI will mean creating another JDBC Request test step. Name this as Verify Expense Record – Create Expense Record (here Create Expense Record is the name of the BPMN activity in the process model which executes the database update).
  • In the JDBC test step, the connection string this time should be specific to the Expense schema that was created initially. The SQL query to fetch the record for a given employee is determined uniquely by the employee id and the record reference (populated with ECID).
SELECT * FROM EMPLOYEE_EXPENSE_RECORD WHERE EMPLOYEE_ID ='${#TestCase#EMPLOYEEID}' AND RECORD_REF='${#TestCase#ECID}'
  • In the same test step, assertions can be added to verify the expense approval status and also the expense approver, if applicable. A JDBC status validation assertion is also added as this is something that should preferably added to each JDBC Request step.

image

  • For this particular test case, the business rule will determine that a manager approval is required before the expense is presented to the finance user. The business process instances initiates a workflow task that is available to the designated user in his in-tray. In the most basic scenario the task is available to be be actioned by the user(s) assigned to the process swimlane role.
  • At this stage if the audit trail of the instance is viewed it will correctly represent that the process has initiated and is waiting at the Approve Employee Expense human task activity.

image

  • Interacting with the task is very easy and straight forward. Fortunately there is no need for a properly designed UI to be able to test the user interactions with the business process. Oracle BPM Suite 11g provides Human Workflow web services that can be invoked with the proper operations to simulate the UI behaviour.
  • When an instance reaches any human task activity the human workflow engine initiates a task with the appropriate task metadata along with a unique taskId and number.
  • The workflow engine captures this metadata in the WFTASK table in the dehydration store. Querying this table against the task definition name, composite instance id and the ECID will return a task identified for the initiated task.
  • Create a JDBC Request test step yet again and name it as Retrieve Task Identifier – Approve Employee Expense. Use a connection string specifying the details of the SOAINFRA schema and the following sql query to uniquely determine the task identifier for the initiated human task.
SELECT WFTASK.TASKID FROM WFTASK WHERE WFTASK.TASKDEFINITIONNAME='${#TestSuite#MANAGERTASK}' AND WFTASK.ECID = '${#TestCase#ECID}' AND WFTASK.STATE = '${#TestSuite#TASKSTATE}' AND WFTASK.COMPOSITEINSTANCEID= '${#TestCase#INSTANCEID}'

image

  • The task identifier that is fetched as a result of the sql query will be used in the future test steps to query and update task details using the workflow APIs. In order to use it forward, a property transfer step (Capture Human Task Identifier – Approve Employee Expense) has to be created to copy this value to the TASKID property of the test case.

image

  • This task id will now be used in the forthcoming test steps to operate on the human task. In order to do so, first of all we will need to add two more WSDL files to our soapUI test project. These WSDL files can be located at the following end points after substituting the appropriate values for the host and port where the soa server is running. This will add all the operations in these services to the soapUI project catalogue.
http://<host>:<soaserverport>/integration/services/TaskQueryService/TaskQueryService?wsdl
http://<host>:<soaserverport>/integration/services/TaskService/TaskServicePort?wsdl
  • Once the WSDLs have been added, add another SOAP Request test step. When prompted for a name put Get Task Details from Task Indetifier – Approve Employee Expenses (long but very descriptive, pinning down the actual human task activity in the process model for which this test step is performed).
  • Select the getTaskDetailsByID operation in the TaskQueryService wsdl in order to retrieve the task details by providing the task identifier. Invoking this operation also requires appropriate credentials. The task identified is captured in the test case property TASKID. The operation returns the entire human workflow task consisting of the payload as well as additional task metadata. In order to ascertain that all functionality requirements are met, a couple of assertions could be added. As this is a synchronous service invocation, add assertions for a valid soap response, schema compliance and that no soap fault is encountered. These assertions are available in the Compliance, Status and Standards tab of the Add Assertion window.
  • Apart from the web service assertions, also verify that the task state is correctly set to a value of Assigned. Also verify that the task version reason is equal to TASK_VERSION_REASON_INITIATED confirming that the task is in an initiated state at this stage. These elements are child of the systemAttributes element in the task data and can be queried with the following XPath expressions.
/*:Envelope/*:Body/*:task/*:systemAttributes/*:state
/*:Envelope/*:Body/*:task/*:systemAttributes/*:versionReason

image

  • The task metadata returned from the service call has a section called task payload. This represents the initial arguments passed to the human task when it is initiated. As part of a manager approval or any other interaction with a form based user interface, process participants may change data in the forms (editable arguments) after which they may action the task (reassign, delegate, approve, reject, suspend etc.). Depending upon the functional requirements of task management, we may have to create appropriate steps at this point and validate the behaviour. For instance, tasks may always be reassigned to a team leader who may then reassign them to actual participants who have to then action them. In this case a test step to invoke the reassignTask operation in the TaskService service may be required. This test case assumes that a team lead simply puts in his comments (which is part of the payload here and saved to the expense record table) and approves the expenses.
  • Add a property transfer step and name it as Capture Human Task Payload – Approve Employee Expense. The purpose of this step is to take the task payload from the response of the previous step, modify the comments child in the approval status element (replace it with a value such as “Manager Approved”) and transfer this to the TASKPAYLOAD property in the test case. In this example, an XQuery function is used to achieve this however any other smarter way can be employed too.

image

  • Once the payload has been modified (mimicking the user’s behaviour in the form), the task metadata has to be updated in the dehydration store so that the workflow engine is aware of the changes made. This can be done by passing the modified task data to the updateTask operation in the TaskService. Add another SOAP Request test step to call it as Update Human Task – Approve Employee Expense. Choose updateTask as the operation from the TaskService. soapUI will populate the default request but the service will work with only the credential of the user trying to perform the update and the task itself (which is now available in the TASKPAYLOAD property). Apart from the usual soap based assertions an important assertion to be determined here is the state of the verionReason element in the task. Verify that it is now modified to TASK_VERSION_REASON_UPDATED.

image

  • The final step in moving this task out of the manager’s inbox is to emulate his action, which can either be approve or reject. Task outcomes can be committed to the workflow engines through the updateTaskOutcome operation in the TaskService. Once an outcome has been received by the human task engine it will send a call-back to the business process with the response, which is waiting for it at the human task step. The business process will then proceed ahead, update the expense record table with the status of manager’s decision and evaluate the manager’s outcome at the exclusive gateway. As we are interested in the scenario where the manager approves the expenses the business process will initiate a task for the finance approver.
  • The operation only requires the outcome flag and the task identifier along with the user credential in able to submit the request. There is one important thing to be kept in mind in this test step. Only those users who have been mapped to the swimlane role to which the task is assigned or reassigned to can update the task outcome.

image

  • Create another JDBC Request step to verify that the expense record is updated with the expected value.
  • As the task moves to the Finance Approver’s inbox, the WFTASK table can be again queried to determine the task id corresponding to this task. The following query (similar to the one used before) can be used to retrieve the new task identifier. The only difference in this query is the TASKDEFINITONNAME filter, which basically have to be set with how the finance approval task is called.
SELECT WFTASK.TASKID FROM WFTASK WHERE  WFTASK.TASKDEFINITIONNAME='${#TestSuite#FINANCETASK}' AND WFTASK.ECID = '${#TestCase#ECID}' AND WFTASK.STATE = '${#TestSuite#TASKSTATE}' AND WFTASK.COMPOSITEINSTANCEID= '${#TestCase#INSTANCEID}'
  • At this point, a few of the test steps that were created to fetch the task id from WF task table, copy it to the test case TASKID, retrieve the task payload, modify it and then update the task outcome has to be repeated for the finance approval task too. To maintain visibility, append the name of the human task i.e Administer Expense Payment to each of these steps. After these steps are created the functional test case will reach a point where it has emulated all human steps in the business process, asserted the functional points, and verified that the instance is behaving as expected.

image

  • Once a member of the finance approval team approves the expenses, notice that the business process model updates the payment register as well as the expense record tables. The business process then ends with a message based event viz. expense paid. This triggers an asynchronous call-back to the client calling the business process if it is waiting for one.
  • As a next logical step in the functional testing scenario is to verify that the updates to the end systems has correctly happened and also to somehow receive the process end response to assert if it has the right status. It is also recommended to have some additional test steps to even verify that the instance state is marked as completed in the dehydration store to fully satisfy all functional aspects.
  • To begin with these assertions, add a Mock Response test step in the test case and name it Receive Process Callback Response – expenses paid. Select the expensePaid callback operation from the ApproveEmployeeExpensePortTypeCallBackBinding interface.
  • The port and the path has to be 2222 and ApproveEmployeeExpensePortTypeCallBackBinding respectively. This however depends upon what was entered in the original/initial soap request in the Reply To field. This step will automatically be populated with the callback response containing the approval status of the expense. Add the normal web services assertions to this test step as well an Xpath assertion to verify that the approval status is equal to FINANCE APPROVE

image

  • It is also worth while to add a property transfer step to Capture the Expense Amount total and assign it to the EXPENSETOTAL test case property. This is done by summing all the amounts in the expense list submitted by the employee.

image

  • As indicated earlier, the final steps in the test case is to create a couple of additional JDBC test steps to verify that the updates to the tables are consistent with the instance data by adding the necessary assertions.

image

  • Optionally it can also be verified that the process instance has successfully completed. This can be queried against the dehydration store tables. The CUBE_INSTANCE and the COMPOSITE_INSTANCE table stores the process instance header and the state data. The state column has a numerical value which can be translated into a very informative physical state of the instance. The given query can be executed to retrieve and decode the status from these two tables.
SELECT (CASE
WHEN CU.STATE=0 THEN 'INITIATED'
WHEN CU.STATE=1 THEN 'OPEN AND RUNNING'
WHEN CU.STATE=2 THEN 'OPEN AND SUSPENDED'
WHEN CU.STATE=3 THEN 'OPEN AND FAULTED'
WHEN CU.STATE=4 THEN 'CLOSED AND PENDING'
WHEN CU.STATE=5 THEN 'CLOSED AND COMPLETED'
WHEN CU.STATE=6 THEN 'CLOSED AND FAUTED'
WHEN CU.STATE=7 THEN 'CLOSED AND CANCELLED'
WHEN CU.STATE=8 THEN 'CLOSED AND ABORTED'
WHEN CU.STATE=9 THEN 'CLOSED AND STALE'
WHEN CU.STATE=10 THEN 'NON-RECOVERABLE'
ELSE CU.STATE || 'UNKNOWN'
END) AS CUBE_INSTANCE_STATE, (CASE
WHEN CO.STATE=0 THEN 'RUNNING'
WHEN CO.STATE=1 THEN 'COMPLETED'
WHEN CO.STATE=2 THEN 'RUNNING WITH FAULTS'
WHEN CO.STATE=3 THEN 'COMPLETED WITH FAULTS'
WHEN CO.STATE=4 THEN 'RUNNING WITH RECOVERY REQUIRED'
WHEN CO.STATE=5 THEN 'COMPLETED WITH RECOVERY REQUIRED'
WHEN CO.STATE=6 THEN 'RUNNING WITH FAULTS AND RECOVERY REQUIRED'
WHEN CO.STATE=7 THEN 'RUNNING WITH FAULTS AND RECOVERY REQUIRED'
WHEN CO.STATE=8 THEN 'RUNNING WITH SUSPENDED'
WHEN CO.STATE=9 THEN 'CLOSED WITH SUSPENDED'
WHEN CO.STATE=10 THEN 'RUNNING WITH FAULTS SUSPENDED'
WHEN CO.STATE=11 THEN 'COMPLETED WITH FAULTS SUSPENDED'
WHEN CO.STATE=12 THEN 'RUNNING WITH RECOVERY REQUIRED AND SUSPENDED'
WHEN CO.STATE=13 THEN 'COMPLETED WITH RECOVERY REQUIRED AND SUSPENDED'
WHEN CO.STATE=14 THEN 'RUNNING WITH FAULTS, RECOVERY REQUIRED AND SUSPENDED'
WHEN CO.STATE=15 THEN 'COMPLETED WITH FAULTS, RECOVERY REQUIRED AND SUSPENDED'
WHEN CO.STATE=16 THEN 'RUNNING WITH TERMINATED'
WHEN CO.STATE=17 THEN 'COMPLETED WITH TERMINATED'
WHEN CO.STATE=18 THEN 'RUNNING WITH FAULTS AND TERMINATED'
WHEN CO.STATE=19 THEN 'COMPLETED WITH FAULTS AND TERMINATED'
WHEN CO.STATE=20 THEN 'RUNNING WITH RECOVERY REQUIRED AND TERMINATED'
WHEN CO.STATE=21 THEN 'COMPLETED WITH RECOVERY REQUIRED AND TERMINATED'
WHEN CO.STATE=22 THEN 'RUNNING WITH FAULTS, RECOVERY REQUIRED AND TERMINATED'
WHEN CO.STATE=23 THEN 'COMPLETED WITH FAULTS, RECOVERY REQUIRED AND TERMINATED'
WHEN CO.STATE=24 THEN 'RUNNING WITH SUSPENDED AND TERMINATED'
WHEN CO.STATE=25 THEN 'COMPLETED WITH SUSPENDED AND TERMINATED'
WHEN CO.STATE=26 THEN 'RUNNING WITH FAULTED, SUSPENDED AND TERMINATED'
WHEN CO.STATE=27 THEN 'COMPLETED WITH FAULTED, SUSPENDED AND TERMINATED'
WHEN CO.STATE=28 THEN 'RUNNING WITH RECOVERY REQUIRED, SUSPENDED AND TERMINATED'
WHEN CO.STATE=29 THEN 'COMPLETED WITH RECOVERY REQUIRED, SUSPENDED AND TERMINATED'
WHEN CO.STATE=30 THEN 'RUNNING WITH FAULTED, RECOVERY REQUIRED, SUSPENDED AND TERMINATED'
WHEN CO.STATE=31 THEN 'COMPLETED WITH FAULTED, RECOVERY REQUIRED,SUSPENDED AND TERMINATED'
WHEN CO.STATE IN (32,64) THEN 'UNKNOWN'
ELSE CO.STATE || ''
END) AS COMPOSITE_INSTANCE_STATE FROM CUBE_INSTANCE CU, COMPOSITE_INSTANCE CO WHERE CU.ECID=CO.ECID
AND CU.ECID='${#TestCase#ECID}'
AND CU.CMPST_ID='${#TestCase#INSTANCEID}'
  • As a last step verify that the state in the cube instance table is CLOSED AND COMPLETED and COMPLETED in the composite instance table.

The test case is now complete in a way that it can successfully test all the major functionality of the business process. This increases operational as well as developer efficiency since continuous changes to business processes can be supported without too much worrying about them breaking existing logic. From here it is also possible to create functional test cases to test all possible paths of the business process. These test cases can be ran together as part of an integrated test suite. Creating additional test case is very easy as it mostly involves cloning an existing test case and making some necessary modifications with test data and steps. Once they are created they can be run together as part of the original Expense Approval Functional Tests suite either sequentially or in parallel.

image

Conclusion

soapUI also provides a way to run and extract reports out of these tests through ant/maven and hence it is extremely easy to incorporate them in the release cycle. If you can use soapUI pro, you have the out of the box ability to generate a wide variety of reports for the executed tests too. The basic version can be used to publish JUnit type tests, which I believe are good enough if considering running these tests in a continuous integration fashion.

Business process testing covers how well the system executes the functions it is supposed to execute; including user actions, data manipulation and integrations. It is done to validate the solution as defined in the business requirements and detailed technical specifications document for each processes. It may often contain the highest overall number of test cases, as it is focused on testing the entire system, not just small bits or units of code. Business processes are tested to ensure that the edits, changes, and outputs conform to the required specifications and/or expectations. Business process testing is also often the most complex area of testing as it involves testing processes during development cycles as it occurs and also when processes evolve through multiple changes post deployment. Automating this testing process will provide extreme productivity, quality and add dynamism to business processes. This article can be used as a reference to carve out a strategy to do so. Off course there may be may way but in my experience, I have found out this way to be effective, easy and without any financial obligation to invest in expensive tools.

The composite project, setup scripts, soapUI projects and some sample reports used in this article can be found here. Please feel free to share opinions, feedbacks and suggestions and I will be glad to provide any help.

.