Message Correlation is not a new concept and has been around in distributed systems for a long time. It becomes very significant in business processes that converse in an asynchronous mechanism.
Correlation means managing the Message Context across the conversations of different partners in the business process.
Correlation is based on the assumption that :
-
The Message Context remains unchanged during the entire lifetime of the conversation.
-
The request and the response share the same Message Context
Correlation can be custom or implicit
-
Custom correlation is based on common information in the message that both partners understand e.g. bookingID, customerID, etc. It can also be a set of multiple attributes (combinations of multiple properties) of the message. This type of a set is known as a Correlation Set.
-
Automatic or out-of-the-box correlation between two partners is based on a unique token called the CorrelationID (or ConversationID), that is passed between the partners in the business process and that is unique to each Message Context. Web Service Addressing by default correlates callback messages based on the conversationID.
Previous releases of Oracle BPM Suite 11g relied on WS-Addressing for correlation of messages sent and received asynchronously.
In WS-Addressing, when a asynchronous service is invoked, the BPM engine injects a unique message ID in the WS-Addressing headers which then float around the with SOAP Packets. These are then used internally by the BPM Engine to correlate the flowing request/response.
Starting with Oracle BPM Suite 11g PS4 Feature Pack there is a mechanism to create Correlation Sets and is a custom way of doing correlation.
In Correlation Sets, one can correlate asynchronous messages based on message body contents. The Developer has total control and responsibility to create a unique correlation set.
Now since both mechanisms are available the question is – when to use which.
Custom Correlation should used when
-
The asynchronous service invoked by BPM does not support or understand WS-Addressing headers. Typically legacy systems.
-
BPM instance is expecting messages from a third party, for example, a file or database poller.
-
The asynchronous call has multiple hops and reply is received directly from end system bypassing any intermediaries.
Before the release of BPM 11g PS4 Feature pack custom correlation in a BPM component could have been achieved by introducing a BPEL component that has support for custom correlation either before or after a BPM.
Scenario 1 : BPEL “in front” of BPM
In this case BPEL forwards all messages to BPM as a proxy for the client.
Benefits of BPEL Fronting BPMN
- Not visible to the BPM design (e.g. non-intrusive)
Limitations
- Must be able to determine start/end events from existing messages
- This approach will not work when there is initiator Human Task (BPEL can’t see start)
Scenario 1 : BPEL “behind” BPM
This approach is best used when BPM process includes initiator Human Task Activity. The BPM process throws an event to initialize BPEL and throws an event to stop BPEL that in turn delivers correlated events as callback messages.
Benefits of BPM Fronting BPEL
Single point for initializing BPEL process
Limitations
Interaction(s) with BPEL are visible in the BPMN model
With the release of Oracle SOA Suite 11g PS4 Feature Pack we can create correlations within our BPM processes and hence overcome the limitations with either of the two approached above.
This blogpost will demonstrate through the use of a simple example as how we can do it.
Prerequisites
- JDeveloper 11g PS4
- PS4 FP SOA and BPM Extensions for JDeveloper
- Oracle BPM Suite 11g PS4 Feature Pack
More information about downloading it can be found here
The Use Case
ABC Car Rental Company has a business process defined to help customers plan and book their car travels online.
The BPM Process has a front ending User Interface that does most of the interaction with the user viz getting his details, travel itinerary, booking preferences etc.
ABC company has a CabBookingProcess BPM Process that provides the back end business logic execution to the UI.
Customers can also choose to cancel their bookings in which case the cancellation request is correlated with his booking request and booking is cancelled subject to cancellation rules. [As we will see this is where Custom Correlation kicks in].
In case of any exceptions during the booking a BookingClerk is notified with the booking details and exception and may choose to process the booking offline.
CabBookingProcess BPMN Execution Model
Take a look at the below process to get more detailed idea about the overall process.
The process is initiated with a message start event and checks for availability of a Cab based on customer input.
Custom Correlation Case 1
If you take a careful look at the happy path of Get Booking Availability sub process (highlighted below), it has a Signal Throw End that throws a booking confirmation detail signal.
Correspondingly it also has a Signal Catch event to catch the booking confirmation response. However the response that is received has to be correlated with the original instance.
Custom Correlation is required in this case as a Signal Event is a one way publish and doesn’t have any asynchronous callback port.
Defining Custom Correlation in BPM is now a piece of cake.
Double Click on the “Send Booking Confirmation” throw signal event and open the Implementation tab. Now click on the Correlations link. Create a Correlation Key and a Property Alias(es).
The Property aliases are used to create the message context for correlation. This context should be unique so as to determine each instance uniquely. The message context is fetched from the input variables to this event.
Also since the correlation is being initiated here the Mode is set to Initiated.
Similarly double click on the Get Booking Confirmation Response catch signal event and click on the Implementation tab. Use the same Correlation Key and Property Alias defined above. The value assigned to the alias should be same as the original message context.
For example let say we use the value of Employee Number to define the unique correlation context. However it can be referred to as EmpNo in one system where as EmployeeNum in another.
The correlation is not initiated in this activity but used here. Hence the Mode is set to Uses.
Custom Correlation Case 2
Another place in this process where we would require the use of Custom Correlation is when the Customer places a request to cancel his booking.
When a cancel booking request is received the first thing that is required is to pick the instance that needs to be cancelled. Out of the “n” running instances how would the engine determine the one to be cancelled.
Correlation is defined in a similar way as in the previous case.
The Booking Request Message Start Event initiates a correlation. A correlation key is defined based on the phone number of the customer requesting the booking.
The BPM model has another Event Sub Process that starts with a Cancel Message. The event message also uses the same correlation key to define a correlation. The correlation Mode in this cases is “Uses” as it would use the existing correlation key to correlate with the instance that needs to be cancelled.
The entire BPM Project and Composite workspace can be downloaded from here.
Deploy the project to a running bpm server and test the composite application.
Running the Process
Running the process with a simple booking request to initiate Cab Booking. The instance moves to a running state. It is has sent a booking confirmation and is awaiting a response callback.
The process flow trail shows the process path with a Green highlight.
Now while the process is awaiting a booking confirmation back if we Cancel the in-flight process using some input parameters from the original request, we will see that the original instance is terminated by this mid process cancel event message.
We can see that original instance terminated after the cancel operation was invoked. See the Audit and Flow trail below for a more better understanding of what happened.
What if we use Improper Correlation Keys
Lets run the cab booking process once again and then cancel the booking midway. Only this time we would use incorrect correlation keys i.e customerPhone number of the booking and cancel request aren’t matching.
The booking instance is created and is in Running state. Now let us try to invoke the cancel operation. But this time we dont use the same correlation key.
We would see that there are now two running instances in the composite dashboard that are unable to correlate with each other. The audit trail of the Cancel booking operation just shows the invoked service state.
I hope with this tutorial correlation in Oracle BPM Suite 11g will not be scary beast anymore. There is once catch in this example however.
The example uses customer phone number as the correlation key. What if the same person has multiple bookings? How do we correlate instances in this case.
We would need to have a correlation key based on multiple property liases in this case. For example, the correlation key can have three aliases that makes it unique viz Customer Phone Number, booking Date and booking time. Feel free to try this out yourself.
The entire JDeveloper workspace can be downloaded from here.
.
Hi arun,
The process example uploaded in the post is not opening in the JDeveloper 11.1.1.4.0 and raising exception when i was trying to open the process in the JDeveloper. The exception is :Oracle.bpm.project.model.exception.ProjectException
oracle.bpm.project.io.serializers.model.bpmn.BPMN20Loader load
SEVERE: unexpected element (uri:”http://www.omg.org/spec/BPMN/20100524/MODEL”, local:”definitions”). Expected elements are ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
javax.xml.bind.UnmarshalException: unexpected element (uri:”http://www.omg.org/spec/BPMN/20100524/MODEL”, local:”definitions”). Expected elements are ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallingContext.handleEvent(UnmarshallingContext.java:642)
LikeLike
Hi,
The example project cannot be opened with JDeveloper 11.1.1.4.o. You need to have JDeveloper 11.1.1.5.0 (and then install the bpm and soa extensions available with the PS4 feature pack.) You can read more about Oracle BPM Suite 11g PS4 Feature pack in here
http://www.oracle.com/technetwork/middleware/bpm/learnmore/whatsnewbpmfeaturepack-487968.html
LikeLike
Ok arun. Thanks for the reply
LikeLike
Good post. Helpful.
Clear on Correlation in BPM but I have this general question :
I have heard about Signal End event ,Intermediate signal event which throws the Signal and Signal Start Event to Recieve the broadcasted Signal , but can you give me pointers to the Signal Catch Event.
LikeLike
Hi Sridhar,
The Signal Catch Event is exactly same as Signal Start event. The only difference is that a Signal Catch event is used to subscribe to events in between a running process while a signal Start event is used to initialize a process.
For example if you have a Signal Catch Event in your mid process and the flow hits this event, the token will wait for any broadcasted event (that can be published by another BPEL or BPMN process) and continue processing.
LikeLike
Thanks. This is really good blog on Correlation in BPM , However I guess FP4 is only for Customers of Oracle. So I am not able to make this work in my local system.
LikeLike
Yes thats true. You need to have an Oracle Support Identifier to be able to download the Patch. However it will probably be available soon to download from OTN,
LikeLike
Pingback: Correlation in BPM | RedStack
Pingback: BPM技術の効果的学習法のすすめ « 岩田研究所
hi, in installed jdev Studio Edition Version 11.1.1.5.0, Build JDEVADF_11.1.1.5.0_GENERIC_110409.0025.6013. When opening the process i run into the error:
Oracle.bpm.project.model.exception.ProjectException
oracle.bpm.project.io.serializers.model.bpmn.BPMN20Loader load
SEVERE: unexpected element (uri:”http://www.omg.org/spec/BPMN/20100524/MODEL”, local:”definitions”). Expected elements are ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
javax.xml.bind.UnmarshalException: unexpected element (uri:”http://www.omg.org/spec/BPMN/20100524/MODEL”, local:”definitions”). Expected elements are
What version of 11.1.1.5.0 do i need?
LikeLike
Hi Aldo,
You need to Have Oracle BPM Suite 11g PS4 FP i.e Feature Pack installed to get this application working.
You can find more information about this at
http://www.oracle.com/technetwork/middleware/bpm/learnmore/whatsnewbpmfeaturepack-487968.html
To get the Feature Pack installed please refer to the below blogpost
http://niallcblogs.blogspot.com/2011/10/installing-bpm11g-feature-pack-on.html
LikeLike
Pingback: Using Intermediate Events and Advanced TaskService Interactions in Oracle BPM Suite 11g « Oracle Technologies Premier
Hi Arun…. I think this post is pretty usefull. My question is: Correlations can be handled in Oracle BPM Suite 11g but without PS4 FP?? I mean is there another way to manage process correlations in an non PS4 FP enviroment ??
Thanks in advance
LikeLike
Hi Isabel,
You can use Correlation in Oracle BPM Suite 11g versions earlier than PS4FP with a slight workaround. This has been blogged in details here
http://blog.andrade.inf.br/2011/01/implementing-correlation-in-bpm-11gr1.html
Hope that helps
LikeLike
Thanks so much, Arun… but that link does not open, at least in my office. I don’t know the reason. Could you redirect me to another documentation?
I’ll be forever gratefull.
Regards.
LikeLike
Please go through my email where I have tried to address your concern.
Thanks
LikeLike
Hello
Very nice article – I got src of this sample but i dont see database code. Could You share with sql with database tabe and simple data to full replicate your sample on my local laptop.
Regards Kuba
LikeLike
Hi,
Thanks for the appreciation but i dont reckon there was any Database code that is required for this exercise. Can you tell me in details as what is exactly that you struggling with.
Cheers
Arun
LikeLike
Hello
In your bpmn example you are using service call to bpel for example – saveOrderDetail. In this bpel service you have database adapter (SaveBookDetails) – so could you share sql script for this database to replocate this sample. We alsow have AvailabilityService.wsdl whit point to custom IP address – do Yo have any mock (soapui) for this endpoint. – This two mising endpoint is configured in composit.xml.
Regards Kuba
LikeLike
Hi,
The way I execute this example is by making both the Service Activities that invokes the WSDL and SaveBookDetails as draft activities. If you mark them as draft, they are not executed.
Thanks
Arun
LikeLike
Hi Arun
In the example you create two Event Subprocess . One catch a specific exception and the other catch all business exceptions. My question is when this happens the specific exception is not captured by the two Event Subprocess. In the example specifically in scenario 1 exception should be captured by the two.
Ths a lot
LikeLike
hi i am frank.
thanks for this presentation. i would like toa s if it is possible to have the source code of this application.
i’m suppose to do a complete travel booking process using Oracle Bpm , so i think this presentation can help me.
thk s
LikeLike
Arun, I have a question regarding the cancel booking flow. If the cancel operation is initiated before the actual booking with key to be used, the cancel operations waits until you receive the booking and immediately cancels it. Is there a way to prevent this from happening?
LikeLike
How can we keep correlation active even if the instances are completed/suspended/aborted so that duplicate instances are not created for same correlation Id??
LikeLike
Hi Hemant,
As far as I know, this is not possible. Correlation is against in-flight instances and not against completed ones but I am only 99.99 percent sure. I will try to check something and eliminate the doubt that I have and update you here.
Thanks
LikeLike
Very Useful posts in your blog… covering almost all topics in 11g…Within short span of time I understand many things with your clear examples….. Thanks for your time for blogging…Keep posting.. 🙂
LikeLike
Pingback: Correlation in BPM
Pingback: BPM: Using Intermediate Events and Advanced TaskService Interactions in Oracle BPM Suite 11g – Chien Nguyen's BPM Blog