BPM System Architecture

The BPM System Architecture is the workhorse of an organization’s BPM vision to fulfill its strategic BPM needs and objectives. This article aims to provide insights into the BPM System Architecture and how it should be devised and evolved to fully support a seamless, scalable and dynamic platform for process implementation. Most organizations in today’s world would purchase a packaged BPM suite which has abundant features out of the box that support’s the entire BPM lifecycle, rather than an in-house built custom BPM platform. The need for a BPM System Architecture is even more imperative for a packaged BPM suite, since the packaged products must be judiciously aligned within the scope of the enterprise architecture (EA) of an organization. A home grown BPM platform would have a lesser challenge in terms of fitment into enterprise architecture since it would have evolved within the EA scope right from the beginning.

 

The packaged BPM suites would seem to be more exciting and successful when implemented on a small scale like proof of concepts. However when these suites are stretched for a full fledged implementation, its short comings may lead to a colossal failure.  A BPM suite can handle 10 process implementations with ease, but when it comes to hundreds and thousands of processes, a trivial mistake or a minuscule short coming in the BPM suite would proportionately increase the rate of failure.

It is imperative that a BPM System Architecture is built so that a packaged BPM suite can be seamlessly integrated into an Enterprise Architecture. This article will provide insights and glimpses of BPM reference architecture.

Process Data versus Business Data

The process data is the one which supplements the process, but not crucial for the actual completion of the process. KPI, SLA and metrics are few examples of process data. A KPI or SLA might be based on a business data but as such it may not be required for process execution. Business data is the actual driver of the process and would take it to completion. In an Insurance industry the business data would look like, an insurance id, a claim id, insurance expiry date etc. Business data is critical for process execution and the process directly depends on the business data for the process to complete.

The BPM System Architecture should clearly differentiate between the process data and the business data. The business data would most probably reside in an independent repository (database) and would be accessed via an abstraction layer like SOA. The process data would be derived from the business data by the BPM suite and would reside in the BPM suite database. 

The BPM System Architecture must clearly define the business data identifiers that a process should be aware of. For instance, an insurance process must be only aware of insurance id. Anything beyond that would be a nightmare to sync the process and business data. If the process data is defined with insurance id and insurance expiry date, then any changes to the insurance expiry date in the external systems must be synced with the process data, which would be a nightmare if the number of business attributes in the process data grows. Instead the process should only be aware of the insurance id and when the process requires the insurance expiry date, it needs to be pulled up from the external system using the insurance id. In other words, the best practice to define process business attributes is to define it with unique business identifiers. An insurance id would be a unique identifier and all relevant details pertaining to the insurance policy can be pulled from the insurance id.

Process data like KPI and SLA can be derived again from business data. If an SLA needs to be defined on the insurance expiry date from the process, with the insurance id, the insurance expiry date required for SLA computation can be pulled from the external system and SLA can be derived. The SLA data for the insurance expiry date can reside in the process repository.

The BPM System Architecture should define a framework (like an integration component) which would act like a nerve centre to pull business date from the process with the help of unique business key attributes.

Long running process

Failure of process instance cannot be always avoided. But what would be unavoidable, is of course the graceful recovery of the process instance. A process instance can fail for variety of reasons ranging from unavailability of system resources, data issues, network outages and technical glitches. The more prominent of these are the data issues.

A process instance in a telecom infrastructure provider would typically execute for months. If the telecom provider has a process implementation, to lay trans-Atlantic cables from continental United States to Europe, such a process instance can get executed for multiple months. In due course of time, if the process instance fails, the situation may not demand to create the process instance altogether from the beginning of the process and it is more likely it may not be possible at all, since the business data has undergone changes in due course of the process traversal. In such scenarios the process instance no matter what has to be gracefully recovered.

Let us assume that for the process described above, the process instance gets stuck at an activity where the process instance expects the length of the cable between pier A and pier B to be of certain length, whereas the business data for cable length is out of that range for what the process instance expects.  For the process instance to execute, the business data for cable length needs to be updated in the source record. The BPM System Architecture should have a mechanism to refresh the process with latest business data after the update and should recover gracefully.

Also, all long running processes which are vulnerable to failures and are critical to business wherein the re-creation of the instance is not possible, the actual business process must be supplemented with a background recovery process. The background recovery process must be designed in such a way that it must be capable enough to replicate any process instance and can hold the same state as the failed process instance before the failure. The input to the recovery process must be the business key attribute and the activity of the process from where the process instance has to start. This input activity would be the same as the failed process instance step. In order to design such a recovery process, it is absolutely essential to separate the business data and process data. Without this it would be very difficult to create a process instance from a specific activity within the process, since the newly created process instance should reflect the exact state of the failed process instance. This is only possible by associating the business data of the failed process instance to the newly created instance.

BPM Portal

A large scale BPM implementation would require a unified user interface for the business user community. The out of the box portal packaged in the BPM suite may not be flexible enough for extreme customizations. Also, process instance search is a ubiquitous functionality which would act as a first interfacing point for the user.

The BPM System Architecture is responsible for a process instance search framework. The challenge to build such a framework would be to integrate both the process and the business data in one single view. A process instance can be searched with a business attribute as a parameter and also the search can be performed with process data such as SLA.

One possible technical solution is to combine the process data and business data together as Database view, by which expensive database joins can be avoided between multiple tables of the business data schema and process data schema.

BPM Exception Handling


Exception handling is one of the prime aspects of software design which attributes to the robustness of the application. Exception handling gains more significance in BPM space, since exceptions had to be dealt both at a system level and at a business level.

The significance of exception handling at a process modeling step cannot be ignored. There is a need to handle ACID transactions in BPM, with a business solution and it has to be captured in the model with business semantics. For instance, a classic example of flight, hotel and rental car booking when deemed as a single atomic transaction cannot always be dealt at a system level. If flight booking and hotel booking are two different applications in two different domains (if flight carrier and hotel are different companies), then system transactions cannot be propagated and even if it seems to be possible, system level locks for transaction may not be possible to be obtained since the whole transaction may exceed beyond days to complete. The ideal way to deal with this situation is to have three distinct transactions as business process and even if one fails the other transactions must be canceled using a business process. If flight booking has been confirmed and if hotel booking transaction fails, then flight booking should also be canceled, wherein flight cancellation must be modeled as a separate business process.


It becomes imperative for a BPM solution to handle system level exceptions gracefully. Since process instances can span across business days, in case of system exceptions the process state has to be persisted and should be able to recover from the state where it was left. This may not be possible always for each and every process instance. Sometimes there is a good chance that a process instance may get into an irrecoverable state. In such scenarios, it may not be desirable for the users to create a new process instance altogether. The alternatives for irrecoverable instances due to system exceptions are few, and one probable solution could be to create a new process instance and associate the
application process data with the new instance, and programmatically navigate process the instance to the workflow step where it failed.

Another alternative is to create an exception business process. Any system level exception occurring in any of the application business process would trigger the exception business process, which would notify the user about the exception. Also exception process can notify a repair/admin team about the exception. Having a repair queue of exceptions can help the repair/admin team to get first hand information about the exception without the need for notification from the business users about what had happened. Also, irrecoverable process instances can be dealt internally within the IT team, without burdening the business users.

Process Versioning

A process undergoes lot of changes in a BPM lifecycle with multiple iterative implementations. The BPM System Architecture must be potent enough to handle multiple versions of process instances. Most of the modern BPM suites come pre-packaged with utilities to migrate the process instances from an obsolete version to higher versions. However, a process instance in an older version may not be always possible to migrate to a higher version, due to external system dependencies, when the external system itself would have undergone a version change along with the process. This is again in-line with the process data versus business data debate. The battle lines between process data and business data have to be carefully drawn in such way that any data on external system should not be a part of process data. A better way to hold reference to external application data is to hold reference to the unique business key attribute of the external system.

Conclusion

These are the some of the many items that must be considered for a BPM System Architecture. If the segregation between process data and application data can be achieved then having the BPM System Architecture would bring the additional flexibility in terms of solution design.