Friday, November 10, 2017

Building Business Applications with DMN and BPMN

A couple weeks ago our own Matteo Mortari delivered a joint presentation and live demo with Denis Gagné from Trisotech at the BPM.com virtual event.

During the presentation, Matteo live demo'd a BPMN process and a couple DMN decision models created using the Trisotech tooling and exported to Red Hat BPM Suite for seamless execution.

Please note that no glue code was necessary for this demo. The BPMN process and the DMN models are natively executed in the platform, no Java knowledge needed.

Enough talking, hit play to watch the presentation... :)



Share/Bookmark

Thursday, October 12, 2017

5 Pillars of a Successful Java Web Application

Last week, Alex Porcelli and I had the opportunity to present at JavaOne San Francisco 2017 two talks related to our work: "5 Pillars of a Successful Java Web Application” and The Hidden Secret of Java Open Source Projects.

It was great to share our cumulative experience over the years building the workbench and the web tooling for the Drools and jBPM platform and both talks had great attendance (250+ people in the room).


In this series of posts, we’ll detail our "5 Pillars of a Successful Java Web Application”, trying to give you an overview of our research and also a taste of participating in a great event like Java One.
There are a lot of challenges related to building and architecting a web application, especially if you want to keep your codebase updated with modern techniques without throwing away a lot of your code every two years in favor of the latest trendy JS framework.
In our team we are able to successfully keep a 7+ year old Java application up-to-date, combining modern techniques with a legacy codebase of more than 1 million LOC, with an agile, sustainable, and evolutionary web approach.
More than just choosing and applying any web framework as the foundation of our web application, we based our web application architecture on 5 architectural pillars that proved crucial for our platform’s success. Let's talk about them:

1st Pillar: Large Scale Applications

The first pillar is that every web application architecture should be concerned about the potential of becoming a long-lived and mission-critical application, or in other words, a large-scale application. Even if your web application is not exactly big like ours (1mi+ lines of web code, 150 sub-projects, +7 years old) you should be concerned about the possibility that your small web app will become a big and important codebase for your business. What if your startup becomes an overnight success? What if your enterprise application needs to integrate with several external systems?
Every web application should be built as a large-scale application because it is part of a distributed system and it is hard to anticipate what will happen to your application and company in two to five years.
And for us, a critical tool for building these kinds of distributed and large-scale applications throughout the years has been static typing.

Static Typing

The debate of static vs. dynamic typing is very controversial. People who advocate in favor of dynamic typing usually argue that it makes the developer's job easier. This is true for certain problems.
However, static typing and a strong type system, among other advantages, simplify identifying errors that can generate failures in production and, especially for large-scale systems, make refactoring more effective.
Every application demands constant refactoring and cleaning. It’s a natural need. For large-scale ones, with codebases spread across multiple modules/projects, this task is even more complex. The confidence when refactoring is related to two factors: test coverage and the tooling that only a static type system is able to provide.
For instance, we need a static type system in order to find all usages of a method, in order to extract classes, and most importantly to figure out at compile time if we accidentally broke something.
But we are in web development and JavaScript is the language of the web. How can we have static typing in order to refactor effectively in the browser?

Using a transpiler

A transpiler is a type of compiler that takes the source code of a program written in one programming language as its input and produces equivalent source code in another programming language.
This is a well-known Computer Science problem and there are a lot of transpilers that output JavaScript. In a sense, JavaScript is the assembly of the web: the common ground across all the web ecosystems. We, as engineers, need to figure out what is the best approach to deal with JavaScript’s dynamic nature.
A Java transpiler, for instance, takes the Java code and transpiles it to JavaScript at compile time. So we have all the advantages of a statically-typed language, and its tooling, targeting the browser.

Java-to-JavaScript Transpilation

The transpiler that we use in our architecture, is GWT. This choice is a bit controversial, especially because the GWT framework was launched in 2006, when the web was a very different place.
But keep in mind that every piece of technology has its own good parts and bad parts. For sure there are some bad parts in GWT (like the Swing Style Widgets, multiple permutations per browser/language), but keep in mind that for our architecture what we are trying to achieve is static typing on the web, and for this purpose the GWT compiler is amazing.
Our group is part of GWT steering committee, and the next generation of GWT is all about JUST these good parts. Basically removing or decoupling the early 2000 legacy and keeping only the good parts. In our opinion the best parts of GWT are:
  • Java to JavaScript transpiler: extreme JavaScript performance due to compiling optimizations and static typing in the web;
  • java.* emulation: excellent emulation of the main java libraries, providing runtime behavior/consistency;
  • JS Interop: almost transparent interoperability between Java <-> Javascript. This is a key aspect of the next generation of GWT and the Drools/jBPM platform: embrace and interop (two way) with JS ecosystem.

Google is currently working on a new transpiler called J2CL (short for Java-to-Closure, using the Google Closure Compiler) that will be the compiler used in GWT 3, the next major GWT release. The J2CL transpiler has a different architecture and scope, allowing it to overcome many of the disadvantages of the previous GWT 2 compiler.

Whereas the GWT 2 compiler must load the entire AST of all sources (including dependencies), J2CL is not a monolithic compiler. Much like javac, it is able to individually compile source files, using class files to resolve external dependencies, leaving greater potential for incremental compilation.
These three good parts are great and in our opinion, you should really consider using GWT as a transpiler in your web applications. But keep in mind that the most important point here is that GWT is just our first pillar implementation. You can consider using other transpilers like Typescript, Dart, Elm, ScalaJS, PureScript, or TeaVM.
The key point is that every web application should be handled as a large-scale application, and every large-scale application should be concerned about effective refactoring. The best way to achieve this is using statically-typed languages.
This is the first of three posts about our 5 pillars of successful web applications. Stay tuned for the next ones.

[I would like to thank Max Barkley and Alexandre Porcelli for kindly reviewing this article before publication, contribute with the final text and provided great feedback.]



Share/Bookmark

Wednesday, September 20, 2017

Watch all the sessions from Red Hat Drools Day LIVE from your desktop or mobile, Sept 26th

We will be streaming all the sessions of the Drools Day in NYC, on Sep 26th, live!

Use the following link to watch:

https://www.facebook.com/RedHatDeveloperProgram/videos/1421743381254509/

Or watch it here:




Share/Bookmark

Wednesday, September 13, 2017

Drools, jBPM and Optaplanner Day: September 26 / 28, 2017 (NY / Washington)

Red Hat is organizing a Drools, jBPM and Optaplanner Day in New York and Washington DC later this year to show how business experts and citizen developers can use business processes, decisions and other models to develop modern business applications.
This free full day event will focus on some key aspects and several of the community experts will be there to showcase some of the more recent enhancements, for example:
  • Using the DMN standard (Decision Model and Notation) to define and execute decisions
  • Moving from traditional business processes to more flexible and dynamic case management
  • The rise of cloud for modeling, execution and monitoring
IT executives, architects, software developers, and business analysts who want to learn about the latest open source, low-code application development technologies.

Detailed agenda and list of speakers can be found on each of the event pages.

Places are limited, so make sure to register asap !

Share/Bookmark

Wednesday, September 06, 2017

Is Optimization AI or OR?

With the renewed interest in AI the same conversations are starting to come up again, about what is or isn't AI.  My recent discussion was on whether optimisation products, such as OptaPlanner, are considered AI as some considered it more Operations Research (OR). For some background, OptaPlanner started out as a Tabu Solver implementation, but has since added other techniques like Simulated Annealing.

Although I'd like to add that no single technique is AI, they are all tools and techniques that are quite typically used together in a blended, hybrid or integrated AI solution. So it's the right tool or tools for the job.

The answer is that Optimisation is both an AI and an OR problem. It is a technique used and researched by both groups,  the two different disciplines tend to take different approaches to the problem, having differing use cases and have historically used different techniques, with a lot of cross pollination from both sides.

I'll start with a consumer oriented answer to the question. StaffJoy has a nice blog article on the overlap of OR and AI, and I'll quote from that below:
https://blog.staffjoy.com/how-operations-research-and-artificial-intelligence-overlap-b128a3efee2e
"Startups are using OR techniques in products like OnFleet, Instacart, and Lyft Line. However, when similar techniques are being exposed externally as services, they are often described as AI — e.g. x.ai, Atomwise, and Sentient. Very few companies describe algorithms that they sell as optimization (with the exception of SigOpt) because the end goal of customers is automating decisions. With StaffJoy, we have found that customers better understand our product when we describe it as an “artificial intelligence” tool rather than an “optimization” or “operations” tool. We think that this is because customers care more about what a product achieves, rather than the means it uses to achieve it."

In short consumers do not see the difference between OR and AI, when applied to real world problems and it is commonly marketed as AI.

I'll go a little more technical now, to further demonstrate it's more than just marketing - as that side is only touched on in the above blog post.

While the two groups (OR and AI) may have once been distinct, it's been well established that the OR and AI groups overlap in this space and have collaborated for years. Glover (1986) states them as "the recent remarriage of two disciplines that were once united, having issued from a common origin, but which became separated" - see final paper link at end.

A cursory google with terms "operations research" and "artificial intelligence" will more than prove this. Some techniques, like Linear Programming, are strongly on the OR side, others like Local Search (which OptaPlanner falls under) are shared. Optimisation, and local search (along with other techniques), is a core fundamental taught in every AI course without fail, and will be covered in every general AI book, used in schools - such as "AI:  A Modern Approach"- see chapter 4, page 120
http://aima.cs.berkeley.edu/contents.html

The book "Artificial Intelligence Methods and Applications" also makes it clear the two (OR and AI) are linked:
"Local search, or local optimisation, is one of the primitive forms of continuous optimisation in a discrete problem space. It was one of the early techniques proposed during the mid sixties to cope with the overwhelming computational intractability of NP-hard combinatorial optimisation problems. Unlike continuous optimisation techniques, local search has often been used in AI research and has established a strong link between AI and the operational research area."
https://books.google.co.uk/books?id=0a_j0R0qh1EC&pg=PA67&lpg=PA67&dq=%22local+search%22+operations+research+artificial+intelligence&source=bl&ots=h2jGquBm4d&sig=2V7CKRIs3ZL-NKzqL3Dnkx33_NI&hl=en&sa=X&redir_esc=y#v=onepage&q=%22local%20search%22%20operations%20research%20artificial%20intelligence&f=false

Lastly I'll quote directly from the original Tabu Solver paper "These developments may be usefully viewed as a synthesis of the perspectives of operations research and artificial intelligence... ... The foundation for this prediction derives, perhaps surprisingly, from the recent remarriage of two disciplines that were once united, having issued from a common origin, but which became separated and maintained only loose ties for several decades: operations research and artificial intelligence. This renewed reunion is highlighting limitations in the frameworks of each (as commonly applied, in contrast to advocated), and promises fertile elaborations to the strategies each has believed fruitful or approaching combinatorial complexity." Glover (1986). Note the paper is from "The Center of Applied Artificial Intelligence". 
http://leeds-faculty.colorado.edu/glover/TS%20-%20Future%20Paths%20for%20Integer%20Programming.pdf

So I hope that clears that up - AI is a very broad church :)



Share/Bookmark

Friday, August 18, 2017

Capture your decisions with DMN

Donato Marrazzo published a very nice and comprehensive introduction to DMN with Drools and Trisotech.

It is worth a read for anyone interested in DMN:

Capture your decisions with DMN

Happy Drooling!


Share/Bookmark

Wednesday, August 09, 2017

Talking about Rule Engines at Software Engineering Radio

I had the pleasure of talking to Robert Blumen, at Software Engineering Radio, about Drools and Rule Engines in general.

http://www.se-radio.net/2017/08/se-radio-episode-299-edson-tirelli-on-rules-engines/

If you don't know this podcast, I highly recommend their previous episodes as well. Very informative, technically oriented podcast.

Hope you enjoy,
Edson
Share/Bookmark

Tuesday, August 08, 2017

Drools Canonical Model - Pure Java Rules

Rule engines, like Drools, typically  make use of a custom languages to define a set of rules. For example, he Drools compiler translates a drl file to an internal representation (the KiePackages) that is subsequently used to generate the ReteOO/Phreak network that will perform the rules evaluation.

This internal representation was never really intended to be generated or consumed by end users. This complication makes it difficult to write rules programmatically and the suggestion is instead to generate text rules at the DRL level.  This means that drl itself is currently the only practical formal notation to define a set of rules in Drools.

Drools internals were developed with several assumptions at the time, that are no longer true or desirable. Prior to Java 8, perm gen was a concern, so various solutions were utilized to address this - such as MVEL for reflective based evaluation. Java 8 now puts code on the heap, so this is no longer necessary.  A the engine level it also inspected and produced indexes, which tied it to the expressions produced by DRL - this makes polyglot impractical. Lastly it liberally uses classloaders and reflection, which makes it difficult to transpiler for execution on different environments.

An engine independent rule model


To overcome this limitation and offer the possibility of programmatically define a set of rule in pure Java we developed a model aimed to provide a canonical representation of a rule set and a fluent DSL to conveniently create an instance of this model. The model itself is totally independent from Drools and could in theory be re-used by other engines, It also introduces layers that fully separates the engine from having to be aware of any language. For example it will not inspect and generate indexes, instead it expects to be provided those indexes, from the layers above. Another advantage is it now means Drools has a developer friendly view of what it's executing, as it's all just pure low-level pojo rules.

This model, other than giving to all Java developers a clean way to write rules in plain Java, will also enable our team to experiment with new features faster, freeing us from the burden of also implementing the corresponding parser and compiler parts that integrate the new feature with the drl notation.
Let's see a practical example of the model of a rule defined using the before mentioned DSL:
 
Rule rule = rule( "Persons older than Mark" )
    .view(
        expr("exprA", markV, p -> p.getName().equals("Mark"))
            .indexedBy( String.class, ConstraintType.EQUAL, Person::getName, "Mark" )
            .reactOn( "name", "age" ), 
        expr("exprB", olderV, p -> !p.getName().equals("Mark"))
            .indexedBy( String.class, ConstraintType.NOT_EQUAL, Person::getName, "Mark" )
            .reactOn( "name" ),
        expr("exprC", olderV, markV, (p1, p2) -> p1.getAge() > p2.getAge())
            .indexedBy( int.class, ConstraintType.GREATER_THAN, Person::getAge, Person::getAge )
            .reactOn( "age" )
    )
    .then(on(olderV, markV)
         .execute((p1, p2) -> System.out.println( p1.getName() + " is older than " + p2.getName())));


This is the equivalent of the following rule expressed in drl:
 
rule "Persons older than Mark" when
    $p1 : Person(name == \"Mark\")
    $p2 : Person(name != \"Mark\", age > $p1.age)
then
    System.out.println($p2.getName() + \" is older than \" + $p1.getName());
end

It is evident that the DSL version is much more verbose and requires to specify many more details that conversely are inferred by the drools compiler when it parses the rule written in drl.
As anticipated this has been done on purpose because the model has to explicitly contain all the information required to generate the section of ReteOO/Phreak network intended to evaluate that rule, without the need of performing any bytecode inspection or using any other complex, brittle and non-portable introspection technique. In those simple examples the index contains the same logic as the expr, as per DRL expressions can contain more than just the index. The index would be inferred and implicitly added by the upper language layers.
To clarify this aspect let's analyze a single LHS constraint in a bit more detail:

expr("exprA",                                                                 [1] 
     markV,                                                                   [2]
     p -> p.getName().equals("Mark") )                                        [3]  
    .indexedBy( String.class, ConstraintType.EQUAL, Person::getName, "Mark" ) [4]
    .reactOn( "name", "age" )                                                 [5]


In this statement you can notice the following parts
[1] This is a label for the constraint used to determine its identity. Two identical constraint must have the same label and two different ones must have different labels in order to let the engine properly implement the node sharing where possible. This label can be optionally omitted and in this case it will be generated from an automatic introspection of the bytecode of the lambda expression implementing the constraint. However, as anticipated, it's preferable to avoid any introspection mechanism and then to explicitly provide a constraint label whenever is possible.
[2] This is the Variable defined before creating the rule and used to bind an actual fact with the formal parameter of the lambda expression used to evaluate a condition on it. It is the equivalent of the variable declaration in rule written with the drl notation.
[3] The lambda expression performing the constraint evaluation.
[4] The specification of the type of this constraint and how it has to be indexed by the engine.
[5] The name of the properties of the Person object for which this constraint has to be reactive.

Building and executing the model


You can programmatically add this and other rules of your rule base to a model:

Model model = new ModelImpl().addRule( rule );


Once you have this model, which again is totally independent from Drools algorithms and data structures, you can use a second project, this time dependent both on Drools and on the model itself, to create a KieBase out of it.

KieBase kieBase = KieBaseBuilder.createKieBaseFromModel( model );


What this builder internally does is recreating the KiePackages out of the model and then building them using the existing drools compiler. The resulting KieBase is identical to the one you may obtain by compiling the corresponding drl and then you can use it exactly in the same way, creating a KieSession out of it and populating it with the facts of your domain. Here you can find more test cases showing the Drools features currently supported by the model and the DSL constructs allowing to use them, while here you can find a more complete example.

Embedding the executable model inside the kjar


At the moment a kjar contains some pregenerated classes implementing the constraints and the consequences. However all of the drl files need to be parsed and compiled from scratch, making limited savings on building it from source only.The executable model will be also useful to speed this process up. Embedding the classes implementing the model of the rule base inside the kjar, makes possible to quickly recreate the KiePackages, instead of having to restart the whole compilation process from the drl files. A proof concept of how this could work is available here.

DRL compiler to Executable Model


We are working on a new compiler that will take existing DRL and directly output the executable model, which is stored in the kjar. This will result in much faster loading and building times. We are also looking into other ways to speed this process up, by pre-caching more information in the kjar, which can be determined during the initial kjar build.

Pojo, Polyglot and DRLX Rules


The executable model is low level and because you have to provide all the information it needs, this makes it a little verbose. This is done by design, to support innovation in the language level. While it is technically still pojo-rules, it is not desirable for everyday use. In the near future we will work on a less verbose pojo rules layer, that will use an annotation processor to inject things like indexes and reactOn. We also hope that it will result in several rule languages, not just one - scala rules, closure rules, mini-dsl rules etc. Nor is it necessary for a language to expose all of the engine capabilities, people can write mini-dsls exposing the subset.

Finally we are also working on a next generation DRL language (DRLX), that will be a superset of Java. This is still in the design phases and will not be available for some time, but we will publish some of the proposals for this, once an early draft spec is ready.



Share/Bookmark

Wednesday, August 02, 2017

Drools, jBPM and Optaplanner Day: September 26 / 28, 2017 (NY / Washington)

Red Hat is organizing a Drools, jBPM and Optaplanner Day in New York and Washington DC later this year to show how business experts and citizen developers can use business processes, decisions and other models to develop modern business applications.
This free full day event will focus on some key aspects and several of the community experts will be there to showcase some of the more recent enhancements, for example:
  • Using the DMN standard (Decision Model and Notation) to define and execute decisions
  • Moving from traditional business processes to more flexible and dynamic case management
  • The rise of cloud for modeling, execution and monitoring
IT executives, architects, software developers, and business analysts who want to learn about the latest open source, low-code application development technologies.

Detailed agenda and list of speakers can be found on each of the event pages.

Places are limited, so make sure to register asap !
Share/Bookmark

Monday, July 03, 2017

Drools, jBPM and Optaplanner are switching to agile delivery!

Today we would like to give everyone in the community a heads up at some upcoming changes that we believe will be extremely beneficial to the community as a whole.

The release of Drools, jBPM and Optaplanner version 7.0 a few weeks ago brought more than just a new major release of these projects.

About a year ago, the core team and Red Hat started investing on improving a number of processes related to the development of the projects. One of the goals was to move from an upfront planning, waterfall-like development process into a more iterative agile development.

The desire to deliver features earlier and more often to the community, as well as to better adapt to devops-managed cloud environments, required changes from the ground up. From how the team manages branches to how it automates builds and how it delivers releases. A challenge for any development team, but even more so to a team that is essentially remote with developers spread all over the world.

Historically, Drools, jBPM and Optaplanner aimed for a cadence of 2 releases per year. Some versions with a larger scope took a bit longer, some were a bit faster, but on average that was the norm.

With version 7.0 we started a new phase in the project. We are now working with 2-week sprints, and with an overall goal of releasing one minor version every 2 sprints. That is correct, one minor version per month on average.

We are currently in a transition phase, but we intend to release version 7.1 at the end of the next sprint (~6 weeks after 7.0), and then we are aiming to release a new version every ~4 weeks after that.

Reducing the release timeframe brings a number of advantages, including:
  • More frequent releases gives the community earlier access to new features, allowing users to try them and provide valuable feedback to the core team. 
  • Reducing the scope of each release allows us to do more predictable releases and to improve our testing coverage, maintaining a more stable release stream.
  • Bug fixes as usual are included in each release, allowing users more frequent access to them as well. 
It is important to note that we will continue to maintain backward compatibility between minor releases (as much as possible - this is even more important in the context of managed cloud deployments as well where seamless upgrades are the norm) and the scope of features is expected to remain similar to what was before. That has two implications:
  • If before, we would release version 7.1 around ~6 months after 7.0, we now will release roughly 6 new versions in those 6 months (7.1, 7.2, ..., 7.6 ), but the amount of feature will be relatively equivalent. I.e., the old version 7.1 is roughly equivalent in terms of features as the scope of the new versions 7.1,..., 7.6 combined. It just splits the scope in smaller chunks and delivers earlier and more often.
  • Users that prefer to not update so often will not lose anything. For instance, a user that updated every 6 months can continue to do so, but instead of jumping from one minor version to the next, he will jump 5-6 minor versions. This is not a problem, again, because the scope is roughly the same as before and the backward compatibility between versions is the same.
This is of course work in progress and we will continue to evolve and adapt the process to better fit the community's and user's needs. We strongly believe, though, that this is a huge step forward and a milestone on the project maturity level.

Share/Bookmark

Monday, June 26, 2017

RuleML+RR with DecisionCamp - July 12-14 201, London

RuleMLWeb Rules and Reasoning as well as DecisionCamp are all coming together (as well as being collocated with BICOD) this year in London.
RuleML+RR home, schedule, registration
DecisionCamp home, schedule, registration

Explore that latest AI happenings at RuleML+RR and keep up to date with the latest Decision Model and Notation (DMN) at Decision Camp.

When: July 12-14 2017
Where: Birkbeck, University of London, London, UK
Malet St, London WC1E 7HX, UK

A number of Red Hat Engineers will be there and presenting:
Mark Proctor - Drools co-founder, BRMS and BPMS Platform Architect:
Edson Tirelli - Drools project lead: DMN Technology Compatibility Kit (TCK), Demystifying the Decision Model and Notation Specification
Geoffrey De Smet - OptaPlanner founder, project lead: Real-time Constraint Solving with OptaPlanner

DecisionCamp:
"DecisionCAMP-2017 will include presentations from leading decision management authorities, vendors, and practitioners. The event will explore the current state in Decision Management, the real-world use of the DMN standard, and solutions to various business problems using Decision Management  tools and capabilities. The event will include a special Open Discussion “What you Like and What you Do Not Like about DMN” and a QnA Panel “Real-world Business Decision Management: Vendor and Practitioner Perspectives”.

RuleML+RR:
"2017 is the leading international joint conference in the field of rule-based reasoning, and focuses on theoretical advances, novel technologies, as well as innovative applications concerning knowledge representation and reasoning with rules."

Keynotes and Speeches:
Bob Kowalski (Imperial College London): Logic and AI – The Last 50 Years
Stephen Muggleton (Imperial College London): Meta-Interpretive Learning: Achievements and Challenges
Jordi Cabot (IN3-UOC, Barcelona): The Secret Life of Rules in Software Engineering (sponsored by EurAI)
Jean-Francois Puget (IBM): Machine Learning and Decision Optimization
Elena Baralis (Politecnico di Torino): Opening the Black Box: Deriving Rules from Data



Share/Bookmark

Monday, May 29, 2017

New KIE persistence API on 7.0

This post introduce the upcoming drools and jBPM persistence api. The motivation for creating a persistence api that is to not be bound to JPA, as persistence in Drools and jBPM was until the 7.0.0 release is to allow a clean integration of alternative persistence mechanisms to JPA. While JPA is a great api it is tightly bound to a traditional RDBMS model with the drawbacks inherited from there - being hard to scale and difficult to get good performance from on ever scaling systems. With the new api we open up for integration of various general NoSQL databases as well as the creation of tightly tailor-made persistence mechanisms to achieve optimal performance and scalability.
At the time of this writing several implementations has been made - the default JPA mechanism, two generic NoSQL implementations backend by Inifinispan and MapDB which will be available as contributions, and a single tailor made NoSQL implementation discussed shortly in this post.

The changes done in the Drools and jBPM persistence mechanisms, its new features, and how it allows to build clean new implementations of persistence for KIE components is the basis for a new soon to be added MapDB integration experimental module. The existing Infinispan adaptation has been changed to accommodate to the new structure.
Because of this refactor, we can now have other implementations of persistence for KIE without depending on JPA, unless our specific persistence implementation is JPA based. It has implied, however, a set of changes:

Creation of drools-persistence-api and jbpm-persistence-api

In version 6, most of the persistence components and interfaces were only present in the JPA projects, where they had to be reused from other persistencies. We had to refactor these projects to reuse these interfaces without having the JPA dependencies added each time we did so. Here's the new set of dependencies:
<dependency>
 <groupId>org.drools</groupId>
 <artifactId>drools-persistence-api</artifactId>
 <version>7.0.0-SNAPSHOT</version>
</dependency>
<dependency>
 <groupId>org.jbpm</groupId>
 <artifactId>jbpm-persistence-api</artifactId>
 <version>7.0.0-SNAPSHOT</version>
</dependency>

The first thing to mention about the classes in this refactor is that the persistence model used by KIE components for KieSessions, WorkItems, ProcessInstances and CorrelationKeys is no longer a JPA class, but an interface. These interfaces are:
  • PersistentSession: For the JPA implementation, this interface is implemented by SessionInfo. For the upcoming MapDB implementation, MapDBSession is used.
  • PersistentWorkItem: For the JPA implementation, this interface is implemented by WorkItemInfo, and MapDBWorkItem for MapDB
  • PersistentProcessInstance: For the JPA implementation, this interface is implemented by ProcessInstanceInfo, and MapDBProcessInstance for MapDB
The important part is that, if you were using the JPA implementation and wish to continue doing so with the same classes as before. All interfaces are prepared to work with these interfaces. Which brings us to our next point

PersistenceContext, ProcessPersistenceContext and TaskPersistenceContext refactors

Interfaces of persistence contexts in version 6 were dependent on the JPA implementations of the model. In order to work with other persistence mechanisms, they had to be refactored to work with the runtime model (ProcessInstance, KieSession, and WorkItem, respectively), build the implementations locally, and be able to return the right element if requested by other components (ProcessInstanceManager, SignalManager, etc)
Also, for components like TaskPersistenceContext there were multiple dynamic HQL queries used in the task service code which wouldn’t be implementable in another persistence model. To avoid it, they were changed to use specific mechanisms more related to a Criteria. This way, the different filtering objects can be used in a different manner by other persistence mechanisms to create the queries required.

Task model refactor

The way the current task model relates tasks and content, comment, attachment and deadline objects was also dependent on the way JPA stores that information, or more precisely, the way ORMs related those types. So a refactor of the task persistence context interface was introduced to do the relation between components for us, if desired. Most of the methods are still there, and the different tables can still be used, but if we just want to use a Task to bind everything together as an object (the way a NoSQL implementation would do it) we now can. For the JPA implementation, it still relates object by ID. For other persistence mechanisms like MapDB, it justs add the sub-object to the task object, which it can fetch from internal indexes.
Another thing that was changed for the task model is that, before, we had different interfaces to represent a Task (Task, InternalTask, TaskSummary, etc) that were incompatible with each other. For JPA, this was ok, because they would represent different views of the same data.
But in general the motivation behind this mix of interfaces is to allow optimizations towards table based stores - by no means a bad thing. For non table based stores however these optimizations might not make sense. Making these interfaces compatible allows implementations where the runtime objects retrieved from the store to implement a multitude of the interfaces without breaking any runtime behavior. Making these interfaces compatible could be viewed as a first step, a further refinement would be to let these interfaces extending each other to underline the model  and make the implementations simpler
(But for other types of implementation like MapDB, where it would always be cheaper to get the Task object directly than creating a different object, we needed to be able to return a Task and make it work as a TaskSummary if the interface requested so. All interfaces now match for the same method names to allow for this.)

Extensible TimerJobFactoryManager / TimerService

On version 6, the only possible implementations of a TimerJobFactoryManager were bound in the construction by the values of theTimeJobFactoryType enum. A refactor was done to extend the existing types, to allow other types of timer job factories to be dynamically added

Creating your own persistence. The MapDB case

All these interfaces can be implemented anew to create a completely different persistence model, if desired. For MapDB, this is exactly what was done. In the case of the MapDB implementation that is still under review, there are three new modules:
  • org.kie:drools-persistence-mapdb
  • org.kie:jbpm-persistence-mapdb
  • org.kie:jbpm-human-task-mapdb
That are meant to implement all the Task model using MapDB implementation classes. Anyone with a wish to have another type of implementation for the KIE components can just follow these steps to get an implementation going:
  1. Create modules for mixing the persistence API projects with a persistence implementation mechanism dependencies
  2. Create a model implementation based on the given interfaces with all necessary configurations and annotations
  3. Create your own (Process|Task)PersistenceContext(Manager) classes, to implement how to store persistent objects
  4. Create your own managers (WorkItemManager, ProcessInstanceManager, SignalManager) and factories with all the necessary extra steps to persist your model.
  5. Create your own KieStoreServices implementation, that creates a session with the required configuration, and adding it to the classpath

You’re not alone: The MultiSupport case

MultiSupport is a Denmark based company that has used this refactor to create its own persistence implementation. They provide an archiving product that is focused on creating a O(1) archive retrieval system, and had a strong interest in getting their internal processes to work using the same persistence mechanism they used for their archives.
We worked on an implementation that allowed for an increase in the response time for large databases. Given their internal mechanism for lookup and retrieval of data, they were able to create an implementation with millions of active tasks which had virtually no degradation in response time.
In MultiSupport we have used the persistence api to create a tailored store, based on our in house storage engine - our motivation has been to provide unlimited scalability, extended search capabilities, simple distribution and a performance we struggled to achieve with the JPA implementation. We think this can be used as a showcase of just how far you can go with the new persistence api. With the current JPA implementation and a dedicated SQL server we have achieved an initial performance of less than 10 ‘start process’ operations per second, now with the upcoming release we on a single application server have a performance more than 10 fold.

Share/Bookmark

Wednesday, May 10, 2017

An Executable DMN Solution for Business Users - bpmNEXT presentation

The video recording from the bpmNEXT presentation we did a few weeks ago is up!

In this presentation, Bruce and myself do a demo of the end-to-end, full (level 3) DMN solution built in partnership with Trisotech and Method&Style.




Here it is:



Share/Bookmark

Wednesday, April 26, 2017

End to end BPM (with a splash of DMN)

Red Hat Summit next week is shaping up to be one of the best ever!

And if you are a Drools or jBPM enthusiast, you will be busy: another top presentation that we have lined up for you comes from a partnership between Signavio and Red Hat. Duncan Doyle and Tom Debevoise will be driving the show on this one with a great example of how do model processes (and a few decisions) with the BPMN and DMN standards using the awesome tools from Signavio, and then deploying those models into the solid Drools and jBPM engines for execution!

This is End to End BPM: from Process Modeling to Execution with Signavio and Red Hat !

Join us on Wednesday, May 3rd, at 3:30pm!

And here is some extra detail from Tom:

End to End BPM


For nearly a decade designing processes in Business Process Model Notation (BPMN) has been a best practice for aligning business and technical objectives. With BPMN, the business analyst or subject matter expert can precisely define the interactions of customers, systems and trading partners with the activities and events that drive them. Because the notation is a standard, the meaning of the process model is unambiguous.
Business uses BPMN to define
·       The roles of the participants
·       Their responsibilities
·       The timing and sequence of events
·       How to handle errors and exceptions

Figure1 Example BPMN process in Signavio
With the Signavio Process Manager, all stakeholders can collaborate on the process model using an ability to commutate comments and concerns and a shared definition of terms. As shown in the figure 1, BPMN activities can denote where forms, services and scripts are needed. BPMN is more than a drawing convention. Compliant software can export the diagram in an XML format that other systems can read. Signavio and Red Hat have leveraged this capability so that processes and more can be exchanged.

Figure 2, the same BPMN process in BPM Suite’s KIE Workbench
To create an executable process, the technical team would then and the code for user forms, scripts and services. So processes in the Signavio Process Manager can be exported to the BPM Suite for this objective.
Most business analysts are not concerned with ‘Code’, except in the areas of compliance where very detailed logic, including quantities, dates and computational logic is critical. Recently BPMN has been extended to include decision modeling with the decision modeling notation (DMN). While separate from BPMN, DMN has been designed to work with BPMN. With decision modeling the business analysts can control a process by determining the logic for:
·       What needs to be done next
·       Who need to do it
·       When and where it is done
·       And importantly, were any important rules broken
Figure 3, Decision logic for the process in DMN
Decision logic can be exported from the Signavio Process Manager and incorporated into the KIE workbench. The process in figure 1 and 2 is controlled by the decision in figure 3.

-->
The teamwork of Signavio and Red Hat is a perfect separation of concerns between the business and IT. Because it is designed to be easy to use and collaborative, the Signavio Process Manager is the perfect environment for developing the business view of a process or a decision. Similarly, because it can leverage the power and scalability of the entire Red Hat middleware stack, the BPM Suite is the perfect environment for turning these decisions into an executable form and hosting them.

Share/Bookmark