Monday, September 30, 2013

Project Job Scheduling (a form of Job Shop Scheduling)

I've created a new video to demonstrate the Project Job Scheduling (a form of Job Shop Scheduling) in OptaPlanner 6.0.0.CR4.

Finish projects earlier by optimizing the order (and the execution mode) of each job, to use the available resources more efficiently.

Monday, September 23, 2013

Vehicle Routing with Time Windows

A new video to demonstrate the Vehicle Routing problem with Time Windows (CVRPTW) in OptaPlanner 6.0.0.CR4:

Sunday, September 22, 2013

Pluggable Belief Systems in Drools 6.0

Drools has supported simple truth maintenance for a long time, and followed a similar approach as that in Jess and Clips.

In 6.0 we abstracted the TMS system to allow for pluggable belief systems. This allows a sub system to control what the main working memory can see; i.e. what is inserted for the user to write rules to join against.

There are two interfaces for this the BeliefSystem and the BeliefSet. The BeliefSystem is global to the engine and provides the handling for logical inserts or deletes. It also has a constructor method to provide the LogicalDependency instance; this allows BeliefSystem to have it's own implementation.
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/beliefsystem/BeliefSystem.java

The BeliefSet is the set of "equals" beliefs; i.e. logical insertions. If you remember in 5.x a TMS system a belief is a one or more logical insertions; but only one will ever be visible in the system. Logical means they have a supported rule, which is tracked by a counter. Only when there are no supporters and that counter is zero, is the belief deleted. We've extended this so a logical insertion have an additional value associated with it; which becomes useful in our JTMS implementation, that I'll cover in a moment. Further the BeliefSystem provides control over what is or is not propagated into the main engine - it could be one of the logical inserted items from the set, or even a derived value determined from the set.

We have a "simple" implementation, that emulates what we had in 5.x, and is still the default.
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/beliefsystem/simple/SimpleBeliefSystem.java
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/beliefsystem/simple/SimpleBeliefSet.java

We've added am experimental JTMS implementation, which allows a logical insertion to have a positive or a negative label. This allows for contradiction handling. A logical insertion will only exist in the main working memory, as long as there is no conflict in the labelling - i.e. it must be one or more positive labels, and no minus label.
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/beliefsystem/jtms/JTMSBeliefSystem.java
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/beliefsystem/jtms/JTMSBeliefSetImpl.java

I've covered the bus pass system before, here. The code is still the same, the only difference now with the JTMS plugin is that each logical insertion defaults to a positive label.


rule "IsChild" when
    p : Person( age < 16 )
then
    logicalInsert( new IsChild( p ) )
end


rule "IsAdult" when
    p : Person( age >= 16 )
then
    logicalInsert( new IsAdult( p ) )
end


rule "Issue Child Bus Pass" when
    p : Person( )
         IsChild( person == p )
then
  logicalInsert(new ChildBusPass( p ) );
end

rule "Issue Adult Bus Pass" when
    p : Person()         IsAdult( person == p )
then
  logicalInsert(new AdultBusPass( p ) );
end


In the case of someone who is a child, it results in a tree that looks like below.
These are called your "default" rules. Now what happens if you want to add an exception, that contradicts the default rule. JTMS allows a negative insertion for a fact, doing this causes a conflict an the fact will be held in limbo, and not available in the working memory, until the conflict is resolved. For instance we might want an exception rule, that does not allow a bus pass to be issued to someone who is banned.

rule "Do not issue to banned people" when
  p : Person( )
       Banned( person == p )
then
  logicalInsertnew ChildBusPass( p ) , “neg” );
end


If the person is banned, it results in a tree with one positive and one negative label.  The belief system is incremental and cascading, so at any time the exception rule can become true which would result in a cascading undo operation.


We've also added another experimental implementation for Defeasible logic. Interestingly it turned out that Defeasible logic can be derived from the JTMS implementation, using the same BeliefSystem implementation but a custom BeliefSet implementation. The DefeasibleSet can be found here, clearly it is a lot more complicated than the JTMS one. We use mask operations to try and keep it optimal. We haven't added tracking for recursion yet, that is a TODO, and ideally done at compile time.
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/beliefsystem/defeasible/DefeasibleBeliefSet.java

Defeasible augments the JTMS with annotations to provide declarative resolving of conflicts.

  • @Strict // rule cannot be defeated
  • @Defeasible // rule can be defeated
  • @Defeater // rule can defeat other rules, but it's result is not propagated into the working memory
  • @Defeats // takes list of rules it defeats

rule "Do not issue to banned people" @Defeasible when
  p : Person( )       Banned( person == p )
then
  logicalInsert(new ChildBusPass( p ) , “neg” );
end

rule "Exception for children with minor offences" @Defeats("Do not issue to banned people") when
  p : Person( )
       IsChild( person == p )
       Banned( person == p, offence == "minor" )
then
  logicalInsert(new ChildBusPass( p )  );
end

In defeasible logic the exception rule here is called a counter argument, and it is possible for another rule to rebut the counter-arguents, creating an argumentation chain, that rebuttal can also be rebutted. A good presentation on this can be found here.

We are currently working on other Belief System implementations. One is based on the Belief Logic Programming idea, which uses the concepts of belief combination functions as inspired by Dempster-Shafer. This will allow each logical insertion to have a degree of belief, and the BeliefSystem will be able to process those chains of logical insertions, applying the combination functions.

The other idea we are working on is Bayesian network integration. The BeliefSystem can back onto a Bayesian network, which can control which facts are inserted, or not inserted, into the engine. As the data changes over time, the sub system can add or remove what the main engine sees.

If you find this interesting, and what to have a go at implementing your own and need some help, then don't hesitate to drop onto irc and ask:
http://www.jboss.org/drools/irc

While the system is pluggable, the registration process is currently hard coded into an Enum, which you'll need to update with your implementation:
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/BeliefSystemType.java


Saturday, September 21, 2013

Drools and jBPM Public Training London 2013

I'm preparing a workshop that introduces Business Process Management and prepares you to be immediately effective in using both Drools and jBPM to improve your applications. You will learn how to utilize the different stages of BPM where development is involved, for both versions 5 and 6 of the Drools and jBPM platform.
We'll discuss several best practices that allow for effective BPM, and how the jBPM components are more suitable placed into those best practices. We'll also cover how is the best way to perceive the software writing work related to running effective Business Processes and rules, and see how this allows a best fit from an End User perspective.
Where? London, England, Number 1 Poultry, EC2R 8JR
When? October 21-25 , filled with Q&A sessions and workshops, from 10:00 AM to 18:00 PM, with the last two hours for specialized questions and workshopping everyday.
What will it cover? Full theoretical and technical overview of Drools and jBPM. You can download the full agenda from here
We offer different options depending on your interest:

Introduction: October 21. Full theoretical introduction to Drools and jBPM components. USD 500.00
Drools: October 21-23. Introduction + full technical coverage of Drools. USD 1350.00
jBPM: October 21, 24 and 25. Introduction + full technical coverage of jBPM. USD 1350.00
Full: October 21 to 25. USD 1728.00, and 1929.00 after 9/21/13. Register now and get the early bird pricing!

You can pay from Paypal here, or send us an email at trainings@plugtree.com for other payment methods. See you at London!

Saturday, September 07, 2013

Rollbackable Processes + Public Training London 2013!!

(Original post here)

This is a topic I've wanted to discuss for a long time. This post is to show you how to use a new component I've made called jbpm-rollback-api, a configurable module that allows you to rollback persistent jBPM process instances to a previous step. It makes it possible by just adding an environment variable, a process event listener and an extra class to the jBPM persistence unit. I'll discuss it in as much detail as possible in Plugtree's next Public Training in London, I invite you to register

Why?
When running process instances, especially during the first runs in a new BPM based project, you might get to a point where you wished you had done something different along the steps of your business process (maybe specifying a different value for a variable, or you end up in a path you didn't wish in the first place. If you can't change that aspect of the process instance, you need to drop it altogether. This isn't an issue when running from a JUnit test case, but if you find this issue in a running system, you might not want to drop the process instance and start again, specially when it involves other people's work. The possibility of rollbacking the tasks of a process allows you to get to a previous state of the process instance without having to start over again.

How?
The whole idea spins around the way the process instances are persisted today in the database. Here's a nice explanation of the database persistence if you wish to go into detail about it. In short, there is a small blob of data marshalled into each ProcessInstance row in the database. Since it is overwritten every time the process instance changes, the rollback module takes a copy of that blob and stores it aside to have it available after the process changes. It can't just copy it every time it pleases when it is inside a database transacted operation, so it does it whenever the session reaches a safe state (that is, after the transaction is finished and the session method is ready to return). And it can't just do it for all live process instances, that would be too expensive performance-wise. So it does it only for the process that changed during the last transaction.

Overall the configuration looks like this:

Configuration for rollback processes
The way it works is by four simple components:
  • ProcessSnapshotLogger: This class acts as two things:
    • A process event listener to monitor for any process instance changes within a persistent session. If a process instance changes, we mark it as a candidate to persist a snapshot after the knowledge session transaction is done. Also if a process instance is completed, we mark the process snapshots of that instance for deletion to keep the database runtime  at a steady size.
    • It also works as an interceptor to wait for all safe states in a persistent session, to persist any changed process instances. The interceptor is added as a step every time a command of the command based knowledge session finishes executing.
  • ProcessSnapshot: A database entity designed to store the full blob representation of a process instance every time it changes, to be able to reload it on demand afterwards.
  • ProcessSnapshotAcceptor: Taking snapshots of process instances can affect performance, so this class provides a very simple interface to configure what process instances to monitor for rollback, omitting by default. You have a few implementations available that allow you to select all instances of a given process definition ID, all the instances, or you can implement your own by implementing the method boolean accept(String processId, long processInstanceId).
  • ProcessRollback: A utility class to query for old snapshots of a process instance and to paste them on top of the preexisting process instance. You can use the goBack(KieSession ksession, long processInstanceId) static method to go back one step, or the overridden static method goBack(KieSession ksession, long processInstanceId, int numberOfSteps) to go back as many steps as you like
Internally, the rollback recreates the old process instance and reactivates any nodes that were alive at the moment of the snapshot. At the moment (and in this form) you don't send any signals to external systems that steps taken in the process need to be rolled back. However, this would be a next step for this module; by creating a RollbackableWorkItemHandler interface with a rollbackWorkItem(WorkItem item, WorkItemManager manager) method that people could implement, the rollback could go one step at a time and notify any external systems about a rollback being effected. This is one of the many subjects we would love to discuss and teach about:
We at Plugtree are organizing a Public Training in London on October 21st to 25th at N 1 Poultry. We'll cover Drools 5, 6, and jBPM 5 and 6 with as much detail as possible. We'll introduce both the BPM and AI theory, as well as all technical specifications for the different components, in order to take the most advantage of Drools and jBPM. Here's an overall agenda:
  • Day 1: Introduction to all components, for technical and non-technical folks alike
  • Days 2 and 3: Focus on Drools components, both versions 5 and 6, from theory to practice in as much detail as possible. We'll also cover rule writing in most formats and best practices
  • Days 4 and 5: Focus on jBPM components, both versions 5 and 6, from theory to practice in as much detail as possible. We'll also cover BPMN2 writing in as much detail as possible.
You can download the full detailed agenda. We offer different packages for group registrations and different interests. If you wish to hear more about special packages, feel free to contact us.
You can click here to register. Take advantage of the early bird pricing!

If you wish to download and play with the code discussed here, you can download it from here