Thursday, July 23, 2015

Validation and Verification for Decision Tables Update

The decision table verification and validation has come to a point where it is time to take a pause on adding new features, but I'm hoping to continue the work before the end of this year. One big missing feature is finding missing ranges and reporting about incomplete decision tables.

This blog entry will be an update to a previous entry that can be found here.

Features that made it into the next Final release

The features are currently in so any future release will include them. Here is a simple demo video showing the V&V issue panel and how it works in real time.



The different issue levels are:
  • Error - Serious fault. It is clear that the author is doing something wrong. Conflicts are a good example of errors.
  • Warning - These are most likely serious faults. They do not prevent the dtable from working, but need to be double checked by the dtable author. Redundant/subsumptant rules for example, maybe the actions need to happen twice in some cases.
  • Info - The author might not want to have any conditions in the dtable. If the conditions are missing each action gets executed. This can be used to insert a set of facts into the working memory. Still it is good to inform that the conditions might have been deleted by accident.  

The verification and validation looks for the following issues:


Redundancy 

To put it simple: two rows that are equal are redundant, but redundancy can be more complicated. The longer explanation is: redundancy exists when two rows do the same actions when they are given the same set of facts.

Redundancy might not be a problem if the redundant rules are setting a value on an existing fact, this just sets the value twice. Problems occur when the two rules increase a counter or add more facts into the working memory. In both cases the other row is not needed.




 

 

Subsumption

Subsumption exists when one row does the same thing as another, with a sub set of the values/facts of another rule. In the simple example below I have a case where a fact that has the max deposit below 2000 fires both rows.

The problems with subsumption are similar to the case with redundancy.




 

 

Conflicts

Conflicts can exists either on a single row or between rows.
A single row conflict prevent the row actions from ever being executed.

Single row conflict - second row checks that amount is greater than 10000 and below 1






Conflict between two rows exists when the conditions of two rules are met with a same set of facts, but the actions set existing fact fields to  different values. The conditions might be redundant or just subsumptant.

The problem here is, how do we know what action is made last? In the example below: Will the rate be set to 2 or 4 in the end? Without going into the details, the end result may be different on each run and with each software version. 
Two conflicting rows - both rows change the same fact to a different value

Deficiency

Deficiency gives the same kind of trouble that conflicts did. The conditions are too loose and the actions conflict.

For example:
If the loan amount is less than 2000 we do not accept it.
If the person has a job we approve the loan.
The problem is, we might have people with jobs asking for loans that are under 2000. Sometimes they get them, sometimes they do not.

Missing Columns

In some cases, usually by accident, the user can delete all the condition or action columns.

When the conditions are removed all the actions are executed and when the actions columns are missing the rows do nothing.
The action columns are missing
The condition columns are missing




Wednesday, July 01, 2015

Extending UberFire with AngularJS for the BPMS Domain

You can now see Alex’s talk, originally created by Kris, showing how to extend UberFire (UF) with AngularJS for the BPMS Domain, using Red Hat BPMS (productized version of jBPM). Everything is built live and in real time.

We are making great progress with our UF documentation, including tutorials. UF forms the core of our extensible UI architecture, it can be used standalone or to extend Drools or jBPM workbenches.

This work was made possible by our polyglot interoperability work for our UF framework, which we show in detail here:

The full UF docs are here, there has been a big update recently, as well as new tutorials. The empty sections will be filled in over next few days:

A recent generated PDF can be found here:

We should be close to a formal launch in a few weeks. The remaining items are:
-GWT 2.8 upgrade
-Move to new L&F theme
-Merge in Tool Windows.
-Move GWTExport to @JsType
-Mege in JS-UI

Tuesday, June 30, 2015

Uberfire, Drools and jBPM High Level Roadmap Slides and Videos from Red Hat Summit 2015

I did a short, 20 minute, high level presentation on some road map items for Uberfire, Drools and jBPM. I have provided the slides here, and it includes some early POC videos.

You can see the ongoing L&F work in the link below. The design is sleeker and more minimal. It now has a compact mode, that collapses the perspective switcher and sub-perspective menus -  see the “simple perspective”. Click the user name to switch modes. It automatically goes into compact mode when a panel is enlarged. 
user : admin
pass : admin

Mark

Friday, June 19, 2015

Improved multi-threading behaviour with Drools 6.3 SNAPSHOT

We’ve rewritten the internal parts of our code that deal with multi-threading to remove a large number of synchronisation points and to improve stability and predictability. We believe that what we have done is now far more robust for the interaction of the User, Timer and Engine threads. Our initial benchmarking is showing that this has led to mild performance improvements too. We’d really like to get this hardened, before we do 6.3 final, so if you have an application that users Timers or Time Windows, especially when using FireUntilHalt, could you give it a good hammering? Especially those using the TimedRuleExecutionFilter, which allows a timer to fire reactively when the engine is in passive mode (not fireUntilHalt).

For this iteration we just focused on the engine internals, we have not yet touched the outer lock and sync points, i.e. the ksession and kbase locks that threads go through when they do an insert/update/delete action. These apparently can create contention for lots of small lived ksessions. We believe with the latest work we've been doing we can soon improve this area too.

You should find all this work in the latest snapshot, for drools-core and drools-compiler.
https://repository.jboss.org/nexus/content/repositories/snapshots/org/drools/drools-core/6.3.0-SNAPSHOT/
https://repository.jboss.org/nexus/content/repositories/snapshots/org/drools/drools-compiler/6.3.0-SNAPSHOT/

For those interest, we have done two things. The first part was to properly separate the User insert/update/delete thread actions with he Engine network evaluations thread.  The second part is to remove most of the internal sync points and replace with a state machine.

The User/Engine thread separation has been made possible by our move away from Rete to Phreak. With Rete the network evaluation is done during the User insert/update/delete action, meaning each user action locks the entire engine. With phreak the insert/update/delete is separated and network evaluation happens when fireAllRules or fireUntilHalt is called. We've added a queue, SynchronizedPropagationList, that stores up the user actions as commands, in a thread safe queue. The engine thread then takes all the entries on each of its iterations. We found our custom queue outperformed the JDK concurrent queues, but I think that is due to our specialist implementation. Instead of the engine taking just the HEAD entry, it does a takeAll and the processes that returned linked list as a batch. This reduces the amount of times the Engine thread hits the queue for each of the elements it processes. We can also efficiently handle when to park and when to notify the engine to spin up again, which was alway a bit hit and miss before. Now it simply parks when takeAll returns null, and it notifies if a Timer or User adds work be done and the engine is known to be parked.

The second part introduces a state machine for the User, Timer and Engine thread interactions. This now provides us with a system that we can ca be documented, due to it's simplification, and also this will help explain the various thread interactions and behaviours. This was missing before, and understanding the behaviour could be a be bit confusing for users. It also means we now have a better behaviour for the interactions of calling fireAllRules and fireUntilHalt and when they overlap, or are called twice. i..e what happens if you call fireUntilHalt while fireAllRules is currently operating? or you call fireAllRules twice, or fall fireAllRules when fireUntilHalt is operating? Our state machine now more cleanly handles this with describable behaviour.

The bulk of the work is contained within the DefaultAgenda:
https://github.com/droolsjbpm/drools/blob/master/drools-core/src/main/java/org/drools/core/common/DefaultAgenda.java

There are three threads that can interact. A User thread doing an insert/update/delete, the  Timer thread, for timers and time windows and the engine thread for network evaluations.  We have now changed this so that the timer thread no longer does network evaluations, blocking other threads, instead it submits a job and notifies the Engine thread (if it's not already running) to process it. You can see this in PhreakTimerNode. When the Timer now triggers it'll submit a job tot he queue that I introduced in the previous paragraph.
public void execute(JobContext ctx) {
    TimerNodeJobContext timerJobCtx = (TimerNodeJobContext) ctx;
    InternalWorkingMemory wm = timerJobCtx.getWorkingMemory();
    wm.addPropagation( new TimerAction( timerJobCtx ) );
}

When a timer thread is kicked off it has no idea if the engine thread is evaluating or parked. It could be parked because fireAllRules has returned and it's waiting for the next fireAllRules. Or it could be parked because fireUntilHalt currently has no work to do. If for instance the engine is parked in fireUntilHalt it needs to notify the engine thread to unpark and process the timer work. If however engine thread is working (be it fireUntilRules or fireUntilHalt) it should just put it into the queue for the engine thread to process and not do the notification. These interactions are subtle, but they must be solid and avoid contention or excessing syncing. The behaviour is complicated further by the TimedRuleExecutionFilter.

To handle this we introduced the following enum to represent the available states of the engine:
private enum ExecutionState {  // fireAllRule | fireUntilHalt | executeTask <-- action="" br="" required="">    INACTIVE( false ),         // fire        | fire          | exec    FIRING_ALL_RULES( true ),  // do nothing  | wait + fire   | enqueue    FIRING_UNTIL_HALT( true ), // do nothing  | do nothing    | enqueue    REST_HALTING( false ),     // wait + fire | wait + fire   | enqueue    FORCE_HALTING( false ),    // wait + fire | wait + fire   | wait + exec    EXECUTING_TASK( false );   // wait + fire | wait + fire   | wait + exec
    private final boolean firing;

    ExecutionState( boolean firing ) {
        this.firing = firing;
    }

    public boolean isFiring() {
        return firing;
    }
}

You can now see this state machine being used by fireAllRules and fireUntilHalt. Notice the new method waitAndEnterExecutionState. This allows threads to either park or return straight away - i.e. if you call fireAllRules and fireUntilHalt is running, just return straight away. If you call fireUntilHalt while fireAllRules is running, wait until fireAllRules finishes, then start fireUntilHalt.

public int fireAllRules(AgendaFilter agendaFilter,
                        int fireLimit) {
    synchronized (this) {
        if (currentState.isFiring()) {
            return 0;
        }
        waitAndEnterExecutionState( ExecutionState.FIRING_ALL_RULES );
    }

public void fireUntilHalt(final AgendaFilter agendaFilter) {
    synchronized (this) {
        if (currentState == ExecutionState.FIRING_UNTIL_HALT) {
            return;
        }
        waitAndEnterExecutionState( ExecutionState.FIRING_UNTIL_HALT );
    }

private void waitAndEnterExecutionState( ExecutionState newState ) {
    if (currentState != ExecutionState.INACTIVE) {
        try {
            wait();
        } catch (InterruptedException e) {
            throw new RuntimeException( e );
        }
    }
    currentState = newState;
}

Previously you saw the Timer thread submitted a job into a queue, this is also handled by the state machine.
public void executeTask( ExecutableEntry executable ) {
    synchronized (this) {
        if (isFiring() || currentState == ExecutionState.REST_HALTING) {
            executable.enqueue();
            return;
        }
        waitAndEnterExecutionState( ExecutionState.EXECUTING_TASK );
    }

    try {
        executable.execute();
    } finally {
        immediateHalt();
    }
}

A key aspect we had to support here was what if a Timer thread triggers some work while the Engine thread is just returning. You end up with gaps, so that's work that doesn't fire, that the user was expecting. This is a problem people have seen in previous Drools releases. The combination of this task system halting statuses, allow the engine to restart again before properly halting. You can think of it as a two phase halting system. You an see that with the main do loop and then the second while loop, ensuring we get a clean shut down - i.e. the engine cannot park, unless there are no timer actions, before it returns and sets the state machine to INACTIVE.
this.workingMemory.flushPropagations();
int returnedFireCount;
do {
    returnedFireCount = fireNextItem( agendaFilter, fireCount, fireLimit );
    fireCount += returnedFireCount;
    this.workingMemory.flushPropagations();
} while ( ( isFiring() && returnedFireCount != 0 && (fireLimit == -1 || fireCount < fireLimit) ) );

PropagationEntry head = tryHalt();
while (head != null) {
    fireCount += fireNextItem( agendaFilter, fireCount, fireLimit );
    SynchronizedPropagationList.flush(workingMemory, head);
    head = workingMemory.takeAllPropagations();
}

private PropagationEntry tryHalt() {
    synchronized (this) {
        PropagationEntry head = workingMemory.takeAllPropagations();
        if (head == null) {
            currentState = ExecutionState.INACTIVE;
            notify();
        } else if (currentState != ExecutionState.FORCE_HALTING) {
            currentState = ExecutionState.REST_HALTING;
        }
        return head;
    }
}

One of the key aspects here is the takeAll action. We can use this to atomically both check if there is work to do, and return that work within a sync point. But process the work outside of the sync point. So you can see it it will only finally halt, if takeAll returns null. Note the Timer thread would have to go through this sync point to add more work - ensuring there are no gaps.

There is a lot to take in here, and it's a bit of a brain dump. But I hope it proves useful to those wanting to understand how we are improving our engine, and how the prior work we did with the Phreak algorithm has enabled this.

Thursday, June 18, 2015

Drools & jBPM get Dockerized

Docker is becoming a reference to build, ship and run container-based applications. It provides an standard, easy and automated way to deploy your applications.

Since latest 6.2.0.Final community release you can use Docker to deploy and run your Drools & jBPM applications in an easy and friendly way. Do not worry about operation system, environment and/or application server provisioning and deployments ... just use the applications!

The images are already available at Docker Hub:

Please refer to next "Drools & jBPM community Docker images" section for more information about what's contained in each image.

Why are these images helpful for me and my company?

To understand the advantages of using these Docker images, let's do a quick comparison with the deployment process for a manual installation of a Drools Workbench application.

If you do it by yourself:
  1. Install and prepare a Java runtime environment
  2. Download the workbench war (and other resources if necessary), from the official home page or from JBoss Nexus
  3. Download and prepare a JBoss WildFly server instance
  4. Configure the WildFly instance, including for example configuring the security subsystem etc.
  5. Deploy Drools into the WildFly instance
  6. Start the application server and run your Drools application
As you can notice, manual installation already takes quite a few steps.  While this process can be automated in some way (as the jbpm-installer for example does), some questions arise at this point ... What if I need a more complex environment? Are other colleagues using the same software versions and configuration? Can I replicate exact same environment? Could someone else easily run my local example easily during a customer demo? And if I need to deploy several identical runtime environments? What about removing my local installation from my computer? ...

Software containers & Docker are a possible solution and help providing an answer to some of these questions.

Both Drools & jBPM community Docker images include:
  • The OpenJDK JRE 1.7 environment 
  • A JBoss WildFly 8.1.0.Final application server
  • Our web-based applications (Drools Workbench, KIE server and/or jBPM Workbench) ready to run (configurations and deployments already present)
You don't have to worry about the Java environment, the application server, the web applications or configuration ... just run the application using a single command:

  docker run -p 8080:8080 -d --name drools-wb jboss/drools-workbench-showcase:6.2.0.Final

Once finished, just remove it:

   docker stop ...

At this point, you can customize, replicate and distribute the applications! Learn more about Docker, its advantages and how to use it at the offical site.

The environment you need

Do not worry about Java environments, application servers or database management systems, just install Docker:

   # For RHEL/Fedora based distributions:
   sudo yum -y install docker

More installation information at the official Docker documentation.

Are you using Windows? 

For windows users, in order to use Docker, you have to install Boot2Docker. It provides a Linux basic environment where Docker can run. Please refer to the official  documentation for the Docker installation on Windows platforms.

You are ready to run!

Drools & jBPM community Docker images

For the 6.2.0.Final community release six Docker images have been released.  They can be categorized in two main groups: Base images provide the base software with no custom configurations. They are intended to be extended and customized by Docker users.   Showcase images provide applications that are ready to run out-of-the-box (including for example some standard configuration).  Just run and use it!  Ideal for demos or evaluations / getting started.
  • Base images 
    • Drools Workbench
    • KIE Execution Server
    • jBPM Workbench
       
  • Showcase images
    • Drools Workbench Showcase
    • KIE Execution Server Showcase
    • jBPM Workbench Showcase


Let's dive into a detailed description of each image in the following sections.


Drools Workbench

This image provides the standalone Drools web authoring and rules management application for version 6.2.0.Final.  It does not include any custom configuration, it just provides a clean Drools Workbench application running in JBoss WildFly 8.1.  The goal of this image is to provide the base software and allow users to extend it, and apply custom configurations and build custom images.

Fetch the image into your Docker host:

   docker pull jboss/drools-workbench:6.2.0.Final

Customize the image by creating your Dockerfiles:

   FROM jboss/drools-workbench:6.2.0.Final
   ...

Please refer to Appendix C for extending this image.

Run a Drools Workbench container:

docker run -p 8080:8080 -d --name drools-wb jboss/drools-workbench:6.2.0.Final

Navigate to your Drools Workbench at:

   http://localhost:8080/drools-wb # Linux users
   http://<boot2docker_ip>:8080/drools-wb # Windows users

Refer to Appendix A for more information about IP address and port bindings.

Drools Workbench Showcase

See it in Docker Hub

This image provides the standalone Drools web authoring and rules management application for version 6.2.0.Final plus security configuration and some examples.
Tip: This image inherits from the Drools Workbench one and adds custom configurations for WildFly security subsystem (security realms) and system properties for enabling the use of the examples repository. 
The goal for this image is to provide a ready to run Drools Workbench application: just pull, run and use the Workbench.

1. Pull the image:

  docker pull jboss/drools-workbench-showcase:6.2.0.Final

2. Run the image:

  docker run -p 8080:8080 -d --name drools-wb-showcase jboss/drools-workbench-showcase:6.2.0.Final

3. Navigate to the workbench at:

   http://localhost:8080/drools-wb # Linux users
   http://<boot2docker_ip>:8080/drools-wb # Windows users

Refer to Appendix A for more information about IP address and port bindings.

You can use admin/admin for default logging in - Refer to Appendix B for default users and roles included

KIE Execution server

This image provides the standalone rules execution component for version 6.2.0.Final, to handle rules via remote interfaces.
More information for the KIE Execution Server can be found at the official documentation.
This image does not include any custom configuration, it just provides a clean KIE Execution Server application running in JBoss WildFly 8.1.  The goal for this image is to provide the base software and let the users to extend it, and apply custom configurations and build custom images.

Fetch the image into your Docker host:

   docker pull jboss/kie-server:6.2.0.Final

Customize the image by creating your Dockerfiles:

   FROM jboss/kie-server:6.2.0.Final
   ...

Please refer to Appendix C for extending this image.
Run a KIE Execution Server container:

   docker run -p 8080:8080 -d --name kie-server jboss/kie-server:6.2.0.Final

The KIE Execution Server is located at:

   http://localhost:8080/kie-server # Linux users
   http://<boot2docker_ip>:8080/kie-server # Windows users


Refer to Appendix A for more information about IP address and port bindings.

Example: use the remote REST API to perform server requests :

 http://localhost:8080/kie-server/services/rest/server # Linux
 http://<boot2docker_ip>:8080/kie-server/services/rest/server # Win

KIE Execution Server Showcase

See it in Docker Hub

This image provides the standalone rules execution component version 6.2.0.Final to handle rules via remote interfaces plus a basic security configuration (include a default user and role).
More information for the KIE Execution Server can be found at the official documentation. 
Tip: This image inherits from the KIE Execution Server one and adds custom configuration for WildFly security subsystem (security realms).

The goal of this image is to provide a ready to run KIE Execution Server: just pull, run and use the remote services.

1. Pull the image:

   docker pull jboss/kie-server-showcase:6.2.0.Final

2. Run the image:

   docker run -p 8080:8080 -d --name kie-server-showcase jboss/kie-server-showcase:6.2.0.Final

3. The server is located at:

   http://localhost:8080/kie-server # Linux users
   http://<boot2docker_ip>:8080/kie-server # Windows users


    The REST API service is located at:
  
 http://localhost:8080/kie-server/services/rest/server # Linux  
 http://<boot2docker_ip>:8080/kie-server/services/rest/server # Win  

Refer to Appendix A for more information about IP address and port bindings.

You can use kie-server/kie-server for default logging - Refer to Appendix B for default users and roles included
  

jBPM Workbench


This image provides the standalone version 6.2.0.Final of the jBPM Workbench: web-based authoring and management of your processes.  It does not include any custom configuration, it just provides a clean jBPM Workbench application running in JBoss WildFly 8.1.  The goal of this image is to provide the base software and let the users to extend it, and apply custom configurations and build custom images.

Fetch the image into your Docker host:

   docker pull jboss/jbpm-workbench:6.2.0.Final

Customize the image by creating your Dockerfiles:

   FROM jboss/jbpm-workbench:6.2.0.Final
   ...

Please refer to Appendix C for extending this image.
Run a jBPM Workbench container:

   docker run -p 8080:8080 -d --name jbpm-wb jboss/jbpm-workbench:6.2.0.Final

Navigate to your jBPM Workbench at:

   http://localhost:8080/jbpm-console # Linux users
   http://<boot2docker_ip>:8080/jbpm-console # Windows users

Refer to Appendix A for more information about IP address and port bindings.

jBPM Workbench Showcase


This image provides the standalone version 6.2.0.Final of the jBPM Workbench: web-based authoring and management of your processes. It includes the security and persistence configurations and some examples too.
Tip: This image inherits from the jBPM Workbench one and adds custom configurations for WildFly security subsystem (security realms) and system properties for enabling the use of the examples repository. 
The goal of this image is to provide a ready to run jBPM Workbench application: just pull, run and use the Workbench:

1. Pull the image:

   docker pull jboss/jbpm-workbench-showcase:6.2.0.Final

2. Run the image:

   docker run -p 8080:8080 -d --name jbpm-wb-showcase jboss/jbpm-workbench-showcase:6.2.0.Final

3. Navigate into the workbench at:

   http://localhost:8080/jbpm-console # Linux users  
   http://<boot2docker_ip>:8080/jbpm-console # Windows users

Refer to Appendix A for more information about IP address and port bindings.

You can use admin/admin for default logging - Refer to Appendix B for default users and roles included

Appendix

 

Appendix A - IP address and ports bindings for Docker containers

 
Port bindings
By default, when using any of the Drools & jBPM Docker images, the port 8080 is exposed for the use of the HTTP connector. This port is not exposed to the Docker host by default, so in order to expose it and be able to navigate through the applications please read the following instructions.

The recommended use for running containers is specifying in the docker client the -p argument as:

  docker run -p 8080:8080 -d ....

Doing this way, the docker daemon binds the internal container's port 8080 to the Docker host machine's port 8080. So you can navigate into the applications at:

   http://<docker_host>:8080/jbpm-console
   http://<docker_host>:8080/kie-server
   http://<docker_host>:8080/drools-wb

If your Docker host machine's port 8080 is not available, run the containers with the -P command line argument. Docker binds the internal 8080 port to an available free exposed port in the Docker host, so in order to access the application you have to discover the bind port number.

To discover running container's ports type the following command:

   docker ps -a

This command will output the processes and the port mappings for each running container:

CONTAINERID  IMAGE             COMMAND  CREATED STATUS PORTS                     NAMES
2a55fb....   jboss/drools-w..  ...      ...     ..     0.0.0.0:49159->8080/tcp.. drools-wb
The PORTS column shows that the internal container's port 8080 is bound to port 49159 on the Docker host, so you can navigate into the applications at:

   http://<docker_host>:49159/jbpm-console
   http://<docker_host>:49159/kie-server
   http://<docker_host>:49159/drools-wb


Docker hostname & IP address
The Docker hostname or IP address have to be specified in order to navigate through the container's applications.

If you are running Docker in your localhost and using Linux based OS, it defaults to localhost:

   http://localhost:8080/jbpm-console
   http://localhost:8080/kie-server
   http://localhost:8080/drools-wb

If you are running Docker on another machine or in Windows environments, where Boot2Docker is required,  you have to specify the host name (if DNS available for it) or the IP address for it:

   http://192.168.1.156:8080/jbpm-console
   http://192.168.1.156:8080/kie-server
   http://192.168.1.156:8080/drools-wb

Appendix B - Default applications users & roles

The Showcase images Drools Workbench Showcase and jBPM Workbench Showcase include default users & roles:

Drools & jBPM Workbench Showcase roles
Role Description
admin The administrator
analyst The analyst
developer The developer
manager The manager
user The end user
kiemgmt KIE management user
Accounting Accounting role
PM Project manager role
HR Human resources role
sales Sales role
IT IT role

Drools & jBPM Workbench Showcase users
Username Password Roles
admin admin admin,analyst,kiemgmt
krisv krisv admin,analyst
john john analyst,Accounting,PM
mary mary analyst,HR
sales-rep sales-rep analyst,sales
katy katy analyst,HR
jack jack analyst,IT
salaboy salaboy admin,analyst,IT,HR,Accounting

For KIE Execution Server Showcase there is a single user and role:
Username Password Roles
kie-server kie-server kie-server

Appendix C - Extending base images

The Base images are intended to be inherited from, for adding your custom configurations or deployments.

In order to extend the images, the Dockerfile must start with: 

    FROM jboss/drools-workbench:6.2.0.Final 
    FROM jboss/kie-server:6.2.0.Final
    FROM jboss/jbpm-workbench:6.2.0.Final

At this point, custom configurations and deployments can be added. Some notes:
  • JBoss WildFly is located at the path given by $JBOSS_HOME environment variable
  • $JBOSS_HOME points to /opt/jboss/wildfly/
  • Applications are using the server in standalone mode:
    • Configurations located at $JBOSS_HOME/standalone/configuration/
    • Configuration files for the standalone-full profile are used
    • Deployments are located at $JBOSS_HOME/standalone/deployments/
You can find more information at each official image page at Docker Hub:

Wednesday, June 17, 2015

Drools & jBPM meeting space needed in Barcelona

We are looking to try and organise a team meeting in Barcelona, towards the end of this year. We have limited to no budget for meeting space :( So I thought I'd see if anyone out there would like to volunteer this space - in return you'll have all the core Drools and jBPM developers on hand for a week :) We need a large room suitable for around 25-30 people sitting at tables, and then one or two break out rooms with around 10 people or so per room.

Mark

Wednesday, May 27, 2015

More Eclipse Tooling enhancements

The biggest complaint from our customers about the eclipse tooling for B*MS is that the cost of entry is too high; not only must a user be familiar with several different technologies, such as Git, maven, REST services and how these technologies are exposed by the eclipse tooling, but s/he must also understand the various Drools and jBPM configuration and definition files. Since there are only a few user-friendly/graphical editors that hide underlying file details, the user must become familiar with most of these file formats, and where in the Project or Repository hierarchy the file resides.

One of the enhancements I have been working on will hopefully ease some of this burden by providing a "navigator" similar to the Eclipse Project Explorer, but designed specifically for Drools/jBPM projects (see below).
At the root of this tree viewer are the app servers that have Drools/jBPM installed. Servers are managed (start, stop, debug) from the WST Servers view. At the next level is the Organizational Unit, then Repositories and finally Projects. Essentially, this viewer mimics the web console with the addition of multiple servers.

The tree structure is cached whenever a connection to the server can be established. This allows the view to be used in "offline" mode if the server is down or network connection is unavailable. When the server is available again, the viewer synchronizes its cache with the server.

Repositories are automatically cloned, and Projects are imported as they are requested by the user with a context menu action.

I'm still in the design/experimenting phase right now, so if there's a feature you'd like to see, or if you have suggestions for improving this interface please post your comments here.

You can also see a related post, showing my work on improving the wizards and runtime generation and configuration.

Improved Drools & jBPM Eclipse wizard

Bob has been working on improving our Drools & jBPM Eclipse wizards.

  • The user no longer needs to create runtimes. They can now be created automatically on the fly by the new project wizard.
  • The project wizard will now list examples from the github repository and allow them to be selected and dowloaded as part of the wizard.
You cans see a video for this here:

Currently all the downloadable examples are jBPM, we still need to migrate the Drools examples over to this repository format.

Mark

Wednesday, May 20, 2015

A Comparative Study of Correlation Engines for Security Event Management

This just paper came up on my google alerts, you can download the full text from ResearchGate.
"A Comparative Study of Correlation Engines for Security Event Management"

 It's an academic paper, published in the peer reviewed journal.
"10th International Conference on Cyber Warfare and Security (ICCWS-2015)"

Th paper is evaluating the correlation performance for large rule sets and large data sets in different open source engines. I was very pleased to see how well Drools scaled at the top end. I'll quote this from the conclusion and copy the results charts.
"As for the comparison study, it must be said that if the sole criteria was raw performance Drools would be considered the best correlation engine, for several reasons: its consistent behaviour and superior performance in the most demanding test cases."

In Table 2 (first image) we scale form 200 rules to 500 rules, with 1mil events with almost no speed loss - 67s vs 70s.

In Table 1 (second image) our throughput increases as the event sets become much larger.

I suspect the reason why our performance is less for for the lower rule and event set numbers, is due to the engine initialisation time for all the functionality we provide and for all the indexing we do. As the matching time becomes large enough, due to larger rule and data sets, this startup time becomes much less significant on the over all figure.




Tuesday, May 12, 2015

Validation and Verification for Decision Tables

The decision tables are getting even more improvements than the UI work Michael has been working on.
Zooming and Panning between Multiple Huge Interconnected Decision Tables
Cell Merging, Collapsing and Sorting with Multiple Large Interconnected Decision Tables

I am currently working on improving the validation and verification of the decision tables. Making it real time and improving the existing V&V checks.

Validation and verification are used to determine if the given rules are complete and to look for any bugs in the dtable authors logic. More about this subject.

Features coming in the next release


Real time Verification & Validation

Previously the user had to press a button to know if the dtable was valid or not. Now the editor does the check in real time, removing the need to constantly hit the Validate-button. This also makes the V&V faster, since there is no need to validate the entire table, just check how the change of a field affected the rest of the table.




Finding Redundancy 

To put it simple: two rows that are equal are redundant, but redundancy can be more complicated. The longer explanation is: redundancy exists when two rows do the same actions when they are given the same set of facts.

Redundancy might not be a problem if the redundant rules are setting a value on an existing fact, this just sets the value twice. Problems occur when the two rules increase a counter or add more facts into the working memory. In both cases the other row is not needed.




 

 

Finding Subsumption

Subsumption exists when one row does the same thing as another, with a sub set of the values/facts of another rule. In the simple example below I have a case where a fact that has the max deposit below 2000 fires both rows.

The problems with subsumption are similar to the case with redundancy.






Finding Conflicts

Conflicts can exists either on a single row or between rows.
A single row conflict prevent the row actions from ever being executed.

Single row conflict - second row checks that amount is greater than 10000 and below 1






Conflict between two rows exists when the conditions of two rules are met with a same set of facts, but the actions set existing fact fields to  different values. The conditions might be redundant or just subsumptant.

The problem here is, how do we know what action is made last? In the example below: Will the rate be set to 2 or 4 in the end? Without going into the details, the end result may be different on each run and with each software version. 
Two conflicting rows - both rows change the same fact to a different value

 

Reporting Missing Columns

In some cases, usually by accident, the user can delete all the condition or action columns.

When the conditions are removed all the actions are executed and when the actions columns are missing the rows do nothing.
The action columns are missing
The condition columns are missing










What to expect in the future releases?


Better reporting

As seen on the examples above. Reporting the issues is currently poor.
The report should let the user know how serious the issue is, why it is happening and how to fix it.

The different issue levels will be:
  • Error - Serious fault. It is clear that the author is doing something wrong. Conflicts are a good example of errors.
  • Warning - These are most likely serious faults. They do not prevent the dtable from working, but need to be double checked by the dtable author. Redundant/subsumptant rules for example, maybe the actions need to happen twice in some cases.
  • Info - The author might not want to have any conditions in the dtable. If the conditions are missing each action gets executed. This can be used to insert a set of facts into the working memory. Still it is good to inform that the conditions might have been deleted by accident.  

 

Finding Deficiency

Deficiency gives the same kind of trouble that conflicts did. The conditions are too loose and the actions conflict.

For example:
If the loan amount is less than 2000 we do not accept it.
If the person has a job we approve the loan.
The problem is, we might have people with jobs asking for loans that are under 2000. Sometimes they get them, sometimes they do not.


 

Finding Missing Ranges and Rows

Is the table complete? In our previous examples we used the dtable to see if the loan application gets approved. One row in the dtable should always activate, no matter how the user fills out his loan application. Either rejecting or approving the loan or else the applicant does not get a loan decision.
The goal of the V&V tool will be to find these gaps for the dtable author.

 

Finding Cycles

The actions can insert new facts and the conditions trigger the actions when new facts are inserted. This can cause an infinite number of activations.
This issue is a common mistake that the goal is to pick it up in the authoring phase with the V&V tool.