Tuesday, August 19, 2014

Drools Mailing List migration to Google Groups

Drools community member,

The Drools team are moving the rules-usesrs and rules-dev list to Google Groups. This will allow users to have a combined email and web access to the group.
New Forum Information : http://drools.org/community/forum.html (click link to view)

The rules-users mailing list has become high volume and it seems natural to split the group into those asking for help with setup, configuration, installation and administration and those who are asking for help with authoring and executing of rules. For this reason rules-users will be split into two groups - drools-setup and drools-usage.  

Drools Setup - https://groups.google.com/forum/#!forum/drools-setup (click link to subscribe)
Drools Usage - https://groups.google.com/forum/#!forum/drools-usage (click link to subscribe)

The rules-dev mailing list will move to drools-development. 

Drools Development - https://groups.google.com/forum/#!forum/drools-development (click link to subscribe)

Google Groups limits the number of invitations, so we were unable to send invitations. For this reason you will need to manually subscribe. 

The Drools Team

Friday, August 15, 2014

Drools Execution Server demo (6.2.0.Beta1)

As some of you know already, we are introducing a new Drools Execution Server in version 6.2.0.

I prepared a quick video demo showing what we have done so far (version 6.2.0.Beta1). Make sure you select "Settings -> 720p" and watch it in full screen.


Monday, August 11, 2014

JUDCon 2014 Brazil: Call for Papers

The International JBoss Users and Developer Conference, and premier JBoss developer event “By Developers, For Developers,” is pleased to announce that the call for papers for JUDCon: 2014 Brazil, which will be held in São Paulo on September 26th, is now open! Got Something to Say? Say it at JUDCon: 2014 Brazil! Call for papers ends at 5 PM on August 22nd, 2014 São Paulo time, and selected speakers will be notified by August 29th, so don't delay!

http://www.jboss.org/events/JUDCon/2014/brazil/cfp

Monday, July 21, 2014

Drools Executable Model (Rules in pure Java)

The Executable Model is a re-design of the Drools lowest level model handled by the engine. In the current series (up to 6.x) the executable model has grown organically over the last 8 years, and was never really intended to be targeted by end users. Those wishing to programmatically write rules were advised to do it via code generation and target drl; which was no ideal. There was never any drive to make this more accessible to end users, because extensive use of anonymous classes in Java was unwieldy. With Java 8 and Lambda's this changes, and the opportunity to make a more compelling model that is accessible to end users becomes possible.

This new model is generated during the compilation process of higher level languages, but can also be used on its own. The goal is for this Executable Model to be self contained and avoid the need for any further byte code munging (analysis, transformation or generation); From this model's perspective, everything is provided either by the code or by higher level language layers. For example indexes etc must be provided by arguments, which the higher level language generates through analysis, when it targets the Executable model.
   
It is designed to map well to a Fluent level builders, leveraging Java 8's lambdas. This will make it more appealing to java developers, and language developers. Also this will allow low level engine feature design and testing, independent of any language. Which means we can innovate at an engine level, without having to worry about the language layer.
   
The Executable Model should be generic enough to map into multiple domains. It will be a low level dataflow model in which you can address functional reactive programming models, but still usable to build a rule based system out of it too.

The following example provides a first view of the fluent DSL used to build the executable model
         
DataSource persons = sourceOf(new Person("Mark", 37),
                              new Person("Edson", 35),
                              new Person("Mario", 40));
                     
Variable<Person> markV = bind(typeOf(Person.class));

Rule rule = rule("Print age of persons named Mark")
        .view(
            input(markV, () -> persons),
            expr(markV, person -> person.getName().equals("Mark"))
        )
        .then(
            on(markV).execute(mark -> System.out.println(mark.getAge())
        )
);

The previous code defines a DataSource containing a few person instances and declares the Variable markV of type Person. The rule itself contains the usual two parts: the LHS is defined by the set of inputs and expressions passed to the view() method, while the RHS is the action defined by the lambda expression passed to the then() method.

Analyzing the LHS in more detail, the statement
         
input(markV, () -> persons)
binds the objects from the persons DataSource to the markV variable, pattern matching by the object class. In this sense the DataSource can be thought as the equivalent of a Drools entry-point.

Conversely the expression
         
expr(markV, person -> person.getName().equals("Mark"))
uses a Predicate to define a condition that the object bound to the markV Variable has to satisfy in order to be successfully matched by the engine. Note that, as anticipated, the evaluation of the pattern matching is not performed by a constraint generated as a result of any sort of analysis or compilation process, but it's merely executed by applying the lambda expression implementing the predicate ( in this case, person -> person.getName().equals("Mark") ) to the object to be matched. In other terms the former DSL produces the executable model of a rule that is equivalent to the one resulting from the parsing of the following drl.
         
rule "Print age of persons named Mark"
when
    markV : Person( name == "Mark" ) from entry-point "persons"
then
    System.out.println(markV.getAge());
end
It is also under development a rete builder that can be fed with the rules defined with this DSL. In particular it is possible to add these rules to a CanonicalKieBase and then to create KieSessions from it as for any other normal KieBase.
         
CanonicalKieBase kieBase = new CanonicalKieBase();
kieBase.addRules(rule);

KieSession ksession = kieBase.newKieSession();
ksession.fireAllRules();
Of course the DSL also allows to define more complex conditions like joins:
         
Variable<Person> markV = bind(typeOf(Person.class));
Variable<Person> olderV = bind(typeOf(Person.class));

Rule rule = rule("Find persons older than Mark")
        .view(
            input(markV, () -> persons),
            input(olderV, () -> persons),
            expr(markV, mark -> mark.getName().equals("Mark")),
            expr(olderV, markV, (older, mark) -> older.getAge() > mark.getAge())
        )
        .then( 
            on(olderV, markV)
                .execute((p1, p2) -> System.out.println(p1.getName() + " is older than " + p2.getName())
        )
);
or existential patterns:
 
Variable<Person> oldestV = bind(typeOf(Person.class));
Variable<Person> otherV = bind(typeOf(Person.class));

Rule rule = rule("Find oldest person")
        .view(
            input(oldestV, () -> persons),
            input(otherV, () -> persons),
            not(otherV, oldestV, (p1, p2) -> p1.getAge() > p2.getAge())
        )
        .then( 
            on(oldestV)
                .execute(p -> System.out.println("Oldest person is " + p.getName())
        )
);
Here the not() stands for the negation of any expression, so the form used above is actually only a shortcut for
 
not( expr( otherV, oldestV, (p1, p2) -> p1.getAge() > p2.getAge() ) )
Also accumulate is already supported in the following form:
 
Variable<Person> person = bind(typeOf(Person.class));
Variable<Integer> resultSum = bind(typeOf(Integer.class));
Variable<Double> resultAvg = bind(typeOf(Double.class));

Rule rule = rule("Calculate sum and avg of all persons having a name starting with M")
        .view(
            input(person, () -> persons),
            accumulate(expr(person, p -> p.getName().startsWith("M")),
                       sum(Person::getAge).as(resultSum),
                       avg(Person::getAge).as(resultAvg))
        )
        .then(
            on(resultSum, resultAvg)
                .execute((sum, avg) -> result.value = "total = " + sum + "; average = " + avg)
);
To provide one last more complete use case, the executable model of the classical fire and alarm example can be defined with this DSL as it follows.
 
Variable<Room> room = any(Room.class);
Variable<Fire> fire = any(Fire.class);
Variable<Sprinkler> sprinkler = any(Sprinkler.class);
Variable<Alarm> alarm = any(Alarm.class);

Rule r1 = rule("When there is a fire turn on the sprinkler")
        .view(
            input(fire),
            input(sprinkler),
            expr(sprinkler, s -> !s.isOn()),
            expr(sprinkler, fire, (s, f) -> s.getRoom().equals(f.getRoom()))
        )
        .then(
            on(sprinkler)
                .execute(s -> {
                    System.out.println("Turn on the sprinkler for room " + s.getRoom().getName());
                    s.setOn(true);
                })
                .update(sprinkler, "on")
);

Rule r2 = rule("When the fire is gone turn off the sprinkler")
        .view(
            input(sprinkler),
            expr(sprinkler, Sprinkler::isOn),
            input(fire),
            not(fire, sprinkler, (f, s) -> f.getRoom().equals(s.getRoom()))
        )
        .then(
            on(sprinkler)
                .execute(s -> {
                    System.out.println("Turn off the sprinkler for room " + s.getRoom().getName());
                    s.setOn(false);
                })
                .update(sprinkler, "on")
);

Rule r3 = rule("Raise the alarm when we have one or more fires")
        .view(
            input(fire),
            exists(fire)
        )
        .then(
            execute(() -> System.out.println("Raise the alarm"))
                .insert(() -> new Alarm())
);

Rule r4 = rule("Lower the alarm when all the fires have gone")
        .view(
            input(fire),
            not(fire),
            input(alarm)
        )
        .then(
            execute(() -> System.out.println("Lower the alarm"))
                .delete(alarm)
);

Rule r5 = rule("Status output when things are ok")
        .view(
            input(alarm),
            not(alarm),
            input(sprinkler),
            not(sprinkler, Sprinkler::isOn)
        )
        .then(
            execute(() -> System.out.println("Everything is ok"))
);

CanonicalKieBase kieBase = new CanonicalKieBase();
kieBase.addRules(r1, r2, r3, r4, r5);

KieSession ksession = kieBase.newKieSession();

// phase 1
Room room1 = new Room("Room 1");
ksession.insert(room1);
FactHandle fireFact1 = ksession.insert(new Fire(room1));
ksession.fireAllRules();

// phase 2
Sprinkler sprinkler1 = new Sprinkler(room1);
ksession.insert(sprinkler1);
ksession.fireAllRules();

assertTrue(sprinkler1.isOn());

// phase 3
ksession.delete(fireFact1);
ksession.fireAllRules();
In this example it's possible to note a few more things:

  • Some repetitions are necessary to bind the parameters of an expression to the formal parameters of the lambda expression evaluating it. Hopefully it will be possible to overcome this issue using the -parameters compilation argument when this JDK bug will be resolved.
  • any(Room.class) is a shortcut for bind(typeOf(Room.class))
  • The inputs don't declare a DataSource. This is a shortcut to state that those objects come from a default empty DataSource (corresponding to the Drools default entry-point). In fact in this example the facts are programmatically inserted into the KieSession.
  • Using an input without providing any expression for that input is actually a shortcut for input(alarm), expr(alarm, a -> true)
  • In the same way an existential pattern without any condition like not(fire) is another shortcut for not( expr( fire, f -> true ) )
  • Java 8 syntax also allows to define a predicate as a method reference accessing a boolean property of a fact like in expr(sprinkler, Sprinkler::isOn)
  • The RHS, together with the block of code to be executed, also provides a fluent interface to define the working memory actions (inserts/updates/deletes) that have to be performed when the rule is fired. In particular the update also gets a varargs of Strings reporting the name of the properties changed in the updated fact like in update(sprinkler, "on"). Once again this information has to be explicitly provided because the executable model has to be created without the need of any code analysis.

Sunday, July 20, 2014

jBPM6 Developer Guide coming out soon!

Hello everyone. This post is just to let you know that jBPM6 Developer Guide is about to get published, and you can pre-order it from here and get from a 20% to a 37% discount on your order! With this book, you can learn how to:
  • Model and implement different business processes using the BPMN2 standard notation
  • Understand how and when to use the different tools provided by the JBoss Business Process Management (BPM) platform
  • Learn how to model complex business scenarios and environments through a step-by-step approach
Here you can find a list of what you will find in each chapter:  

Chapter 1, Why Do We Need Business Process Management?, introduces the BPM discipline. This chapter will provide the basis for the rest of the book, by providing an understanding of why and how the jBPM6 project has been designed, and the path its evolution will follow.  
Chapter 2, BPM Systems Structure, goes in depth into understanding what the main pieces and components inside a Business Process Management System (BPMS) are. This chapter introduces the concept of BPMS as the natural follow up of an understanding of the BPM discipline. The reader will find a deep and technical explanation about how a BPM system core can be built from scratch and how it will interact with the rest of the components in the BPMS infrastructure. This chapter also describes the intimate relationship between the Drools and jBPM projects, which is one of the key advantages of jBPM6 in comparison with all the other BPMSs, as well as existing methodologies where a BPMS connects with other systems.
Chapter 3, Using BPMN 2.0 to Model Business Scenarios, covers the main constructs used to model our business processes, guiding the reader through an example that illustrates the most useful modeling patterns. The BPMN 2.0 specification has become the de facto standard for modeling executable business processes since it was released in early 2011, and is recommended to any BPM implementation, even outside the scope of jBPM6.  
Chapter 4, Understanding the Knowledge Is Everything Workbench, takes a look into the tooling provided by the jBPM6 project, which will enable the reader to both define new processes and configure a runtime to execute those processes. The overall architecture of the tooling provided will be covered as well in this chapter.
Chapter 5, Creating a Process Project in the KIE Workbench, dives into the required steps to create a process definition with the existing tooling, as well as to test it and run it. The BPMN 2.0 specification will be put into practice as the reader creates an executable process and a compiled project where the runtime specifications will be defined.
Chapter 6, Human Interactions, covers in depth the Human Task component inside jBPM6. A big feature of BPMS is the capability to coordinate human and system interactions. It also describes how the existing tooling builds a user interface using the concepts of task lists and task forms, exposing the end users involved in the execution of multiple process definitions’ tasks to a common interface.
Chapter 7, Defining Your Environment with the Runtime Manager, covers the different strategies provided to configure an environment to run our processes. The reader will see the configurations for connecting external systems, human task components, persistence strategies and the relation a specific process execution will have with an environment, as well as methods to define their own custom runtime configuration.
Chapter 8, Implementing Persistence and Transactions, covers the shared mechanisms between the Drools and jBPM projects used to store information and define transaction boundaries. When we want to support processes that coordinate systems and people over long periods of time, we need to understand how the process information can be persisted.  
Chapter 9, Integration with other Knowledge Definitions, gives a brief introduction to the Drools Rule Engine. It is used to mix business processes with business rules, to define advanced and complex scenarios. Also, we cover Drools Fusion, and added feature of the Drools Rule Engine to add the ability of temporal reasoning, allowing business processes to be monitored, improved and covered by business scenarios that require temporal inferences.  
Chapter 10, KIE Workbench Integration with External Systems, describes the ways in which the provided tooling can be extended with extra features, along with a description of all the different extension points provided by the API and exposed by the tooling. A set of good practices is described in order to give the reader a comprehensive way to deal with different scenarios a BPMS will likely face.
Appendix A, The UberFire Framework, goes into detail about the based utility framework used by the KIE Workbench to define its user interface. The reader will learn the structure and use of the framework, along with a demonstration that will enable the extension of any component in the workbench distribution you choose. Hope you like it! Cheers,

Friday, July 18, 2014

Kie Uberfire Social Activities

The Uberfire Framework, has a new extension: Kie Uberfire Social Activities. In this initial version this Uberfire extension will provided an extensible architecture to capture, handle, and present (in a timeline style) configurable types of social events.


  • Basic Architecture
An event is any type of "CDI Event" and will be handled by their respective adapter. The adapter is a CDI Managed Bean, which implements SocialAdapter interface. The main responsibility of the adapter is to translate from a CDI event to a Social Event. This social event will be captured and persisted by Kie Uberfire Social Activities in their respectives timelines (basically user and type timeline). 

That is the basic architecture and workflow of this tech:

Basic Architecture


  • Timelines

There is many ways of interact and display a timeline. This session will briefly describe each one of them.

a-) Atom URL

Social Activities provides a custom URL for each event type. This url is accessible by: http://project/social/TYPE_NAME.



The users timeline works on the same way, being accessible by http://project/social-user/USER_NAME .

Another cool stuff is that an adapter can provide his pluggable url-filters. Implementing the method getTimelineFilters from SocialAdapter interface, he can do anything that he want with his timeline. This filters is accessible by a query parameter, i.e. http://project/social/TYPE_NAME?max-results=1 .


B-) Basic Widgets

Social Activities also includes some basic (extendable) widgets. There is two type of timelines widgets: simple and regular widgets.

Simple Widget

Regular Widget

The ">" symbol on 'Simple Widget' is a pagination component. You can configure it by an easy API. With an object SocialPaged( 2 ) you creates a pagination with 2 items size. This object helps you to customize your widgets using the methods canIGoBackward() and canIGoForward() to display icons, and  forward() and backward() to set the navigation direction.
The Social Activities component has an initial support for avatar. In case you provide an user e-mail for the API, the gravatar image will be displayed in this widgets.


C-) Drools Query API

Another way to interact with a timeline is throught the Social Timeline Drools Query API. This API executes one or more DRLs in a Timeline in all cached events. It's a great way to merge different types of timelines.



  • Followers/Following Social Users

A user can follow another social user.  When a user generates a social event, this event is replicated in all timelines of his followers. Social also provides a basic widget to follow another user, show all social users and display a user following list.


It is important to mention that the current implementation lists socials users through  a "small hack". We search the uberfire default git repository for branch names (each uberfire user has his own branch),  and extract the list of social users.

This hack is needed as we don’t have direct access of the user base (due the container based auth).



  • Persistence Architecture

The persistence architecture of Social Activities is build on two concepts: Local Cache and File Persistence. The local cache is a in memory cache that holds all recent social events. These events are kept only in this cache until the max events threshold is reached. The size of this threshold is configured by a system property org.uberfire.social.threshold (default value 100).

When the threshold is reached, the social persist the current cache into the file system (system.git repository - social branch). Inside this branch there is a social-files directory and this structure:



  • userNames: file that contains all social users name
  • each user has his own file (with his name), that contains a Json with user data.
  • a directory for each social type event .
  • a directory "USER_TIMELINE" that contains specific user timelines


Each directory keeps a file "LAST_FILE_INDEX" that point for the most recent timeline file.




Inside each file, there is a persisted list of Social Events in JSON format:

({"timestamp":"Jul16,2014,5:04:13PM","socialUser":{"name":"stress1","followersName":[],"followingName":[]},"type":"FOLLOW_USER","adicionalInfo":["follow stress2"]})

Separating each JSONs there is a HEX and the size in bytes of the JSON. The file is read by social in reverse order.

The METADATA file current hold only the number of social events on that file (used for pagination support).

It is important to mention that this whole structure is transparent to the widgets and pagination. All the file structure and respective cache are MERGED to compose a timeline.

  • Clustering
In case that your application is using Uberfire in a cluster environment, Kie Social Activities also supports distributed persistence. His cluster sync is build on top of UberfireCluster support (Apache Zookeeper and Apache Helix).


Each node broadcast social events to the cluster via a cluster message  SocialClusterMessage.NEW_EVENT containing Social Event data. With this message, all the nodes receive the event and can store it on their own local cache. In that point all nodes caches are consistent.
When a cache from a node reaches the threshold, it lock the filesystem to persist his cache on filesystem. Then the node sends a SOCIAL_FILE_SYSTEM_PERSISTENCE message to the cluster notifying all the nodes that the cache is persisted on filesystem.
If during this persistence process, any node receives a new event, this stale event is merged during this sync.

  • Stress Test and Performance

In my github account, there is an example Stress Test class used to test the performance of this project.  This class isn't imported to our official repository.

The results of that test, find out that Social Actitivies can write ~1000 events per second in my personal laptop (Mb Pro,  Intel Core i5 2.4 GHZ, 8Gb 1600MHz DDR3, SSD). In a single instance enviroment, it writes 10k events in 7s, writed 100k in 48s, and 500k events in 512s.
  • Demo
A sample project of this feature can be found at my GitHub account or you can just download and install the war of this demo. Please take a note that this repository moved from my account to our official uberfire extensions repository.

  • Roadmap
This is an early version of Kie Uberfire Social Activities. In the nexts versions we plan to provide:

  • A "Notification Center" tool, inspired by OSX notification tool; (far term)
  • Integrate this project with dashbuilder KPI's;(far term)
  • A purge tool, able to move old events from filesystem to another persistence store; (short term)
  • In this version, we only provide basic widgets. We need to create a way to allow to use customized templates on this widgets.(near term)
  • A dashboard to group multiple social widgets.(near term)

If you want start contributing to Open Source, this is a nice opportunity. Fell free to contact me!

Friday, June 27, 2014

Compiling GWT applications on Windows

If you're a developer using Microsoft Windows and you've ever developed a GWT application of any size you've probably encountered the command-line length limitation (http://support.microsoft.com/kb/830473).

The gwt-maven-plugin constructs a command line statement to invoke the GWT compiler containing a list of what can be a very extensive classpath declaration. The length of the command line statement can easily exceed the maximum supported by Microsoft Windows; leaving the developer unable to compile their GWT application without resorting to tricks such as mapping a drive to their local Maven repository thus shortening the classpath entries.

Hopefully this will soon become a thing of the past!

I've submitted a Pull Request to the gwt-maven-plugin project to provide a more concrete solution. With this patch the gwt-maven-plugin is able to compile GWT applications of any size on Microsoft Windows without developers needing to devise tricks.

Until the pull request is accepted and merged you can compile kie-drools-wb or kie-wb by fetching my fork of the gwt-maven-plugin and building it locally. No further changes are then required to compile kie-wb.

Happy hunting!


Friday, May 23, 2014

Running drools-wb with GWT's SuperDevMode

Like most, I like surprises!

Some surprises aren't always welcome though; and one such surprise bit me yesterday.

As a good citizen I upgraded my installation of Google Chrome when advised a new version was available. With hind-sight I don't know why I so gleefully went along with the upgrade (after all, I'd recently removed the latest version from my mobile telephone as it didn't "feel" as good... anyway I digress).

The surprise was that Chrome 35 stops supporting GWT's "DevMode" (something I'd long been used to with FireFox) and as from GWT 2.6.0 support for "DevMode" is to come to an end ("GWT Development Mode will no longer be available for Chrome sometime in 2014, so we improved alternate ways of debugging. There are improvements to Super Dev Mode, asserts, console logging, and error messages.")

Options were to find an installation of Chrome 34, or switch to SuperDevMode (that seems inevitable). Electing for the latter, I present my findings on how to configure your webapp, IDE and run (or debug) it in "SuperDevMode".

These instructions are for IDEA (NetBeans will probably follow a similar route).

(1) Create a regular GWT Launcher:


(2) Create a new GWT Launcher for SuperDevMode:



(3) Add the following to your webapp's gwt.xml (module) file:

  <!-- SuperDevMode -->
  <add-linker name="xsiframe"/>
  <set-configuration-property name="devModeRedirectEnabled" value="true"/>
  <set-property name="compiler.useSourceMaps" value="true"/>
  <set-configuration-property name='xsiframe.failIfScriptTag' value='false'/>

(4) Launch your regular webapp (the "classic" GWT Launcher):

... <tick>, <tock>, <tick>, <tock> while it compiles and launches...


(5) Launch the SuperDevMode code server (the "SuperDevMode" GWT Launcher):

... <tick>, <tock>, <tick>, <tock> while it compiles and launches...


​(6) Drag the "Dev Mode On" and "Dev Mode Off" buttons to your bookmark bar (as advised) - but we don't normally read these sort of things, right! ;)

(7) Go back to the webapp's browser tab

(8) Click on the "Dev Mode On" bookmark you created in step (6)



(9) Click on "compile"



(10) Delete the "codesvr" part of the URL and press enter (dismiss the popups that appear; which ones depends on what browser your GWT module targets; e.g. I had to dismiss a popup about using Chrome but the GWT model targets FireFox).



​(11) Done!



(12) What's that? You want to debug your application?!?

This isn't too bad. Just launch both your "classic" GWT Launcher in debug mode and the "SuperDevMode" GWT Launcher in normal mode.

Server-side code needs break-points in IDEA, and client-side break-points need to be added using Chrome's Developer Tools (you'll need to make sure "sourceMaps" are enabled, but this appears to be the default in Chrome 35).

Accessing Chrome's debugger:



Debugging:



Simple!

It takes a bit of getting used to debugging client-side stuff in Chrome, and server-side stuff in IDEA, but it's not terrible (although don't expect to be able to introspect everything in Chrome like you used to in IDEA).

I hope this helps (probably more so as others find "DevMode" stops working for them.... and when we move to GWT 2.6.1 --- for IE10 support --- so it is coming sooner than later).

Have fun!

Mike

Thursday, May 22, 2014

London (May 26th) Drools & jBPM community contributor meeting

London, Chiswick, May 26th to May 30th

During next week a large percentage of the Drools team, some of the jBPM team and some community members will be meeting in London (Chiswick). There won’t be any presentations, we’ll just be in a room hacking, designing, exchanging ideas and planing. This is open to community members who wish to contribute towards Drools or jBPM, and want help with those contributions. This also includes people working on open source or academic projects that utilise Drools or jBPM. Email me if you want to attend, our locations may very (but within chiswick) each day. 

We will not be able to make the day time available to people looking for general Drools or jBPM guidance (unless you want to buy us all lunch). But we will be organising evenings things (like bowling) and could make wed or thu evening open to people wanting general chats and advice. Email me if you’re interested, and after discussing with the team, I’ll let you know.

Those currently attending:
Mark Proctor (mon-fri) Group architect
Edson Tirelli (mon-fri) Drools backend, and project lead
Mario Fusco (mon-fri) Drools backend
Davide Sottara (wed-fri) Drools backend
Alex Porcelli (mon-fri) Drools UI
Michael Anstis (thu-fri) Drools UI
Kris Verlaenen (wed-thu) jBPM backend, and project lead
Mauricio Salatino (mon-fri) jBPM tasks and general UI

Wednesday, May 07, 2014

Drools - Bayesian Belief Network Integration Part 3


This follows my earlier Part 2 posting in April,

Things now work end to end, and I have a clean separation from the Creation of the JunctionTree and initialisation of all state, and the state that change after evidence insertion. This separation ensures that multiple instances of the same bayesian network can be created cheaply.

I'm now working on integrating this into the belief system. One issue I have is that I can automate the update of the bayesian network as soon as the evidence changes. The reason for this is updating of the  network is expensive, if you insert three pieces of evidence, you only want it to update one not three times. So for now I will add a dirty check, and allow users to call update. For best practice I will recommend people separate reasoning of the results of bayesian and entering new evidence, so that it becomes clearer when it's efficient to call update.

For now I'm only dealing with hard evidence. We will be using superiority rules to resolve conflict evidence for a variable. Any unresolved conflicts will leave a variable marked as "Undecided". Handling of soft or virtual evidence would be nice, this would add way to resolve conflicted evidence statistically; but for now this is out of scope. There is a paper here on who to do it, if anyone wants to help me :)

I'll be committing this to github in a few days, for now if anyone is interested,  here is the jar in a zip form from dropbox.

--update--
The XMLBIF parser provided by Horacio Antar is now integrated and tested. I'm just working on refactoring Drools for pluggable knowledge types, to fully integrate Bayesian as a new type of knowledge.

Graph<BayesVariable> graph = new BayesNetwork();

GraphNode<BayesVariable> burglaryNode   = graph.addNode();
GraphNode<BayesVariable> earthquakeNode = graph.addNode();
GraphNode<BayesVariable> alarmNode      = graph.addNode();
GraphNode<BayesVariable> johnCallsNode  = graph.addNode();
GraphNode<BayesVariable> maryCallsNode  = graph.addNode();

BayesVariable burglary   = new BayesVariable<String>("Burglary", burglaryNode.getId(), new String[]{"true", "false"}, new double[][]{{0.001, 0.999}});
BayesVariable earthquake = new BayesVariable<String>("Earthquake", earthquakeNode.getId(), new String[]{"true", "false"}, new double[][]{{0.002, 0.998}});
BayesVariable alarm      = new BayesVariable<String>("Alarm", alarmNode.getId(), new String[]{"true", "false"}, new double[][]{{0.95, 0.05}, {0.94, 0.06}, {0.29, 0.71}, {0.001, 0.999}});
BayesVariable johnCalls  = new BayesVariable<String>("JohnCalls", johnCallsNode.getId(), new String[]{"true", "false"}, new double[][]{{0.90, 0.1}, {0.05, 0.95}});
BayesVariable maryCalls  = new BayesVariable<String>("MaryCalls", maryCallsNode.getId(), new String[]{"true", "false"}, new double[][]{{0.7, 0.3}, {0.01, 0.99}});

BayesVariableState burglaryState;
BayesVariableState earthquakeState;
BayesVariableState alarmState;
BayesVariableState johnCallsState;
BayesVariableState maryCallsState;

JunctionTreeNode jtNode1;
JunctionTreeNode jtNode2;
JunctionTreeNode jtNode3;

JunctionTree jTree;

BayesEngine engine;

@Before
public void setUp() {
    connectParentToChildren(burglaryNode, alarmNode);
    connectParentToChildren(earthquakeNode, alarmNode);
    connectParentToChildren(alarmNode, johnCallsNode, maryCallsNode);

    burglaryNode.setContent(burglary);
    earthquakeNode.setContent(earthquake);
    alarmNode.setContent(alarm);
    johnCallsNode.setContent(johnCalls);
    maryCallsNode.setContent(maryCalls);

    JunctionTreeBuilder jtBuilder = new JunctionTreeBuilder(graph);
    jTree = jtBuilder.build();
    jTree.initialize();

    jtNode1 = jTree.getRoot();
    jtNode2 = jtNode1.getChildren().get(0).getChild();
    jtNode3 = jtNode1.getChildren().get(1).getChild();

    engine = new BayesEngine(jTree);

    burglaryState = engine.getVarStates()[burglary.getId()];
    earthquakeState = engine.getVarStates()[earthquake.getId()];
    alarmState = engine.getVarStates()[alarm.getId()];
    johnCallsState = engine.getVarStates()[johnCalls.getId()];
    maryCallsState = engine.getVarStates()[maryCalls.getId()];
}

@Test
public void testInitialize() {
    // johnCalls
    assertArray(new double[]{0.90, 0.1, 0.05, 0.95}, scaleDouble( 3, jtNode1.getPotentials() ));


    // maryCalls
    assertArray( new double[]{ 0.7, 0.3, 0.01, 0.99 }, scaleDouble( 3, jtNode2.getPotentials() ));

    // burglary, earthquake, alarm
    assertArray( new double[]{0.0000019, 0.0000001, 0.0009381, 0.0000599, 0.0005794, 0.0014186, 0.0009970, 0.9960050 },
                 scaleDouble( 7, jtNode3.getPotentials() ));
}

@Test
public void testNoEvidence() {
    engine.globalUpdate();

    assertArray( new double[]{0.052139, 0.947861},  scaleDouble(6, engine.marginalize("JohnCalls").getDistribution()) );

    assertArray( new double[]{0.011736, 0.988264 },  scaleDouble( 6, engine.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.001, 0.999},  scaleDouble(3, engine.marginalize("Burglary").getDistribution()) );

    assertArray( new double[]{ 0.002, 0.998},  scaleDouble( 3, engine.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.002516, 0.997484},   scaleDouble(6, engine.marginalize("Alarm").getDistribution()) );
}

@Test
public void testAlarmEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode3,  alarmNode, new double[] { 1.0, 0.0 }) );

    nue.globalUpdate();

    assertArray( new double[]{0.9, 0.1}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.7, 0.3 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.374, 0.626}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{ 0.231, 0.769}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) ); }

@Test
public void testEathQuakeEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood(new BayesLikelyhood(graph, jtNode3, earthquakeNode, new double[]{1.0, 0.0}));
    nue.globalUpdate();

    assertArray( new double[]{0.297, 0.703}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.211, 0.789 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{.001, 0.999}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.291, 0.709}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}

@Test
public void testJoinCallsEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode1,  johnCallsNode, new double[] { 1.0, 0.0 }) );
    nue.globalUpdate();

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.04, 0.96 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.016, 0.984}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{0.011, 0.989}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.043, 0.957}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}

@Test
public void testEarthquakeAndJohnCallsEvidence() {
    BayesEngine nue = new BayesEngine(jTree);
    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode1,  johnCallsNode, new double[] { 1.0, 0.0 }) );

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode3,  earthquakeNode, new double[] { 1.0, 0.0 }) );
    nue.globalUpdate();

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.618, 0.382 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.003, 0.997}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{ 1.0, 0.0}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.881, 0.119}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}