Monday, July 21, 2014

Drools Executable Model (Rules in pure Java)

The Executable Model is a re-design of the Drools lowest level model handled by the engine. In the current series (up to 6.x) the executable model has grown organically over the last 8 years, and was never really intended to be targeted by end users. Those wishing to programmatically write rules were advised to do it via code generation and target drl; which was no ideal. There was never any drive to make this more accessible to end users, because extensive use of anonymous classes in Java was unwieldy. With Java 8 and Lambda's this changes, and the opportunity to make a more compelling model that is accessible to end users becomes possible.

This new model is generated during the compilation process of higher level languages, but can also be used on its own. The goal is for this Executable Model to be self contained and avoid the need for any further byte code munging (analysis, transformation or generation); From this model's perspective, everything is provided either by the code or by higher level language layers. For example indexes etc must be provided by arguments, which the higher level language generates through analysis, when it targets the Executable model.
   
It is designed to map well to a Fluent level builders, leveraging Java 8's lambdas. This will make it more appealing to java developers, and language developers. Also this will allow low level engine feature design and testing, independent of any language. Which means we can innovate at an engine level, without having to worry about the language layer.
   
The Executable Model should be generic enough to map into multiple domains. It will be a low level dataflow model in which you can address functional reactive programming models, but still usable to build a rule based system out of it too.

The following example provides a first view of the fluent DSL used to build the executable model
         
DataSource persons = sourceOf(new Person("Mark", 37),
                              new Person("Edson", 35),
                              new Person("Mario", 40));
                     
Variable<Person> markV = bind(typeOf(Person.class));

Rule rule = rule("Print age of persons named Mark")
        .view(
            input(markV, () -> persons),
            expr(markV, person -> person.getName().equals("Mark"))
        )
        .then(
            on(markV).execute(mark -> System.out.println(mark.getAge())
        )
);

The previous code defines a DataSource containing a few person instances and declares the Variable markV of type Person. The rule itself contains the usual two parts: the LHS is defined by the set of inputs and expressions passed to the view() method, while the RHS is the action defined by the lambda expression passed to the then() method.

Analyzing the LHS in more detail, the statement
         
input(markV, () -> persons)
binds the objects from the persons DataSource to the markV variable, pattern matching by the object class. In this sense the DataSource can be thought as the equivalent of a Drools entry-point.

Conversely the expression
         
expr(markV, person -> person.getName().equals("Mark"))
uses a Predicate to define a condition that the object bound to the markV Variable has to satisfy in order to be successfully matched by the engine. Note that, as anticipated, the evaluation of the pattern matching is not performed by a constraint generated as a result of any sort of analysis or compilation process, but it's merely executed by applying the lambda expression implementing the predicate ( in this case, person -> person.getName().equals("Mark") ) to the object to be matched. In other terms the former DSL produces the executable model of a rule that is equivalent to the one resulting from the parsing of the following drl.
         
rule "Print age of persons named Mark"
when
    markV : Person( name == "Mark" ) from entry-point "persons"
then
    System.out.println(markV.getAge());
end
It is also under development a rete builder that can be fed with the rules defined with this DSL. In particular it is possible to add these rules to a CanonicalKieBase and then to create KieSessions from it as for any other normal KieBase.
         
CanonicalKieBase kieBase = new CanonicalKieBase();
kieBase.addRules(rule);

KieSession ksession = kieBase.newKieSession();
ksession.fireAllRules();
Of course the DSL also allows to define more complex conditions like joins:
         
Variable<Person> markV = bind(typeOf(Person.class));
Variable<Person> olderV = bind(typeOf(Person.class));

Rule rule = rule("Find persons older than Mark")
        .view(
            input(markV, () -> persons),
            input(olderV, () -> persons),
            expr(markV, mark -> mark.getName().equals("Mark")),
            expr(olderV, markV, (older, mark) -> older.getAge() > mark.getAge())
        )
        .then( 
            on(olderV, markV)
                .execute((p1, p2) -> System.out.println(p1.getName() + " is older than " + p2.getName())
        )
);
or existential patterns:
 
Variable<Person> oldestV = bind(typeOf(Person.class));
Variable<Person> otherV = bind(typeOf(Person.class));

Rule rule = rule("Find oldest person")
        .view(
            input(oldestV, () -> persons),
            input(otherV, () -> persons),
            not(otherV, oldestV, (p1, p2) -> p1.getAge() > p2.getAge())
        )
        .then( 
            on(oldestV)
                .execute(p -> System.out.println("Oldest person is " + p.getName())
        )
);
Here the not() stands for the negation of any expression, so the form used above is actually only a shortcut for
 
not( expr( otherV, oldestV, (p1, p2) -> p1.getAge() > p2.getAge() ) )
Also accumulate is already supported in the following form:
 
Variable<Person> person = bind(typeOf(Person.class));
Variable<Integer> resultSum = bind(typeOf(Integer.class));
Variable<Double> resultAvg = bind(typeOf(Double.class));

Rule rule = rule("Calculate sum and avg of all persons having a name starting with M")
        .view(
            input(person, () -> persons),
            accumulate(expr(person, p -> p.getName().startsWith("M")),
                       sum(Person::getAge).as(resultSum),
                       avg(Person::getAge).as(resultAvg))
        )
        .then(
            on(resultSum, resultAvg)
                .execute((sum, avg) -> result.value = "total = " + sum + "; average = " + avg)
);
To provide one last more complete use case, the executable model of the classical fire and alarm example can be defined with this DSL as it follows.
 
Variable<Room> room = any(Room.class);
Variable<Fire> fire = any(Fire.class);
Variable<Sprinkler> sprinkler = any(Sprinkler.class);
Variable<Alarm> alarm = any(Alarm.class);

Rule r1 = rule("When there is a fire turn on the sprinkler")
        .view(
            input(fire),
            input(sprinkler),
            expr(sprinkler, s -> !s.isOn()),
            expr(sprinkler, fire, (s, f) -> s.getRoom().equals(f.getRoom()))
        )
        .then(
            on(sprinkler)
                .execute(s -> {
                    System.out.println("Turn on the sprinkler for room " + s.getRoom().getName());
                    s.setOn(true);
                })
                .update(sprinkler, "on")
);

Rule r2 = rule("When the fire is gone turn off the sprinkler")
        .view(
            input(sprinkler),
            expr(sprinkler, Sprinkler::isOn),
            input(fire),
            not(fire, sprinkler, (f, s) -> f.getRoom().equals(s.getRoom()))
        )
        .then(
            on(sprinkler)
                .execute(s -> {
                    System.out.println("Turn off the sprinkler for room " + s.getRoom().getName());
                    s.setOn(false);
                })
                .update(sprinkler, "on")
);

Rule r3 = rule("Raise the alarm when we have one or more fires")
        .view(
            input(fire),
            exists(fire)
        )
        .then(
            execute(() -> System.out.println("Raise the alarm"))
                .insert(() -> new Alarm())
);

Rule r4 = rule("Lower the alarm when all the fires have gone")
        .view(
            input(fire),
            not(fire),
            input(alarm)
        )
        .then(
            execute(() -> System.out.println("Lower the alarm"))
                .delete(alarm)
);

Rule r5 = rule("Status output when things are ok")
        .view(
            input(alarm),
            not(alarm),
            input(sprinkler),
            not(sprinkler, Sprinkler::isOn)
        )
        .then(
            execute(() -> System.out.println("Everything is ok"))
);

CanonicalKieBase kieBase = new CanonicalKieBase();
kieBase.addRules(r1, r2, r3, r4, r5);

KieSession ksession = kieBase.newKieSession();

// phase 1
Room room1 = new Room("Room 1");
ksession.insert(room1);
FactHandle fireFact1 = ksession.insert(new Fire(room1));
ksession.fireAllRules();

// phase 2
Sprinkler sprinkler1 = new Sprinkler(room1);
ksession.insert(sprinkler1);
ksession.fireAllRules();

assertTrue(sprinkler1.isOn());

// phase 3
ksession.delete(fireFact1);
ksession.fireAllRules();
In this example it's possible to note a few more things:

  • Some repetitions are necessary to bind the parameters of an expression to the formal parameters of the lambda expression evaluating it. Hopefully it will be possible to overcome this issue using the -parameters compilation argument when this JDK bug will be resolved.
  • any(Room.class) is a shortcut for bind(typeOf(Room.class))
  • The inputs don't declare a DataSource. This is a shortcut to state that those objects come from a default empty DataSource (corresponding to the Drools default entry-point). In fact in this example the facts are programmatically inserted into the KieSession.
  • Using an input without providing any expression for that input is actually a shortcut for input(alarm), expr(alarm, a -> true)
  • In the same way an existential pattern without any condition like not(fire) is another shortcut for not( expr( fire, f -> true ) )
  • Java 8 syntax also allows to define a predicate as a method reference accessing a boolean property of a fact like in expr(sprinkler, Sprinkler::isOn)
  • The RHS, together with the block of code to be executed, also provides a fluent interface to define the working memory actions (inserts/updates/deletes) that have to be performed when the rule is fired. In particular the update also gets a varargs of Strings reporting the name of the properties changed in the updated fact like in update(sprinkler, "on"). Once again this information has to be explicitly provided because the executable model has to be created without the need of any code analysis.

Sunday, July 20, 2014

jBPM6 Developer Guide coming out soon!

Hello everyone. This post is just to let you know that jBPM6 Developer Guide is about to get published, and you can pre-order it from here and get from a 20% to a 37% discount on your order! With this book, you can learn how to:
  • Model and implement different business processes using the BPMN2 standard notation
  • Understand how and when to use the different tools provided by the JBoss Business Process Management (BPM) platform
  • Learn how to model complex business scenarios and environments through a step-by-step approach
Here you can find a list of what you will find in each chapter:  

Chapter 1, Why Do We Need Business Process Management?, introduces the BPM discipline. This chapter will provide the basis for the rest of the book, by providing an understanding of why and how the jBPM6 project has been designed, and the path its evolution will follow.  
Chapter 2, BPM Systems Structure, goes in depth into understanding what the main pieces and components inside a Business Process Management System (BPMS) are. This chapter introduces the concept of BPMS as the natural follow up of an understanding of the BPM discipline. The reader will find a deep and technical explanation about how a BPM system core can be built from scratch and how it will interact with the rest of the components in the BPMS infrastructure. This chapter also describes the intimate relationship between the Drools and jBPM projects, which is one of the key advantages of jBPM6 in comparison with all the other BPMSs, as well as existing methodologies where a BPMS connects with other systems.
Chapter 3, Using BPMN 2.0 to Model Business Scenarios, covers the main constructs used to model our business processes, guiding the reader through an example that illustrates the most useful modeling patterns. The BPMN 2.0 specification has become the de facto standard for modeling executable business processes since it was released in early 2011, and is recommended to any BPM implementation, even outside the scope of jBPM6.  
Chapter 4, Understanding the Knowledge Is Everything Workbench, takes a look into the tooling provided by the jBPM6 project, which will enable the reader to both define new processes and configure a runtime to execute those processes. The overall architecture of the tooling provided will be covered as well in this chapter.
Chapter 5, Creating a Process Project in the KIE Workbench, dives into the required steps to create a process definition with the existing tooling, as well as to test it and run it. The BPMN 2.0 specification will be put into practice as the reader creates an executable process and a compiled project where the runtime specifications will be defined.
Chapter 6, Human Interactions, covers in depth the Human Task component inside jBPM6. A big feature of BPMS is the capability to coordinate human and system interactions. It also describes how the existing tooling builds a user interface using the concepts of task lists and task forms, exposing the end users involved in the execution of multiple process definitions’ tasks to a common interface.
Chapter 7, Defining Your Environment with the Runtime Manager, covers the different strategies provided to configure an environment to run our processes. The reader will see the configurations for connecting external systems, human task components, persistence strategies and the relation a specific process execution will have with an environment, as well as methods to define their own custom runtime configuration.
Chapter 8, Implementing Persistence and Transactions, covers the shared mechanisms between the Drools and jBPM projects used to store information and define transaction boundaries. When we want to support processes that coordinate systems and people over long periods of time, we need to understand how the process information can be persisted.  
Chapter 9, Integration with other Knowledge Definitions, gives a brief introduction to the Drools Rule Engine. It is used to mix business processes with business rules, to define advanced and complex scenarios. Also, we cover Drools Fusion, and added feature of the Drools Rule Engine to add the ability of temporal reasoning, allowing business processes to be monitored, improved and covered by business scenarios that require temporal inferences.  
Chapter 10, KIE Workbench Integration with External Systems, describes the ways in which the provided tooling can be extended with extra features, along with a description of all the different extension points provided by the API and exposed by the tooling. A set of good practices is described in order to give the reader a comprehensive way to deal with different scenarios a BPMS will likely face.
Appendix A, The UberFire Framework, goes into detail about the based utility framework used by the KIE Workbench to define its user interface. The reader will learn the structure and use of the framework, along with a demonstration that will enable the extension of any component in the workbench distribution you choose. Hope you like it! Cheers,

Friday, July 18, 2014

Kie Uberfire Social Activities

The Uberfire Framework, has a new extension: Kie Uberfire Social Activities. In this initial version this Uberfire extension will provided an extensible architecture to capture, handle, and present (in a timeline style) configurable types of social events.


  • Basic Architecture
An event is any type of "CDI Event" and will be handled by their respective adapter. The adapter is a CDI Managed Bean, which implements SocialAdapter interface. The main responsibility of the adapter is to translate from a CDI event to a Social Event. This social event will be captured and persisted by Kie Uberfire Social Activities in their respectives timelines (basically user and type timeline). 

That is the basic architecture and workflow of this tech:

Basic Architecture


  • Timelines

There is many ways of interact and display a timeline. This session will briefly describe each one of them.

a-) Atom URL

Social Activities provides a custom URL for each event type. This url is accessible by: http://project/social/TYPE_NAME.



The users timeline works on the same way, being accessible by http://project/social-user/USER_NAME .

Another cool stuff is that an adapter can provide his pluggable url-filters. Implementing the method getTimelineFilters from SocialAdapter interface, he can do anything that he want with his timeline. This filters is accessible by a query parameter, i.e. http://project/social/TYPE_NAME?max-results=1 .


B-) Basic Widgets

Social Activities also includes some basic (extendable) widgets. There is two type of timelines widgets: simple and regular widgets.

Simple Widget

Regular Widget

The ">" symbol on 'Simple Widget' is a pagination component. You can configure it by an easy API. With an object SocialPaged( 2 ) you creates a pagination with 2 items size. This object helps you to customize your widgets using the methods canIGoBackward() and canIGoForward() to display icons, and  forward() and backward() to set the navigation direction.
The Social Activities component has an initial support for avatar. In case you provide an user e-mail for the API, the gravatar image will be displayed in this widgets.


C-) Drools Query API

Another way to interact with a timeline is throught the Social Timeline Drools Query API. This API executes one or more DRLs in a Timeline in all cached events. It's a great way to merge different types of timelines.



  • Followers/Following Social Users

A user can follow another social user.  When a user generates a social event, this event is replicated in all timelines of his followers. Social also provides a basic widget to follow another user, show all social users and display a user following list.


It is important to mention that the current implementation lists socials users through  a "small hack". We search the uberfire default git repository for branch names (each uberfire user has his own branch),  and extract the list of social users.

This hack is needed as we don’t have direct access of the user base (due the container based auth).



  • Persistence Architecture

The persistence architecture of Social Activities is build on two concepts: Local Cache and File Persistence. The local cache is a in memory cache that holds all recent social events. These events are kept only in this cache until the max events threshold is reached. The size of this threshold is configured by a system property org.uberfire.social.threshold (default value 100).

When the threshold is reached, the social persist the current cache into the file system (system.git repository - social branch). Inside this branch there is a social-files directory and this structure:



  • userNames: file that contains all social users name
  • each user has his own file (with his name), that contains a Json with user data.
  • a directory for each social type event .
  • a directory "USER_TIMELINE" that contains specific user timelines


Each directory keeps a file "LAST_FILE_INDEX" that point for the most recent timeline file.




Inside each file, there is a persisted list of Social Events in JSON format:

({"timestamp":"Jul16,2014,5:04:13PM","socialUser":{"name":"stress1","followersName":[],"followingName":[]},"type":"FOLLOW_USER","adicionalInfo":["follow stress2"]})

Separating each JSONs there is a HEX and the size in bytes of the JSON. The file is read by social in reverse order.

The METADATA file current hold only the number of social events on that file (used for pagination support).

It is important to mention that this whole structure is transparent to the widgets and pagination. All the file structure and respective cache are MERGED to compose a timeline.

  • Clustering
In case that your application is using Uberfire in a cluster environment, Kie Social Activities also supports distributed persistence. His cluster sync is build on top of UberfireCluster support (Apache Zookeeper and Apache Helix).


Each node broadcast social events to the cluster via a cluster message  SocialClusterMessage.NEW_EVENT containing Social Event data. With this message, all the nodes receive the event and can store it on their own local cache. In that point all nodes caches are consistent.
When a cache from a node reaches the threshold, it lock the filesystem to persist his cache on filesystem. Then the node sends a SOCIAL_FILE_SYSTEM_PERSISTENCE message to the cluster notifying all the nodes that the cache is persisted on filesystem.
If during this persistence process, any node receives a new event, this stale event is merged during this sync.

  • Stress Test and Performance

In my github account, there is an example Stress Test class used to test the performance of this project.  This class isn't imported to our official repository.

The results of that test, find out that Social Actitivies can write ~1000 events per second in my personal laptop (Mb Pro,  Intel Core i5 2.4 GHZ, 8Gb 1600MHz DDR3, SSD). In a single instance enviroment, it writes 10k events in 7s, writed 100k in 48s, and 500k events in 512s.
  • Demo
A sample project of this feature can be found at my GitHub account or you can just download and install the war of this demo. Please take a note that this repository moved from my account to our official uberfire extensions repository.

  • Roadmap
This is an early version of Kie Uberfire Social Activities. In the nexts versions we plan to provide:

  • A "Notification Center" tool, inspired by OSX notification tool; (far term)
  • Integrate this project with dashbuilder KPI's;(far term)
  • A purge tool, able to move old events from filesystem to another persistence store; (short term)
  • In this version, we only provide basic widgets. We need to create a way to allow to use customized templates on this widgets.(near term)
  • A dashboard to group multiple social widgets.(near term)

If you want start contributing to Open Source, this is a nice opportunity. Fell free to contact me!

Friday, June 27, 2014

Compiling GWT applications on Windows

If you're a developer using Microsoft Windows and you've ever developed a GWT application of any size you've probably encountered the command-line length limitation (http://support.microsoft.com/kb/830473).

The gwt-maven-plugin constructs a command line statement to invoke the GWT compiler containing a list of what can be a very extensive classpath declaration. The length of the command line statement can easily exceed the maximum supported by Microsoft Windows; leaving the developer unable to compile their GWT application without resorting to tricks such as mapping a drive to their local Maven repository thus shortening the classpath entries.

Hopefully this will soon become a thing of the past!

I've submitted a Pull Request to the gwt-maven-plugin project to provide a more concrete solution. With this patch the gwt-maven-plugin is able to compile GWT applications of any size on Microsoft Windows without developers needing to devise tricks.

Until the pull request is accepted and merged you can compile kie-drools-wb or kie-wb by fetching my fork of the gwt-maven-plugin and building it locally. No further changes are then required to compile kie-wb.

Happy hunting!


Friday, May 23, 2014

Running drools-wb with GWT's SuperDevMode

Like most, I like surprises!

Some surprises aren't always welcome though; and one such surprise bit me yesterday.

As a good citizen I upgraded my installation of Google Chrome when advised a new version was available. With hind-sight I don't know why I so gleefully went along with the upgrade (after all, I'd recently removed the latest version from my mobile telephone as it didn't "feel" as good... anyway I digress).

The surprise was that Chrome 35 stops supporting GWT's "DevMode" (something I'd long been used to with FireFox) and as from GWT 2.6.0 support for "DevMode" is to come to an end ("GWT Development Mode will no longer be available for Chrome sometime in 2014, so we improved alternate ways of debugging. There are improvements to Super Dev Mode, asserts, console logging, and error messages.")

Options were to find an installation of Chrome 34, or switch to SuperDevMode (that seems inevitable). Electing for the latter, I present my findings on how to configure your webapp, IDE and run (or debug) it in "SuperDevMode".

These instructions are for IDEA (NetBeans will probably follow a similar route).

(1) Create a regular GWT Launcher:


(2) Create a new GWT Launcher for SuperDevMode:



(3) Add the following to your webapp's gwt.xml (module) file:

  <!-- SuperDevMode -->
  <add-linker name="xsiframe"/>
  <set-configuration-property name="devModeRedirectEnabled" value="true"/>
  <set-property name="compiler.useSourceMaps" value="true"/>
  <set-configuration-property name='xsiframe.failIfScriptTag' value='false'/>

(4) Launch your regular webapp (the "classic" GWT Launcher):

... <tick>, <tock>, <tick>, <tock> while it compiles and launches...


(5) Launch the SuperDevMode code server (the "SuperDevMode" GWT Launcher):

... <tick>, <tock>, <tick>, <tock> while it compiles and launches...


​(6) Drag the "Dev Mode On" and "Dev Mode Off" buttons to your bookmark bar (as advised) - but we don't normally read these sort of things, right! ;)

(7) Go back to the webapp's browser tab

(8) Click on the "Dev Mode On" bookmark you created in step (6)



(9) Click on "compile"



(10) Delete the "codesvr" part of the URL and press enter (dismiss the popups that appear; which ones depends on what browser your GWT module targets; e.g. I had to dismiss a popup about using Chrome but the GWT model targets FireFox).



​(11) Done!



(12) What's that? You want to debug your application?!?

This isn't too bad. Just launch both your "classic" GWT Launcher in debug mode and the "SuperDevMode" GWT Launcher in normal mode.

Server-side code needs break-points in IDEA, and client-side break-points need to be added using Chrome's Developer Tools (you'll need to make sure "sourceMaps" are enabled, but this appears to be the default in Chrome 35).

Accessing Chrome's debugger:



Debugging:



Simple!

It takes a bit of getting used to debugging client-side stuff in Chrome, and server-side stuff in IDEA, but it's not terrible (although don't expect to be able to introspect everything in Chrome like you used to in IDEA).

I hope this helps (probably more so as others find "DevMode" stops working for them.... and when we move to GWT 2.6.1 --- for IE10 support --- so it is coming sooner than later).

Have fun!

Mike

Thursday, May 22, 2014

London (May 26th) Drools & jBPM community contributor meeting

London, Chiswick, May 26th to May 30th

During next week a large percentage of the Drools team, some of the jBPM team and some community members will be meeting in London (Chiswick). There won’t be any presentations, we’ll just be in a room hacking, designing, exchanging ideas and planing. This is open to community members who wish to contribute towards Drools or jBPM, and want help with those contributions. This also includes people working on open source or academic projects that utilise Drools or jBPM. Email me if you want to attend, our locations may very (but within chiswick) each day. 

We will not be able to make the day time available to people looking for general Drools or jBPM guidance (unless you want to buy us all lunch). But we will be organising evenings things (like bowling) and could make wed or thu evening open to people wanting general chats and advice. Email me if you’re interested, and after discussing with the team, I’ll let you know.

Those currently attending:
Mark Proctor (mon-fri) Group architect
Edson Tirelli (mon-fri) Drools backend, and project lead
Mario Fusco (mon-fri) Drools backend
Davide Sottara (wed-fri) Drools backend
Alex Porcelli (mon-fri) Drools UI
Michael Anstis (thu-fri) Drools UI
Kris Verlaenen (wed-thu) jBPM backend, and project lead
Mauricio Salatino (mon-fri) jBPM tasks and general UI

Wednesday, May 07, 2014

Drools - Bayesian Belief Network Integration Part 3


This follows my earlier Part 2 posting in April,

Things now work end to end, and I have a clean separation from the Creation of the JunctionTree and initialisation of all state, and the state that change after evidence insertion. This separation ensures that multiple instances of the same bayesian network can be created cheaply.

I'm now working on integrating this into the belief system. One issue I have is that I can automate the update of the bayesian network as soon as the evidence changes. The reason for this is updating of the  network is expensive, if you insert three pieces of evidence, you only want it to update one not three times. So for now I will add a dirty check, and allow users to call update. For best practice I will recommend people separate reasoning of the results of bayesian and entering new evidence, so that it becomes clearer when it's efficient to call update.

For now I'm only dealing with hard evidence. We will be using superiority rules to resolve conflict evidence for a variable. Any unresolved conflicts will leave a variable marked as "Undecided". Handling of soft or virtual evidence would be nice, this would add way to resolve conflicted evidence statistically; but for now this is out of scope. There is a paper here on who to do it, if anyone wants to help me :)

I'll be committing this to github in a few days, for now if anyone is interested,  here is the jar in a zip form from dropbox.

--update--
The XMLBIF parser provided by Horacio Antar is now integrated and tested. I'm just working on refactoring Drools for pluggable knowledge types, to fully integrate Bayesian as a new type of knowledge.

Graph<BayesVariable> graph = new BayesNetwork();

GraphNode<BayesVariable> burglaryNode   = graph.addNode();
GraphNode<BayesVariable> earthquakeNode = graph.addNode();
GraphNode<BayesVariable> alarmNode      = graph.addNode();
GraphNode<BayesVariable> johnCallsNode  = graph.addNode();
GraphNode<BayesVariable> maryCallsNode  = graph.addNode();

BayesVariable burglary   = new BayesVariable<String>("Burglary", burglaryNode.getId(), new String[]{"true", "false"}, new double[][]{{0.001, 0.999}});
BayesVariable earthquake = new BayesVariable<String>("Earthquake", earthquakeNode.getId(), new String[]{"true", "false"}, new double[][]{{0.002, 0.998}});
BayesVariable alarm      = new BayesVariable<String>("Alarm", alarmNode.getId(), new String[]{"true", "false"}, new double[][]{{0.95, 0.05}, {0.94, 0.06}, {0.29, 0.71}, {0.001, 0.999}});
BayesVariable johnCalls  = new BayesVariable<String>("JohnCalls", johnCallsNode.getId(), new String[]{"true", "false"}, new double[][]{{0.90, 0.1}, {0.05, 0.95}});
BayesVariable maryCalls  = new BayesVariable<String>("MaryCalls", maryCallsNode.getId(), new String[]{"true", "false"}, new double[][]{{0.7, 0.3}, {0.01, 0.99}});

BayesVariableState burglaryState;
BayesVariableState earthquakeState;
BayesVariableState alarmState;
BayesVariableState johnCallsState;
BayesVariableState maryCallsState;

JunctionTreeNode jtNode1;
JunctionTreeNode jtNode2;
JunctionTreeNode jtNode3;

JunctionTree jTree;

BayesEngine engine;

@Before
public void setUp() {
    connectParentToChildren(burglaryNode, alarmNode);
    connectParentToChildren(earthquakeNode, alarmNode);
    connectParentToChildren(alarmNode, johnCallsNode, maryCallsNode);

    burglaryNode.setContent(burglary);
    earthquakeNode.setContent(earthquake);
    alarmNode.setContent(alarm);
    johnCallsNode.setContent(johnCalls);
    maryCallsNode.setContent(maryCalls);

    JunctionTreeBuilder jtBuilder = new JunctionTreeBuilder(graph);
    jTree = jtBuilder.build();
    jTree.initialize();

    jtNode1 = jTree.getRoot();
    jtNode2 = jtNode1.getChildren().get(0).getChild();
    jtNode3 = jtNode1.getChildren().get(1).getChild();

    engine = new BayesEngine(jTree);

    burglaryState = engine.getVarStates()[burglary.getId()];
    earthquakeState = engine.getVarStates()[earthquake.getId()];
    alarmState = engine.getVarStates()[alarm.getId()];
    johnCallsState = engine.getVarStates()[johnCalls.getId()];
    maryCallsState = engine.getVarStates()[maryCalls.getId()];
}

@Test
public void testInitialize() {
    // johnCalls
    assertArray(new double[]{0.90, 0.1, 0.05, 0.95}, scaleDouble( 3, jtNode1.getPotentials() ));


    // maryCalls
    assertArray( new double[]{ 0.7, 0.3, 0.01, 0.99 }, scaleDouble( 3, jtNode2.getPotentials() ));

    // burglary, earthquake, alarm
    assertArray( new double[]{0.0000019, 0.0000001, 0.0009381, 0.0000599, 0.0005794, 0.0014186, 0.0009970, 0.9960050 },
                 scaleDouble( 7, jtNode3.getPotentials() ));
}

@Test
public void testNoEvidence() {
    engine.globalUpdate();

    assertArray( new double[]{0.052139, 0.947861},  scaleDouble(6, engine.marginalize("JohnCalls").getDistribution()) );

    assertArray( new double[]{0.011736, 0.988264 },  scaleDouble( 6, engine.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.001, 0.999},  scaleDouble(3, engine.marginalize("Burglary").getDistribution()) );

    assertArray( new double[]{ 0.002, 0.998},  scaleDouble( 3, engine.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.002516, 0.997484},   scaleDouble(6, engine.marginalize("Alarm").getDistribution()) );
}

@Test
public void testAlarmEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode3,  alarmNode, new double[] { 1.0, 0.0 }) );

    nue.globalUpdate();

    assertArray( new double[]{0.9, 0.1}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.7, 0.3 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.374, 0.626}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{ 0.231, 0.769}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) ); }

@Test
public void testEathQuakeEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood(new BayesLikelyhood(graph, jtNode3, earthquakeNode, new double[]{1.0, 0.0}));
    nue.globalUpdate();

    assertArray( new double[]{0.297, 0.703}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.211, 0.789 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{.001, 0.999}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.291, 0.709}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}

@Test
public void testJoinCallsEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode1,  johnCallsNode, new double[] { 1.0, 0.0 }) );
    nue.globalUpdate();

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.04, 0.96 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.016, 0.984}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{0.011, 0.989}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.043, 0.957}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}

@Test
public void testEarthquakeAndJohnCallsEvidence() {
    BayesEngine nue = new BayesEngine(jTree);
    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode1,  johnCallsNode, new double[] { 1.0, 0.0 }) );

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode3,  earthquakeNode, new double[] { 1.0, 0.0 }) );
    nue.globalUpdate();

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.618, 0.382 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.003, 0.997}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{ 1.0, 0.0}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.881, 0.119}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}

Wednesday, April 30, 2014

Decision Camp 2014 : Call for Speakers : Oct 13-15, San Jose



Decision Camp is on again for 2014, registration is now open.

Picture

Who Should Attend

Practitioners are Business Analysts or Business Experts, Developers or Architects that
use or consider using Decision Management technologies such as Business Rules, Predictive Analytics, Business Intelligence
and Decision Optimization

Join practitioners like you as well as the renowned experts from industry, coming from consulting companies and technology vendors
Picture

Why Attend

Decision CAMP is the
first event for Decision Management practitioners.

It is filled with hands-on activities and insightful experience-sharing sessions.​

If you are new to Business Rules or Predictive Analytics, join us to speed up your learning curve and get more out of those technologies

The Call for speakers is now also open:

Who should submit an abstract

We are looking for keynotes, case studies, general sessions, and technical workshops. We are particularly looking for case studies.If you are a Business Analyst, Rules Writer, Rules Analyst or Rules Architect, and your job function includes harvesting, eliciting, capturing business rules, or more generally speaking decision logic, 
then you are the perfect speaker for the event!

If you are an Enterprise Architect, head of Software Development, or Software Architect, and your job function includes the integration of business rules / decision management technologies in your systems,
then you are a wonderful speaker too!

We are looking for practitioners from both sides.


Submissions will not be selected if they appear to directly promote any products or services, or are of a commercial nature.

Wednesday, April 23, 2014

jBPM accepted on Google Summer of Code 2014!

I'm thrilled to share with the whole Drools & jBPM community that once again a student has being accepted in the Google Summer Of Code Program to work in the projects for the Summer. This year Google will be funding  Nicolas Gomes (from Argentina) to work on the jBPM project. Nicolas task will be to work towards the integration of the KIE Workbench with a Content Management System such as Magnolia.
GSoC 2013
The integration will involve the backend services and front end screens to work with documents from end to end.
Here you can find all the accepted projects this year (2014):
Nicolas has also started a blog where he will sharing the integration progress,
You can also follow him on twitter:
twitter: @nicolasegomez
As I will be mentoring the work, I will be also sharing some updates and videos about how the work is being done. So stay tuned and feel free to leave comments in Nicolas' blog regarding his proposals for the work that needs to be done. If you are looking to do something similar please get in touch with Nicolas or with myself so we can coordinate the work.



Tuesday, April 22, 2014

New feature overview : PMML

Today, I'll introduce a new 6.1 experimental feature, just being released from our incubator: Drools-PMML.

I'll spend the next days trying to describe this new module from the pages of this blog. Some of you, early adopters of Drools-Scorecards, will probably have heard the name. Thanks to the great work done by Vinod Kiran, PMML was already living inside that module. However, the Predictive Model Markup Language (www.dmg.org) is much more than that. It is a standard that can be used to encode and interchange classification and regression models such as neural networks or decision trees. These quantitative models complement nicely the qualitative nature of business rules.

So, how does it work? Exactly like with all other KIE assets such as processes, rules or decision tables. First, you have to create the model: there are plenty of statistical/data mining softwares that generate or consume PMML. In the future, even Drools might be able to do that!
Once you have your model, just deploy it in your Kie Module and let it become part of your Kie Base. Betting you are now familiar with the kmodule.xml configuration, let me show an example using the programmatic APIs:
String pmmlSource = // ... path to your PMML file
KieServices ks = KieServices.Factory.get(); KieFileSystem kfs = ks.newKieFileSystem(); kfs.write( ResourceFactory.newClassPathResource( pmmlSource ) .setResourceType( ResourceType.PMML ) ); ks.newKieBuilder( kfs ).buildAll().getResults(); KieSession kSession = ks.newKieContainer( ks.getRepository().getDefaultReleaseId() ) .newKieSession();
Let's imagine that you have a predictive model called "MockColdPredictor". It is used to predict the probability of catching a cold given the environmental conditions. It has one input called "temperature" and one output "coldProbability". The most basic way to invoke the model is to insert the input value(s) using a generated entry-point:

ksession.getEntryPoint( "in_Temperature" ) // "in_" + input name (capitalized)
        .insert( 32.0 );                   // the actual value
ksession.fireAllRules();

The result will be generated and can be extracted, e.g., using a drools query:

QueryResults qrs = kSession.getQueryResults(
                       "ColdProbability",   // the name of the output (capitalized)
                       "MockColdPredictor", // the name of the model
                       Variable.v );        // the Variable to retrieve the result
Double probability = (Double) qrs.iterator().next().get( "$result" );

In the future posts, we'll discuss in detail how PMML defines models, input and output fields. Based on this, we'll see how to integrate models and which options are available.

At the moment, these model types are supported:
  • Clustering
  • Simple Regression
  • Naive Bayes
  • Neural Networks
  • Scorecards
  • Decision Trees
  • Support Vector Machines
The version of the standard is 4.1. The support for these models is being consolidated, and more will be added soon. 

Behind the scenes: the models are interpreted: that is, they are converted to an appropriate combination of rules and facts that emulate the calculations. A compiled version, where the models are evaluated directly, is possible and will be added in the future. So, the evaluation is slower than a "native" engine. The  goal is not to prove that production rules can outperform matrix operations at... doing matrix math :) Rather, the idea is to provide a uniform abstraction of a hybrid knowledge base, making it easier to integrate rule and non-rule based reasoning.

So, my next post will describe the structure of a PMML model.
Stay tuned!
-- Davide

Disclaimer: as a new feature, it is likely to suffer from bugs and issues. Feedback will be welcome.

Acknowledgments :
Thanks to the University of Bologna (Bologna, Italy), the KMR2 Project and Arizona State University (Scottsdale, AZ) which, over time, have supported this project.

Publications (more to follow) :
- D. Sottara, P. Mello, C. Sartori, and E. Fry. 2011. Enhancing a Production Rule Engine with Predictive Models using PMML. In Proceedings of the 2011 workshop on Predictive markup language modeling (PMML '11). ACM, New York, NY, USA, 39-47. DOI=10.1145/2023598.2023604 http://doi.acm.org/10.1145/2023598.2023604