Friday, May 23, 2014

Running drools-wb with GWT's SuperDevMode

Like most, I like surprises!

Some surprises aren't always welcome though; and one such surprise bit me yesterday.

As a good citizen I upgraded my installation of Google Chrome when advised a new version was available. With hind-sight I don't know why I so gleefully went along with the upgrade (after all, I'd recently removed the latest version from my mobile telephone as it didn't "feel" as good... anyway I digress).

The surprise was that Chrome 35 stops supporting GWT's "DevMode" (something I'd long been used to with FireFox) and as from GWT 2.6.0 support for "DevMode" is to come to an end ("GWT Development Mode will no longer be available for Chrome sometime in 2014, so we improved alternate ways of debugging. There are improvements to Super Dev Mode, asserts, console logging, and error messages.")

Options were to find an installation of Chrome 34, or switch to SuperDevMode (that seems inevitable). Electing for the latter, I present my findings on how to configure your webapp, IDE and run (or debug) it in "SuperDevMode".

These instructions are for IDEA (NetBeans will probably follow a similar route).

(1) Create a regular GWT Launcher:


(2) Create a new GWT Launcher for SuperDevMode:



(3) Add the following to your webapp's gwt.xml (module) file:

  <!-- SuperDevMode -->
  <add-linker name="xsiframe"/>
  <set-configuration-property name="devModeRedirectEnabled" value="true"/>
  <set-property name="compiler.useSourceMaps" value="true"/>
  <set-configuration-property name='xsiframe.failIfScriptTag' value='false'/>

(4) Launch your regular webapp (the "classic" GWT Launcher):

... <tick>, <tock>, <tick>, <tock> while it compiles and launches...


(5) Launch the SuperDevMode code server (the "SuperDevMode" GWT Launcher):

... <tick>, <tock>, <tick>, <tock> while it compiles and launches...


​(6) Drag the "Dev Mode On" and "Dev Mode Off" buttons to your bookmark bar (as advised) - but we don't normally read these sort of things, right! ;)

(7) Go back to the webapp's browser tab

(8) Click on the "Dev Mode On" bookmark you created in step (6)



(9) Click on "compile"



(10) Delete the "codesvr" part of the URL and press enter (dismiss the popups that appear; which ones depends on what browser your GWT module targets; e.g. I had to dismiss a popup about using Chrome but the GWT model targets FireFox).



​(11) Done!



(12) What's that? You want to debug your application?!?

This isn't too bad. Just launch both your "classic" GWT Launcher in debug mode and the "SuperDevMode" GWT Launcher in normal mode.

Server-side code needs break-points in IDEA, and client-side break-points need to be added using Chrome's Developer Tools (you'll need to make sure "sourceMaps" are enabled, but this appears to be the default in Chrome 35).

Accessing Chrome's debugger:



Debugging:



Simple!

It takes a bit of getting used to debugging client-side stuff in Chrome, and server-side stuff in IDEA, but it's not terrible (although don't expect to be able to introspect everything in Chrome like you used to in IDEA).

I hope this helps (probably more so as others find "DevMode" stops working for them.... and when we move to GWT 2.6.1 --- for IE10 support --- so it is coming sooner than later).

Have fun!

Mike

Thursday, May 22, 2014

London (May 26th) Drools & jBPM community contributor meeting

London, Chiswick, May 26th to May 30th

During next week a large percentage of the Drools team, some of the jBPM team and some community members will be meeting in London (Chiswick). There won’t be any presentations, we’ll just be in a room hacking, designing, exchanging ideas and planing. This is open to community members who wish to contribute towards Drools or jBPM, and want help with those contributions. This also includes people working on open source or academic projects that utilise Drools or jBPM. Email me if you want to attend, our locations may very (but within chiswick) each day. 

We will not be able to make the day time available to people looking for general Drools or jBPM guidance (unless you want to buy us all lunch). But we will be organising evenings things (like bowling) and could make wed or thu evening open to people wanting general chats and advice. Email me if you’re interested, and after discussing with the team, I’ll let you know.

Those currently attending:
Mark Proctor (mon-fri) Group architect
Edson Tirelli (mon-fri) Drools backend, and project lead
Mario Fusco (mon-fri) Drools backend
Davide Sottara (wed-fri) Drools backend
Alex Porcelli (mon-fri) Drools UI
Michael Anstis (thu-fri) Drools UI
Kris Verlaenen (wed-thu) jBPM backend, and project lead
Mauricio Salatino (mon-fri) jBPM tasks and general UI

Wednesday, May 07, 2014

Drools - Bayesian Belief Network Integration Part 3


This follows my earlier Part 2 posting in April,

Things now work end to end, and I have a clean separation from the Creation of the JunctionTree and initialisation of all state, and the state that change after evidence insertion. This separation ensures that multiple instances of the same bayesian network can be created cheaply.

I'm now working on integrating this into the belief system. One issue I have is that I can automate the update of the bayesian network as soon as the evidence changes. The reason for this is updating of the  network is expensive, if you insert three pieces of evidence, you only want it to update one not three times. So for now I will add a dirty check, and allow users to call update. For best practice I will recommend people separate reasoning of the results of bayesian and entering new evidence, so that it becomes clearer when it's efficient to call update.

For now I'm only dealing with hard evidence. We will be using superiority rules to resolve conflict evidence for a variable. Any unresolved conflicts will leave a variable marked as "Undecided". Handling of soft or virtual evidence would be nice, this would add way to resolve conflicted evidence statistically; but for now this is out of scope. There is a paper here on who to do it, if anyone wants to help me :)

I'll be committing this to github in a few days, for now if anyone is interested,  here is the jar in a zip form from dropbox.

--update--
The XMLBIF parser provided by Horacio Antar is now integrated and tested. I'm just working on refactoring Drools for pluggable knowledge types, to fully integrate Bayesian as a new type of knowledge.

Graph<BayesVariable> graph = new BayesNetwork();

GraphNode<BayesVariable> burglaryNode   = graph.addNode();
GraphNode<BayesVariable> earthquakeNode = graph.addNode();
GraphNode<BayesVariable> alarmNode      = graph.addNode();
GraphNode<BayesVariable> johnCallsNode  = graph.addNode();
GraphNode<BayesVariable> maryCallsNode  = graph.addNode();

BayesVariable burglary   = new BayesVariable<String>("Burglary", burglaryNode.getId(), new String[]{"true", "false"}, new double[][]{{0.001, 0.999}});
BayesVariable earthquake = new BayesVariable<String>("Earthquake", earthquakeNode.getId(), new String[]{"true", "false"}, new double[][]{{0.002, 0.998}});
BayesVariable alarm      = new BayesVariable<String>("Alarm", alarmNode.getId(), new String[]{"true", "false"}, new double[][]{{0.95, 0.05}, {0.94, 0.06}, {0.29, 0.71}, {0.001, 0.999}});
BayesVariable johnCalls  = new BayesVariable<String>("JohnCalls", johnCallsNode.getId(), new String[]{"true", "false"}, new double[][]{{0.90, 0.1}, {0.05, 0.95}});
BayesVariable maryCalls  = new BayesVariable<String>("MaryCalls", maryCallsNode.getId(), new String[]{"true", "false"}, new double[][]{{0.7, 0.3}, {0.01, 0.99}});

BayesVariableState burglaryState;
BayesVariableState earthquakeState;
BayesVariableState alarmState;
BayesVariableState johnCallsState;
BayesVariableState maryCallsState;

JunctionTreeNode jtNode1;
JunctionTreeNode jtNode2;
JunctionTreeNode jtNode3;

JunctionTree jTree;

BayesEngine engine;

@Before
public void setUp() {
    connectParentToChildren(burglaryNode, alarmNode);
    connectParentToChildren(earthquakeNode, alarmNode);
    connectParentToChildren(alarmNode, johnCallsNode, maryCallsNode);

    burglaryNode.setContent(burglary);
    earthquakeNode.setContent(earthquake);
    alarmNode.setContent(alarm);
    johnCallsNode.setContent(johnCalls);
    maryCallsNode.setContent(maryCalls);

    JunctionTreeBuilder jtBuilder = new JunctionTreeBuilder(graph);
    jTree = jtBuilder.build();
    jTree.initialize();

    jtNode1 = jTree.getRoot();
    jtNode2 = jtNode1.getChildren().get(0).getChild();
    jtNode3 = jtNode1.getChildren().get(1).getChild();

    engine = new BayesEngine(jTree);

    burglaryState = engine.getVarStates()[burglary.getId()];
    earthquakeState = engine.getVarStates()[earthquake.getId()];
    alarmState = engine.getVarStates()[alarm.getId()];
    johnCallsState = engine.getVarStates()[johnCalls.getId()];
    maryCallsState = engine.getVarStates()[maryCalls.getId()];
}

@Test
public void testInitialize() {
    // johnCalls
    assertArray(new double[]{0.90, 0.1, 0.05, 0.95}, scaleDouble( 3, jtNode1.getPotentials() ));


    // maryCalls
    assertArray( new double[]{ 0.7, 0.3, 0.01, 0.99 }, scaleDouble( 3, jtNode2.getPotentials() ));

    // burglary, earthquake, alarm
    assertArray( new double[]{0.0000019, 0.0000001, 0.0009381, 0.0000599, 0.0005794, 0.0014186, 0.0009970, 0.9960050 },
                 scaleDouble( 7, jtNode3.getPotentials() ));
}

@Test
public void testNoEvidence() {
    engine.globalUpdate();

    assertArray( new double[]{0.052139, 0.947861},  scaleDouble(6, engine.marginalize("JohnCalls").getDistribution()) );

    assertArray( new double[]{0.011736, 0.988264 },  scaleDouble( 6, engine.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.001, 0.999},  scaleDouble(3, engine.marginalize("Burglary").getDistribution()) );

    assertArray( new double[]{ 0.002, 0.998},  scaleDouble( 3, engine.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.002516, 0.997484},   scaleDouble(6, engine.marginalize("Alarm").getDistribution()) );
}

@Test
public void testAlarmEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode3,  alarmNode, new double[] { 1.0, 0.0 }) );

    nue.globalUpdate();

    assertArray( new double[]{0.9, 0.1}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.7, 0.3 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.374, 0.626}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{ 0.231, 0.769}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) ); }

@Test
public void testEathQuakeEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood(new BayesLikelyhood(graph, jtNode3, earthquakeNode, new double[]{1.0, 0.0}));
    nue.globalUpdate();

    assertArray( new double[]{0.297, 0.703}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.211, 0.789 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{.001, 0.999}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.291, 0.709}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}

@Test
public void testJoinCallsEvidence() {
    BayesEngine nue = new BayesEngine(jTree);

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode1,  johnCallsNode, new double[] { 1.0, 0.0 }) );
    nue.globalUpdate();

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.04, 0.96 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.016, 0.984}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{0.011, 0.989}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.043, 0.957}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}

@Test
public void testEarthquakeAndJohnCallsEvidence() {
    BayesEngine nue = new BayesEngine(jTree);
    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode1,  johnCallsNode, new double[] { 1.0, 0.0 }) );

    nue.setLikelyhood( new BayesLikelyhood( graph, jtNode3,  earthquakeNode, new double[] { 1.0, 0.0 }) );
    nue.globalUpdate();

    assertArray( new double[]{1.0, 0.0}, scaleDouble( 6, nue.marginalize("JohnCalls").getDistribution() ) );

    assertArray( new double[]{0.618, 0.382 }, scaleDouble( 6, nue.marginalize("MaryCalls").getDistribution() ) );

    assertArray( new double[]{0.003, 0.997}, scaleDouble( 3, nue.marginalize("Burglary").getDistribution() ) );

    assertArray( new double[]{ 1.0, 0.0}, scaleDouble( 3, nue.marginalize("Earthquake").getDistribution() ) );

    assertArray( new double[]{0.881, 0.119}, scaleDouble( 6, nue.marginalize("Alarm").getDistribution() ) );
}