Friday, December 21, 2007

Drools Extensible Process Definition Language (ePDL) and the Semantic Module Framework (SMF)

Old school Drools 2.x users might remember the Semantic Module Framework and how it would allow for domain specific XML representations, such as the House Example DSL.

One of the issues with the SMF in Drools 2.0 was that rules are too hard to build programmatically, so the complexity was great. In contrast process nodes are trivial to build and wire together, so I have decided to resurrect the SMF for our process language; actually I have done this for rules too so they both using the same framwork, but I'm not expecting users to extend the rules side. I have forward ported the XML framework from Drools 2.0 and simplified it.

The parser creates DOM like fragments which are processed by builder handlers. The parser also maintains a parent hierarchy back to the root node. The SemanticModule is a configuration of handlers for elements in a given namespace (uri). We keep a registry called SemanticModules, with the uri as a key for each SematnicModule. The code to retrieve a SemanticModule is:
public SemanticModule getSemanticModule(String uri) {
return this.modules.get( uri );
}
The SemanticModules can be built programmaticaly and/or via discovered configuration files. The default process model handlers are already in the SemanticModules registry, via concrete implementations RulesSemanticModule and ProcessSemanticModule, discovered modules are wired together using the DefaultSemanticModule class.

For each xml element the parsers reads in it looks up the Handler, by first getting the SemanticModule for the namespace and then retrieving the handler from the SemanticModule with the getHandler(...) method:
public Handler getHandler(String name) {
return this.handlers.get( name );
}
So how does a Handler work? It has two methods, start(...) and end(...), called when the parsers enters and exits the element; inbetween which a DOM like fragment is built. The start method has access to the element attributes and the end method has access to the DOM like fragment that was built.

The start method initialises the DOM fragment build process using the startConfiguration() method:
reader.startConfiguration( localName, attrs );
The end method finalises the returns the DOM fragment using the endConfiguration() method:
Configuration config = reader.endConfiguration();
Lets look at a complete start method now for an ActionNode:
public Object start(String uri, String localName,
Attributes attrs, ExtensibleXmlParser reader) throws SAXException {
xmlPackageReader.startConfiguration( localName, attrs );
RuleFlowProcessImpl process = ( RuleFlowProcessImpl) reader.getParent();
ActionNodeImpl actionNode = new ActionNodeImpl();
actionNode.setName( attrs.getValue( "name" ) );
process.addNode( actionNode );
((ProcessBuildData)reader.getData()).addNode( actionNode );
return actionNode;
}
Above you can see we start the configuration for the DOM fragment and create an ActionNode setting its properties; note we don't yet set the content body, as we don't have that information yet, this is created end the end method:
public Object end(String uri, String localName,
ExtensibleXmlParser reader) throws SAXException {
Configuration config = reader.endConfiguration();
RuleFlowProcessImpl process = ( RuleFlowProcessImpl ) reader.getParent();
ActionNodeImpl actionNode = ( ActionNodeImpl ) xmlPackageReader.getCurrent();
actionNode.setAction( "logger.Warn(" + config.getText() + ")") );
return actionNode;
}
Both properties files and programmatic apis can be used to register handlers for a SemanticModule. For instance if we wanted to programmatically register a logger handler we would do it as follows:
SemanticModule module = new DefaultSemanticModule( "http://domain/org/mydsl" );
module.addHandler( "logger", new LoggerHandler() );
Or we can do it via properties file:
uri=http://domain/org/mydsl
store=org.drools.xml.StoreHandler
Note to use a properties file the PackageBuilderConfiguration needs to be told about these configurations:
semanticModules = mydsl.conf
So the above LoggerHandler and its registration allows for domain specific XML like below:
<process  name="process name" id="process name" package-name="org.domain"
xmlns="http://drools.org/drools-4.0/process"
xmlns:mydsl="http://domain/org/mydsl"
xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
xs:schemaLocation="http://drools.org/drools-4.0/process drools-processes-4.0.xsd" >

<nodes>
<start name="start node" />

<action name="action node" dialect="java">
list.add( "action node was here" );
</action>

<mydsl:logger name="test logger" type="warn">
This is my message
<mydsl:logger>

<end name="end node" />
</nodes>

<connections>
<connection from="start node" to="action node" />
<connection from="action node" to="test logger" />
<connection from="test logger" to="end node" />
</connections>

</process>

Share/Bookmark

Thursday, December 20, 2007

When to use Rules and Processes and their overlap

Paul Haley, founder and chairman of haley.com, has just written a very in depth blog covering rules and processes and when to use which and the various ways they overlap. It's pretty much a "must read" for people trying to understand this space; heh, and he mentions our work in this efforts at the end :)
Business Rules Process Management
Share/Bookmark

Modify Block for Java Dialect

In one more post in the series: "What are you guys doing right now?", :) we just added support to modify block in Java dialect consequences.

This is a feature that was already supported by MVEL and we decided to port it to Java so that it would benefit everyone, independently of the chosen dialect.

Basically, a modify block is a block of code that allow attributes of a given fact to be changed in the consequence of your rules, even with shadow facts turned off. In other words, if you don't want to pay for the shadow fact performance and memory cost, you can disabled them as long as you only change fact attributes inside modify blocks.

Example:

rule "Eat Cheese"
when
$p: Person( status == "hungry" )
$c: Cheese( )
then
retract( $c );
modify( $p ) {
setStatus( "full" ),
setAge( $p.getAge() + 1 )
}
end

So, inside of the modify block, you may write a comma separated list of Java expressions (set attribute values, call methods, etc) that are applied to the object scoped to that modify call. All of them are executed in sequence and inside a single fact update scope.

The grammar pseudo BNF looks like this:

modify ::= 'modify' '(' <expression> ')' '{' [ <expression> [ ',' <expression> ]* ] '}'

This feature was added both to trunk and 4.0.x branches and will be released in Drools 4.0.4 and Drools 4.1.0.

Happy drooling,
Edson
Share/Bookmark

Tuesday, December 18, 2007

Drools and JBoss World

JBoss World is on for February the 13th 2008 at Orlando. I'll be doing a Drools talk there and I'll also be on Bob McWhirter's OSS round table, where we will talk about successful strategies for OSS communities.
Share/Bookmark

Monday, December 17, 2007

Drools Javapolis BOF on our Business Logic Platform

I did a Drools BOF at Javapolis, where I presented what I believe is the future :) our vision on a unified and integrated business logic platform. You won't go to separate proprietary vendors for rules, processes and cep - each with their own runtime, api and server side management/deployment; your business logic is too important to be tied into closed source proprietary frameworks. We will provide the end users with a single platform with fully integrated rules, processes and cep with a single runtime, api and server side management/deployment. I have made available the slides from the presentation here, which are at a technical level, and you'll see this delivered in phases of 2008.
Share/Bookmark

Friday, December 14, 2007

Stream support in Drools - a glimpse

As some of you, our users, already know, we are working on a series of features for CEP and ESP support for the next major release of Drools. We weren't talking much about it (busy researching and writing code), but I think it is time to share with you a little bit of what we developed on the Stream support side.

The first thing to understand is the concept of an "Entry Point".

When we work with streams, where the volume of data is usually huge, it is mandatory that we develop a way of scoping where that data will evaluated. Another mandatory requirement is the ability to effectively work with multiple streams in parallel, and for that, we need to effectively parallelize parts of the network, so that we don't create bottlenecks among different streams.

So an "Entry Point" is a channel through which you can assert data into the engine. All facts asserted through one entry point are only visible by patterns associated with that entry point and all entry points are independent of each other, thread safe and can run in parallel.

So, to illustrate the idea, nothing better than a simple rule:


import event org.drools.StockTradeEvent;

rule "Correlate Trade Order"
when
Customer( $id : id )
StockTradeEvent( customer == $id, $asset : asset ) from entry-point HomeBrokerStream
StockTradeEvent( asset == $asset, status == Trade.CONFIRMED ) from entry-point StockExchangeStream
then
// correlate events
end


So, in the above example, you have one single rule correlating events from the working memory (Customer) and from 2 different streams (HomeBrokerStream and StockExchangeStream) at the same time it keeps each of the streams processing in parallel.

The good thing about the implementation we did is that we leverage the characteristics of RETE to support the streams, like node sharing, indexing etc. So, if you have multiple rules with patterns that are associated with the same streams, the patterns will share nodes, index facts, etc transparently.

Another interesting characteristics is that we handle heterogeneous streams. I.e., each stream is not limited to a single type of event/fact and this is all transparent to the user. The engine select the events/facts from the stream based on the patterns the rules are looking for.

From a code perspective, all the user needs to do is get the entry point interface from the session for each of his streams and it to insert the events/facts:


StatefulSession session = ruleBase.newStatefulSession();

// one thread may do this:
EntryPointInterface broker = session.getEntryPoint( "HomeBrokerStream" );
// insert each event as usual
broker.insert( event );

// another thread may do this:
EntryPointInterface stock = session.getEntryPoint( "StockExchangeStream" );
// insert each event as usual
stock.insert( event );


There is still a lot to do, but we are getting there.

Happy drooling,
Edson
Share/Bookmark

Wednesday, December 12, 2007

Drools solver Javapolis 2007 slides

The BOF was a success and as promised, I 've posted the slides online:
People asked a bunch of interesting questions afterwards.
One question I felt I left a bit unanswered: how do you weigh your constraints in DRL? So, here's a code example that should answer that question too:

rule "twoExamsInARow"
when
// ...
then
insertLogical(new IntConstraintOccurrence(
"twoExamsInARow",
10, // weight
$leftExam, $rightExam));
end

rule "twoExamsInADay"
when
// ...
then
insertLogical(new IntConstraintOccurrence(
"
twoExamsInADay",
1, // weight
$leftExam, $rightExam));
end

rule "softConstraintsBroken"
when
$total : Double() from accumulate(
IntConstraintOccurrence($weight : weight),
sum($
weight)
);
then
scoreCalculator.setSoftConstraintsBroken($total.intValue());
end

So the 2 in a row constraint is 10 times heavier than the 2 in a day constraint.
Share/Bookmark

Friday, December 07, 2007

Drools November Download Stats

For 3 consecutive months in a row we've had approximately 20K download attempts for -bin.zip, what is interesting though is that the BRMS has a lot more download attempts this month, likewise Eclipse, especially if you combine the eclipse3.3 and eclipse3.2 download attempts; that's almost 22K eclipse download attempts. so over all I'm really pleased :)

20230 drools-4.0.3-bin.zip
14822 drools-4.0.3-brms.zip
14297 drools-4.0.3-eclipse3.3.zip
13477 drools-4.0.3-src.zip
8313 drools-4.0.3-examples.zip
7501 drools-4.0.3-eclipse3.2.zip


Let's hope that the features we have planned in 4.1.0 in Q1 will help us break that 20K barrier for the -bins :)
Share/Bookmark

Thursday, December 06, 2007

Life and Death, Rules and RuleFlow

Yesterday I told a friend what I do for a living, and she's still in shock. I don't think the words 'grabbing people's brains and shoving them into a PC' was the most tactful explanation. I wouldn't recommend explaining Rules and RuleFlow to your boss in those terms either. Unless they already think you're some sort of Frankenstein and you have best friend called Igor.

A better way might be to use the workflow example from the recent IJTC conference. It explains the importance of Rules and Ruleflow in a way that even your boss can understand. Strangely enough for a Drools blogpost, you don't even need a computer to understand it;

The Health services in Bangladesh (like many elsewhere) can't get enough doctors. Training more is not a solution ; those that do qualify tend to leave for higher rates of pay elsewhere. So, given the desparate need for trained medical staff in rural areas(to curb child mortality rates), what are health workers to do?

The solution that the Health workers came up with was IMCI - or Integrated Management of Childhood Illness. It takes what the Knowledge in Doctor's head's and writes it down as a simple guide that health workers can follow. When a sick child is brought into the remote clinic the health worker is able to follow the simple step-by-step instructions to make quite a sophisticated diagnosis.

I've no medical training beyond simple CPR (and if you're relying on that then you're in real trouble) but even I can understand it.

Look at the pale blue box. It's a set of medical rules: Are there any danger signs? What are the main symptoms? What combination of these symptoms are there? What is the age of the child? How long have they been ill? Depending on the outcome of the rules, go to the next set (the pink / yellow /green) boxes and apply the rules that you find there.

That's Rules and RuleFlow.

  • Rules are 'when something is present , then do this'. And not just single rules, but many of them. Together, loads of simple rules allow you to come up with quite a sophisticated diagnosis.

  • Ruleflow allows you to group your rules. If you're a health worker with a sick child you want to do the most important checks first. Depending on the outcome, you then apply the next set of medical rules: Pink if they need urgent referral to the hospital, yellow if the clinic can cope , or green if the child can be looked after at home.
As gory as it sounds, everybody, including the doctors, are happy that their 'brains have been put into a PC' (or in this case , a set of paper cards). The Doctors are happy because they can (guilt free) move to better paying jobs. The medical workers using the system are happy because they can help the sick children that they see every day. And the children gain because the better availability of medical knowledge (via Rules and RuleFlow) is literally the difference between life and death.

Paul Browne also writes on the People and Technology blog.
Share/Bookmark

Testing rules - introduction

Quite some time ago, I created the fit-for-rules project, which was built using the FIT framework (Framework for Integrated Tests). I was quite a fan of fit - however for reasons unknown it has not yet taken off in the mainstream (perhaps its just too strange, I think its brilliant, but then I like egg nogg lattes at Christmas - no idea why, its not something I would normally like at all).

Recently we have been working on an integrated testing tool to encourage people to write tests for rules the TDD people do for code. This tool, still doesn't have a name, so feel free to suggest some names. In terms of the tool itself, I spent some time looking into BDD (behavioral driven development - not the disease), and tried to adopt the BDD nomclamenture.

So, the basic test item is a Scenario. A scenario is essentially a sequence of data followed by expectations (ie in a given scenario, expect things). Expectations can be data, or rules firing etc.

Its not that different to a rule:

Given some data -> Then fire the rules -> Expect some results -> Modify some data -> Fire the rules -> Expect some results.

You may only want one given/expect cycle, but as rule session are (can be) stateful, this will allow it. Well a picture is worth a thousand words:

You can see it looks not unlike the guided editor. You set up facts, and then simulate rules firing (actually they do fire, unless you tell it to not fire certain rules). You can then check the data and rules after the fact (rinse, repeat). These tests are just as important as the rules, and will be stored as such.



Simulating a date is optional, but handy as often rules are dependent on time.

Suggestions for a name for this?
Share/Bookmark

Wednesday, December 05, 2007

Rule Constraints and Code Constraints now works for Splits

Recently I got the Drools dialect system working for processes, as discussed in this previous blog article. The dialect system is a pluggable framework to allow code to be written with different languages. Previously Split constraints would only work with rules, so the next stage was naturally to allow for constraints to work with code, like traditional process engines - ofcourse the code constraint would also work with the dialect system. Below is a screenshot of a Split node with both MVEL and Java code constraints and MVEL and Java rule constraints.


Share/Bookmark

Tuesday, December 04, 2007

Showcase: Chooz-it (choosing the right technology)

Paul Smith, a Drools user and long time mailing list lurker, has launched a new service that he has been cooking up for some time : www.chooz-it.com.au.

Choose it is a web site/service that connects suppliers with consumers (particular for service industries). It allows suppliers to provide their availability, and also does bookings/fulfillment. What this means is if you offer say a dog washing service, you can provide this service through chooze-it, and people can book your services (according to your availability) and you won't be double booked, payment dealt with etc. Obviously there is much more to it then this, so see below for a more detailed explanation of Chooze-it in Paul's own words:

Choozing the right technology


A little under two years ago I set out to build a system that would allow for users to search for service suppliers and allow those suppliers to expose their services, availabilities and take bookings for those availabilities. The system had to be generic enough to encompass as many businesses as possible and needed to be built on very solid foundations as scalability was going to be one of the top requirements. I also wanted to showcase some of the latest ‘bells and whistles’ that the Java/JEE world had to offer. Early in the piece I had been tackling the issue of how to separate the business logic so that it became a nice separate pluggable module leaving the rest of my persistence/service code relatively unpolluted. Drools was then in its early days being just before version 3 was released. I played with it and straight away realised that this had some seriously powerful implications. The use of a logic or rules engine separated logic nicely and the inclusion of domain specific languages made the rules very easy to read and understand. Back then the Eclipse plugin wasn’t yet available so debugging was a pain but not insurmountable. All of the logic became easily unit testable because I separated it totally from the database. This left the cactus tests free to just test that things actually get put in the database. Another relatively new player was EJB3 and JPA. Having gone through the pain of EJB2 but not yet confident enough to use Spring, I opted to give the EJB3 path a go as there was good support already in JBoss AS 4. It had a few initial teething troubles (mainly down to the spec not being finalised) but on the whole it was very stable and easy to use. No config files (hallelujah), just annotate the files and let JBoss do its thing at deploy time. The session facades worked well too. Once I’d ironed out the security domain issues it worked a treat. This gave me the framework to build a high performance service layer. It’s broadly speaking based on SOA because I wanted to allow for the possibility of integration with PC based booking systems. This is for later development but I’d like to allow for the transfer of booking and availability information between the online system and the customers booking system. So a service architecture was a definite must have.


Notifications came into the picture sometime later in development because I needed to have guaranteed email and sms delivery. This came right at the time when JBoss MQ was on the way out and JBoss Messaging was coming in, so naturally I opted to go with the latest and greatest despite the pain that generally comes along with this. The results were not favourable initially as for some reason, the queue JNDI lookups kept failing. Everything looked to be configured correctly as per the doco but it just wouldn’t work. I raised a Jira issue but didn’t get a resolution. It took til version 4.21GA of JBoss AS before I got it working. Somebody fixed something in that version of the code and it all started working like a charm. When the messaging descriptor is configured correctly JBossAS it just picks up on the create SQL and generates the tables if they aren’t already there. I’m confident it will work well because when I first started building the server, the SMTP config was wrong and nothing was happening but as soon I fixed the config all the emails etc just got picked up from the queue and sent out when the app server came back up.


The front end isn’t really that flash. It’s struts tiles utilising a smattering of Ajax. Nothing really new there but we do have a little bit of stuff to interact with the Google Calendar API. I used the sslext tags to handle the changes from HTTP to HTTPS and back again. It generally seems to work well but has got a couple of quirks.


Functionality wise, the system offers suppliers the ability to signup and configure their business details, services and service availabilities. Users can come in and browse for suppliers, look at their service offerings and check for availabilities. Once logged in they can either book a specific availability or make a booking enquiry for a service. Depending on the supplier configuration for the service they can also pay the deposit for the service. The booking can then be managed from both the customer and supplier perspective eg) the status can be confirmed or the customer can request cancellation. Manual bookings taken by phone can also be entered. There is also the facility for employees to be configured and added to teams. Each service can then have a team of service providers or just one provider. Automatic invoicing was built in to trigger on a monthly basis so the supplier can look at their invoice as a .pdf and then pay it online. I used itext to generate the .pdf dynamically when they click on the link. It’s pretty quick but the API is a bit clunky. I also implemented a generic payment gateway interface over the gateway suppliers API that allows me to easily change payment gateway providers if I need to. I could have just used PayPal or something similar but I didn’t like the idea of having the users go outside of the site.


It’s easy to make technology decisions based on how new something is or how great it might sound to say that it’s being used. The proof of how good the technology is really doesn’t happen until it makes it to production, and doesn’t fall over, but up until now I’ve been very happy with my technology choices.


So that, in a nutshell, is www.chooz-it.com.au and the technologies behind it.

Share/Bookmark

Saturday, December 01, 2007

The Missing BRMS tutorial

James Childers (of the macjavadev blog) has helpfully come up with a (slightly humorous) Real Quickstart guide (I like to call it the "missing manual"). Its good advice.

We do have plans to have more 5, 10 minute tutorials. Of course to save some work, we are changing some aspects of the user interface to try and get rid of the annoying confusing bits (especially around package management, and navigation). So with the BRMSv2, expect to see more of these "ease into it" style tutorials (I was quite fond of the olde codehaus 1, 2 and 5 minute tutorials, it was what got me into drools in the first place).

In the mean time, sit back, relax, or something, and take a look at the following.

Quickstart: Drools 4.0.3 Web Admin (BRMS)

The documentation on Drools is a little bit light on the "where the hell do I start" part, so here's a quickstart guide. I'll walk you through getting the equivalent of a "Hello, World" app up in Drools, i.e. the simplest setup possible. This will use the Drools web management interface, and will not require any actual code whatsoever. This assumes that you, much like myself, in fact know next to nothing about Drools, and just want to play with it before getting into Rete algorithms and truth maintenance and all that crap.

Isn't that nice.

There are only three requirements.

  1. You have your own servlet container: Tomcat, Glassfish, whatever.

  2. You have in your possession a JavaBean in some .jar flie somewhere. Any bean will do.

  3. You don't require screenshots. Sorry, I'm lazy. Maybe in the future.

That's it.

  1. Download the Drools BRMS from JBoss.

  2. Deploy it to your app server. I like Glassfish.
    [0932][jchilders@Ivan:~/webapps]$ unzip drools-4.0.3-brms.zip
    [0932][jchilders@Ivan:~/webapps]$ sudo asadmin
    Use "exit" to exit and "help" for online help.
    asadmin> start-domain domain1
    asadmin> deploydir ./drools-brms/

    Log in to it. (Mine deployed to http://localhost:8080/drools-jbrms/). User ID is "", password is "", without the quotes.

  3. You should see five options on the left: Info, Rules, Packages, Deployment, and Admin. Click on Admin.

  4. Click on "Create New Category".

  5. Name your category something clever. We'll call ours "BannerAds". Save it.

  6. Click on "Packages". There is a button on this page that will allow you to create a new package. It is small and has no label, but does have a tooltip. Click on it.

  7. Give your package a name and optional description. We'll call ours "MyPackage". Save it.

  8. There is another tiny, hard-to-find button on this page that allows you to create a new model. Click on it.

  9. This model will reference the jar file that has your bean it in I mentioned earlier. Name it something witty like "MyBean" and save it.

  10. You should be at a screen that will let you upload your jar. Do it.

  11. Click on the "Save Changes" button.

  12. Holy crap we're almost there! Now click on the "MyPackage" package in the tree, then "Edit Package Configuration" in the main window.

  13. In the "Header" section in the main window you need to actually tell Drools about your bean. To do so:
    import com.mycompany.MyBean

    No semicolon at the end. If you want to get crazy and are lucky enough to own more than one JavaBean, do more than one import.

  14. Wow we should actually be able to make a rule now! AWESOME!

  15. Click on the massive 16px button that will let you "Create New Rule"

  16. Give it a name and category and type it as "Business rule". Save.

  17. Ok, now comes the real test. Click on the big green plus sign to the right of "WHEN". You should see a window that says "Add a condition to the rule..." If everything was set up correctly -- and that's a big if -- then you should see two selects: "Fact", and "Condition Type". If all it says is "Add a condition to the rule..." and nothing else, then you screwed up. Also these instructions may be wrong. It is much more likely that you screwed up because I have a blog. If it doesn't work please go to step RTFM, else continue.

  18. Fact. Click on it. Select your bean.

  19. Figure out the rest on your own. You're now to the point where you can create rules based upon your bean. Doing so using the visual editor is pretty self-explanatory.

Share/Bookmark

Wednesday, November 28, 2007

Drools blog reaches over 500 subscribers :)

Today is a milestone for this blog, our rss/atom subscribers just broke 500 with 507 subscribers :)

Last week we saw our highest web hits with 726 unique views, which is on top of the subscribers and the views at syndication sites like JBoss.

I know these figures are small compared to techs like Hibernate, but the main thing is we are growing the message. Two years ago most people didn't know what a rule engine was, hopefully we are starting to change that :)
Share/Bookmark

Tuesday, November 27, 2007

Drools solver @ Javapolis

I 'll hold a Drools Solver BOF at Javapolis 2007 on Tuesday December 11th at 20:00. You're all invited :)
Take a look at the schedule here.
Take a look the contents of the BOF here.

The drools solver manual has also been expanded with more info about the examples:
Take a look at the updated manual here.

The manual now contains some insight about the problem size of the examples. Did you know that the traveling tournament example nl16 finds a feasible solution out of 2,45064610271441678267620602e+259 possible solutions?

And of course I 've added some more eye candy:


Share/Bookmark

Pluggable Dialects for Drools Processes now work :)

Many of you would have read my blog on unifying rules and processes, which was also featured at infoq. Unifying these technologies is not just about the modelling paradigm it's also about the infrastructure overlaps. Today I just finished my first end to end test for dialectable actions in a processes definition - which we call ruleflow, indicating its a melding of the power of processes and rules - so what does this mean?

Pluggable Dialects has been apart of the Drools framework for a while now, what it means is that any eval conditions and the consequence of the rule and can be written in any language; we currently support Java and MVEL as dialect plugins.

One of the extra bits of plumbing that makes this worth while is that a Dialect, at compile time, returns the identifiers that it needs - i.e. none local variables - this allows us to do variable inject in compiled languages like Java, which means no manually retrieving and assignment variables from a context :)

Scripting language plugins like MVEL are very easy to integrate although compiled languages like Java add extra levels of complexity - this is because we want to compile all our consequences, and now actions, in a single pass. The compilation is at a later date than when the rule/action itself was built and thus we need an additional wiring process to hook the compiled code back up to the rule/action.

As we've already built all this for the rules framework with a bit of tweaking the process framework gets it for free - and thus we start to see the value of a unified core.

The image below (click to enlarge) shows a screenshot from our ruleflow editor in Eclipse. It contains just two actions, but of different dialects, one Action is MVEL the other in Java - both populate a String value in the List. The displayed Action is of the Java dialect, notice it has variable inject, so you don't need to do assign the variables manually from a context, i.e.:
List list = (List) context.getVariable("list");


A Java Dialect Action in Ruleflow

Notice as well how the rules and process apis are complimentary to each other. The image below is from the unit test for the process definition in the above screenshot.



Dialect Unit Test for Ruleflow

Share/Bookmark

Monday, November 26, 2007

Pigeons, Complex Event Processing and how to make millions with JBoss Drools

I'm still waiting for the JBoss bouncers to hand me my coat and ask me to leave this blog. Mark gets to talk about Unifying Rules and Processes. Fernando and Michael are very proud of the 2nd version of the Business Rules WebApp . And I get to talk about pigeons. Yep, Pigeons; birds that fly, sometimes useful for carrying messages and have one hidden talent.

During the cold war, the Soviets (allegedly) trained pigeons to inspect ball-bearings on the production line . The Pigeons would sit in comfortable little boxes while the shiny silver ball bearings steamed past on a conveyor belt. If the pigeon spotted any that were defective, they would peck a button and the broken bearing was gone. Since the fall of the Berlin wall, all the pigeons have been gainfully re-employed over at Google.

Thankfully the pigeons didn't go to work at a Bank in the City (have you ever seen anything with feathers drive a Ferrari?) . While the pigeons would be very good at responding to simple market events events (Market up , sell; Market Down , Buy). more complex analysis escapes them; For example ; if the market is down for the 30 mins, and Shares in Acme corp are down more than 10% than the average ,and I have seen 3 buy orders for that share in the last 60 seconds = I think the market is about to turn = buy shares in Acme corp.

Never mind pigeons; most humans would find that difficult - think about trying to read the stock ticker prices (the ones you see rolling across the screen at MSNBC) for all stocks, while trying to hold the buy and sell information for the last 30 minutes in your head. And do that not only for one , but for the couple of hundred different types of shares in the market. And while keeping an eye on your own trading position so that you're not exposed to one sector of the market (e.g. keeping enough cash , not too many property or technology shares. No wonder most traders make their millions and burn out before they're 30 - that sort of Complex Event Processing (CEP) will wear you out.

Most IT applications are like pigeons; they can only handle simple events. Press Button. Do something.

The way to make millions is to design applications that can handle these complex events, and apply sophisticated business rules to the (evolving) situation. And do it quickly enough (milliseconds) to seize the opportunity before somebody else does. A keep on doing it as long as the market is open.

Funnily enough, Complex Event Processing is part of the vision for Drools. With enough support, I'm sure we could convince Mark to stand up at JavaPolis and use a set of Pigeons on his slides. I suppose it's better than using pictures of lego people to explain how to do projects using Agile.
Share/Bookmark

Monday, November 19, 2007

A Vision for Unified Rules and Processes

Since Drools 4.0 I've been demonstrating our ruleflow stuff, which includes a graphical designer and basic rules and process integration for stateful rule orchestration. What is ruleflow? Ruleflow is the integration of rules and processes, which can predominantly be used to orchestrate the execution of rules. Additionally what 4.0 also provides is a prototype to prove that we don't need to have a process oriented or a rule oriented view of the world. I believe that any company that isn't able to truly unify rules and processes into a single modelling system, how PegaSystems have done, will not be in a suitable position for the future - actually Microsoft get this too with their Workflow Foundations offering, although their rule offering is still very weak. Typically the market has a strong process company with weak rules, or strong rules with weak processes - rules and processes must exist as first class citizens within the modelling and execution environment. The current "modus operandi" for the rule industry is a focus on stateless decision services, where the separate workflow engine at some point calls out to the separate stateless rule engine to assist in some decision making - after 30 years of research and development is that the best we have to offer, multi-million pound license deals that effectively boil down to glorified spreadsheets called via a stateless web services from some workflow engine. Not to mock this model, as it is actually quite useful, but this level of integration is superficial at most and we need to make sure that we unify these models and allow it to go to much greater depths. Once rules and processes are fully integrated that modelling environment will benefit from any other declarative systems added, such as our plans to add Complex Event Processing (CEP) to the engine and language - this means we can have rules monitoring streams of data and triggering process milestones.


Drools 4.0 ruleflow diagram

Partly the reason for the rule industry flocking to this model of stateless decision services is their existing tech is hard to understand and implement and thus more difficult to sell, decision services are simple to understand and thus easier toimplement and sell - it's the rule engine industry's stab at trying to grow their measly market share, compared to the workflow industry.

The jBPM team have put out their vision of the "Process Virtial Machine" (PVM), but it is a process centric vision. The PVM was a term used by Tom Baeyens, in the linked paper, to present the idea of a generic engine for executing different process models, the "virtual machine" term may not be totally appropriate, and already irks some purists, but we have continued with this terminology for mean while - so we can get apples to oranges comparisons, instead of apples to giraffes :) Mike Brock suggests that Virtual Process Engine, or Generic Process Engine - I prefer something away from the terms process and rules, to something that focuses on the unified modelling concepts, so hopefully someone out there can put in an argument for something more appropriate :)

What we lay out here is what we have started to put in place with Drools, our vision of a PVM+ with rules and processes as first class citizens, tightly integrated modelling GUIs, single unified engine and apis for compilation/building, deployment and runtime execution.

So with the base PVM in place what we are working on now? We'll we still have a lot to do, I've found our compilation framework is too coupled to rules, so I'm busy trying to refactor this so it can compile and build both rules and actions. The engine variables are currently only scoped at two levels, globals and rules; we need to make sure that we can scope variables by both process and sub process, and have the rules executed in those processes also scoped to that level. I need to extend our concept of a rule "duration", which is basically a simple timer, to allow for cron type definitions and allow rules to execute each time, if its still true - this will allow for rich conditional timers. I have plans for stateful high availability, via JBoss Cache, and also I need to put in an optimal framework for persistence and restoring - ideally I want all this done, and more by Q1 :) We do not plan to do the BPEL, BPM etc layers and instead hope the jBPM team will become consumers of our tech, and also core developers (a joining of the two teams), and work on these parts of the domain.

The rest of this blog is a small paper put together by our ruleflow lead Kris Verlaenen, but exemplifies the whole Drool's team vision and commitment to declarative programming, via multiple modelling paradigms - no one tool fits all solutions.

The Process Virtual Machine (PVM)
This is an attempt to clarify our vision on an integrated approach for modelling business logic using rules and processes on top of the Drools Platform. It is intended to serve as a glossary, to create a common set of terms that might help in simplifying future discussions and creating a combined vision regarding this matter.

Figure 1 shows an overview of our approach to unify rules and processes by integrating a powerful process virtual machine (PVM+) into the Drools Platform. This allows us to support the execution of rules as well as the execution of processes based on this PVM+ within the Drools Platform. We believe that creating a unified approach for handling rules and processes (for the end user) will result in a much more powerful business logic system than what can be achieved by simply linking separate rules and workflow products. It will also allow us to create a lot of additional services on top of this unified platform (IDE, web-based management system, etc.), which can then be applied easily for both rules and processes, giving a much more unified experience for the end users of the platform. Each of the terms used in the figure will be explained in more detail in the subsequent sections.

Figure 1
PVM
The Process Virtual Machine defines a common model that supports multiple process models. It is the basis for implementing workflow process models, and their implementation. It represents a state machine that can be embedded into any software application. Therefore it defines:
  • A process (definition) model: Defines concepts like a process, variables, nodes, connections, work definitions, etc.
  • A runtime model: Runtime instances corresponding to each of the elements in the process model, like process instance, variable instance, node instance, work item, etc.
  • API: Process instances can be started, aborted, suspended, the value of variable instances can be retrieved, work items can be completed or aborted, etc.
  • Services: The PVM also implements (non-functional) services which are useful for most process language implementations, like persistence, transaction management, asynchronous continuations, etc. These services should all be pluggable (do not have to be used, minimal overhead if not used) and configurable (different strategies could be used for each of these services, this should be configurable and extensible so people can plug in their own implementation).
On top of this process model, the PVM also defines/shows how to use the concepts of process (instance), node (instance), connection, etc. to implement common workflow patterns (in control flow, data, resource, exceptions) like a sequence of nodes, parallelism, choice, synchronization, state, subprocess, scoped variables, etc. These node implementations can be used as a basis for implementing different process languages.

PVM+
Extends the PVM and integrates it into the Drools Platform. This allows:
  • Integration of rules and processes: Processes can include (the power of) rules in their process model whenever appropriate, e.g. split decisions, assignment of actors to work items, rules as expression language, etc. vice-versa, rules can start processes during their execution.
  • Processes and rules share one common data contextl, no need to integrate two (or more) different systems, continuously pass information between those two systems, synchronize data, etc.
  • Processes (and rules) can use other functionality that is offered by the Drools Platform: a unified audit system, unified API to start processes / rules, single build and deployment infrastructure etc.
  • One engine session can execute multiple different process instances in parallel, where each process can interact with the other processes and rules via changes to the shared variable context.
This PVM+ also defines additional node implementations that show the power of integrating rules and processes and how that power can be used inside a process model, e.g. choice using rules to evaluate conditions, milestones (a state where rules decide when to progress to the next state), timers with built in conditionals, actions supporting pluggable dialects, etc.

Specific workflow languages
On top of the PVM+, different (domain-)specific workflow languages can be implemented:
  • jPDL: the general purpose, expressive workflow language for the Java developer
  • PageFlow: workflow language for specifying the control flow in web pages
  • RuleFlow: a workflow language for specifying the order in which large rule sets should be
  • evaluated
  • WS-BPEL: an implementation of the WS-BPEL standard for web service orchestration
  • ...
These languages each define a process model and implementation for each of their nodes. These
implementations will be based in a lot of cases on (a combination of) common node implementations of the PVM(+).

Pluggability: New node implementations can be added to existing process languages, existing
process languages can be extended with new functionality (e.g. time constraints), or entirley new process languages can be plugged in into the PVM+.

Work Definitions
All communication with the external world is handled by using work items, which are an abstract representation of a unit of work that should be executed. Work item handlers are then responsible for executing these work items whenever necessary during the execution of a process instance. This approach has the following advantages:
  • A much more declarative way of programming, where you only define what should executed (using an abstract work item), not how (no code)
  • Hides implementation details
  • Different handlers can be used in different contexts:
    • A workflow can be reused without modifications in different runtime execution contexts (e.g. different companies or different hospitals in the context of clinical workflow) by creating custom handlers for each of these settings
    • A workflow can behave differently depending on its stage in the life cycle. For example, for testing, handlers that do not actually do anything but simply test the execution of the workflow could be registered. For simulation, some visualization of the work items that should be executed, and the possibility for the person doing the simulation to complete/abort these work items is possible.
  • Work item definitions and handler implementations can be reused across nodes, across
    processes, and even across process models.
The different work items that are available in a specific workflow languages should be defined (by defining a unique id for that type of work item, and parameters for that work item). Different sets of work definitions can be defined:
  • Generic work definitions (and their handler implementation) can be defined for common
    task that might be useful in different workflow languages, e.g. related to communication
    (sending a mail, SMS, etc.), invoking a web service, logging a message, etc.
  • People can define their own domain-specific work items (and their handler implementation), which can then be used for modeling processes in that domain. For example, a clinical workflow language could define work items like “nursing order”, “medication order”, “contact general practitioner”, etc.
Extensions
When a unified approach to processes and rules is used, as part of the Drools Platform, extensions on top of these concepts and APIs can easily be reused for all rules and process
languages:
  • Eclipse-based IDE supports developing applications on top of the Drools Platform
    supporting the use of rules and processes. This IDE includes
    • a graphical workflow editor
    • unified error handling
    • integrated debugging
    • unified simulation
    • pluggability of process languages, custom property panels, etc.
    • ...
  • B(R)MS: Business (Rules) Management System, a web-based application that serves as the repository for all business knowledge. Supports unified packaging, versioning, management, quality assurance, etc.
  • Security management: who is allowed to perform which operations on the Drools Platform.
  • (Human) task list management component that can be shared across rules and process
    languages, for integrating human tasks.
  • Reasoning on business logic, which is a combination of all rules and processes of a business.



Pluggable work items (currently in svn trunk)

I'll be at Javapolis this year presenting a BOF on the concepts of Declarative Programming with Rules, Processes and CEP which will cover most of this blog and more. The BOF is on Monday 10th from 21:00 to 22:00. So please do come along, if you want to talk about this in more detail.

Share/Bookmark

Friday, November 16, 2007

How Big is too Big?

The JBoss Drools boys have something cool brewing. They already have the most useful GWT (Google Web Toolkit) App that I've seen outside of the Googleplex. That's a fully fledged app ready and waiting to be used in anger, not some example widget, or a test case thrown together by somebody playing with the technology. The BRMS app itself, targeted at business / professional users, allows them to catch the knowledge that they have in their heads and share it with the team.

Here's the problem; If you're into Rules, you've probably already downloaded the BRMS - ( Business Rules management system). If you're just a casual browser, as cool a GWT app it may be, but you're not going to download. Why? Unless you have Tomcat or JBoss 4 ready and waiting on your PC you're not going to bother (note ; this will work on other application servers like weblogic , with another couple minutes work, but that just proves my point).

In order to make things easy for us lazy people, the nice people at Drools are preparing a standalone BRMS; download , unzip , start and play. That's presuming you have Java installed. You do have Java installed don't you? But being lazy, how big a download is too big?

I'm not the best person to answer the question - I've already downloaded the (largish - 530mb) Red Hat Developer Studio (now at release candidate 1). So what's your download speed and how big (small) does the BRMS have to be before you'd consider trying it out?

Answers on a postcard below.




Share/Bookmark

Drools User Mailing List Growth Problems



As the above graph shows, ignoring spikes, there has been a steady uptrend in the growth for usage of the Drools user mailing list. One of the things the core developers are proud of is our community support, we try our best to answer all emails and to date have mostly succeeded in this endevour. However it now seems there is a constant stream of emails and regrettably we are no longer able to reply to all of them, the community itself is starting to answer more of its own questions which is helping a little, but many questions are now going unanswered.

I know that myself I tend to work in bursts, scanning the unanswered messages and replying where possible. Obviously the less vague and easier to answer questions tend to get replies. So now more than ever if people need a response they need to be more diligent in how they phrase their questions and the supporting information they supply.

A little while ago we put up on the mailing list page a few pointers on how to have your emails ignored, its worth repeating those here:
  1. Start the email with "URGENT"
  2. Tell everyone how desparate you and how you need an immediate response for your really important project.
  3. Don't wait a minimum of 3 days and resend your email within minutes of hours.
  4. Send emails directly to mailing list members, especially the developers.
  5. Paste pages of code and then say "it doesn't work, please help".
  6. Paste a long stack trace and say "it doesn't work, please help".
  7. Start your email with "please sirs" or include "do the needful".
  8. Ask dumb questions that are in the manual.
  9. Ask basic java questions.
  10. Ask questions about JRules
  11. Reply to an existing thread but starting a new topic
  12. Start your email with "I'm a member of the drools mailinglist, so please answer me"
  13. General begging and pleading
  14. Say some thing to the effect of "Please tell me how I can call a function on the LHS. (hurry my assignment is due today!)"
The positive side is that it shows the Drools community is growing, so we must be doing something right, and as we become unable to answer all questions more people wil have to turn to support subscriptions from JBoss with guaranteed response times - so hopefully this will atleast make my managers happy :)
Share/Bookmark

Thoughts for the Business Rules Forum and RuleML Conference

This year I got invited to attend two expert panels, one for the BRF and one for the RuleML conference, I also gave two presentations. One demonstrating Drools' ruleflow capabilities extensions to rules and the other on an interoperability exercise I did with ilog and Mismo.

One of the things I like most about going to the Business Rules Forum is the people I meet, especial the old school guys who helped founded the technology and the industry. Last year I had dinner with Charles Forgy and this year Paul Haley - both occasions were the highlight of my trip. Paul Haley was involved in the development of ART, which was a hugely powerful state of the art Expert System for it's time (with features we still have yet to do in Drools) and the author of the Eclipse rule engine (not IDE), which formed the tech foundations for Haley Systems Inc - "The Evolution of Rules Languages". Which reminds me, if anyone has any ART or Eclipse manuals hanging around, please do email/post my way :)

What's great about meeting people like Charles and Paul is that they've been thinking about rule engines for 20+ years, while I've just been doing it 5 years, so when I tell them about my latest R&D they already know the various possible solutions and pitfalls - so drawing out nuggets of gold from these guys is immensely beneficial :) For instance I mentioned to Paul that our RuleBases are fully stateless allowing multiple sessions to share the same RuleBase, without concurrency issues, the sessions themseles are light weight negating the need to have any form of pooling. If you have a large number of sessions executing on the same RuleBase, how do you manage a RuleBase update? We currently iterate and lock all sessions, apply the change, and then iterate and unlock all. For most situations this works fine, but if you have a very large number of sessions this can create quite a pause. Paul suggested that instead of locking all the working memories we allow the RuleBase two exist in two states, so sessions don't get locked and only see the updated RuleBase when they are ready, when all sessions are viewing the new state of the RuleBase, we can stop maintain two states in-favour of the most recent. As soon as he said it I realised what he meant, which gave me a doh! moment :) We don't have time to implement this now, but it's certainly gonna simmer away until we do. Thanks Paul, looking forward to extracting more nuggets from you - now where are those thumb screws ;)

While on the subject of Paul the Drools user mailing list today received an email from him, which I take as a huge compliment and I hope he doesn't mind if I share that here:
"Haley Systems (www.haley.com), the company that I founded many years ago (hopefully, some of you have heard of it!), is in the process of being acquired by Ruleburst. I am not going with the acquisition but have started up my own "vendor neutral" business practice in which I anticipate helping improve and support Drools and the emerging standards in rules and web semantics. "


While there I met the infamous "Smart Enough Systems" author James Taylor, who was expectedly as large and entertaining in life as he is in his blogs :) I also met Paul Vincent for the first time, having spoken to him once before and followed his emails on the RIF mailing list, was good to see that Paul's dry humour extended from his online communication to his offline :) Although it seems the thumbscrews didn't work on him and I didn't learn all of Tibco's secrets, heh mental note "more drink needed next time". Although he did give me an interesting insight into a new R&D area where Tibco use ghant charts to represent processes and use cep to effect re-organisation of those changes, sounded kinda cool.

Pega Systems where there as usual. I really like what they do, and have been very much inspired by their "modelling" approach, like them I don't subscribe to a rules or process view of the world that has dogged the current industry main streamers, but a unified "modelling" view where rules and processes are first class citizens. I again met Jon Pellan, who while being very busy, took a few moments to show me over their rule engine (which I really appreciated), it was only a quick glimpse and I didn't gain enough information to draw any conclusions - but their approach looked interesting, and it certainly was different to what anyone else in the industry is doing (I think), it was more like the data flow approach that Stanislav Shor once told me about. At compile time you determine for each change on a fact which are the possible rules to evaluate that fact against, and pull in just those rules for evaluation - PegaRules creates a hash of those rules against the field changes, it then uses subgoaling to match the data against the chained conditions (hope I got that right). I'd really like to learn more about their system, maybe next year, as I was left with not understanding how it would handle joins and a better understand of their subgoaling.

I also spent quite a lot of time with Benjamin Grosof and Said Tabet. Benjamin is the Senior Research Program Manager, from Vulcan and the author of systems like SweetRules, Benjamin was also previously an MIT Sloan Professor - as you can expect he's wickedly smart in real life :) Said Tabet is best known for his RuleML work but also has a huge wealth of experience in the commercial application of production rules systems. We talked a lot about the role that Drools could play in helping with RuleML and Benjamin took me over the SweetRules system and explained his innovative way of deal with the inconsistent way that PR systems deal with 'not' compared to prolog when porting rule syntaxes to different engines. It was a delight spending so much time with these two, and I hope I get to do it again some time.

One of the other events going on, that I really enjoyed, was the RuleML conference - which is where I meet-up with Benjamin and Said. This was a much more academic conference of like minded individuals sharing information. I also finally got to meet the Mandarax author, Jens Dietrich, and the Prova author, Adrian Paschke. Jens is now working on backward chaining derivation rules based system called Take - which I'm going to research for possible integration into Drools this xmas.

I met plenty of other people, but this blog is already getting way too long and its now late, so time for me to draw it to a close and go to bed :)
Share/Bookmark

Friday, November 09, 2007

IJTC and new Production Rule (Rete) explanation in slides

Just got back from the Irish Java Technology Conference, for this presentation I had another go at trying to explain how a production rule engine works - the behaviour, not the algorithm. As I've mentioned in the past I'm finding it easier to talk about SQL to begin with, to frame people's minds. If you jump into an example, straight away they are thinking, or asking, "how is this different from an 'if' statement in java". By taking the SQL approach you hopefully break that problem.

The presentation takes 3 tables with data and shows the resulting rows for 2 different views, and how data might change if we had triggers on those views. So it gets people thinking in terms of cross products and rows of data (tuples). I then show the same data against rules and the resulting rows, which is identical to the views. Showing that basically a rule is a view on data, resulting in rows (tuples) of matched facts. The consequence is executed for each resulting row. This concept is taken further to say that if each rule is a view then the agenda is just an aggregate view of all the rule views. As you insert, retract and update data, rows are added and removed from the "agenda view".

I then introduce the idea of conflict resolution and salience, along with two phase execution, as a way to determine which of the rows in the agenda view have their consequences fired first; a new simple rule with a salience is added, along with the resulting agenda view tables show the impact of this. The presentation then goes on to touch first order logic, specifically 'not' and 'accumulate' and details our ruleflow work, the normal screen shots are supplied for explaining the rest of the capabilities of the system. I also did a populated BRMs demo at the end.

Do give me feedback on my approach to helping people understand, via the sql and views analogy, production rule systems - I'd certainly like to try and improve the slides to help explain this better.

You can get the slides here
Share/Bookmark

IJTC and new Production Rule (Rete) explanation in slides

Just got back from the Irish Java Technology Conference, for this presentation I had another go at trying to explain how a production rule engine works - the behaviour, not the algorithm. As I've mentioned in the past I'm finding it easier to talk about SQL to begin with, to frame people's minds. If you jump into an example, straight away they are thinking, or asking, "how is this different from an 'if' statement in java". By taking the SQL approach you hopefully break that problem.

The presentation takes 3 tables with data and shows the resulting rows for 2 different views, and how data might change if we had triggers on those views. So it gets people thinking in terms of cross products and rows of data (tuples). I then show the same data against rules and the resulting rows, which is identical to the views. Showing that basically a rule is a view on data, resulting in rows (tuples) of matched facts. The consequence is executed for each resulting row. This concept is taken further to say that if each rule is a view then the agenda is just an aggregate view of all the rule views. As you insert, retract and update data, rows are added and removed from the "agenda view". I then introduce the idea of conflict resolution and salience, along with two phase execution, as a way to determine which of the rows in the agenda view have their consequences fired first; a new simple rule with a salience is added, along with the resulting agenda view tables show the impact of this. The presentation then goes on to touch first order logic, specifically 'not' and 'accumulate' and details our ruleflow work, the normal screen shots are supplied for explaining the rest of the capabilities of the system. I also did a populated BRMs demo at the end.

Do give me feedback on my approach to helping people understand production rule systems, via the sql and views analogy, I'd certainly like to try and improve the slides to help explain this better.

You can get the slides here
Share/Bookmark

Tuesday, November 06, 2007

Drools now has 1725 unit and integration tests

One of the great things about Open Source is we are totally open and transparent, so it's very easy to make a judgement on the level of quality of the software and the efforts gone into QA. On this note we would like to bring to everyone's attention that Drools now has 1725 unit and integration tests - which I think is high by anyone's standard - none of these tests were produced by code generation. This report is shown as part our Hudson built test results page, https://hudson.jboss.org/hudson/job/drools/983/testReport/.

Our Hudson build server, https://hudson.jboss.org/hudson/job/drools/, builds Drools after every commit and makes distribution zips publicly available here , so you can always get the latest trunk build for your own testing.
Share/Bookmark

Irish Java Technology Conference - 9th of November

I'm at the Irish Java Technology Conference doing a talk on Drools. My talk is this Friday the 9th of November from 11:00 to 12:15, where I'll be demoing our BRMS.
Share/Bookmark

More great articles

John Dunning has produced a fantastic blog entry summarising his perforamance characteristics research at EBRD, the European Bank of Reconstruction and Development - "Benchmarking Drools" which was a follow up on his "Banking on Drools" blog. In this he takes a simple problem and solves it a number of different ways, benchmarking each, he details all the variations with several perf charts comparing the approaches. Someone else has recently taking a variation (making it harder) of his benchmarks and run it under JRules, with very surprisingly results, I'll provide more details later on when the results have been better verified :) I'll also try and get a zip where you can run this benchmark for Drools and JRules yourself, as luckily ILog have now made their software easily obtainable for trial purposes. Lets's put it this way, which system do you think scales easily to over 500K, even 1mill, objects, and which one didn't :)

Steve Shabino has started a blog on his experience with Drools, as they have an extreme problem to solve involving the need to reason over 2mill objects in a stateful session. I'm really looking forward to seeing his findings.

The mysterious Yaakov Kohen has started blogging again with an insightful blog on the problems with todays acadmic benchmarks.
Share/Bookmark

Tuesday, October 30, 2007

Drools BRMS version two design and ideas

Hi folks

After delivering 4.0.3 Michael and I had some conversations about adding more features and a complete re-design of the BRMS web interface. Basically we are looking at how to break things up into "perspectives" kind of like eclipse has (but simpler), just to make it easier to present the user with the information they need for their current work example:
  1. BA type users (no package configuration, follow templates, read only sometimes etc)
  2. Developers (configure stuff - eventually should only need eclipse so this doesn't have to be too strong on the web side)
  3. Admins/operator types (migrate, security setup etc).

Nothing that complex, but we do need to have it scale up and down to these skill levels.

I think we can do something like merging the previous BRMS organization with the new tree feature. We do like the tree/explorer motif, fairly familiar to most people and easy to make it filterable based on both server side and user settings. Also, its more like ideas that the jboss web console is moving to.


The new package feature



Categorization



Administration


Also with this refactory we will enable I18n internationalization then people can easily edit the bundle file/config to add they own language setting. Other key features like rule based authorization and custom queries are also some of the new features.

We are messing around with a few designs and also very receptive on ideas and feature requests you can find our task list on Jira and suggest something, also you can ping the dev mailing list on rules-dev@lists.jboss.org

The new BRMS design concept

Share/Bookmark

Monday, October 29, 2007

Drools and SBVR

During my time at the Business Rules Forum and RuleML I met a big company, one of America's largest medical insurance companies, and they are very interested in funding a SBVR authoring environment for Drools, possible utilising the existing (but inactive) SBeaVer project. For those that don't know SBVR is a standard for representing rules using a structured natural language approach.



We are currently looking to create a group of stake holders, managed by the existing SBVR group, who would be interested in funding this work to make a usable reference implementation authoring environment for the SBVR spec. If you are interested and able to help, or know someone that might be interested and able to help, then please email me mproctor a@t redhat d0t com. The idea is to finding initial phased funding for the work and to provide an ongoing maintenance fund. The SBVR group are currently evaluating a number of 3rd parties to undertake this development effort, once funding has been resolved.
Share/Bookmark

Wednesday, October 24, 2007

Business Rules Forum - Quick Notes

As Mark mentioned in a previous blog, we are attending the 10th Business Rules Forum in Orlando. Tuesday (October 23rd) was the first presentations day and I had the opportunity to watch a few interesting presentations.

In the morning I watched the keynote with Ronald Ross: "The Hitchhicker's guide: from Business Rules to Enterprise Decisioning"
It was an interesting presentation whose keyword, as emphasized by Ron was "balance". He presented the needs to change the current focus:
  • from BPM to balanced Enterprise Decision Management
  • from Data Quality and Accessibility to balanced Decision Deployment
  • from Business Rules to Balanced Decisions

The real deal, in his words is to understand that Business Rules are the means to an end, and as such we need to stop focusing and justifying the means and start focusing and justifying the ends, i.e., the implementation and deployment of complete Enterprise Decisioning solutions.
Another point I thought was also interesting was his statement, that I completely agree, that the decisions we need to automate are not the big strategic and broad company decisions, but the small, frequent, everyday, thousands of operational decisions, as these are the ones that will give us the greatest returns and allow us to quickly answer to trend changes.

Another great presentation I attended and worth mentioning was "Business Rules, Decision Management and Smarter Systems" by Neil Raden and James Taylor, authors of the recently released "Smart (enough) Systems".
They are very good presenters and talked a lot about what are "Smart (enough) Systems", the challenges and opportunities, but again, the highlight point for me was the realization that today, at least 80% of the decisions made everyday in a business environment are what they call "hidden decisions". I.e., the small decisions manually taken by people involved in the business that are not documented and automated that inevitably lead to inconsistencies and unpredictability on the business results.
The idea is that expert decisions will continue to be made by experts for a long time still, some big/broad decisions will continue to be made by people in charge, but to ensure consistency, predictability, trusted statistics and allow business optimization, these hidden decisions need to be uncovered and automated.

The other presentation I would like to mention and that was in my opinion the best presentation so far in the event was Stephen Hendrick's (IDC Group VP) keynote: "Intelligent Process Automation: The Key to Business Process Optimization".
Stephen presented us with the big picture for the coming years. Where to go, from a business process optimization perspective, what are the challenges and the gains of implementing IPA, a few success cases of companies that already have IPA solutions in place and an illustration case of what may happen to those that don't.
In his words, the key IPA constructs are:
- Event Processing
- Decisioning
- BPM
- Advanced Analytics
He also emphasized that solutions for all the key constructs above already exist, but we still fail in delivering completely integrated solutions that minimize risks and maximize gains of customers implementing them.

At least, we know that us in Drools development are heading to the right direction. As you probably know, we are tackling the Event Processing and the BPM integration side in our solution and improving our rules engine product to become more than a rules engine: a real Enterprise Decisioning tool in a complete Behavioral Modeling platform.
Anyone interested in helping us with the Advanced Analytics side? :)

Happy Drooling,
Edson
Share/Bookmark

Tuesday, October 23, 2007

Drools 4.0.3 Released

We just released Drools v4.0.3. This is a minor release with a few improvements on existing features and some bug fixes.

Release Notes - JBoss Drools - Version 4.0.3

We would like to really thanks all the contributors that helped on getting this release out. From those contributing patches and docs, to those testing and reporting bugs and providing feedback. The list is a bit long to post all names here and I may incur in a mistake forgetting someone, so our open public thank you to you all!

Follows the release notes.

Happy Drooling
Drools Team

Bug


  • [JBRULES-1264] - NPE at BaseObjectClassFieldExtractor.getLongValue with null fields

  • [JBRULES-1266] - Composite facts types (OR, AND) not rendering correctly

  • [JBRULES-1272] - DSL : String index out of range: -1

  • [JBRULES-1279] - Memory leak in release 4.0.2

  • [JBRULES-1281] - ExecutorService cannot be shared

  • [JBRULES-1282] - Problems uploading models etc. into the BRMS

  • [JBRULES-1283] - Unable to serialize rule base ( NotSerializableException: org.drools.base.FireAllRulesRuleBaseUpdateListener)

  • [JBRULES-1295] - DSL Mapping files does not support comments and empty lines

Feature Request

Task


Share/Bookmark

Sunday, October 21, 2007

Business Rules Forum

Edson and myself are at the Business Rules Forum 2007 conference. We arrived yesterday and have settled into the hotel, which btw is really nice. If anyone else is here and wants to chat tech stuff, just ping us. Also if anyone is up for it maybe we can organise a pub crawl for a drink or three :)

Mark
Share/Bookmark

Tuesday, October 09, 2007

Drools 4.0.2 Released

We just released Drools v4.0.2. This is a minor release with a few improvements on existing features and several bug fixes. We also included a lot of new documentation, specially in the examples section.
Important: there is one backward compatibility break when using Rule Flow. If you use Rule Flow, please check the section 1.2.5 of the manual for details.

We would like to really thanks all the contributors that helped on getting this release out. From those contributing patches and docs, to those testing and reporting bugs and providing feedback. The list is a bit long to post all names here and I may incur in a mistake forgetting someone, so our open public thank you to you all!

This is now the stable version of Drools and from now on, we expect to only release eventual bug fixes in this branch. We are starting now the development of the new major release of Drools and new features shall all be added to it.

For those curious about what is coming, please check previous posts on this blog. Mark's slides presented at Synasc also talk about several things we want to do for the coming versions.

Follows the release notes.

Happy Drooling
Drools Team

Release Notes

Sub-task

  • [ JBRULES-779 ] Better error reporting in the GUI
  • [ JBRULES-996 ] Admin screen option to remove archive items
  • [ JBRULES-1210 ] Implement support to multi-pattern accumulate to the XML parser.

Feature Request

  • [ JBRULES-910 ] upgrade GWT to 1.4
  • [ JBRULES-944 ] Support in Rule Workbench for custom colors
  • [ JBRULES-1040 ] Multi-pattern accumulate support
  • [ JBRULES-1098 ] Add action support in ruleflow
  • [ JBRULES-1107 ] Support for multiple pattern in accumulate source pattern
  • [ JBRULES-1118 ] Add support for rf, xml and dsrl into drools-ant task
  • [ JBRULES-1138 ] MVEL code completion does not work for nested expressions
  • [ JBRULES-1155 ] In Decision Table actions, cannot reference variables that were not referenced in LHS
  • [ JBRULES-1165 ] StatelessSessionResult should have getGlobal
  • [ JBRULES-1175 ] soundslike operator
  • [ JBRULES-1178 ] trivial jbrms issues
  • [ JBRULES-1183 ] Add global support in ruleflow
  • [ JBRULES-1184 ] Refresh snapshot binaries if needed
  • [ JBRULES-1185 ] Add support to RuleBase events on the Event Support system
  • [ JBRULES-1189 ] Auto coercion of String to date in MVEL actions
  • [ JBRULES-1196 ] Cannot specify file and dir paths with spaces using RuleAgent properties file
  • [ JBRULES-1198 ] Add a launch shortcut for debugging drools applications
  • [ JBRULES-1201 ] Allow rule agent to pick up DRL files and compile as needed
  • [ JBRULES-1227 ] Add drools-analytics support to drools-ant
  • [ JBRULES-1228 ] Enums to allow reference to another fields value
  • [ JBRULES-1236 ] Sort getters alphabetically in drop-down
  • [ JBRULES-1241 ] Dynamic rules should be able to lock for a series of updates and fireAllRules should be optional
  • [ JBRULES-1252 ] DrlDumper does not dump import functions
  • [ JBRULES-1255 ] add alternating coloured rows for rule analysis reporting and allow for user provided css
  • [ JBRULES-1256 ] publish source jars with maven
  • [ JBRULES-1257 ] Add Banking Example from John Dunning

Bug

  • [ JBRULES-723 ] Eclipse plugin detects error inconsistently and without location information
  • [ JBRULES-1013 ] Two functions with Janino compiler end up with out of memory error
  • [ JBRULES-1100 ] BRMS Bug on view source in weblogic
  • [ JBRULES-1124 ] Code completion does not work for 'with' blocks
  • [ JBRULES-1141 ] context assist no longer works for functions in the drl.
  • [ JBRULES-1144 ] Toggling breakpoint gives an exception
  • [ JBRULES-1145 ] Break points do not work at all
  • [ JBRULES-1146 ] No variables are shown for Debug points
  • [ JBRULES-1148 ] Out of memory exception asserting more than 100 facts
  • [ JBRULES-1151 ] Bulding sources from root folder fails
  • [ JBRULES-1152 ] my unit tests are hanging with drools 4.0.1
  • [ JBRULES-1153 ] Accumulate/Collect can't be used in queries
  • [ JBRULES-1154 ] ClassCastException retrieving query parameters
  • [ JBRULES-1156 ] WorkingMemoryFileLogger is raising NPE for rules containing unbalanced OR CEs
  • [ JBRULES-1157 ] threads using one ruleset throws exception
  • [ JBRULES-1159 ] Function calls or static method calls do not work for the MVEL dialect
  • [ JBRULES-1164 ] DSL is not working well when mapping includes ()
  • [ JBRULES-1166 ] BRL not rendering inpattern eval() correctly (for a predicate)
  • [ JBRULES-1169 ] PredicateConstraint and/or MVEL Constraint issue?
  • [ JBRULES-1170 ] ClassCastException using dynamic salience
  • [ JBRULES-1171 ] Nested accessors in "from" clause are not supported in depth more then 2
  • [ JBRULES-1172 ] In nested accessors "this" keyword is required in front
  • [ JBRULES-1174 ] debug steps into "when" statement
  • [ JBRULES-1177 ] Problem when adding dynamic rules to networks with shared AlphaNodes
  • [ JBRULES-1179 ] Premature rule execution
  • [ JBRULES-1180 ] java.lang.ClassCastException: org.drools.reteoo.RightInputAdapterNode
  • [ JBRULES-1182 ] serialized rules using MVEL depend on a compiler class
  • [ JBRULES-1187 ] Bug in guided editor when you add a boolean restriction field
  • [ JBRULES-1190 ] Can't use "=" in a DSL expression
  • [ JBRULES-1191 ] compilation error when consequence has no space between rule mappings
  • [ JBRULES-1193 ] Getting org.mvel.PropertyAccessException in rule flow actions when using class names without package
  • [ JBRULES-1202 ] primitive types all return Longs for getValue
  • [ JBRULES-1203 ] Shadow facts of final classes not working correctly
  • [ JBRULES-1204 ] Problem with memberOf and/or eval used in collect statement
  • [ JBRULES-1205 ] AND works in a rule, but OR gives a classCastException
  • [ JBRULES-1207 ] RuleBase serialization does not handle the process field
  • [ JBRULES-1208 ] Errors for some OR junctions in LHS conditional elements
  • [ JBRULES-1211 ] Rules with accumulators cannot be serialized
  • [ JBRULES-1212 ] Rete throws NoClassDefFoundError when asserting facts with classess loaded by inaccessible classloader.
  • [ JBRULES-1217 ] RuleFlow Editor -> Constraint Editor -> Imports Editor context help
  • [ JBRULES-1220 ] Integration with jBPM does not work
  • [ JBRULES-1221 ] org.drools.base.accumulators.CountAccumulateFunction error when building a package with rules including an accumulate construct in Drools BRMS 4.0.1
  • [ JBRULES-1223 ] Concurrent problem in MVEL
  • [ JBRULES-1224 ] regular expressions inside mvel expressions require escaping
  • [ JBRULES-1231 ] Functions do not get added to new BRMS package when package imported from drl file
  • [ JBRULES-1239 ] Fix class loaders so that creating multiple rulebase with separate class loader hirarchy works
  • [ JBRULES-1240 ] Remove Dialect state from PackageBuilderConfiguration
  • [ JBRULES-1244 ] Ruleflows should have a package name
  • [ JBRULES-1245 ] class cast exception
  • [ JBRULES-1248 ] requires an 'value' attribute, but it can't be null
  • [ JBRULES-1253 ] drools-examples-drl does not work under Java 6 as-is

Task


Share/Bookmark