Monday, October 27, 2008

DSL Widgets in Guvnor

The Drools DSL format has been updated to allow for the addition of metadata to provide for user friendly widgets when using a DSL in combination with the guided editor. The DSL sentence can now display a dropdown for an enumeration, a date selector, a checkbox, and allows for the restriction of the value that can be entered in a text field via a regular expression. The regular expression validates against the contents of the text box and displays a message to the user when an invalid value is entered.Examples of each format are shown below along with screenshots from Guvnor

Defining a dropdown driven by a Guvnor Enum:

[when]When the credit rating is {rating:ENUM:Applicant.creditRating} = applicant:Applicant(credit=="{rating}")

Define a date selector via DSL:

[when]When the applicant dates is after {dos:DATE:dd-MMM-YYYY}=applicant:Applicant(applicationDate>"{dos}")

Defining a checkbox via DSL:
{<variablename>:BOOLEAN:defaultValue[checked | unchecked]}

[when]When the applicant approval is {bool:BOOLEAN:checked} =applicant:Applicant(approved=={bool})

Define a restriction on a textbox via a regular Expression:

[when]When the age is less than {num:1?[0-9]?[0-9]} = applicant:Applicant(age<{num})

Sunday, October 26, 2008

Symmetrical and Asymmetrical Rete

In my endeavours to understand Clips I realised that it's Rete implementation was different to the more classical approach, as used by Drools 4. I haven't read anything in the past that attempted to give this difference a name, so I adopted the terminology of symmetrical Rete and asymmetrical Rete. It was really nice to see this decision validated by Gary Riley, the author of Clips, who used the terminology in his presentation on Rete enhancements in Clips at the Rules Fest in Texas.

Here is a quick explanation of the difference, without getting into the details on merit, in Drools 4 we use the classical approach of the retract being the same work as assert, except on the assert data is added to the memory on retract data is removed. By this I mean that the fact propagates through the network, for each join node the constraints are evaluated to determine the joins, the join is made and the fact plus the new join fact are propagated, each propagation into a new join node is added to the join node's memory. The retract is exactly the same, the constraints are re-evaluated to determine the previous joins so that the join and propagation can be remade, this time the facthandle and it's joined data are removed from the join node memory. With the work done for the attract and assert being the same I dubbed this symmetrical Rete.

In Drools 5 we adopted the Clips algorithm. Here the assert uses a lot of linked reference so that the propagation creates a chain of tokens. The retract can now iterate this chain of references removing data from the node memories, the joins do not need to be recalculated to determine which facts we joined against, we already know this as we have the references. As the work done for the retract is now different to the assert I dubbed this assymetrical Rete.

While everyone was having fun at ORF... (some of us got some work done !)

I (Michael) was not able to travel to texas for the meetup, due to an imminent addition to the family.

However, with the wonders of 2003 technology I was able to "attend" in a sense - Skype and Elluminate (skype had nicest voice quality - at least to my ears).

Just a quick update of some new features that fell out of some work that happened during ORF/rule meetup:

Jeff Delong requested that the Scenario runner be able to show an "audit log" of events (tracing the execution of the rules - this is similar to what you can do in the IDE using the logging event listener and the plugin. This was relatively easy to to:

So after you run a scenario, you can show the event log to trace the execution - very useful for explaining how the rules came to a certain conclusion. This shows all events (including ruleflow should it be invoked). Note that if you are using "pojo" facts, be sure to put in a sensible toString() method to show the state (the above ones did not, hence the "pointer address" style messages !).

Michael from Franklin American was at the rules meetup, and contributed a feature called "category rules" (more details on this in future) - it is a really clever idea: you specify that rules tagged in a certain category "extend" a specified rule. Before he could do this he had to add the "extends" feature to rules - so a rule can extend another rule (it "inherits" the conditional part - just single inheritance - very simple !).
(excuse my random keyboard-mashing names - unseasonal cold weather has locked up my joints for typing !). This is quite a neat idea - you could use it to simplify rule authoring so that the mere fact that a rule belongs in a certain category implicitly applies extra filtering constraints to it (that was the case Michael had in mind, I am sure he will provide a more concrete example in the future).

Also, Joe of Recondotech contributed an enhancement to DSL's to allow drop downs, dates and such to be configured in a DSL and an editor rendered accordingly - which provides a more user friendly way of building form-style UIs for business users.

Thursday, October 23, 2008

October Rules Fest (day 1) and the upcoming RuleML conference

As mentioned before, we are all attending the October Rules Fest in Dallas. The very first thing that I said when I saw the presentation list was:

- Wow!!

The October Rules Fest group managed to host the best technical convention on BR I ever attended to, and managed to have not only the most known products and vendors attending, but specially some of the most known researchers in the rules engines field. Take a look at the speakers and the agenda: we have Charles Forgy, Gary Riley, Gopal Gupta, .... you name it!

Yesterday was the first day and it was a very good day. The presentation that most intrigued me was "Why systems fail, Why systems work" by Rolando Hernandez. I need to say that I was not familiar with the Zachman Enterprise Architecture Framework (wikipedia), but when he (and also Dr Leon A. Kappelman in the previous presentation) showed the diagram of the framework, it just made sense.

Rolando went a step further stating that Business Rules are not represented as a column or line the framework, but are the glue that binds together all the columns and permeates the enterprise architecture. He showed a "donuts like" hexagram, where each edge represents one of the columns in the framework and the center of the donut represents the business rules "glue". His presentation will be here, when he uploads it.

From my perspective, this is something that I firmly believe and matches the Drools vision of a platform that allows you to express your business rules in a declarative way. It was natural then to understand his diagram showing processes and event processing as component models of the whole architecture.

Now if you think about the problem, how can you write your rules, or model your business if you will, if your rules engine does not know the concept of events and processes? That is why we, at the Drools team, are working to expand the Drools engine so that processes and events can be handled as first class citizens of your Business Model.

If your rule involves events, just state it. If your rule involves processes, just state it.

The good news is that we are not the only ones saying it, and that proves we are not crazy (or at the very least not the only ones that are crazy). Take a look at the RuleML conference agenda for next week. Even being a rules related conference, look at both keynotes for the first day:

For those not familiar with them, Paul Haley is one of the fathers of current Business Rules technologies and David Luckham is one of the fathers of current Complex Event Technology. There is no way I will miss!

Can we get a better endorsement than that?

Happy Drooling,

PS: the presentation I did is available at the conference site.

[edit: fixed the reference to the actual conference organization group. Thank you for pointing that out James.]

Monday, October 20, 2008

Drools Boot Camp T-Shirts

T-Shirts Finally Arrived today, so everyone was very excited :) Thought I'd put up some photo's of our motley crew - Asif and Andrea (who have been here the other days) could not make it today, so they missed out on the photo. Just click any of the photos to enlarge.

Saturday, October 18, 2008

Drools Flow and BPMN

One of the main things we want to offer to end users is the ability to specify themselves what their processes look like. This includes the ability to extend the process engine with custom-defined nodes, to add domain-specific work items, etc. We now have introduced the concept of a (process) skin, which controls how the different nodes are visualized. This allows you to change the visualization of the different node types the way you like them (by implementing your own SkinProvider).

BPMN is a popular language used by business users for modeling business processes. BPMN defines terminology, different types of nodes, how these should be visualized, etc. People who are familiar with BPMN might find it easier to implement an executable process (possibly based on a BPMN process diagram) using a similar visualization. We have therefore created a BPMN skin that maps the Drools Flow concepts to the equivalent BPMN visualization.

For example, the following figure shows a process using some of the different types of nodes in the RuleFlow language using the default skin ...

Simply by switching the preferred process skin in the Drools preferences ...

and then reopening the editor shows the same process using the BPMN skin ...

Tell us what you like! Or more in general, let us know what you think the other features are that are important when deciding how to visualize (executable) business processes (as flow charts).

Friday, October 17, 2008

Rules and BPEL (Joe White @ Recondo Technologies)

Joe is here with us in Texas for the Drools Boot Camp, his company is a big BPEL user and he's been sharing his pain with us. They turned to BPEL, having drunk the koolaid, with the aim to simplify the management of a collection of services and their input and output of data. He tells us the result is a fairly complex system, that lives in Speghetti hell, that is difficult to manage on its own. Joe has been kind enough to share a screen shot and his own thoughts (all pasted below) on where he'd like to go with his company to better address this issue in the future, with the hope that Drools can help.

The image below is a graphical representation of a business process in BPEL. The boxes and circles along the right are services, the lines running to those services are invocations, and the boxes in the middle are steps in the BPEL process (assign, invoke, copy etc.)

Spaghetti BPEL, welcome to hell!!!
(click to enlarge)

An alternative is to use rules to manage your business process. A purely rules based approach would use a rule set as a content based router and every decision point becomes an evaluation of your business routing rules. In addition, rules can manage service invocation by making the service invocation the consequence of firing into another rule set. Coupled with a BRMS like Guvnor the rules based process orchestration of services becomes accessible by business users in a managed environment. Coupled with a workflow or integration engine like Drools-Flow or Apache Camel the rules based business process management will allow for the management of complex long running business processes without some of the complexity and development overhead introduced by BPEL. The rules manage the decision points and the workflow engine helps manage the progression through your business process. The rules based approach won’t provide everything that you get with BPEL. For example state management, persistence, and ease of integration with WSDL are all advantages that BPEL provides that you wouldn’t get for free with a rules driven approach. In the end a rules based approach to business process orchestration should provide simplicity, modularity, and ease of development. As an architect it is worth considering a rules driven approach to business process management.

Drools Boot Camp - Day 1

Day 1 of the Drools Boot Camp is going well, unfortunately Michael Neal from the core team couldn't make it in the end :( However Edson, Kris and Toni are all here :)

The Boot Camp runs until Wed 22nd of October where the Rules Fest conference starts, which we will all be attending. The Boot camp is open to all, so if you are in the Texas area, do please come and join us, we are staying in the Best Western Hotel, Addison.

Today we've been joined by:
  • Joel White from Recondo.
  • Michael Rhoden from American Franklin.
  • Asif Akram from Imperial College.
  • Andrej Sasko from the independent IT consulting
Jens Ruehmkorf from German Aerospace Centre is in Texas but is tracking down his luggage, which has been lost, hopefully he will be with us tomorrow.

It's been an interesting day and I'll do a separate project later on topics covered. We have discovered the best hotel ever with free food and beer :) Here are some photos of the day:

Free food and the beer, the beer cooler is at the square box in the left of the photo.

Kris and Toni caught enjoying the free beer :) and a general shot of the room

Some more general shots of everyone hard at work.

And finally a picture of us at lunch.

Wednesday, October 15, 2008

Drools Boot Camp in Texas is now being Twittered

At the request of others I've setup a twitter account and will send out communications via Twitter on the Drools Boot Camp, Rules Fest, Business Rules Forum and RuleML conferences over the next 2 and a half weeks.

First Look - Drools 5.0 (James Taylor @ Smart Enough Systems)

I met up with James Taylor, author of Smart Enough Systems, the other week and showed him over the Drools 5.0 work. He's written a blog entry on his thoughts, which you can read here.

Sunday, October 12, 2008

Drools 5.0 M2 New and Noteworthy Summary

5.o M2 New and Noteworthy Summary

The complete JIRA release notes can be found here.

Guvnor (the BRMS component)

  • Fine grained security (lock down access to the app per package or per category). Users who only have category permissions have limited UI capability (ideal for business users)
  • Import/export individual packages (to XML)
  • Execution server - access rules via XML or JSON for execution
  • Many more (check the JIRA logs !)
  • Many fixes (resource leak in M1 was fixed early on)

Drools Flow

Human task management is very important in the context of processes. While we allow users to plug in any task component they prefer, we have developed a human task management component that supports the entire life cycle of human tasks based on the WS-HumanTask specification, which is discussed in the following two blows:
Drools and WS-HumanTask
Creating a DSL for WS-HumanTask and when not to use a Rule Engine
Sample Eclipse views allow users to manipulate their tasks. The code can be found here.

The Drools Flow language itself has also been extended with new powerful functionality, including:
  • Event nodes that allow a process to respond to external events
  • Exception handlers and exception handler scopes to handle exceptions that could be thrown
  • A ForEach node allows instantiating a section of your flow multiple times, for each element in a collection
  • Data type support has been extended
  • Timers are integrated with common node types
As a result, new node types and properties have been added to the Drools Flow editor in Eclipse. You can also find examples of these new features in the integration tests (e.g. ProcessExceptionHandlerTest, ProcessTimerTest, etc.).

Finally, our pluggable work item approach allows you to plug in domain-specific work in your process in a declarative manner. We plan to build a library of common work items and already provide an implementation for sending emails, finding files, archiving, executing system commands, logging and human tasks.

Drools Expert and Fusion

Drools now supports "fireUntilHalt()" feature, that starts the engine in a reactive mode, where rules will be continually fired, until a halt() call is made. This is specially useful for CEP scenarios that require what is commonly known as "active queries".

Rule Base partitioning and multi-thread propagation
Drools ReteOO algorithm now supports an option to start the rule base in a multi-thread mode, where Drools ReteOO network is split into multiple partitions and rules are then evaluated concurrently by multiple threads. This is also a requirement for CEP where there usually are several independent rules running concurrently, with near realtime performance/throughput requirements and the evaluation of one can not interfere with the evaluation of others. This is a big topic and has several implications that I will discuss in a follow up blog post and the upcoming documentation.

XSD Model Support
Drools now supports XSD models, you can look at the unit test here to see how it works. Remember though the XSD model is generated as pojos local to the Drools classloader. A helper class is there to assist in the creation of the model in the packagebuilder. Once the data model is generated you'll typically use the JAXB dataloader to insert data.

Data Loaders

Drools now supports two data loaders, Smooks and JAXB. Smooks is an open source data transformation tool for ETL and JAXB a standard sun data mapping tool. Unit tests showing Smooks can be found here and JAXB here.

Thursday, October 09, 2008

Nationwide Insurance to give Drools Blog

GoogleAlert just let me know that Nationwide
are doing a Drools presentation at the Central Ohio Java Users Group on the 14th of October. The details can be found here, and I've pasted them below. It's really nice to see other members in the community promoting the use of Drools, especially a big brand name like Nationwide:

when: 10/14/2008 | 11:30 - 12:30
Event title JBoss (Drools) Rules Engine in J2EE Enterprise Applications
Where: OCLC's Kilgour building private dinning room - Dublin
Category: 2008 Meetings


Provides a glossary view of rules engine. Some comparison between BRMS and Traditional Rules Engine (embedded). A quick look at JBoss Rules, different ways to represent the rules and some highlights of the overall programming environment.


Name: Keith Ebare
JobTitle: Consulting, IT Architect
Company: Nationwide P&C
Technical Area of Interest: J2EE and usage/applicability of open source products

Name: Anu Alwar
Job Title: Consulting IT Architect
Company: Nationwide P&C

Implementation of rules engines for externalizing business rules in a j2ee application.
Masters in Computer Science from Ohio State University. About 13 years of IT experience with about 10 in j2ee.

Venue OCLC's Kilgour building private dinning room
Street: 6565 Kilgour Place
ZIP: 43017-3395
City Dublin
Country: OH