Monday, March 31, 2008

Drools Quick Video Interview with Mark Proctor

While at JBoss World 2008, Orlando, the in-house propaganda team caught up with me to ask me a few questions. They've published this in the Red Hat magazine here. I've also embedded it below.


Download this video: [Ogg Theora]

Thursday, March 27, 2008

Sponship Request for RuleML

I will be at the RuleML conference again this year. Last year was a great event with lots of good presentations and knowledge sharing. While the conference is organised by the RuleML group it is not specific to the RuleML standard and various standards and technologies are discussed at this event - which is what makes it so important.

As always these things can't happen without funding. This year Red Hat will be a RuleML sponsor and I strongly encourage others to help sponsor this fantastic event. Sponsorship starts from as little as $500 for Bronze. You can view the pdf detailing these sponsorship packages here.

International RuleML Symposium on Rule Interchange and Applications

The 2008 International RuleML Symposium on Rule Interchange and Applications, RuleML-2008 (http://2008.ruleml.org), is the primary industrial-oriented meeting place to discuss the applicability and commercialisation of web rule technologies for enterprises and public organizations. At this, RuleML-2008 addresses users, system providers, practitioners, developers and researchers as well as representatives of rule/event standardization efforts and open source rule communities. After last year's highly successful RuleML-2007 (http://2007.ruleml.org), RuleML-2008 again joins the established, practically oriented Business Rules Forum (http://www.businessrulesforum.com) to help cross-fertilizing between Web and Business Logic technology.

The symposium will be again held in Orlando, Florida, US, October 30/31, 2008. Participants will be offered an exciting venue to exchange new ideas, practical developments and experiences on issues related to the engineering, management, integration, interoperation and interchange of rules in open distributed environments such as the Web. RuleML-2008 will enable delegates to better understand web rules and event processing technologies and their potential, and how to exploit these technologies in their organizations

The RuleML-2008 highlights will include:
  1. Plenary keynotes and a panel discussion featuring prominent and visionary speakers.
  2. A RuleML-2008 Challenge with prizes to demonstrate tools, use cases, and applications.
  3. Scientific research, industry, and demo papers and presentations advancing and assessing the state of the art in event and rule-based systems selected in a peer-reviewed fashion by an international program committee.
  4. Invited talks given by world-class scientists and industrials featuring conceptual topics on event and rule-based computing and industry success stories.
  5. Social events to promote networking among the symposium delegates in an informal setting.
The RuleML-2008 Challenge with prestigious prizes provides an excellent opportunity to showcase your organization and become associated with this innovation driven technology market, characterized by strong market growth, visibility and commercial opportunities.
To share your ideas, results and experiences, and to learn more about rules technology, RuleML-2008 invites all of you, including:
  • IT Services, Rules and Event Processing system providers
  • Representatives of, and participants in, rule standardization efforts
  • Members of open source rules communities
  • CTOs and Strategists of technology firms
  • Information Architects and Strategic Technology Planners
  • CIOs and Information Technology Managers
  • Venture Capitalists and Investors
  • CEOs of software start-ups
  • Product Managers from software and hardware companies
  • Data Analysts from major public and private organizations
  • Academics and Researchers
  • Software Developers
  • Press and Analysts
  • Technology Consultants
For further details please see the Symposium website http://2008.ruleml.org/.
For more information please contact: ruleml2008@easychair.org.

Wednesday, March 26, 2008

Complexity and code count (Michael Neale)

Every now and then I like to take stock of the drools codebase, and look at some stats. One of my favourite tools for this is sloccount. Easy to use command line (yum install sloccount on fedora).

Totals grouped by language (dominant language first):
java: 243736 (99.57%)
jsp: 520 (0.21%)
ruby: 243 (0.10%)
sh: 146 (0.06%)
python: 135 (0.06%)

So, about a quarter of a million lines of code. Thats a lot of complexity (and hopefully power) hidden beneath the surface. Hopefully most people don't have to worry about this complexity (we try to keep the interfaces simple - try).

The thing about complexity is sometimes (just sometimes) it is necessary, and you can roll your own complexity, or use someone elses. Mostly I prefer someone elses.

For many years, sewing machines were a mystery to me. I never knew how they actually used the thread to connect stuff together. I also knew I didn't need to know (in fact, when suffering insomnia I would try and solve it in my mind). But I knew they worked, and I knew it was easier to use them then roll your own.
Well not entirely relevant, but I think its cool that someone made an animated GIF showing how they actually work:


Sunday, March 23, 2008

Implement business logic with the Drools rules engine

Ricardo Olivieri has updated his tutorial at IBM developers:
http://www.ibm.com/developerworks/java/library/j-drools/

He has updated his rule syntax from the Drools 2.x XML syntax to the new DRL and goes into a tutorial on how to use the Drools debugging tools in Eclipse.

"Using a rules engine can lower an application's maintenance and extensibility costs by reducing the complexity of components that implement complex business logic. This updated article shows you how to use the open source Drools rules engine to make a Java™ application more adaptive to changes. The Drools project has introduced a new native rule expression language and an Eclipse plug-in, making Drools easier to use than ever before."
......
"As of this writing, the latest version of the Drools rules engine is 4.0.4. This is a major update. Although some backward-compatibility issues exist, this version's features make Drools even more attractive than before. For instance, the new native language for expressing rules is simpler and more elegant than the XML format some older versions use. This new language requires less coding and has a human-readable form.

Another notable development is that a Drools plug-in for the Eclipse IDE (Versions 3.2 and 3.3) is now available. I highly recommend that you use this plug-in to work with Drools. It simplifies the development of projects that use Drools and will improve your productivity. For instance, the plug-in checks your rules file for syntax errors and offers code completion. It also allows you to debug your rules file, potentially reducing debugging time from hours to minutes. You can add breakpoints to your rules file, which lets you inspect the state of the objects at specific moments during rule execution. This gives you information about the knowledge — a term you'll become familiar with later in this article — that the rules engine possesses at a particular moment in time."

Wednesday, March 19, 2008

Drools 4.0.5 Released

We just released Drools v4.0.5. This is a minor release with a few improvements on existing features and some bug fixes.

Release Notes - JBoss Drools - Version 4.0.5

We would like to really thanks all the contributors that helped on getting this release out. From those contributing patches and docs, to those testing and reporting bugs and providing feedback. The list is a bit long to post all names here and I may incur in a mistake forgetting someone, so our open public thank you to you all!

Follows the release notes.

Release Notes - JBoss Drools - Version 4.0.5

Bug

  • [JBRULES-1286] - Incorrect information in section 2.5.7.3. Agenda Filters
  • [JBRULES-1315] - Rule that uses a 'collect' and 'from' clause together causes problem
  • [JBRULES-1327] - drools-ant task and java.lang.ClassCastException: org.drools.reteoo.ReteooRuleBase
  • [JBRULES-1329] - RuleBase.removeRule() prevents other rules from being applied
  • [JBRULES-1330] - Using RuleBase in multithread application server environment
  • [JBRULES-1357] - RuleBuildContext does not initialize package of rule
  • [JBRULES-1388] - Eval error when using multiple declarations
  • [JBRULES-1389] - using eval after using or causes ClassCastException
  • [JBRULES-1392] - Rules behave incorrectly (randomly) in multi-threaded environment
  • [JBRULES-1414] - Cannot build from source -- missing directory or pom file
  • [JBRULES-1415] - Certain uses of from causes NullPointerException in WorkingMemoryLogger
  • [JBRULES-1423] - ObjectFactory$ObjectEqualsComparator.equals throws NPE if second arg is null
  • [JBRULES-1426] - NPE in ObjectFactory$ObjectEqualsComparator
  • [JBRULES-1428] - ClassCastException when comparing BigDecimal fields
  • [JBRULES-1429] - NPE in ObjectEqualsComparator
  • [JBRULES-1435] - NPE if rule checks a Long field for null, and a fact is passed in with Long field that is NOT null.
  • [JBRULES-1436] - Filescanner throws NPE when there's a compile error.
  • [JBRULES-1446] - ClassCastException when iterating over an array using "from"
  • [JBRULES-1447] - Parser error when using keyword operators
  • [JBRULES-1448] - MVELDataProver throws nullpointer on null return value from expression
  • [JBRULES-1451] - LHS expression comparing a Boolean to a String does not fail but always returns true
  • [JBRULES-1456] - Or using DSL language
  • [JBRULES-1459] - parser/scanner bug: "unterminated literal"
  • [JBRULES-1464] - Comilation error : 'Syntax error on token ",", delete this token' when referencing a global in an accumulate block
  • [JBRULES-1467] - Concurrency errors when parsing strings to dates in rules
  • [JBRULES-1472] - Problem when mixing alpha and beta constraints in a composite constraint
  • [JBRULES-1477] - User exception thrown from inside a rule always causes a stackTrace to console
  • [JBRULES-1479] - Exception compiling rules
  • [JBRULES-1480] - Potential multithreaded problem in MVELClassFieldExtractor
  • [JBRULES-1481] - Stop actionQueue recursion in working memory
  • [JBRULES-1482] - Bug with ReeteoRuleBase and merged package
  • [JBRULES-1487] - access to store property in MapBackedClassLoader should be synchronized
  • [JBRULES-1489] - Working Memory deserialization causes NPE
  • [JBRULES-1490] - org.drools.util.AbstractHashTable$SingleIndex is not serializable
  • [JBRULES-1491] - java.io.NotSerializableException: org.drools.util.AbstractHashTable$SingleIndex
  • [JBRULES-1492] - NPE caused by non-shadowed deep object access
  • [JBRULES-1501] - Multiples modify causes variable duplication exception
  • [JBRULES-1503] - Errors on rulebase serialization scenarios
  • [JBRULES-1505] - Fix CCE when using nested accessors to compare Date values
  • [JBRULES-1506] - RuleAgent will not read binary packages from Apache HTTP Server
  • [JBRULES-1507] - Objenesis too heavily used
  • [JBRULES-1508] - NPE in MVELDataProvider
  • [JBRULES-1510] - ObejnesisFactory.getStaticObjenesis() doesn't set OBJENESIS_INSTANCE

Feature Request

  • [JBRULES-1307] - Update and expand documentation for decision tables to show actual uses for all keywords
  • [JBRULES-1338] - DirectoryScanner should only take into account relevant files
  • [JBRULES-1473] - Add a configuration parameter to allow Drools to dump all generated java source code

Patch

Quality Risk

  • [JBRULES-1475] - Teensy spelling error in drools project template

Task

  • [JBRULES-1502] - update example to use modify block as default
Happy Drooling
Drools Team

Release Notes - JBoss Drools - Version 4.0.5

Drools and Multi Colored Balls

I thought I would share with our readers a thread from our priority support system. It's an interesting thread as it covers some complex constraint problems in good detail, which I think most rules users might find interesting, and it also demonstrates the level of support we provide to our priority customers. This conversation is printed "as is" with only the user and customer names removed. Services have just asked me to mention that when you purchase JBoss Drools priority support subscriptions, you aren't just getting high quality support as shown below, but also a legal commitment from us to respond with guaranteed response times (whose values depend on the type of agreement they purchased of course).

User
I have a query to help me retrieve different combination of 5 balls out of 20 balls.

Here is my simple query,
query "get balls set"
ball1 : Ball();
ball2 : Ball(this != ball1);
ball3 : Ball(this != ball1 && != ball2);
ball4 : Ball(this != ball1 && != ball2 && != ball3);
ball5 : Ball(this != ball1 && != ball2 && != ball3 && != ball4);
end
And now, I want to add one more rule to this query.
The rule is, only 0-2 balls can be (color == 'yellow').

That means, after the rule was added, I may have a set of balls that having,
no balls in yellow color; or
1 ball in yellow color; or
2 balls in yellow color.

So, how should I modify the query?


Edson
In other words, balls 3, 4 and 5 must be different from yellow, right?

query "get balls set"
ball1 : Ball();
ball2 : Ball(this != ball1);
ball3 : Ball(this != ball1 && != ball2, color != "yellow" );
ball4 : Ball(this != ball1 && != ball2 && != ball3, color != "yellow" );
ball5 : Ball(this != ball1 && != ball2 && != ball3 && != ball4, color != "yellow" );
end

Does that help you?


User
Yes, you are right. Thank you.
What if I want to apply some rules on the ball set to ensure some mutually exclusion?

Say,
1. color == 'red' and color == 'blue' cannot be co-exists in the same ball set.
2. when having 2 size == 'medium', no ball with size == 'big' can be selected.

And I am going to have dozen of these mutually exclusion rules.


Edson
Well, all constraints must be expressed in some way. Some are more complex than others, but the language is Turing complete and any constraint can be expressed.

So, for your first question, we can use a simple approach writing down the possible combinations and using a class to represent the mutually exclusive colors. You can obviously hard code the colors, but I'm just trying to show you different approaches to the problem:
query "avoid mutually exclusive colors"
ball1 : Ball( $c1 : color );
ball2 : Ball(this not in ( ball1 ), $c2 : color );
ball3 : Ball(this not in ( ball1, ball2 ), $c3 : color );
ball4 : Ball(this not in ( ball1, ball2, ball3 ), $c4 : color );
ball5 : Ball(this not in ( ball1, ball2, ball3, ball4 ), $c5 : color );
not MutuallyExclusiveColors((color1==$c1 && color2 in ($c2,$c3,$c4,$c5))||
(color1==$c2 && color2 in ($c1,$c3,$c4,$c5))||
(color1==$c3 && color2 in ($c1,$c2,$c4,$c5))||
(color1==$c4 && color2 in ($c1,$c2,$c3,$c5))||
(color1==$c5 && color2 in ($c1,$c2,$c3,$c4)))
end
Now, we can also use a simple trick to add some flexibility, and a whole new set of possibilities are open for us. Since we can create lists using the MVEL dialect, we can define a dummy function to help us.
function java.util.List returnList( java.util.List list ) {
return list;
}
So, we can express your original question with a rule like that:
rule "Select maximum 2 yellow balls"
when
$b1 : Ball( )
$b2 : Ball( this not in ($b1) )
$b3 : Ball( this not in ($b1, $b2) )
$b4 : Ball( this not in ($b1, $b2, $b3) )
$b5 : Ball( this not in ($b1, $b2, $b3, $b4) )
Number( intValue <= 2 )
from accumulate( $b : Ball( color == "yellow" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
then
System.out.println( "Balls: "+$b1+" "+$b2+" "+$b3+" "+$b4+" "+$b5 );
end
The same way, we can express your red/blue constraint by writing something like:
rule "red and blue can not co-exist"
when
$b1 : Ball( )
$b2 : Ball( this not in ($b1) )
$b3 : Ball( this not in ($b1, $b2) )
$b4 : Ball( this not in ($b1, $b2, $b3) )
$b5 : Ball( this not in ($b1, $b2, $b3, $b4) )
Number( $red : intValue )
from accumulate( $b : Ball( color == "red" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
Number( intValue == 0 || eval( $red == 0 ) )
from accumulate( $b : Ball( color == "blue" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
then
System.out.println( "Balls: "+$b1+" "+$b2+" "+$b3+" "+$b4+" "+$b5 );
end
Your second constraint can be expressed as:
rule "if there are 2 or more mediums, no big allowed"
when
$b1 : Ball( )
$b2 : Ball( this not in ($b1) )
$b3 : Ball( this not in ($b1, $b2) )
$b4 : Ball( this not in ($b1, $b2, $b3) )
$b5 : Ball( this not in ($b1, $b2, $b3, $b4) )
Number( $big : intValue )
from accumulate( $b : Ball( size == "big" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
Number( intValue < 2 || eval( $big == 0 ) )
from accumulate( $b : Ball( size == "medium" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
then
System.out.println( "Balls: "+$b1+" "+$b2+" "+$b3+" "+$b4+" "+$b5 );
end
Now I would like to call your attention to the fact that such kind of rules have the potential for combinatory explosion and, as such, a great performance degradation.

What I mean is that if you have for instance 10 balls in the working memory and you want to select 5, you are effectivelly doing permutation without repetitions, whose total possible permutations are given by the formula:
P(10, 5) = 10! / (10-5)! = 30240 possible results.
If you add just one more ball to the working memory, you will get:
P(11, 5) = 11! / (11-5)! = 55440 possible results.
As you can see, it is an exponential growth. Just something to be aware of.


User
OK, I digested your message now.
It helps alot.

I am trying to merge all your 3 rules into a single query.
Like this,
Query "Merging all rules into single query"
$b1 : Ball( )
$b2 : Ball( this not in ($b1) )
$b3 : Ball( this not in ($b1, $b2) )
$b4 : Ball( this not in ($b1, $b2, $b3) )
$b5 : Ball( this not in ($b1, $b2, $b3, $b4) )
Number( intValue <= 2 )
from accumulate( $b : Ball( color == "yellow" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
not ( Number( $red : intValue )
from accumulate( $b : Ball( color == "red" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
&&
Number( intValue == 0 || eval( $red == 0 ) )
from accumulate( $b : Ball( color == "blue" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) ) )
not ( Number( $big : intValue )
from accumulate( $b : Ball( size == "big" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
&&
Number( intValue < 2 || eval( $big == 0 ) )
from accumulate( $b : Ball( size == "medium" ) from returnList( [$b1, $b2, $b3, $b4, $b5] ),
count( $b ) )
Do you think I can merge these 3 rules like this?
And after merging it, would it introduce any new performance issues?


Edson
Technically speaking, you can merge them. The major impact is still the combinatorial explosion the first 5 patterns will cause, as detailed in my previous message, but of course, each additional pattern will make the query a bit heavier.

Also, it is not easy to read it, but if that is what you need, then I don't think there is any other way.

Finally, with a small modification to your dummy function, you can avoid calling it every time to create the list over and over again.

The "from" CE will always iterate over the elements of the collection that is returned by the function call, so, instead of returning a collection of balls, change it to return a collection of collection of balls:
function java.util.List returnWrappedList( java.util.List list ) {
return new ArrayList( list );
}
Then you can write a pattern in your query like:
$balls : List() from returnWrappedList( [$b1,$b2,$b3,$b4,$b5] )
And then you can use $balls instead of recreating the list over and over again. Example:
Number( intValue <= 2 )
from accumulate( $b : Ball( color == "yellow" ) from $balls,
count( $b ) )


------------------------
NB:
With latest MVEL (1.2.24), we don't need a dummy function anymore... we can simply use MVEL's return statement:

from return( [$b1, $b2, $b3, $b4, $b5] )

Or if we need a wrapped list:

from return( [[ $b1, $b2, $b3, $b4, $b5 ]] )

Saturday, March 15, 2008

MIKE - The forgotten hybrid rule engine

Micro Interpreter for Knowledge Engineering (Mike)

Mike is a hybrid rule engine by retired Open University Professor Marc Eisenstraat and Hull University's Dr Mike Brayshaw. The last release was version 2.50 in 1990 - so it's 18 years old. Mike formed part of the Open Universities "knowldge engineering" study pack. What's interesting is that 18 years ago they combined both prolog and rete for a hybrid reasoning system around frames (classes) - they even released the source code. As well as being a hybrid engine it also offers truth maintenance a few varieties of uncertaintity - standard, Bayes, Shortliffe and Buchanan uncertaintity. Marc Eisenstraat released a 1990 BYTE magazine article called "Build your own knowledge engineering toolkit" which was a two part overview of Mike. Mike itself is based on a 1989, almost 20 years old, paper "Building an Expert System in Prolog" by Dennis Merrit, founder of Amzi. You start to get the feeling that the mainstream rule engine development has forgotten more now than it knew 20 years ago.

Thursday, March 13, 2008

Drools and Google Summer Of Code

The Google Summer of Code is beginning. Drools, being a Red Hat project, has to participate under the Fedora banner. So I've put a few project ideas up on the JBoss GSoC page. If you're able to enter as a student please do take a look and don't hesitate to contact if anything interests you.

I've also listed the ideas below, please leave comments if you have any other ideas you think might make good student projects. Or maybe you are a student and you want to propose a GSoC project.
  • Subversion/CVS and JCR synchronisation. This allows our web based BRMS, called Guvnor, to synchronise it's content with what's used in an IDE, such as Eclipse
  • Eclipse file upload tool with meta-data properties editing. You should be able to right click a file or a folder and upload to the web based Guvnor. Previously uploaded files(assets) should be recognised and uploaded as a modify.
  • SBVR, structured natural language, implementation for Drools. Can do either or both the frontend or the backend.
  • Eclipse tooling enhancements including any of the following: search, refactoring, reformatting.
  • Animate the Rete view to represent network propagation.
  • Improve the DSL capabilities of Drools, supporting more complex grammars.
  • Improve the Guvnor, web based BRMS, to handle the management of more asset types. Including images, video, sound etc. Guvnor involves JCR, Seam and GWT work.
  • Create a Web based process designer for Drools ruleflow using GWT Designer.
  • Web based, GWT, Audit viewer.

Thursday, March 06, 2008

Drools blog now has 612 subscribers :)

Feedburner tells me that today I have 612 subscribers, it's great that the blog still continues to grow :)

We also get between 200 and 500 unique visitors a day to the website.

Wednesday, March 05, 2008

The Power of Skunkworks

Google alerts made my day with this blog quote :)

The Power of Skunkworks
While at LPA me and a guy from Amentra were told to make the Blaze Rules Engine to work in ways it was not built to be used. After banging our heads on it for a month, countless conference calls with Fair Isaac, and general disgust with the complexity - I spent a weekend learning Drools and replaced it. It was perfect. Just what we needed. I switched their coffee with mine, and they never noticed. I told them a week later what happened. Told my boss. Then his boss. Several conference calls later my system was officially in - Blaze was out, saving our company $250k/year in license and support fees.

Tuesday, March 04, 2008

ILog releases Java to C# source translator as open source.

Daniel Selman has blogged that ILog (makers of the JRules BRMS) are releasing their Java to C# source converter as open source.

This is great news, and kudos to them for doing it as open source (Eclipse Licence - EPL).

ILog have been supporting the .Net platform for some time, and this tool was obviously key to that. In the .Net "world", they are able to do deep integration with Microsoft Office tools (for example, rules editing in MS word).

Its certainly an interesting tool worth looking into. We have had some success with the IKVM project, allowing java bytecodes to run in .Net (with minimal source porting) for Drools.Net but this source translation approach is also a very interesting (and possibly very efficient) alternative.

Prior to Mark and myself working on Drools fulltime, we both spent some time in .Net land (that was some years ago now - I feel out of touch with progress on that platform). Integration with office productivity tools interest me, something that Microsoft alone seem to have managed to make so easy and effective.

Thanks for ILog for releasing this !