Wednesday, November 29, 2006

JBoss Rules 3.0.5 is released

Version 3.0.5 is now available, from downloads and the plug-in update site. This is the latest release of our current "production" branch of JBoss Rules.

Watch this space for news of a milestone release very soon of the next major version.

Thanks to all out there who submitted bugs/fixes/patches and enhancements.

See below for Release Notes - JBoss Rules - Version 3.0.5 (note you can always get these from our JIRA issue tracking system when you want):


  • [JBRULES-410] - More than one Query definition causes an incorrect Rete network to be built.

  • [JBRULES-440] - Unable remove a rule package when it was added after it has been added and removed

  • [JBRULES-455] - java.lang.NullPointerException at org.drools.util.LinkedList.remove(Unknown Source) in 3.0.4 while not in 3.0.3

  • [JBRULES-459] - Literal constraint using 'null' triggers a NPE.

  • [JBRULES-475] - Error in documentation concerning the joining of constarints

  • [JBRULES-481] - LeftInputAdater.updateNewNode does incorrect propagation

  • [JBRULES-499] - Rule not rendering activation group correctly

  • [JBRULES-505] - Using a GLOBAL value in predicates in more than one rule - rule fails to evaluate correctly.

  • [JBRULES-506] - Exception thrown when retract, reassert and retract again

  • [JBRULES-523] - JoinNode and NotNode with empty binder throw exception with getConstraints()

  • [JBRULES-532] - Query feature doesnt seem to work if there is more than one query.

  • [JBRULES-539] - Combining self-exists() with calling QueryResults twice returns to much

  • [JBRULES-542] - Drools core 3.0.4 - Drools does not remove itself as a listener when Facts are Retracted

  • [JBRULES-548] - NPT when binding a variable and comparing that variable to another property of the same object the variable was bound from

  • [JBRULES-550] - NPE while removing and adding same package

  • [JBRULES-557] - Dynamically generated classes not accessed by drools classloader

  • [JBRULES-562] - Security Permission problem in Websphere 6.1

  • [JBRULES-566] - Not-Constraint is not working properly

Feature Request

  • [JBRULES-442] - Building from source failed: Serialisation / externalisation error

  • [JBRULES-444] - patch to support Notes attribute on decision table Package

  • [JBRULES-483] - Open Decision Table API to allow custom RuleSheetListeners

  • [JBRULES-544] - getQueryResults behavior: return null


Sunday, November 26, 2006

Business Rule System: To use or not to use? (Edson Tirelli)

One of the most frequently asked questions around is "Why would I want to use a Business Rules Engine?". Usually this question is immediately followed and is related to: "When should I use a Business Rules Engine and when should I not?".

Googling out there you will find several posts, papers and books about it. Examples:

From JBoss Rules Documentation
From Jess Documentation

Those questions are usually answered in terms of the benefits of using the core rules engine component (algorithm), and are all true. We can even state that they justify the need for themselves.

Although, one frequently under looked aspect is the benefit of rules governance you achieve when using a BRMS.

Business Rules require lifecycle management in the same way most other current Enterprise Technologies do.

If one looks at classical application development in an enterprise environment, no one can even imagine it happening nowadays without a control process, involving orchestration of activities, with authorization control for each of the steps, does not matter what is the actual used methodology. The whole process is usually supported by specialized tools, varying from Development Tools, Code Versioning Systems, Automated Integration Tests, and specially, Configuration Management tools.

In the same way, once Rules Engines allow us to separate Business Decision Logic from the application core code, it also allows us to implement a governance infrastructure for Business Rules. It means:

  • Allows specialized teams to work on each of the roles in a Business Rules lifecycle. For instance, business rules analysts are responsible for maintaining the business rules, while configuration management teams are the ones responsible for the deployment and application core developers worry only about using (interpreting) rules results.

  • Allows for fine grained control on the access and maintenance of the rules that control the Enterprise Business.

  • Allows for comprehensive auditing of implemented rules and all changes happening in the corporate environment.

  • Allows for rule sharing between different applications and corporate business processes. This as we know, allows for consistency between corporate processes and decision making, as well as reduced maintenance costs: a change in the single rule instance will reflect throughout the whole organization.

  • These advantages are in sync with SOA architecture goals, as well as governance requirements.

    So, when analyzing the advantages of using a BRMS, also think about Business Rules Governance and the benefits it brings to the organization.

    For more details on the requirements and features of such platform, you can refer to Mark's post about the JBoss Rules server.

    [EDIT: Adding related links posted by fellow readers in the comments, as here it is more visible to other readers than staying in the comments.]

    Thank you James for posting:

    "There are a number of reasons why business rules can be better than traditional code and a lot of benefits to be gained.
    There are also many excuses for not taking on business rules as an approach. I wrote more on these topics in my FAQ."

    Also, thank you Rajgo:

    "Especially, considering that rules are intrinsically more structured than code, it makes all the more sense to use a business rules Management System.
    In addition to governance & management, and decision automation, BRMS can provide businesss rules analysis that one could use to refine their policies to meet business goals"

    Saturday, November 18, 2006

    Drools 3.1 milestone 1 is comming

    As some of you already know, Drools 3.1m1 will be out of the door any minute now. The Drools name is used for project development releases, which are also unsupported - eventually the code is hardened and released to production as JBoss Rules 3.2. This new release is a a big improvement over the 3.0.x production branch in many ways and it will be intersting to see where we will go from there.

    For those curious about it, I will briefly talk about the major changes one expect to see when moving to it, from a core/compiler perspective.


    After about 5-6 weeks of struggling led by Mark himself, most of the anchors slowing down JBRules 3.0 were simply left behind. By this I mean, no more internal structures for remembering matches, no more reference linking hell, no more overuse of array copies. Core was basically streamlined with a huge reduction on memory consumption and a significant performance boost. Benchmarks like manners, that stress the engine pattern matching algorithm are seeing improvements in the order of 300%, while memory comsuption is down to 20% of the original.
    Does that means that 3.0 performance was bad? I would say not at all! Performance was actually good on 3.0, specially when compared with previous drools versions and other open source engines. But no big improvement was possible in the 3.1 version without breaking some paradigms we had before. And Mark did an amazing job pushing every little aspect to the limit to find exactly what we needed to change.
    Does that mean real world applications will see improvements like that? Probably not in that order of magnitude, as benchmarks are designed to stress engines to its maximum, consequently generating extreme cases. Real world applications will certanly see great improvements, specially in the memory consumption side, but not in that order of magnitude. Actually I'm quite curious to test it in real use cases and see what will be the improvements.

    Mark has provided a quick summary of the main enhancements:
    • Unrolled loops for constraint evaluation
    • High performance custom collections implementation
    • Tuple is now the entry, avoid Entry object creation.
    • Tuple, a wrapper for FactHandle[] during propagations, now uses a linked list as per Jess from "Implementing A High-Performance Symbolic Reasoning Engine In Java"
    • Composite key indexing for left and right memory
    • Left and right value caching for join nodes, to avoid repeated value retrieval
    • Improved alpha node hashing
    • Extensive use of Single/Composite design to avoid setting up a for loop when there is only 1 child
    • Primitive support, avoids continuous primitive wrapping

    Primitive type support

    As users know, JBRules 3.0 automatically wraps any primitive types he needs to work with in its respective Object Wrappers. That was a limitation that was causing 2 undesired side effects:
    1. Performance really suffers when one needs to wrap/unwrap primitives.
    2. Code using primitive wrappers is much more verbose and cluttered.
    rule "return value"
    $person1 : Person( $age1 : age )
    person2 : Person( age == ( new Integer( $age1.intValue() + 2 ) ) )

    As you can see in the above example, the rule's writer needs to be aware and keep wrapping/unwrapping primitive values to work with the engine.

    JBRules 3.1m1 has now full support to primitive types, not only helping it to perform better and consume less memory but also making rules much more clear. Look at the same rule in 3.1m1:
    rule "return value"
    $person1 : Person( $age1 : age )
    person2 : Person( age == ( $age1 + 2 ) )
    It is now clear and straight forward to understand what the rule writer wanted his rule to do.

    Shadow Facts support

    As it is well known, JBRules uses POJO as its first and preferred fact representation structure. We design the engine targeting it to perform most efficiently on beans, as we understand that this is the most simple and productive way of integrating a rules engine in real world applications: work with user's own business objects model. Although, this kind of integration demands a carefull planing on the engine side to avoid collateral effects, as the engine do not own the beans. The application or the user can change any bean property on the fly and not notify the engine.
    So it is mandatory that the engine implements some kind of control over the values being used for reasoning, keeping the whole engine state consistent. This is implemented by Shadow Facts, that are shallow copies (cache) of the bean values while reasoning over it, allowing updates only on safe points.
    Shadow Facts are implemented in JBRules 3.1m1 as dynamic proxies and besides being mandatory to have from an engine perspective, they are almost invisible for the engine users. I say almost, because being dynamic proxies, there is only one requirement users need to meet: a nonarg default constructor for the classes asserted into the engine. Besides that, users don't have to worry about it. Only to know, their facts are being taken care appropriatelly. The proxy is not exposed to users, not even in consequence block.

    New nodes

    Evolving the engine also requires support to some new conditional expression elements. Three new CEs are included in 3.1m1, being them:

    FROM: allows reasoning over facts not previously asserted into the engine. Specially useful for retrieval of facts on the fly from database tables or any other external source. Example:
    rule "From"
    $cheese : Cheese(type == "stilton" ) from cheesery.getCheeses()
    // do stuff
    COLLECT: allows reasoning over collections of objects from working memory. Example:
    rule "Collect"
    $cheeseList : ArrayList(size > 2) from collect( Cheese( price < 30 ) ) ;
    // do stuff

    ACCUMULATE: a more flexible and customizable version of Collect, allows to iterate and calculate values from a set of facts in the working memory. Example:
    rule "Accumulate"
    $person : Person( $likes : likes )
    $cheesery : Cheesery( totalAmount > 100 )
    from accumulate( $cheese : Cheese( type == $likes ),
    init( Cheesery cheesery = new Cheesery(); ),
    action( cheesery.addCheese( $cheese ); ),
    result( cheesery ) );
    // do stuff

    These are the main features you will se in this milestone from a core/compiler perspective. New milestones will bring even more news, specially related to the Rule Server under development as described in a previous post.

    So, stay tunned, enjoy, and please provide us your feedback.


    Friday, November 17, 2006

    Declarative Behavioural Modelling (DBM) and jBPM/JBoss Rules integration

    Production Rule systems, such as JBoss Rules are just one form of declarative programming; I believe that to meet the challenges of tomorrows technology we need to be looking at a unified tooling and platform approach to declarative programming. Workflow/BPM has been the predominant declarative approach for a while and Production Rules are one of the first technologies from the AI family to go mainstream. One of the nice things about Production Rule systems is they can be expanded to handle a number of domains declaratively and a variety declarative tooling can be built on top - as shown in the diagram below. I hope to use JBoss Rules as a vehicle to bring more of these technologies into the mainstream - Event Stream/Management Processing, Backward chaining and Constraint Programming are all essential items to a modern enterprise and each allows you to model the different behaviours of your system - I see these as shorter term goals. Genetic Algorithms, GIS, Neural Networks and the Semantic Web are all interesting long term goals.

    For now we need to focus on the two mainstream technologies rules and workflow and figure out how provide a much needed unified modelling environment.

    Declarative programming often involves the generation of content which can expose a whole heap of problems. The jBPM way is to own the control of this using their deployment tool to couple the process definition and the java classes. You write your process definition xml file and select its dependant class files which it then deploys as a PAR; which is a JAR containing the xml file and the compiled dependant classes. This gives you control over the java classes, making sure that a particular process definition always works with the correct version of the java classes. We can extend this to allow the process definition xml to reference one ore more drl files which it then makes available as a rulebase using variable reference in the jBPM context. There are two ways we can approach this. Either jBPM compiles the rulebase to a binary and serialises that to the PAR and reducing the runtime dependencies or it can add the drl to the PAR needing a runtime compilation of the drls into Rule Bases. Users can then easily look-up and reference the working memory from their own concrete handler implementations.

    While this is an effective and simple way to solve the deployment problem it has a number of downsides. The PAR has ownership of those java classes and their versions, this may clash with existing deployed classes for other applications - so care needs to be taken here. While processes rarely change and many consider them to be code and thus more suited to the traditional development and deployment, rules on the other hand are both code and data and can have continual updates - this can make traditional deployment systems quite cumbersome. Further to this large organisations can have thousands of processes and tens of thousands of rules - suddenly maintaining this in a standard source control management system doesn't sound so appetising.

    The next release of JBoss Rules will include a BRMS which will allow a more enterprise approach to management of rules, please see my previous blog "JBoss Rules Server" for more details on this. The basic idea is that all rules are stored in the BRMS where they are versioned with full meta data and categorised, I can configure specific versions of those rules into a RuleBase that I can then make available - the rule base itself is also versioned. With this approach the jBPM process definition can specify a URI to the BRMS for the rule base it needs, which is lazy retrieved on first use. While this does provide better management for a large number of rules the key benefit this provides is that the application and the BRMS are now aware of each other and the BRMS can feed the application regular updates to the rule bases its using; allowing rules to live up to their role of being data.

    If this approach proves successful we can look at managing other knowledge assets within the BRMS, such as the classes and the process definitions themselves. We can build versioned bundles which depend upon other versioned bundles which means that during deployment we can have a more fine grained way to solve the initial problem of the PAR owning the classes - with this approach it would be up to the server to manage versions and bundle dependencies. OSGi tries to solve some of these issues, and its something I want to explore as a long term solution to this.

    The previous section covered deployment; which is actually very boring and not of interest to most users. What they are really interested in is the tooling. Currently jBPM nodes are handler based; you create a concrete class that implements an interface and retrieves the variables it needs from the jBPM context, the jBPM node then specifies that class as its handler. While simple this concept is very powerful in its flexibility and is one of jBPM's strong points and why it is favourite with BPM developers. However there are times when you want something a bit more integrated and declarative, especially when working hand in hand with business users. We can't integrate DRLs any more than the previous deployment example provides; however as the above diagram shows Production Rule systems facilitate a number of very good ways to declaratively author rules, the most common of which are decision trees, decision tables, score cards, structured natural language, gui driven rule authoring and rule flow. What users really want is an ability to specify one of these authoring tools and apply them to a jBPM node so that they can declaratively control the decision or task node - this offers real value. Further more the "glue" to hold these together, i.e. the passing of the relevant variables from the jBPM context and the calling and execution of the rules must be handled seamlessly in the background. I expect that the actually author page should be embedded in one of the existing jBPM process tabs in eclipse.

    The above sections cover the stateless use of JBoss Rules with jBPM, this means that the life time of a working memory is for the execution of the current node, there are no long lived Working Memories. The reason for this is because jBPM supports the persistence of long lived processes, it is not generally expected that a Working Memory will be suspended and persisted else where. For this reason we currently consider it an exercise for the end user to manage, although we will try and provide a number of useful enhancements to enable JBoss Rules to play well in this environment.

    So far I have only covered the unification of a Production Rule system and Workflow/BPM system; however this still only allows you to model a specific and limited set of problems. For a bit of background many of you may have heard of the AI Winter where AI was hyped and failed to deliver and has been relegated to mostly academic areas. Production Rules is one of the AI technologies now going mainstream, even though it’s been around for 25 years - what’s enabling this? I think it's mostly due to good tooling and better documentation, mere mortals are simply not able to work in shell environments and be productive, let alone decipher some cryptic paper on how to use and implement a specific technology. Many of the other technologies have worked in silos again with limited or no tooling. Both probability/uncertainty and genetic algorithms have had a number of papers showing how they can work well with a Production Rule system, yet this hasn't had any real uptake. It is my hope that the introduction of those systems into JBoss Rules with good tooling and documentation can help take these technologies such as these mainstream and improve the way software is written. With the way that hardware is scaling with the move to multi core processors and grid computing existing procedural ways of programming will not be enough and we will need paradigms such as these and distributed agent computing to really make the most of these environments.

    Post Comment

    Wednesday, November 15, 2006

    Washington Rules !

    -- Michael Neale
    Both Mark and myself recently returned from the highly informative Business Rules Forum (in Washington DC).

    The business rules industry seems to be in quite good shape, and there was a strong showing from all the vendors. I thought I would start with some summaries in general of what we saw, and what I thought.

    The engine performance panel
    James Owen - an industry commentator and rule practitioner and evangelist of many years experience, kindly hosted a panel on performance challenges for rule engines (for the geeks). It was a real whos-who of rule engine core tech.

    Take a look at the panel of braniacs below (James Owen is missing, he was talking).

    From left to right (Dr. Ernest Friedman (Jess), Dr. Charles Forgey (Fair Isaac, inventor of RETE), Rev. Daniel Selman (ILog JRules product manager), Rabbi Mark Proctor (some bloke, JBoss Rules), Pedram Abrari (CTO Corticon)

    Innovations I found interesting

    There were strong offerings from all the vendors on rule management/editing/higher level tools, and an understanding that a lot of the core tech is being commoditised. Nevertheless, there were many different approaches. Here is an outline of the products/companies that tickled my fancy:
    • Rule Burst
      • I love these guys as they are Australian ;) (same as me for those who don't know). Their approach is rules capture and management inside Word documents, with a strong natural language engine. They used to be called "Softlaw" and specialise in turning complex legistlation into executable rules by marking up the actual legal documentation !
      • I also like them as they are solving hard problems for important customers which really shows the power of inference engines and rule management systems. They have been around for quite some time (> 15 years) and got their start with Prolog ! They tooling is also impressive, and they have a fairly unique way of allowing "backwards" chaining rules to ask questions and automatically generate questionnaires based on the rules.
    • Corticon
      • These guys are really approaching the problem from a different angle. They call it "rule modelling" and their Workbench/IDE/Environment is very impressive, and allows people to quickly model facts (vocabulary I think they call it), rules, and test/verify/analyse/find logic faults and gaps all in in the one desk top tool, and deploy it (for example) as a web service. Their CEO is also a MD (Medical Doctor) and he got his start as he saw the potential of rule based systems in medical diagnosis.
    • Pega Systems
      • Pega weren't so much focusing on rules but also on process management, but even more so as whole new way of building apps "in place" - as in you take a web applications user interface, and switch in to design mode and you can modify the process/rules/fields that are "behind" the screen. Impressive, they have been really going hard with R&D, and it shows. Rules are just one part of their vision.
    • ILog
      • They didn't participate in the "product derby" (kind of an Uber demo that gets shown while we are eating lunch) this year, but I did attend a session or two with Daniel Selman, the JRules Product Manager. I was particularly impressed with his presentation on "Mission Critical applications" and how it applies to rule engines. ILog JRules is clearly used everywhere in really sensitive applications, and Daniel did a great job outlining the real world challenges of maintaining complex applications and evolving them AND THE OBJECT MODEL over time (the JRules "Business Object Model" approach really shines for this, and now supports re factoring etc). ILog have impressive credentials in this space.
    • Haley
      • One of the benefits of this conference is that we get to speak with real world rule practitioners, and I heard great feedback about Haley's implementation of NLP and NLU (Natural Language Processing and Natural Language Understanding) - which is an area that greatly interest me. On the one hand I view NLP as a little like voice recognition (annoying) or handwriting recognition (we all remember *that* version of the Apple newton) - ie hard to get right (and near enough is NOT good enough). But Haley seemed to have made this work for rules - so well done !
    So it was an impressive showing from the Vendors. I am sure I missed some out, but this is all I had time to write notes on.

    SOA and rules
    The "rule service" approach (stateless rule services exposed nominally via WSDL) seems to have become popular, and certainly seems applicable for a large class of rule problems (business decisioning, single point of truth etc.).

    Future challenges
    No one seemed to agree on a single way forward for modelling facts, some were talking MDA (MDA was a strong theme), some were purely WSDL/Web Service, and utilising OWL was even mentioned a few times (personally over the long term, I think OWL ontology's may be The Way, but I am only new to them so I shouldn't show my ignorance just yet). Modelling facts is a real challenge, and very few people were also thinking about how to cope with model evolution over time (as no one will get it right first go).

    Rule standards are another major challenge, I am not expecting anything concrete for some months, perhaps a year. It was great to meet Said Tabet (of RuleML fame) and hear his opinions, and its also great to see vendors like ILog take RIF from W3C very seriously.

    Truth, Justice and the American way
    I found the US to be surprisingly familiar in every way, more so then Britain. The "tipping" culture took a while to get used to, I felt embarrassed as I had to ask people when to tip or not tip, how much etc. A feature of this seams to be to deliberately cause conflict. On the flip side, the service for everything was excellent. It was a hoot hanging around with James Owen, the ultimate Texan ! From my sampling of several Texan acquaintances over the years, I can say that I like Texans !

    Pretty Washington
    The hotel was fantastic, and I was impressed by the array of Autumn colours (Autumn seems to be taking its time in the UK).

    Next week we will be in Berlin for JBoss World (I am arriving a little early to do the tourist thing, as a native to the Land Down Under, there is plenty over this side of the globe I have not yet seen - its all good !). If you are there, make sure you find Mark Proctor or myself, we have t shirts to give away !

    Post Comment

    Thursday, November 09, 2006

    JFDI a new Business Action Scripting Language

    We are working on a new non-imperative conseqeuence language for JBoss Ruless, called JFDI which is a sort of business action scripting language. We have placed this project over at Codehaus as we are hoping that it will get taken up by other declarative systems, whether they are rule or process engines.

    When we first thought about doing this we got a deluge of emails like "have you looked at groovy, rhino, judoscript etc" - the answer was obviously "yes". The last thing wanted to do was yet another language and we looked hard into using those languages, but they all fell short. Not short in features, but short becuase they did too much!!! Here was the main feature set:
    1) A rule consequence should be "when this then that", not "when this maybe that", its quite simply a list of actions to be performed when this situation occurs. Therefor it should not contain any imperative code, no "if", "switch" statements, its purely focused on expressions, method/function calls and we will allow a foreach loop. If someone needs to do some imperative code it should be encapsulated within a method or a function which is then called from the consequence.

    2) The language should not be ambiguous and thus should be typed, although we may eventually allow type inference. Support for refactoring is definitely a long term goal.

    3) A rule base may have thousands of rules, currently with JBoss Rules you end up with a minimum of two classes per rule and then a further class for each function, predicate, return value or 'eval' used. This large number of classes can cause a number of problems from perm gen issues to slowing down the system's classloader. Therefore it must be able to run interpreted, no bytecode generation. A consequence can be serialised to disk and lazy loaded on first use. While this is interpreted, as much as possible is done at "compile" time with no runtime introspection, so it's a tree of cached java.lang.reflect.Method calls. Although eventually we would like optional bytecode generation optimisations, but generally it is not the execution of the consequence that is the bottleneck.

    4) A simple groovy/ruby like expression syntax. We need non-verbose ways to reference sub fields and to declare and reference maps and arrays, with support for inline anonymous maps and arrays. We would like further support for variable interpolation and <<EOF type stream writing.

    5) The traditional way to "inject" variables into a scripting language is to put each key/value pair into a dictionary or context. We use to do this before drools-2.0-beta-14, it is not performant, instead we need to allow the application to determine at compile time how to resolve those external variables.

    6) We need to be able to decorate a variable, at compile time, with additional methods - this way in a consequence a user can do "myVar.getFactHandle()" even if the variable itself does not support that method.

    7) Native support for more complex types like BigInteger, BigDecimal and currency types is needed - so we need more control over that.

    8) In-built support for FactTemplates (a better DynaBean) so they can be used as if they were real classes.

    9) A less verbose way to set setters "instance(field1=z, field2=42)". Ability to call a constructor and set fields in a single statement "new BarBaz(field1 = "val", field2 = "x")".

    10) The dependancy must be small, a few hundred kb.

    Bob McWhirter has been busy beavering away on this and the project is getting to an almost useable state; although there is no content on the website you can start to look over the unit tests to get an idea of how the language is turning out.

    A quick look at the language itself:

    instance.field = value;
    instance(field1=z, field2=42)["key"] = value;
    instance.array[0] = value;

    // method call with an inline map and array
    instance.method( [1, 2,"z", var], {"a" => 2, "b" <= c} );

    // standard constructor
    bar = new BarBaz("x", 42)

    // calls default constructor, THEN setters
    bar = new BarBaz(field1 = "val", field2 = "x")

    We are still trying to decide on the for each loop syntax. Bob wants to offer both the crappy java5 syntax (otherwise people will complain) and something more readable:

    foreach item in collection {
    func2(index) # index available automatically as a counter?

    For ( item : collection ) {
    func2(index) # index available automatically as a counter?

    We don't just plan to use this language for consequences, it will also be used from predicates, return values and 'eval's as well as the upcoming 'accumulate' and 'from' conditional elements.

    $cheese : Cheese( /* additional field constraints go here */ )
    from session.getNamedQuery("thename").setProperties( {"key1" => "value1", "key2" => "value2" }).list()

    So if you like what you see why not roll up your sleaves and get involved. A free T-Shirt to the first person that hassles bob to apply some patches :)

    Post Comment

    Just say no to DynaBeans

    The subject of DynaBeans just came up on the mailing list, and it is one I've been asked about before - so I thought I would do a blog my answer.

    Dynabeans was written as a solution for Struts back in 2000, they are not JavaBean compliant – thus they have no value outside of that provided with commons BeanUtils, thus coupling your enterprise system to BeanUtils forever – no other apps are aware of them, or able to script them like they can JavaBeans. They cannot be used with any database schema mapping tools, so you cannot easily leverage db schema generation tools. JBRules 3.0 has no support for mapping utilities thus you’ll only be able to script those from within evals. If you only use evals you will have crippled the engine to work like a simple chain of command scripting engine – under no circumstances do I recommend this. Evals are a worst use case for conditions that cannot be expressed in field constraints, as field constraints and conditional elements become more powerful the dependency of evals is reduced. Further more DynaBeans are backed by a map, each read or write results in a hashmap read or write – this is not a scalable solution.

    A much better solution that works now for JBRules 3.0 is to have runtime class generation. See this blog entry here:

    Tools like cglib make this trivial now:

    BeanGenerator bg = new BeanGenerator();
    bg.addProperty("foo", Double.TYPE);
    bg.addProperty("bar", String.class);
    Object bean = bg.create();

    That produced bean can have hibernate mappings generated at runtime, so you’ll get runtime hibernate support. JBRules can compile drls at runtime using the classloader those beans where generated with thus providing full field constraint use of them and pojo like access in the consequence - much nicer :)

    For JBRules 3.2 we will provide a feature called FactTemplates – this is akin to jess/clips DefTemplates. These allow people to define Facts without the need for a compiled class, which is preferable to some who want complete encapsulation of their rules and business objects within the drl. FactTemplates are backed by an array and thus read/writes are much faster. FactTemplates support both named and int values to specify the field for the value; named set/get result in a HashMap lookup so have similar performance to DynaBean. However JBRules provides compile time optimisation of those named fields and swaps them to int lookups – we will have support for this in the JFDI language too, which we also hope will be support in jBPM. FactTemplates will also provide mapping facilities which means they can be mapped to any underlying structure – be it JavaBean, a Hashmap or a DynaBean.

    Post Comment

    Sunday, November 05, 2006

    Rete with Lazy Joins

    I've just spent the last four weeks stuck in front jProfiler taking JBoss Rules performance to the next level and the results have been great and well beyond what I hoped. To achieve this I wrote custom collections, unrolled loops and cached variables that are used repeatedly for join attempts. I really feel I've taken traditional Rete implementations with the following well known performance enhancements node sharing, alpha node hashing and beta node hashing to the limit. Yet I'm still way off OPSJ levels of performance for manners128 and waltz50.

    My initial thought was that OPSJ must be doing some kinda of compile time/static agenda, like JRules attempts. This idea was slated for investigation when I next tackle performance again. The idea with compile time/static agenda's is that you arrange the nodes and their memories so that propagations and joins occur in such a way that they mimick the results of a simple lifo style agenda and thus they can fire as soon as they hit the terminal node. You gain speed as you are no longer determining all cross products and conflict sets, this is kind of a cross between Rete and Leaps.

    I have recently had the pleasure of exchanging a few emails with Charles Forgy, Earnest Friedman-Hill and a few others where ofcourse I just had to take the opportunity to quiz Charles Forgy on this. Here was his reply "OPSJ does not do compile-time static analysis to avoid computing conflict sets. It computes complete conflict sets, and it applies full MEA analysis to the conflict set (not some less-expensive composite pseudo-MEA)". I was gob smacked and straight away went to off to verify this:

    Waltz on OPSJ (results given to me):
    Added To Agenda: 29,910
    Fired : 14,067

    Waltz on JBoss Rules:
    Added To Agenda: 31,841
    Fired : 14064

    The differences are most likely due to my "less-expensive composite pseudo-MEA" conflict resolution strategy. These results proved Charles' statement true and after having spent four weeks taking true Rete to what I thought was the limit it left me feeling like the twelve year old kid who thought he could play football with the adults.

    Anyway not to be detered I went back to racking my brains again. One of the great things about systems like OPSJ and Jess is they give you something to aim for, without those markers I would probably have given up on trying to improve Rete long ago and opted for a compile time/static agenda like JRules has done. So here goes my next attempt at double guessing OPSJ :)

    In my latest drools code my hashing system does two things, firstly it allows for indexing of composite fields and secondly it returns a bucket where it guarantees that all iterated values are already true for the indexed fields, you do not need to test them again - only further non "==" fields for that join node. Inside a join node I return this bucket and join and propagate for each iterated value. However imagine if instead of iterating over that bucket you simple give a reference from the incoming token to that bucket and propagate forward – so by the time it reaches the Agenda there have been no joins but it has references to all the buckets that contain potential successful joins – I use the term "potential" as the bucket may contain un-indexed fields due to field constraints with non "==" operators. The joining process is then delayed right until the activation is finally fired. Obviously there are still a huge number of details to work out and I’m still not totally sure if this would work, especially if a join node refers to variable constraints that are in a buckets that contains non "==" operators, as you cannot be sure that you should be testing and joining those.

    For Manners i'm pretty sure this works as the "find_seating" rule as two ‘not’ nodes with only "==" constraints. For waltz it’s more difficult as all joins also have "!=" in them. Still I have enough of an idea now that I can start messing with code to try and do a proof of concept.

    Post Comment