Friday, September 19, 2014

The Birth of Drools Pojo Rules

A few weeks back I blogged about our plans for a clean low level executable mode, you can read about that here.

We now have our first rules working, and you can find the project with unit tests here. None of this requires drools-compiler any more, and allows people to write DSLs without ever going through DRL and heavy compilation stages.

It's far off our eventually plans for the executable model, but it's a good start that fits our existing problem domain. Here is a code snippet from the example in the project above, it uses the classic Fire Alarm example from the documentation.

We plan to build Scala and Clojure DSLs in the near future too, using the same technique as below.

public static class WhenThereIsAFireTurnOnTheSprinkler {
    Variable<Fire> fire = any(Fire.class);
    Variable<Sprinkler> sprinkler = any(Sprinkler.class);

    Object when = when(
            input(fire),
            input(sprinkler),
            expr(sprinkler, s -> !s.isOn()),
            expr(sprinkler, fire, (s, f) -> s.getRoom().equals(f.getRoom()))
    );

    public void then(Drools drools, Sprinkler sprinkler) {
        System.out.println("Turn on the sprinkler for room " + sprinkler.getRoom().getName());
        sprinkler.setOn(true);
        drools.update(sprinkler);
    }
}

public static class WhenTheFireIsGoneTurnOffTheSprinkler {
    Variable<Fire> fire = any(Fire.class);
    Variable<Sprinkler> sprinkler = any(Sprinkler.class);

    Object when = when(
            input(sprinkler),
            expr(sprinkler, Sprinkler::isOn),
            input(fire),
            not(fire, sprinkler, (f, s) -> f.getRoom().equals(s.getRoom()))
    );

    public void then(Drools drools, Sprinkler sprinkler) {
        System.out.println("Turn off the sprinkler for room " + sprinkler.getRoom().getName());
        sprinkler.setOn(false);
        drools.update(sprinkler);
    }
}

Share/Bookmark

16 comments:

  1. Hi Mark,
    Just a few lines to write about our concern about Drools WB.
    1) Size of Drools WB is the dobbled of Guvnor (now >160MB!!!).
    2) Different artifacts for diffeerent AS version // Guvnor was portable (!) to any AS
    3) Seem that a lot a bugs are still bloking issue (e.g. Maven does not support transitive dependencies on WB6.0.1, and other bugs encountered on 6.1)
    4) Even on the Drools client, we need to import a ton of unsueful libs/jar (Maven/Maven dependencies) that is very risky for any application (classloading runtime errors hazard)
    The result.
    I did a POC just me alone with guvnor / drools 5.5 within less than a month.
    Now we are trying to do exactly the same with Drools >6 / Drools WB,
    2 Poeple full time did not reach the same result after more than a month full-time.
    And there are still bugs encountered (confusion with the release rules.jar / snaphot , transitive dependencies, etc).

    I'm very very concerned about the projet.
    Especially with the explosion of complexity / war needin EJBs, coupling with security AS implementation, etc, etc.
    Would we see finally the goal a 'usable' release with Drools WB 6.2. (final)?
    Are you going to reduce the size of the artifact?

    Thank you for your reply and best regards !
    S.K.

    ReplyDelete
  2. Stephen

    Forum is probably a better place for these types of Qs.
    1) We've more than doubled the features of workbench. 5.x was very limited and had a lot of problems with adapting to more enterprise use cases.
    2) "skinny wars" have been set as a priority for the project, and JBoss as a whole. This ensures our artefacts are more minimal and use the dependencies provided by EAP. This gives a faster install and restart time and also ensures we don't have class path conflict issues when users use different JBoss technologies in an integrated way.

    But I do agree that it would be nice to still have a "fat war" too, if we can find some time I'll see if we can do this.

    3) 6.0 did have a lot of bugs, it was a huge update. The maven transitive dependencies is fixed. 6.1 fixes a lot of bugs and 6.2 will fix more. Things will stabilise given time, as the product matures.

    4) You pay for what you need. A simple runtime install needs no more dependencies than 5.x did. You can put all your jars on the class path, or lookup manually via http or file. However if you want a dynamic module system with automatic updates from an external maven repository, with all transitive dependencies handled for you - well then you need embedded Maven. But I stress you pay for what you use.

    To ease the start time we wrote 10 or so examples that cover most of the build, deploy and utilisation architectures. Each dealing with an increasingly more complex use case. These documents are available in GIT and fully documented. These exemples can be used as a copy/paste getting started template - I believe more recently someone started to add Maven archetypes to help. The best thing you can do to help is let us know which use cases are not covered by the examples, and help us add them. Then if we can be sure we have full coverage for expected end user cases, with documentation, we should be address this "getting started" issue.
    http://docs.jboss.org/drools/release/6.1.0.Final/drools-docs/html/KIEChapter.html#KIEExamplesSection

    There is no coupling of AS security. Our stuff runs on Websphere and more recently Weblogic.

    As mentioned 6.0 was a first release after a huge update. We will continue to improve and stabilise the product, as it matures.

    ReplyDelete
    Replies
    1. Thanks Mark for your reply. I know this is not the place; I'm sorry for the inconvenience.

      I'm happy to read that you are focusing on both portabily and reducing the weight of the war... And I would not be worried again about the support of Weblogic!

      We need to prove the hot replacement of the behaviour for our product (the common understanding of a "dynamic" business logic cf. JSR94). It was easy to achieve this goal with Guvnor. This was much more difficult with Wb.
      We had to implement the Agent of Guvnor (my good old friend) ourselves quick and dirty on 6.01; now we need to converge to your solution featured for 6.2.
      We tried to download 6.2Beta, with the promise of having a real suppport of rules hot replace
      But We noticed that 6.2 has a special release of Wildfly / and it was not easy to find ! And this is the reason of my worries.

      We know that we ran too fast towards 6.0 because of the promise of having a better support of versioning (Maven). Binaries Packages were not a good solution on Guvnor, though it works perfectly well, simple to understand.
      (4) We are still running on 6.01, because 6.1 is worse. Currently, we are still not able to maketransitive dependencies working (?!) this is a blocking bug on 6.01 / and as I said, 6.1 is even worse (!)
      The result is that we need to duplicate/merge within a single jar our dependencies for the need of the WB! This is a painful workaround...
      Finally, we observe that before 6.2, we are not able to converge to the basic concepts of 'hot replacement' provided by Guvnor out of the box. And that 6.2 is not easy to find/run and worse : to migrate our repository from 6.01 to 6.2

      I understand that the product has a lot of feature requests; maybe I could help pointing on the basic features that my boss is waiting for :)
      Feature #1 :
      Extracting a Decision Table as an Excel SpreadSheet
      and/or basically printing (!) the rules (even the basic Guvnor print PDF has disapeared).
      Feature#2
      one-click GIT repository dump (export knowlesge base) "à la MySQL"
      ++ retrocompatibility : GIT repository/knowledge base dump of Drools WB v6.x should run on WB6.x+1
      Feature #3
      Promoting function as global / just as Guvnor did support it/ or an other scope (set of packages?)
      We *NEED* functions AND global functions because we are based on a model with a complex object grape with hashmap and collections; we have to check every null on any nodes in any rules... Otherwise, the application crashes with a nullpointer... Sometimes, we have to check the type, etc... The workaround for now is a plain old static java Helper embeedeed in our jar dependeny, with the downside that the user cannot see/modify the implemenation.

      Tell me where I could stack those wishes ?!

      Thank you very much and kind regards

      Delete
    2. Transitive dependencies should have worked since 6.1, if they do not the new need to know urgently. More important you need to raise JIRA's for these things, ideally with unit tests.
      http://docs.jboss.org/drools/release/5.5.0.Final/droolsjbpm-introduction-docs/html/gettingstarted.html

      No Jira, No Unit Test - no moaning ;)

      1) We can load from an XLS, just not read out. The plan for next year is to drop our current XML and move to XLS as durable storage format. So that will unify our web and XLS DTable solutions.

      2) You can already do a dump with any GIT tool. But I'll see if we can have a "save as zip" button added, should be quick.

      A 6.0 GIT repository will work with 6.1 and 6.2. If it does not, it's a bug - we need to hear about these things, so we can fix them. Please make sure you raise JIRA's.

      3) Earlier version of drools allowed functions to be used across packages. However this was the source of classloader memory leaks. When the package is reloaded with new rules, all the rules in other packages still reference the older function and create a leak. This is incredibly complex to solve, as it involves creating traceability and then reloading all dependant packages. And it's a very fragile thing to do. So we instead isolated function use to the package they are declared in.

      .java files can be edited in the workbench. The recommendation is you put static methods in there and use them from your rules, they all rules can access them. Although you cannot hot update this function, but you can add new .java files. This is a limitation of class loaders that we cannot avoid.

      You can create project that have your static functions in - in the workbench. And then depend on those projects via maven. You should then have full access to the functions in your rules.

      Feel free to chat to us on irc or the web forum, we can't fix these things if people don't tell us about them, or help provide reproducers or ideally unit tests. And JIRA's are essential.
      http://drools.org/community/forum.html
      http://drools.org/community/chat.html

      Mark

      Delete
    3. The big problem is that we can reproduce this inside our large corporate (proprietary) code base and rule set, but whenever we try to create sample code to reproduce it, the bug doesn't occur in the sample code, the rule fires just fine in the sample code, meaning it's some bizarre interaction with other rules that's causing the rule not to fire despite the fact that the correct known test data has been inserted to make it fire. And of course we can't give you our corporate crown jewels (the complete rule set and the large application that it is embedded in) because we'd be fired and not in the rules sense. This is making us go back to Drools 5 because regardless of Drools 5's limitations, it at least works (mostly), while Drools 6 has been one inscrutable impossible-to-reproduce-outside-our-giant-application bug after another and no guidance from the Drools folks as to how to properly debug these problems so we can actually nail down a problem and create a test case that reproduces it.

      Delete
    4. Red Hat can operate under NDA while looking at your rule bases, to help you debug your problems. We do this for all our customers. We can probably do this for a non-customer as a one off, as 6 is new. This will allow you to protect your code, while getting help. Could you provide a link to the thread you are having problems with in the mailing list? Mario and Davide are normally very keen to help people, so I'd be surprised if they are not attempting to help you.

      Delete
    5. "This is making us go back to Drools 5 because regardless of Drools 5's limitations, it at least works (mostly), "
      There is a mode to run Drools 6 with Rete mode, which might help for now - this is there as a fallback, to allow people to get the benefit of the new api and deployment model, while easing into the phreak algorithm
      http://grepcode.com/file/repo1.maven.org/maven2/org.kie/kie-internal/6.0.0.CR3/org/kie/internal/builder/conf/RuleEngineOption.java

      Also did you try providing trace log outputs? This can really help with understanding what's going on. We will happily explain to you how to interpret them.
      http://blog.athico.com/2014/10/trace-output-with-drools.html

      All Drools developers are available on irc for real time chat discussions across EU and US timezones.
      http://drools.org/community/chat.html

      Delete
  3. I'd actually recommend the web forum for these types of discussions too, so everyone can benefit.
    https://groups.google.com/forum/#!forum/drools-development

    But keep the discussions technical.

    ReplyDelete
  4. Btw you will not get transitive decencies resolved if you do not have kie-ci on the class path.

    ReplyDelete
  5. Hi Mark,
    It sounds like a great evolution of Drools. Making it more accessible to pure Java developers is a really good idea.
    I have a question about the execution speed of such rules because of the "inputs followed by exprs" pattern.
    Drools 5.4 is currently integrated into our application as a kind of "scripting" engine and I always ask my users to write "fail fast" conditions.
    Maybe it is a misunderstanding of Drools 5.4 behavior but I got to the conclusion (after somes experiments) such rules:
    fact1 : Fact( plenty of conditions on fact1 )
    fac2 : Fact( plenty of conditions on fact2 )
    fact3 : OtherFact ( link fact3 with fact1, link fact3 with fact2)

    ... are faster than their equivalent written like this:
    fact1 : Fact()
    fac2 : Fact()
    fact3 : OtherFact ( plenty of conditions on fact1, plenty of conditions on fact2, link fact3 with fact1, link fact3 with fact2)
    The way I imagine Drools behavior, the second one creates a cross product of the Facts then iterates over a big collection to eventually select only a few tuples.
    If my assumption is correct, do you think pojo rules will behave like the first rule or like the second rule?
    Thanks in advance for your answer.
    Best regards

    PS : and I have the impression this approach will bring a lot of benefits to large rule databases (using rule templates) as far as permgen space is concerned

    ReplyDelete
    Replies
    1. yes that is true, and it hasn't changed in the example above. If you want fail fast behaviour you'll want to avoid one large expr block, and use small ones. We'll look at each expr and analyse it's required inputs and build the tree efficiently. So expr blocks with a single input, will still go in the alpha node. Ones with two inputs will be put into the join node etc.

      So yes there is more separation the way things are declared in the above, but we still build the same network via required input analysis.

      Pergem is no longer an issue in java8, it's all heap now.

      Mark

      Delete
    2. Thanks Mark, crystal clear.
      I'll eventually give the drools-pojorule a try but: would you say it will be possible to set a breakpoint in the lambda-based conditions (if there are many of them) in order to be able determine, when a rule is not firing, which condition is evaluated to false? If it was the case, it would be great!
      Thanks in advance.

      Delete
    3. You will be able to put break points in any lambda to determine if it fires, yes.

      Mark

      Delete
    4. That's great! I am really looking forward to implementing my first pojo rules!

      Delete