Sunday, February 17, 2008

Drools Clips

I've done some more work on Drools Clips in the last few weeks and it's starting to take shape now and the basic shell is working and we should have something useful as part of the next Drools milestone release in a few weeks. You can see the unit tests here:
http://anonsvn.labs.jboss.com/labs/jbossrules/trunk/drools-clips/src/test/java/org/drools/clips/ShellTest.java

Below you can see two of the unit tests demonstrating deffunctions and rules:

public void testRuleCallDeftemplate() {
String function = "(deffunction max (?a ?b) (if (> ?a ?b) then (return ?a) else (return ?b) ) )";
this.shell.eval( function );

this.shell.eval( "(import org.drools.*)" );
this.shell.eval( "(defrule testRule (Person (age ?age) ) => (printout t hello)
(printout t \" \" (max 3 ?age) ) )" );
this.shell.eval( "(assert (Person (name mark) (age 32) ) )" );
this.shell.eval( "(run)" );
assertEquals( "hello 32",
new String( this.baos.toByteArray() ) );
}

public void testTwoSimpleRulesWithModify() {
this.shell.eval( "(import org.drools.*)" );
this.shell.eval( "(defrule testRule ?p <- (Person (name ?name&mark) ) =>
(printout t hello) (printout t \" \" ?name) (modify ?p (name bob) ) )" );
this.shell.eval( "(defrule testRule (Person (name ?name&bob) ) => (printout t
hello) (printout t \" \" ?name))" );
this.shell.eval( "(assert (Person (name mark) ) )" );
this.shell.eval( "(run)" );
assertEquals( "hello markhello bob",
new String( this.baos.toByteArray() ) );
}
We have support for a large set of the Clips LHS rule syntax and functions are easy to add. As Drools DRL also has support for the Jess 7.0 style infix notation we will add support for that too. The goal is to support the full LHS syntax for Jess and Clips. For now we will be missing out logical assertions, Jess slot specific and Clips COOL. Our logical assertion approaches are slightly different, so it's easier to leave that out, and we currently have no way to support slot specific which is also the behaviour of Clips COOL. As we have full support for Java pojos and their approach to OO we have less need for COOL at this stage. It would be nice to add slot specific support soon and I hope to do it some time this year.

We do not yet support deftemplates but the rules will reason directly over pojo classes, as our DRL already does. I will add support for those soon.

One of the cool things about this is that it doesn't just allow you to execute Jess and Clips rules but that it also provides you with a migration path. Using DrlDumper you can now load Jess/Clips rules via drools-clips and have the DrlDumper dump them to the more modern Drools DRL syntax.

I've actually changed my design approach for this. Originally I was building my own Lisp execution engine and then realised that I was creating much of the infrastructure that we already had put into MVEL. As Lisp really is just an abstract syntax tree you can actually take any Lisp statement and dump it to MVEL for execution. So I gave up on my own Lisp execution engine and instead did a Lisp to MVEL converter. This is only a stop gap though as Mike Brock has promised to eventually do a direct S-Expression lexer and parser for MVEL. It's trivial to add new built-in functions to drools-clips by simply creating an MVEL language dumper. You can look at the existing functions like the 'if' function in the link below to see how simple this is:
http://anonsvn.labs.jboss.com/labs/jbossrules/trunk/drools-clips/src/main/java/org/drools/clips/functions/IfFunction.java

12 comments:

  1. It's nice to see progress on CLISP support, but I'm gonna say that DRL is not more modern. It's just different. There are plenty of things in CLIPS that neither JESS or Drools supports. For example, clips has built in profiling, which jess and drools does not have. Also, Clips functions are quite different than drools functions. Clips also makes it much easier to add rule dynamically at runtime than drools. Adding and removing rules in clips is trivial. JRules could learn a lot from clips and still doesn't have many of the features that clips provides.

    There are features in drools that aren't in clips, so there's plenty of room for everyone to learn from each other. For example, Jamocha supports temporal facts, temporal activations and temporal patterns, which no other RETE engine provides. JRules has stream processing features, but they tackle that differently.

    Another cool thing in clips and jess, is that you can build a new rule using the built in string functions and then pass it to "build" command.

    In many ways, the slotted positional design owes it's birth to OPS5 and clips. Modern is a subject description :)

    Also, there's still a ton of stuff everyone could learn from ParaOPS5, OPSJ and CLIPS.

    ReplyDelete
  2. "There are plenty of things in CLIPS that neither JESS or Drools supports. For example, clips has built in profiling, which jess and drools does not have. Also, Clips functions are quite different than drools functions."
    Your confusing the syntax with the features. I'm not denying that Clips is a powerful engine but profiling has nothing to do with drl versus a lisp-like syntax, nor does the depth of functions that Clips supports. Just look at the lengths that Jess 7.0 is going, with infix support, to address the legacy issue of a Lisp like language. Then there is the ability to directly access sub fields with a graph/fluent like notation, that neither jess/clips supports, as well as the various other sugar that Drools supports with regards to bindings to reduce the verbosity of older language approaches.

    The DRL is a custom language built specifically for declarative programming with rules. Everything we do is aimed at reducing verbosity and increasing clarity of those rules. The same cannot be claimed for any system with a lisp-like heritage.

    We could go down the route of how powerful Lisp is, but then neither Jess nor Clips support the power provided by common lisp systems, such as macros and other advances features. And even then I would still maintain that anything lisp like reduces the clarity and thus declarativeness of those rules.

    The only real advantage Lisp has is it's use in a command like shell system as you can define an entire rule in a single line. The complexity of our lexer does indeed make this difficult. But then I don't aspire to have such simple consoles, I would rather have a console that supported a rich rule editor.

    The slotted approach for constraints and binding's is indeed a good approach which is why more modern rule languages such as IRL and DRL have adopted and modernised it, while losing much of the lisp baggage.

    ReplyDelete
  3. I won't speak for others, but infix support is a questionable feature in my mind. Often, using dot notation is a work around for a complex object model that doesn't work well for pattern matching.

    The idea itself isn't new by the way. Many engines have considered it way before drools. The reason it wasn't done before is that many of the designers though it was bad and polutes rule programming.

    Like I said, there's plenty for everyone to learn to from each other. I wasn't confusing the system with the features, just trying to point out there's still alot out there for everyone to learn.

    In terms of language, it's a personal choice, so I don't consider a language modern or old. That's like saying Hindi is old and english is new. Those kind of arguments generally don't go any where and really don't do anything for me.

    the console doesn't really matter to me. What is more important is the design and functionality of the engine. having a solid design for functions and providing model mapping capabilities. For example, JESS and CLIPS both don't have the ability to map an object to a different deftemplate structure. It's a feature that I've blogged about in the past. JRules supports the same thing with their BOM functionality.

    Take for example JESS. Users can write custom beanInfo for a given java class and filter out attributes they don't want in the deftemplate. CLIPS doesn't have that feature. JESS also has run-until-halt, which clips doesn't have. JRules has after, and before for their event processing functionality.

    One could do the same thing as infix with mapping functionality. for example myobject.address.zip could be mapped to a deftemplate as myobject.zip. To me, that is a better approach, but that's my own bias. You're right that clips and jess aren't as powerful as prolog. I believe art and clips consciously made that decision. I've read the old art manuals and they had a clear rationale for it.

    Whether that rationale still applies today is questionable. for example, ART built their own memory management component. We get that for free from the JVM. I've heard from users who switched from ART to JESS and JRules that ART is many cases is much faster. I don't know exactly why it's faster, but it could be the custom memory management.

    ReplyDelete
  4. If you want to see what a nice shell could do, you might want to look at what the Aachen students built for jamocha's main branch. It's got a full editor for deftemplates, functions and facts. It also has a nice RETE viewer that allows you select which rules are shown. They also have full FIPA support and other agent related features.

    I think JESS and CLIPS aren't going anywhere. Both are mature rule engines that have proven themselves hundreds and thousands of times.

    I hope that ernest improves JESS shell and IDE, so that users who prefer a rich GUI have good tooling. The aachen students have been working on an eclipse ide for jamocha, so even if JESS doesn't provide it, I hope jamocha can help. This way, users of JESS can keep using JESS, but use some other editor to write their rules.

    ReplyDelete
  5. It's great to see the language we defined 24 years ago live on, Mark!

    For a history and functional review, check on my post responding to yours today.

    Deftemplate support is pretty important for most CLIPS apps.

    Is there a challenge with the logical dependencies?

    I think you've covered most of the functional advances since CLIPS elsewhere in Drools. I agree that COOL is extraneous (and not relevant to porting most JESS apps either).

    Please let me know if I can help with any details.

    ReplyDelete
  6. "Deftemplate support is pretty important for most CLIPS apps."
    We have an internal structure called FactTemplate, which works much the same and can be used on the LHS now. It's an array backed objecttype that can be defined at runtime. However I'm not totally happy with it and haven't documented it or told users about it. I think this is because I want a model to support ontologies before I expose anything that I have to support.

    "Is there a challenge with the logical dependencies?"
    Not really we have a good TMS implementation, which behaves the same. The difference is that where Jess/Clips uses the logical CE to designate the relationship and then all inserted facts are logically created with that relationship. We instead ask our users to use insertLogical(...) and it creates a logical relationship with the entire rule, i.e. we do this at the TerminalNode. I can extend this so that we have a logical CE too, and it'll re-use the same TMS sub system, it's just been lower on my list of priorities.

    The backward chaining is my favourite thorn and one I know I have to address soon. I see and understand how Jess does it. Charles explained OPSJ to me, which I assume is the same as Haley's Eclipse. But it sounded different to the way Jess does it, must admit I didn't fully understand Charles at the time. So any white papers on this would be great :)

    I think my approach will be to create a unify keyword, as how the current rule propagation binds with the unifying data I think can be made separate from how that data is derived. This would allow me to plugin an OO prolog engine, such as Take, and/or support Jess/EClipse/OPJs style backward chaining too. I really do hope I can takle this one over the summer.

    ReplyDelete
  7. to woolfel: hats off!
    FYI: we will be using ** and contributing to ** jamocha
    looks lean, mean, and clean!

    ReplyDelete
  8. Hi,
    I am trying to make some contributions to Drools, I downloaded source from GTI repositories, and I have issues with building Drools-clips subproject only. Some classes are missing:

    etf_workspace\drools\drools-clips\src\main\java\org\drools\clips\ClipsShell.java:[44,27] cannot find symbol
    symbol : class DroolsMVELFactory

    Are you still active on project? Can I count on your help?

    Please contact me at amerzec@gmail.com

    Regards

    ReplyDelete
  9. GIT HEAD is currently broken as we bring in the new parser. You will want to stay with the 5.2M1 branch.

    Mark

    ReplyDelete
  10. Hello Mark,

    are you planning to finish the "dumper" for CLIPS in order to make possible the round tripping? It would be interesting to see if I could convert some of my drools src to clips - hoeping I will gain some performance, if I execute C at the end.

    Thanks and best regards,
    Reka

    ReplyDelete
  11. "are you planning to finish the "dumper" for CLIPS in order to make possible the round tripping?"

    We have no plans to develop this further. There has been no community uptate of this feature. So if someone wants to see it progress, they will need to role up their sleeves :)

    ReplyDelete
  12. Thanks for the info, I will consider which way is cheaper for me :) write the ClpDumper other do the migration manual (the first one of course is more elegant).
    An other interesting topic for me is to create a grammar file for the Ilog Rule Language to parse it and make available the language for Drools. I have seen your work with apocrif and also found an interesting article from INRIA (http://hal.inria.fr/docs/00/34/40/13/PDF/RR-6747.pdf).
    Based on these I got the conclusion may be the easiest is to create the grammar file for ANTLR and then integrate the generated code with Drools. Have you already similar plans?
    The two language are not far from each other. Do you think I can use the DRL.g as starting point? (Sorry if the question is stupid I have just finished my course about formal languages ;) and now I am looking for a nice home work).

    ReplyDelete