Sunday, December 25, 2011

Less boilerplate in Planner: Generic MoveFactory

The recently released Drools Planner 5.4.0.Beta1 includes 2 Generic MoveFactories. That means it's no longer required to implement  a MoveFactory and Move to use a local search optimization algorithm such as Tabu Search or Simulated Annealing.

For example, the MachineReassignment example configures Tabu Search like this:

  <localSearch>
    <selector>
      <selector>
        <moveFactoryClass>org.drools.planner.core.move.generic.GenericChangeMoveFactory</moveFactoryClass>
      </selector>
      <selector>
        <moveFactoryClass>org.drools.planner.core.move.generic.GenericSwapMoveFactory</moveFactoryClass>
      </selector>
    </selector>
    <acceptor>
      <propertyTabuSize>5</propertyTabuSize>
    </acceptor>
    <forager>
      <minimalAcceptedSelection>1000</minimalAcceptedSelection>
    </forager>
  </localSearch>

Notice that there is no MachineReassignment specific code in there whatsoever. But if I wanted to, I could easily mix in a custom MoveFactory implementation too.

Planner comes with 2 generic move factories out of the box:
  • GenericChangeMoveFactory: A GenericChangeMove changes 1 planning variable of 1 planning entity to another planning value. For example: Given course C1 in room R1 and period P1, change its room to room R2.
  • GenericSwapMoveFactory: A GenericSwapMove swaps all the planning variables of 2 planning entities. For example: Given course C1 in room R1 and period P1 and Course C2 in room R2 and period P2, put course C1 in room R2 and period P2 and put course C2 in room R1 and period P1.
They are slightly slower than a custom implementation, but equally scalable.

Friday, December 23, 2011

The Decision Model IP Trap - Part Deux

part 1 part 2 part 3

A while back I published this article titled the “The Decision Model Trap”, http://blog.athico.com/2011/11/decision-model-ip-trap.html. In short it highlighted the dangers of adopting a patented methodology and my opinion on Red Hat's stance on the matter. The patent is owned by the Knowledge Partners International (KPI) who push TDM.

My article was referenced in a thread, started by Jacob Feldman from Open Rules, in a linkedin group for “The Decision Model”. It's a closed group, A copy is provided online here. The original link is here:

I'll quote Jacob below:
“First I learned about a possible patent for The Decision Model from Mark Proctor – see http://blog.athico.com/2011/11/decision-model-ip-trap.html. But it was impossible to find any references to it on the web. Besides, neither Larry nor Barb ever mentioned anything about the patent (at least to me). So, I thought that was just a misunderstanding.

However, on Dec. 6, 2011 USPTO apparently granted a patent to Larry and Barb – see http://www.freepatentsonline.com/8073801.html. I believe it would be only helpful if the Decision Model authors openly explain their position regarding this patent to all of us. Otherwise, such a “holiday present” may scare the entire decision modelling community to stay away from TDM. “

The thread turned hostile with a KPI representative demanding I clarify my motivations and then resorting to belittling me – but I'll come back to that later. The result was that eventually KPI made an announcement on their objectives and intentions with regards to the patent and TDM. The link for this is here:

Lets look at this briefly:
Objectives of the Patent Policy:
  • To ensure that we are able to evolve what we started without risking an infringement of someone else's patent.
  • To share the ideas behind The Decision Model in an orderly way.
  • To protect its rigor, hence its reputation.”
With the current insanity of the various patent systems, having to play the patent game, just to protect yourself is a reality. However restricting 3rd party use of that patent is not necessary to achieve the later two goals. Trademark and certification is a perfectly adequate, if not far superior and more effective way to achieve those goals. Unless they have other objectives, not listed, I invite them to license their patent under terms similar to that in the Apache Software License.

Then lets get onto their next statement:
"Vendors who provide Open Source Software, and who wish to incorporate TDM can obtain a royalty-free license for Open Source software. There will be a certification fee and process for Open Source vendors who desire this optional software certification."

Talk about the classic honey trap. Ring ring, ring ring..... "hello?... Hey KPI it's 2002 calling, they want their business ideas back". For those that don't get the joke it's a play on the "hey hunny, its the 80s calling, they want their hair back" :) Seriously the world has moved on, it's clued up, they don't fall for that clap trap any more. KPI, there is an awesome website, that covered the SCO débâcle, called groklaw.net - very recommended reading. If you are an OSS vendor and take KPI up on their offer, you aren't not Open Source - end of story. Just don't do it to yourself, you deserve better, your customers deserve better.

It's old news now that KPI through a partner is trying to infect the OMG Decision Model and Notation standard effort, http://www.omgwiki.org/dmn-rfp/doku.php. Private emails have been sent between the various heavyweights in the OMG process. I think the general sentiment was "not a chance in hell". So that's one nail in the coffin. A proprietary and encumbered methodology will die when faced with an un-encumbered official and open standard.

They may however try to argue that their patent covers the resulting DMN standard, regardless of whether the DMN group accepts their proposal. The result on the industry in general could be chilling. I would urge KPI to re-read groklaw.net about what happened to SCO when they tried to enforce bogus patents. Yes that's right, “SC...Who?” - it's doubtful your reputations and company brand would survive if you became hostile on an open standard and/or an open source implementation of that standard. RIP TDM.....

So let's now get back to that linked in forum posting. As the thread was started by quoting my initial blog, when someone asked what impact this could have on the industry, I felt that I had every right to re-iterate a key point from the article. That while TDM continues to be patented the industry will move around and beyond it, and that the work we are doing lifting from the extensive research made available in the Prologa and XTT2 will also make it's patent irrelevant.

Michael Grohs, VP of Business Development @ KPI, jumped in demanding I declare my motivations – as if I was some how being underhanded. I don't think he had taken the time to read my article. I think it makes my stance and motivations very clear. But then I believe he was more interested in posturing than substance. I'll show two key points from the article, I think they show my stance and motivations pretty clearly.
“...snip...
Open Source and Patents do not mix. When you get software from Red Hat you are guaranteed its 100% Open Source, not maybe OS or partly OS. From top to bottom, inside and out 100% OS goodness.
...snip...
In the mean time we in the Drools team will continue to take our inspiration from the excellent and unencumbered research projects; Prologa and XTT2.
http://www.econ.kuleuven.be/prologa/
http://ai.ia.agh.edu.pl/wiki/hekate:xtt2”

The thread continued to regress into noise. With further indications that I was trying to be underhanded, by demanding I declare my motivations. It was insinuated that I don't live in the real world, that the patents I have through Red Hat make me hypercritical for demanding “special rights”:
“but then world is full of people who believe that they are entitled to special rights which they believe other people should not have. “

In general there seemed to be a lack of understanding on the use of defensive patents within OSS, particularly on how they have virtually no restrictions, beyond that defensive clause – as specified in the Apache Software License. There also seemed to be a lack of understanding on the walled garden patents create, shutting off the OSS research world - which is why I oppose this so vehemently.

Finally Michael, a VP @KPI, just decided to get full out snotty on me. I couldn't figure out if he was trying to belittle me or indicate that I was being underhanded by concealing that I work for Red Hat – or maybe both. I guess when you have nothing of substance to say, just use insults:
Michael:”Mark I understand that your and Edson's patent is assigned to your employer Red Hat and not to the World, but correct me if I am wrong. So it is actually Red Hat who does the gifting. “

I apologise in advance for the slightly over pompous use of “I”. Those that know me, know that while I evangelise the technology, that I will big up the Drools community in general and it's achievements - I do not try to add grandeur directly to my personage, that actually I'm a little shy of direct attention. I believe Michael thought I was some peon of a developer, scuttling along to my masters commands. So given the circumstances, I felt that a point should be made:

Michael:"I understand that your and Edson's patent is assigned to your employer Red Hat and not to the World, but correct me if I am wrong. So it is actually Red Hat who does the gifting. "

mark:"I'm really not sure what point you are trying to make with this comment. When you have to reach for semantic interpretations it makes you sound bitter and doesn't become an industry professional. I'm not even sure I should dignify it with a response.... but then I wouldn't be me :)

"So it is actually Red Hat who does the gifting"
I don't make it any secret that I'm employee of Red Hat, I'm very proud to work for the worlds number one Open Source company.

But I don't know if that is the point you are trying to make, or if you are trying to belittle me by arguing semantics on the appropriation of the term "we". Much as Suleiman keeps trying to talk down to me by using terms like "real world" and "special rights". So I guess I should answer both possibilities, neither are becoming for you.

I'm the co-founder and creator of Drools, I did this before joining JBoss. The choice to license Drools under the Apache Software License was mine and done before joining JBoss - JBoss was later acquired by Red Hat. It is this license, that I chose, that grants those free and perpetual rights. In fact it is this license that ensures that neither I nor Red Hat nor anyone else contributing to Drools project may file a patent that is not covered under this free and perpetual rights, when that patent relates to Drools.

While at Red Hat it was my choice to file the patent and my choice to do the work necessary for the patent, I could have chosen not to file a patent. Edson also had those same personal choices and we did the work together.

I would say considering those choices that I made I have a write to use the term "we". We as in myself, Edson and Red Hat.

"and not to the World"
You are trying to argue the points of assignment and usage and gift? I'm not sure which part of the following you don't get:
"a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work" "

It doesn't get more "gifting" than this. Well maybe it can. Having watched a user gorge themselves on christmas pudding, mince pies and port. I could wrap the patent in silver paper with stars on it, tie a red bow around it, put on my Red Fedora and climb down the chimney and ram it down their throats. Is that "gifty" enough for you? :) to clarify the term "their" I mean "the world".

......next? “

Anyway I'm looking forward to what Jan Vanthienen, one of the decision table godfathers, has to say on the matter in the new year. I'm guessing that he's not pleased that someone took his work, changed the names used in the terminology and patented it.

Mark
Disclaimer: This post is made in a personal capacity. Nothing written above should be construed as Red Hat's corporate position.

Tuesday, December 20, 2011

Drools : A Journey towards a Meta Framework for Hybrid Reasoning Systems

Introduction

Recent versions of Drools have started to show our direction towards a hybrid reasoning system, going beyond production rule systems. In 2011 we introduced prolog style derivation trees, with reactive materialised views, we also introduced traits. In 2012 we'll continue to build out more of the prolog like functionality and traits will expand to description logic for semantic reasoning.

To make further progress I feel we need to smash apart our current infrustructure that is hard coded for the rigid way PRD systems where designed 30 years ago. In that time the structure of PRD systems hasn't change that much. In essence they have an agenda + conflict resolution strategy. Simple "groups" have been added by various systems be it a single push/pop stacks or rufeflows. Further execution control has been hard coded via attributes.

If we could break Drools down into smaller components, each with life cycles, event models, hook points and designed for compositability - what would this look like? This would provide more of a meta framework, so the author can define how it's execution should behave and open up Drools to a wider variety of research and problem solving.

Existing PRD behaviour can be easily provided as an "out of the box meta configuration", however it is hoped via "macro's" power users can implement other useful types of behaviours and provide those to end users as fully encapsulated macro's.

By opening up the way that Drools components can be easily brought together we hope to make it easier for people to try out different research ideas - without having to develop yet another engine. This will allow ideas to go beyond that conceived by the core development team.

I hope the introduction of a macro concept for rules will allow for a more pattern oriented approach to rule engine development.

When reading this document please cast aside preconcieved ideas of what a rule engine is or how it works. The document is conceptual in nature and at this stage I would like conceptual input on how to progress the ideas, and not spend 95% of the time argueing over which symbols or keywords to use, or whether it's needed (see http://www.bikeshed.com). Expect many rough sketches, incomplete syntax etc, it is conceptual brain storming and not an implementation spec.

Continuous updates to this article can be found here.

Module

Additional ideas on modules for rules can be read as part of the research on Venus:
Venus: An Object-Oriented Extension of Rule-Based Programming (1998)
An Overview of the VenusDB Active Multidatabase System (1996)

The module concept can be introduced as generic container for rules. The module consists of a name and arguments. Parenthesis can be ommitted if no arguments are specified. There are also further member variables, internal to the module. A module supports attributes. A module can specify a single parent module, there is a MAIN module which is the default parent of all modules. A module may optional extend other modules
<@attribute(....)>*
module <module_name> ( <arguments>*)
<parent module_name>?
<extends <module_name>+>*
<variables plus optional initialisation>*
A DRL can have multiple modules specified. Within the DRL file all rules that come after that module name are associated with the module.
module A
rule r1....
end

rule r2...
end

module B(String s, Person p)
var Book b

rule r3...
end

rule r4...
end
NOTE) Can modules span namespaces? What does a namespce mean for a module, does it live within the namespace or is it global? For now assuming scoped to namespace.:

A module can be called like a java Runnable, the caller's method signature must match the specified arguments or a runtime exception is thrown
Module m = kruntime.packages["org.domain"].modules["A"]
ModuleHandle bhA = m.run()

m = kruntime.packages["org.domain"].modules["B"]
ModuleHandle mhB =m.run( "S", new Person("darth") );
The parameter variables are available to the rules inside of the module, in a similar manner that globals are. An array pattern referencing the arguments is injected as the root pattern to each rule. This not only makes the variables available but acts as a control object for when rules can fire with those variables. The ModuleHandle has a "stop" method that results in that array element being retracted, and thus the rule cannot fire any more for those variables. Even rules associated with that module that do not depend on those variables has the root control fact injected.
mhB.stop()
Note that a module can be called multple times with different arguments, each results in a different ModuleHandle. Should a module be called with existing arguments the previous ModuleHandle is returned. So you can consider a module instance module + variables.

Passed module arguments may or may not be facts, if they are facts they may be modified (unlike globals) and patterns will respect and react to that change as normal. The module's called signature is updated.

The role of member variables is for scoped data avaliable to events, such as counters or intermediary objects.

(NOTE) If we want to make member variables available to rules, we will need to think carefuly about the behaviour, as they have more potentially more complications than globals. Currently they are only available as fields on events, see the activation-group variance.

Match

A Match is the same as the traditional Activation concept in PRD systems. It has an array of FactHandle's for the matched objects. But we recognise the Match can be active or dormant, and use a boolean to represent this, and that even information on dormant matches can be useful in reasoning systems. A Match is considered active if it is ellible for for foring and has not yet done so.
Match 
FactHandle[] facts
boolean active
(NOTE) Maybe instead of "boolean active" we should have an enum, and possible also including "running" as a state. ACTIVE, RUNNING, DORMANT.

(NOTE) It's possible for a rule + facts to fire multiple times without relaxing (made false). We should optional keep a counter for the number of times fired, and even each execution can be time stamped.

Module Properties

Module Properties
activeSize // number of active Matches, queries can be used for more complex cases using rule attributes
dormantSize // number of dormant Matches, queries can be used for more complex cases using rule attributes
activeMatches // Collection of active Matches
dormantMatches // Collection of dormant Matches
(NOTE) The size and and collection properties only contain relevance for eager matching algorithms that compute an entire cross product for each WME change. Future versions of Drools may optionally implement some lazy matching and in those situations the user's understanding of those properties may be a problem.

Module Methods

Module Methods
cancel() // all active matches are made dormant
cancel( Match ) // cancels a match, i.e. sets it dormant, if already dormant this method does nothing.
refresh() // all dormant activates are made active, however filters such as calendars, enabled etc are still obeyed
refresh( Match ) // refreshes a match, i.e. sets it active if it's dormant, if it's already active it is ignored
halt() // The state of the module is preserved, but no
continue() //
addListener() // type inference adds the listener implementation to the correct list(s).
// Listener composition for single instance is allowed, meaning it is added to multiple listener lists.
start(Object[] args)
stop()

Module Events and Lifecycle

A Module has a life cycle to which listeners may be added.
Module Events
onEnter // when a module is called
onExit // when a module is stopped
onMatch // when a rule is matched
onRematch // when a rule is matched and is matched again, without relaxing first (via update)
onUnmatch // when a rule stops being matched
onBeforeFire // before a rule for this module fires
onAfterFire // after a rule for this module fires
onHalt // tells the listener halt has been callled
onResume // tells the listeners resume ahs been called
onEmpty // when the size == 0, all Matches are dormant -- other potential events --
onBeforeRuleEvaluation // before a wme insert/update/delete for this module
onAfterRuleEvaluation// after a wme insert/update/delete for this module

onEmpty will only be triggered after the first rule evaluation phase. i.e. it does not fire after onEnter, before the rule evaluation phase has had a chance to execute.

(NOTE) Other potential events are before/after rule evaluation. Each WME insert/modify/update causes a rule evaluation, and could be listend to via onBeforeRuleEvaluation and OnAfterRuleEvaluation. This poses problems with concurrency where potentially multiple insertions/updates/modifies could be happening at the same time. So at best it's a listener scoped to a fact, we cannot guarantee the resutling matches are assocated with this event, unless serial rule evaluation is enforced - which may be a possible configuration for a module.

DRL will support the ability to declare literal functions attached to these listeners, the exact syntax for this is TBD. But the keyword will probaby be "on.". We will support .Net style delegate operators for = and += when adding or setting listeners.
on.Enter += {
}
Those listeners can also be added from java code.

Simulation the Agenda


A Module itself doesn't do anything other than obey the life cycle of the prescribed events, it doesn't even fire a rule, making it dormant. The only thing the Module does is maintain the list of active and dormant matches.

However this then allows a much more flexible system to which the end user can customise the behaviour. For instance the traditional agenda+conflict resolution strategy can be implemented via the onMatch/onRematch/onUnmatch listeners.
class Agenda implements onMatch, onUnmatch, onRematch { 
....
}
The above class implements a composition of event listener interfaces, but it only needs to be added to the module once
module.addListener( new Agenda() );
This means each module may have it's own conflict resolution strategy. Some may want more traditional lifo execution, but anything else is possible. Such as rule definition order, or async execution. Listeners can be combined, but that means that order IS important. Some listeners may want to preevent a rule from firing, others may want to prever other rules from firing after the current rule has fired.

Async rule execution

Fire each match asyncronously as it matches.
onMatch + { asyncFire( match ); }

Manual Interaction Agenda

Because the listeners are now fully pluggable and whether a rule fires or does not, or is cancelled is fully pragmatic it is possible to just expose the Module via an interactive GUI and the end user can see the conflict set and specify which rule(s) to fire.

As WME actions occur the users GUI will be updated showing the active matches, however none will fire without user selection. This means the user can interrogate the state of each Match and the fact it contains to select whch rules to fire. The user can also see dormant matches and "refresh" them so they can become available for firing again.

This could be taken a step further where a user sees a dash board of modules and all are inactive and the user can manually activate modules they wish to see evaluated.

Simulating existing rule execution behviours

activation-group plus variance

Other rule attribute behaviours can be implemented by combining listeners, although listener order is obviously important. For instance this listener can be added AFTER the agenda listener, and it simulates the existing "activation-group" behaviour, such that only the first activation fires for a conflict.
activation-group
onMatch += { 
module.cancel(); // cancels all other matches
}
We could do a variance on this that fires the first 3 activations, and cancels the rest
onEnter += { count = 0 } // count is a module member variable
onMatch += {
if ( count++ == 3 ) {
module.cancel(); // cancels all other matches ]
}
}

ruleflow-group

ruleflow-group behaviour is very simple to simulate. jBPM injects a trigger member variable, that on exit calls telling jBPM to trigger the next nodes in the flow, which may be ruleflow-group nodes or other jBPM nodes
onEmpty { module.stop() } // stopping the module causes onExit to fire
onExit + {
node.triggerCompleted()
}

agenda-groups

agenda-groups implement a push/pop stack behaviour. Only the stack tip executes, the others are considered "halted". To acehive this the consequence of any rule needs to call the following three methods:
module1.halt();
module2.onExit += { module1.resume() };
module2.call();
So that halts the current module and executes "pushes" the next module, when that module finishes "pops" the caller is resumed.

Simulating agenda-filters
If a match is cancelled, and made dormant it is not propagated to the next listener in the onMatch event.
onMatch += {
if ( <boolean expr> ) {
module.cancel( match );
}
}

Events as facts

Events are also inserted as facts, so can be matched in rules too. Member variables are fields on the events
rule  activationGroupVariance when 
m : OnMatch( count < 3 )
then
m.count++
end

Parent Modules and Module Scoping

Modules can contain sub modules, they can ony be called from within the scope of the parent module.Sub modules will have access othe parent modules member variables and parameter variables.

Module Reuse and Extension

A module can extend other modules. The exact semantics of this will need a lot more extensive though, but i'll outline something to get started. When a module extends another module it effectively copies the definitions of all the contained rules.However those rules will match and fire completely indepedantly of the source.

Macro's to abstract module configuration

All this manual configuration would become very cumbersome to the end user, and potentially far too complex. The idea is that all the above complexity is for power users, this is then encapsulated via macro's for end users. I don't intend to outline a Macro system for Drools yet, as that is a different subject in itself and will need a lot of though. For now I'll use <macro "name">, pontentially these can be listed. But how macro's are defined and applied are completely open to debate. These macro's are then expanded at complie time to provide all the required listener behaviour:
Module xxxx 
macro activationGroup( "g1" )

module xxx
macro agenda module xxx
macro agenda activationGroup( "g1 )

module xxx
macro default

Drools 5.4.0.Beta1 released

We're happy to announce the release of Drools (Expert, Fusion, Planner, Guvnor) 5.4.0.Beta1.

Beta1 finally introduces the simulation and testing work that we've been talking about for so long. This will provide a unified environment for simulation and testing over time for rules, workflow and event processing. Beta1 also demonstrates our continued effort to fully commodotise the decison table paradigm, using the excellent research provided by Jan Vanthienen's Prologa project, some high lights for the decision table improvements are listed below.
  • Limited and Extended entry wizards
  • Automatic decision table generation to expanded form
  • Improved rule templating flexibility
  • Work Item integration (declarative lazy environment bound functions)
  • Impossible Match detection
  • Conflict Match detection

Documtation, Release Notes and Downloads are detailed below:

Try it out and give us some feed-back (user list, issue tracker).

Monday, December 19, 2011

Dynamic typing in rules : Traits (part 2)

In a previous post, we discussed the experimental feature called "traits", a way to combine strong and weak typing in Drools. A trait is an interface which can be attached ("donned"), even temporarily, to a fact. It is particularly suitable to model roles or temporary behaviors. Let's take, for example, the following fact model:

declare Person
  @Traitable
  code         : String
  name         : String
  address      : String
  balance      : long
  registerDate : Date
  orders       : Collection
end

declare OrderItem
  @Traitable
  itemId       : String
  price        : long
end

//new syntax!
declare trait HasDiscountApplied
  discount     : long
end


declare trait Customer
  code         : String
  balance      : long
  registerDate : Date
  orders       : Collection
end

declare trait GoldenCustomer extends Customer, HasDiscountApplied
  maxExpense   : long  
end

declare trait SeniorCustomer extends Customer, HasDiscountApplied
  wasAwarded   : boolean
end


A shop registers visitor to prepare special dedicated offers. When a visitor makes a purchase, they become Customers. For some reasons, the shop cares about Golden customers (those who spent more than a certain amount of money over the last year) and Senior customers (those who have been loyal for a certain time).
While some of the items on sale may have a discount, golden and senior customers always receive an (additional), personalized discount. Senior customers may even receive a public mention on the website, if they wish so.

The toy example is not accurate, but will serve to discuss some issues. Here, people and items are the only concrete domain entities: Customer, Golden and Senior are all statuses gained (or lost) by people according to some conditions. Likewise, being applied a discount is another transient property of customers and items.

We have seen previously that :

when
  $p : Person( ... )        // some conditions apply
  // exists Order( ... )
then
  don( $p, Customer.class ) // a Customer proxy is insert
end


The proxy will allow to write rules against the fields defined by the Customer interface, but the getters/setters will be remapped internally to the concrete fields of the wrapped core object (the Person, in this case). Notice that, as a side benefit, using interfaces allows to exploit multiple inheritance.

Now, one might wonder what happens when a core class does NOT provide the implementation for a field defined in an interface. We call hard fields those trait fields which are also core fields and thus readily available, while we define soft those fields which are NOT provided by the core class. Hidden fields, instead, are fields in the core class not exposed by the interface.
In our example, code, balance, registerDate and orders are hard fields for Customer provided by Person. GoldenCustomer adds discount (from HasDiscountApplied!) and maxExpense, which are soft fields. Eventually, name is a hidden field.

So, while hard field management is intuitive, there remains the problem of soft and hidden fields. The solution we have adopted is to use a two-part proxy.
Internally, proxies are formed by a proper proxy and a wrapper. The former implements the interface, while the latter manages the core object fields, implementing a name/value map to supports soft fields. The proxy, then, uses both the core object and the map wrapper to implement the interface, as needed. So, you can write:

when
  $sc : SeniorCustomer( $c : code, // hard getter
                        $award : wasAwarded == true // soft getter
                      )        
then
  $sc.setDiscount( ... ); // soft setter
end


The wrapper itself is exposed through the fields special getter available to all trait proxies. It is used to access soft fields as well as hard ones. The wrapper, in fact, mimics soft access to hard fields too, for uniformity.

  $sc : SeniorCustomer( $name : fields[ "name" ],
                        $code : fields[ "code"],  
                        $award : fields[ "wasAwarded" ] == true 
                      )        


The wrapper, then, provides a looser form of typing when writing rules. However, it has also other uses. The wrapper is specific to the object it wraps, regardless of how many traits have been attached to an object: all the proxies on the same object will share the same wrapper. Secondly, the wrapper also contains a back-reference to all proxies attached to the wrapped object, effectively allowing traits to see each other. To this end, we have provided the new isA operator:

  $sc : SeniorCustomer( wasAwarded == true, 
                        this isA "GoldenCustomer",
                        $maxExpense : fields[ "maxExpense" ]
                      )        


This rule would be triggered by a SeniorCustomer, but propagated only if the fact is "also" ( i.e. the same core object also has donned the trait of ) a GoldenCustomer. The only possible disadvantage is that this type of cross-access requires loose typing, although if an explicit join is always possible thanks to the core field, again common to all traits:

  $sc : SeniorCustomer( wasAwarded == true, 
                        $core : core
                      )        
  $gc : SeniorCustomer( core == $core,
                        $maxExpense : maxExpense
                      )       


Eventually, the business logic may require that a trait is removed from a wrapped object. To this end, we provide two options. The first is a "logical don", which will result in a logical insertion of the proxy resulting from the traiting operation:

then
  don( $x, // core object
       Customer.class, // trait class 
       true // optional flag for logical insertion
     )


The second is the use of the shed keyword, which causes the retraction of the proxy corresponding to the given argument type:

then
  Thing t = shed( $x, GoldenCustomer.class )


This operation returns another proxy implementing the org.drools.factmodel.traits.Thing interface, where the getFields() and getCore() methods are defined. Internally, in fact, all declared traits are generated to extend this interface (in addition to any others specified). This allows to preserve the wrapper with the soft fields which would otherwise be lost.

(Note: in addition to Thing, we also provide the class org.drools.factmodel.trait.Entity, an empty class with just an id and the data structures to make it @Traitable. We might rename it to Individual in the near future. And if this rings two bells to some of you, yes, the reason would be exactly that. Stay tuned.)

Sunday, December 18, 2011

New feature spotlight: Traits (part 1)

We all know that Drools is Java-oriented, so its fact model is object-oriented, in the sense of what Java makes of classes and objects. Leaving academic discussions out, let's focus on a great advantage and disadvantage: strong typing. Patterns and constraints have to be written against available classes and fields, and the compiler can check the consistency at load time so they can be correctly evaluated at runtime.

All well and good, save for a limitation: a fact, being an object, can have only one type hierarchy. Imagine you're building a business management application (policies? mortgages?): you'll likely need rules for employees and customers, but what if you want to write rules for special classes of customers? A few days ago, I was discussing a data validation application: what if you wanted to "tag" some facts as "invalid" or "inaccurate" and then write specific rules for them? In another example, we had a message-driven application, where we had to apply rules to the members of an organization according to the role they had in a message: sender vs recipient vs subject etc.... Take conan's adventure game: right now he has "hero" vs "monster", but soon he'll add classes, say fighter, mage, ... vs orc, undead, ...: what if he wants a fighter/mage to face an undead orc?

All these applications - and many more - have a common problem: the same fact (customer, employee, data sample, character) may have multiple dynamic types which do not fit naturally in a class hierarchy. Arguably, there are many possible solutions one could adopt, including:


  • Fit all classes in a hierarchy

  • Use labels (strings, enums, etc...) to model types

  • Write rules using interfaces

  • Use proxy facts



The first method is definitely not recommended. Java does not have multiple inheritance, so fitting a complex hierarchy in a simpler one might result in unnatural "isA" relations between classes.

declare Customer
// fields here
end

declare GoldenCustomer extends Customer
// more fields here
end


Even then, there is another problem here: imagine you have a Customer you want to promote to GoldenCustomer (which extends Customer). Probably, you'll have to clone your existing fact into an instance of the subclass, with a considerable effort to maintain consistency.

It seems much easier to model roles with strings or other marker objects:

declare Customer
 roles : Collection
 // more fields here
end


With this solution, one can add multiple types to a fact, but unfortunately those will just be labels, not types, so a change in type will not support a change in behavior. This means that any field which is relevant only for GoldenCustomers will have to moved up to Customer and constantly checked for consistency.

Roles are indeed modeled better using interfaces, which define the fields which should be visible when an object is observed from the point of view of that interface.

declare CustomerImpl implements Customer
 // interface impl here
end


Forgetting that Drools does not support implements, which makes the previous snippet illegal, the problem here is that an interface is attached to a class and not to individual objects. So, our GoldenCustomer can't be an interface unless we want either all Customers to be golden ones, or we provide an implementation class for GoldenCustomers only, effectively going back to the first solution discussed.

A much better solution would be the use of proxy facts:

declare GoldenCustomer
 customer : Customer
 // more fields here
end


The idea is to create an additional fact, modelling the role that an object would have in the particular context. The obvious advantages of wrapping/decorating against cloning is that information is not replicated and, moreover, the role is applied to a particular fact (not to the entire class) and can be removed as needed. The price to pay is that, in order to write the rules, the user must refer explicitly the inner customer to access its fields. This also implies that the status of GoldenCustomer can be applied to instances of Customer only (think of more generic roles, which do not have a single domain, like Sender...)

In order to get the best of all these solutions, Drools now offers an experimental feature : traits

A trait is an interface that can be applied (and eventually removed) to an individual object at runtime.
To create a trait out of an interface, one has to add a @format(trait) annotation to its declaration in DRL:

declare trait GoldenCustomer
  // fields will map to getters/setters
  code     : String
  balance  : long
  discount : int
end

import LegacyInterface;
declare trait LegacyInterface
end


In order to apply a trait to an object, we provide the new don keyword, which can be used as simply as this:

when
$c : Customer()
then
GoldenCustomer gc = don( $c, Customer.class );
end


A few important remarks here. First of all, when a core object dons a trait, a proxy class is created on the fly (one such class will be generated lazily for each core/trait class combination). The proxy instance, which wraps the core object and implements the trait interface, is insert-ed automatically and will possibly activate other rules.
An immediate advantage of declaring and using interfaces, getting the implementation proxy for free from the engine, is that multiple inheritance hierarchies can be exploited when writing rules. The core classes, however, need not implement any of those interfaces statically, also facilitating the use of legacy classes as cores.
In fact, any object can don a trait. For efficiency reasons, however, one can add the @Traitable annotation to a declared bean class to reduce the amount of glue code that the compiler will have to generate. This is optional and will not change the behavior of the engine:

declare Customer
  @Traitable
  code    : String
  balance : long
end


Since the only connection between core classes and trait interfaces is at the proxy level, a trait is not specifically tied to a core class. This means that the same trait can be applied to totally different objects. (The problem of filling the LHS of a donning rule will be the topic of a next post...)
Notice that, for this reason, the trait does not transparently expose the fields of its core object. So, when writing a rule using a trait interface, only the fields of the interface will be available, as usual.
However, any field in the interface that corresponds to a core object field, will be mapped by the proxy class.

when
 $o: OrderItem( $p : price, $code : custCode )
 $c: GoldenCustomer( code == $code, $a : balance, $d: discount )
then
 $c.setBalance( $a - $p*$d );
end


In this case, the code and balance would be read from the underlying Customer object. Likewise, the setAccount will modify the underlying object, preserving a strongly typed access to the data structures.

Friday, December 16, 2011

Why programmers work at night

http://swizec.com/blog/why-programmers-work-at-night/swizec/3198

"....snip...

On the other hand you have something PG calls the maker’s schedule – a schedule for those of us who produce stuff. Working on large abstract systems involves fitting the whole thing into your mind – somebody once likened this to constructing a house out of expensive crystal glassand as soon as someone distracts you, it all comes barreling down and shatters into a thousand pieces.

This is why programmers are so annoyed when you distract them.

...snip...

Keep staring at a bright source of light in the evening and your sleep cyclegets delayed. You forget to be tired until 3am. Then you wake up at 11am and when the evening rolls around you simply aren’t tired because hey, you’ve only been up since 11am!

Given enough iterations this can essentially drag you into a different timezone. What’s more interesting is that it doesn’t seem to keep rolling, once you get into that equilibrium of going to bed between 3am and 4am you tend to stay there."



Thursday, December 15, 2011

Drools Developers get PAID MORE as demand QUADRUPLES!!!

Good news for those with JBoss Drools skills, your in demand!!! Latest results in the UK from itjobswatch.co.uk shows over a three times increase in demand for Drools jobs since Dec 2010. It doesn't take much to imagine where that curve is going to be in 2014.....

The report for this information can be seen here.

But it gets better the average salary in London over the last 3 months is £90K GBP that's around £140K USD. And note that's just the average, I know I've seen jobs over 100K GBP in London for Drools. The demand growth for Drools in London has quadrupled to a 4x increase in the same period as above:


Now lets compare this to the average Java developer, the report for that is here. The results are much more depressing, your demand as a java developer is shrinking, with salaries averaging just 55K GBP or 88K USD for London:

So if you are already a fellow Drools developer, congratualations and enjoy the finer things in life. If you aren't, what are you waiting for, now is the perfect time to dip your toes in - as someone famous once said "Because your worth it" :)

Using Work Items in rules' consequences

Recently I added the ability to use Work Items as function calls in the guided decision table editor in Guvnor. This highlighted that Work Items have always been available as function calls in DRL but this has not been well documented; leaving users to make the connection.

Various Work Item handlers are available "out of the (jBPM) box" in the org.jbpm.process.workitem package that may prove useful to rule authors. In addition Work Item Handlers providing bespoke services for all domain areas can be easily authored and plugged in.

I have added a couple of examples to drools-examples (in the master branch on github and to be included 5.4.0.beta1) that illustrate how to use Work Item handlers from the right-hand side of a rule: one simulates sending an email and the other provides a greeting service; the code for both residing in custom Work Item handlers.

I hope they provide a catalyst to encourage the use of Work Items in rules.

Wednesday, December 14, 2011

jBPM Form Builder follow-up

Greetings. There are a few items I will be adding to the Form Builder in the next few weeks, including the following:

Create user roles for different menu options
It’s important to have different roles with separate functions. Certain profiles, i.e. web designer, could edit any visual component. However, a functional analyst might need to have limited visual components, previously approved by a web designer. At first, these are the profiles in mind:

  • web designer: Can define any customized visual component
  • functional analyst: Can use any customized visual component

Create visual asistants for script development on visual components specific events
Profiles that will use the form builder, at least to begin with, shouldn’t need scripting knowledge to understand what to do on specific visual events or how to handle them. That’s why a series of visual helpers should allow a user with very few technical knowledge of scripting to define behaviour for certain visual events like onClick, onChange, and so on.

Guvnor and jBPM console interaction
I will change current jBPM console interaction to use a specific API exposed by the form builder. Currently we publish specific FTL files in guvnor every time we save a form definition, making it unnecessary to change it. This option, however, could be improved, in order to provide different language options. Here's an example view of one of them:

Instalation scripts
Currently I’m working on expanding the jbpm installer to have the form builder included in it.


Also, we expect to have this as an available module by next release (5.3) If everything goes well, everyone will have this tool pretty soon

Cheers,

Mariano

Monday, December 12, 2011

jBPM 5.2 released

The team is proud to present the next release of jBPM. jBPM is an open-source business process engine, supporting BPMN2. I think we have a few very nice new features in this release, with for example
A full list of features is added below.

You can download the artefacts here.
Documentation can be found here.

To get started, it is probably best to download the full installer and follow the installer documentation, to guide you through the tools with a simple example. You can also import the examples module to have look at the numerous examples included in there.

We'll be updating some of the documentation and adding some quick starts and more examples in the next few weeks, so stay tuned!

Have fun!


Release notes

On top of some optimizations, bug fixes and small improvements, these are the most important new features in jBPM 5.2.0.Final.

Core engine
  • domain-specific service repository: the ability to import domain-specific services from a repository so you can immediately use them in your processes, e.g. twitter integration, FTP, web or REST service, etc. This is ideal for non-technical users to integrate with external services
  • improved persistence support for multiple databases and backwards compatibility
  • jbpm-test: new module that offers improved support for JUnit testing of processes, including the automatic setup of the necessary datasources and services on the fly
  • support for Java7

Installer

  • added support for JBoss AS7, which is now used as the default application server in the jbpm-installer

Web-based designer
  • Integration with the domain-specific service repository
  • Visual Process Validation: new features allows users to visually validate their processes at edit time
  • Ability to view the process in ERDF, JSON, PDF, PNG, BPMN2, and SVG formats
  • New Process Information section: contains information about the process, such as name, creation date, version, etc
  • jBPM 3.2 Process Migration: new feature allows users to migrate existing jBPM 3.2-based processes to BPMN2
  • Ability to import existing bpmn2 processes straight into designer
  • Ability to create "sharable" process image, PDF, and ability to generate code for embedding designer in custom applications
  • Support four boundary events
  • Visual support for Text Annotations, Groups, and Lanes
  • Support for sub-processes
  • Update to latest Guvnor

Human task service
  • introduction of the task service interface, and interface to interact with the (default) human task service (independent of the underlying communication pattern and technology)
  • user group callback: easily integrate with your own service for user / group validation and user-group-management (which users are part of which groups)
  • local task service: a local implementation of the task service that runs locally, next to the process engine, and (re)uses the same transaction of the core engine
  • human task service war: deploy the human task service as a service on your application server

jBPM console
  • updated to latest version of the JBoss BPM console
  • console synchronizes with the process definitions (and other assets) from all packages in the Guvnor repository
  • updated BIRT reporting to latest version

Eclipse plugin
  • jBPM perspective: Eclipse perspective that simplifies workspace configuration
  • more advanced jBPM project wizard

Friday, December 09, 2011

Machine reassignment for Google ROADEF/EURO Challenge 2012

Drools Planner has a new example: machine reassignment.

Problem statement
Assign each process to a machine. All processes already have an original (unoptimized) assignment. Each process requires an amount of each resource (such as CPU, RAM, ...). This is more complex version of the Cloud balancing example.

The problem is defined by the Google ROADEF/EURO Challenge 2012.

Hard constraints:
  • Maximum capacity: The maximum capacity for each resource for each machine must not be exceeded.
  • Conflict: Processes of the same service must run on distinct machines.
  • Spread: Processes of the same service must be spread across locations.
  • Dependency: The processes of a service depending on another service must run in the neighborhood of a process of the other service.
  • Transient usage: Some resources are transient and count towards the maximum capacity of both the original machine as the newly assigned machine.
Soft constraints:
  • Load: The safety capacity for each resource for each machine should not be exceeded.
  • Balance: Leave room for future assignments by balancing the available resources on each machine.
  • Process move cost: A process has a move cost.
  • Service move cost: A service has a move cost.
  • Machine move cost: Moving a process from machine A to machine B has another A-B specific move cost.
Problem size
model_a1_1: 2 resources, 1 neighborhoods, 4 locations, 4 machines, 79 services, 100 processes and 1 balancePenalties with flooredPossibleSolutionSize (10^60).
model_a1_2: 4 resources, 2 neighborhoods, 4 locations, 100 machines, 980 services, 1000 processes and 0 balancePenalties with flooredPossibleSolutionSize (10^2000).
model_a1_3: 3 resources, 5 neighborhoods, 25 locations, 100 machines, 216 services, 1000 processes and 0 balancePenalties with flooredPossibleSolutionSize (10^2000).
model_a1_4: 3 resources, 50 neighborhoods, 50 locations, 50 machines, 142 services, 1000 processes and 1 balancePenalties with flooredPossibleSolutionSize (10^1698).
model_a1_5: 4 resources, 2 neighborhoods, 4 locations, 12 machines, 981 services, 1000 processes and 1 balancePenalties with flooredPossibleSolutionSize (10^1079).
model_a2_1: 3 resources, 1 neighborhoods, 1 locations, 100 machines, 1000 services, 1000 processes and 0 balancePenalties with flooredPossibleSolutionSize (10^2000).
model_a2_2: 12 resources, 5 neighborhoods, 25 locations, 100 machines, 170 services, 1000 processes and 0 balancePenalties with flooredPossibleSolutionSize (10^2000).
model_a2_3: 12 resources, 5 neighborhoods, 25 locations, 100 machines, 129 services, 1000 processes and 0 balancePenalties with flooredPossibleSolutionSize (10^2000).
model_a2_4: 12 resources, 5 neighborhoods, 25 locations, 50 machines, 180 services, 1000 processes and 1 balancePenalties with flooredPossibleSolutionSize (10^1698).
model_a2_5: 12 resources, 5 neighborhoods, 25 locations, 50 machines, 153 services, 1000 processes and 0 balancePenalties with flooredPossibleSolutionSize (10^1698).

Results
The implementation is still pretty basic, I wrote it in about 5 Sundays (including tweaking). These are the results:


The schedules found by Drools Planner (after only 5 minutes) are on average 27% better than the original schedules.

Open Science
In the name of Open Science and Open Source, all my work was public from day one (as people who follow me on Google+ or GitHub will have seen). I hope that researchers who work full-time on these sort of competitions start doing the same.


Qualification phase 1 has just ended. It has been an interesting competition so far: it has given me a good insight on the strengths and - more interestingly - the weaknesses of Drools Planner. Expect those weaknesses to be improved in future Drools Planner versions :)

Thursday, December 08, 2011

Drools & jBPM at ICAART (Portugal) 4th International Conference on Agents and Artificial Intelligence


Droosl & jBPM @ ICAART 2012 is now confirmed, and myself (Mark Proctor) and Dr Davide Sottara will be there. If you have any interesting research on or with Drools & jBPM that you would like to present on the day, let us konw.
6-8 Febuary 2012
Vilamoura, Algarve, Portugal
http://www.icaart.org/tutorials.asp

The day is a tutorial day aimed at all levels. It will start with general introductions to the technology but will slant off to more of our research based projects such as Drools Semantics and Chance, as it's part of an academic conference. We would also like to give an opportunity for the people to present their own research, slots can be anything from 20minutes to 60 - contact me if you are interested mproctor at codehaus d0t org.

There will also be plenty of time for discussions and help with your own projects.

Abstract

Drools is the leading open source, industry focused, rule engine. While Drools started life as a Rete based forward chaining engine, it has since transcended. It's ongoing mission is to explore declarative paradigms from a practical and industrial perspective, to boldly go where no engine has gone before.

The tutorial will start with a gentle introduction, suitable for all level of expertise, covering the core language and functionality slowly expanding into more complex areas. The topics covered include, but are not limited to:

Basic Concepts:

  • Patterns, Constraints and Unification
  • Data Driven and Goal Oriented Inference using Forward Chaining and (Opportunistic) Backward Chaining
  • Truth Maintenance
  • Temporal Reasoning and Complex Event Processing
  • Functional Programming
  • Traits and Declarative Models
Advanced Topics:
  • Decision Tables
  • Rule and Workflow Integration
  • Hybrid Rule-Based Systems
  • Agents and Services
  • Unified Testing

Brief biography of Mark Proctor

Mark Proctor received his B.Eng in Engineer Science and Technology and then his M.Sc. in Business and Information Systems; both from Brunel University, West London. His M.Sc. thesis was in the field of Genetic Algorithms; which is where he discovered his interest for anything AI related.


Mark became involved in the Drools expert system project at an early stage and soon became its project lead. Mark then joined JBoss (later acquired by Red Hat) as an employee when the Drools project was federated into the JBoss middleware stack.
Mark now leads the effort at Red Hat for a unified platform for declarative technologies; with a focus on rules, event processing, workflow, semantics, distributed agents and governance.

Brief biography of Davide Sottara

Davide Sottara received his Ms. Degree in Computer Science(2006) and his Ph.D (2010) in Computer Science, Electronics and Telecommunications from the University of Bologna.
His research and development interests include Artificial Intelligence in general and Decision Support Systems in particular, focusing on hybrid systems combining predictive models and rule-based systems.

Since 2006, he has been working on the development of intelligent DSSs in the environmental and medical field. He is a member of the Drools open source Community, leading a sub-project on the extension of production rule engines to support hybrid and uncertain reasoning, and he's also involved in the RuleML rule language standardization initiative. He is currently working on remote health-care systems enhanced with AI-based predictive, diagnostic and planning features.

Contacts

e-mail: icaart.secretariat@insticc.org

Wednesday, December 07, 2011

Drools&jBPM at ICAART 4th International Conference on Agents and Artificial Intelligence



We have a tentative full day tutorial for Droosl & jBPM in Portugal, Algarve, during Febuary the 6 - 8. I'm just waiting for internal confirmations, but I wanted to start getting prelimary communication out there.

http://www.icaart.org/tutorials.asp

ICAART will cancel the conference if attendance is not high enough, so if you are thinking about attendending, register your interest quickly.


Mark