Wednesday, September 24, 2008

Creating a DSL for WS-HumanTask and when not to use a Rule Engine

As previous discussed Drools is building a task system based on oasis spec WS-HumtanTask (WSHT). The spec has a number of operations to control the status of the task for a defined life cycle, where only certain people execute an operation based on given permissions. For a quick recap WSHT supplies the following:

A Task can be in one of the following states:
Created, Ready, Reserved, In Progress, Completed

And supports the following main actions:
Create, Claim, Start, Stop, Release, Suspend, Skip, Resume, Delegate, Forward, Complete, Fail.

WSHT supports the following role types, which it refers to as People Assignments:
Task Initiator, Task Owner, Potential Owners, Business Administrators, Excluded Owners, Recipients, Task Stakeholders.

Already it's obvious that we have rules and flow involved here. To give you an idea of the business logic involved lets look over some of them now - obviously the spec pdf has the complete set of business logic. The task starts off as Created and then moves to Ready where it can transition to Reserved or InProgress depending if the operation is claim or start. Only potential owners or business administrators may claim or start a task. Once in progress only the owner can fail or complete it. Potential owners can only forward a task if it has a status of Ready, once Reserved or In Progress only the owner or business administrator can forward it, further to that the person forwarding must be specified explicitly in the permission and not implicitly as part of some group.

The first thing that came to mind was "great we can model this with rules and flow using Drools" and after some thought I decided that Flow was a bit over kill for this as the flow was quite basic. Also flow diagrams can obscure the business logic, making you need to inspect each node to get a full understanding of the logic; if the flow is simple I think I'd prefer something that made the business logic more obvious. So I decided to defined a DSL that would define both the flow and the rules in a single way, that was also "self documenting" and that made it obvious what could be done where. I decided to use MVEL to model the DSL, as it allows for compact and minimal syntax when creating large graph objects. So lets look at what this DSL would look like.

We know that we first have an operation command (claim, start, stop, etc) and that those operations are only valid when the task is in given status, someone can only claim a task that is in the status Ready. Further to that we have allowed permissions, who can execute that task for the given status, only potential owners or business administrators may claim a task and then we have the resulting new status, which in this case would be Reserved, and the owner of the task must be set to the new user.

I first create the enum Operation which has entries for each possible operations (claim, start, stop, etc). I also create an enum Allowed which lists the different types of people/groups that may have permission for the operation (owner potential owner, business administrator). Then I create an OperationCommand object that will encapsulate the desired business logic in a declarative manner. As a start that OperationCommand would have the following fields:
public class OperationCommand
List status;
List allowed;
Status newStatus;
boolean setNewOwnerToUser;
So using the above we can use MVEL to declare the first entry for our DSL:
[   Operation.Claim
: [ new OperationCommand().{
status = [ Status.Ready ],
allowed = [ Allowed.PotentialOwner, Allowed.BusinessAdministrator ],
setNewOwnerToUser = true,
newStatus = Status.Reserved
} ]
]
Notice MVEL uses the "." suffix to allow in-line "with" field accessors for very compact syntax. What the above says is that we have a Map of possible operations and each key has an array of possible OperationCommands. In the above case there is only one possible status Claim can execute on and that is Ready.

Now we know that Start can be called while the task is in status Ready or Reserved, so lets look at that DSL entry:
[    Operation.Start
: [ new OperationCommand().{
status = [ Status.Ready ],
allowed = [ Allowed.PotentialOwner ],
setNewOwnerToUser = true,
newStatus = Status.InProgress
},
new OperationCommand().{
status = [ Status.Reserved ],
allowed = [ Allowed.Owner ],
newStatus = Status.InProgress
} ]
]

See the difference? Any potential owner can start a Ready task, where as only the owner can start a Reserved task. So what we have here is a self documenting DSL that is easy to understand and easy to administer. In reality WSHT gets a little more complex than there, where operations suspend and resume need to track previous status and as mentioned previously forwarding has an extra permissions check, and delegating will need to execute an additional command to also then Claim the task in the delegates name. The following links are for the full source code for OperationCommand and the operations-dsl.mvel.

Having written the DSL, forgoing Flow, the next question was - how do I write my rules to process the DSL. Given some more thought I realised that the rules where not large nor complex and the domain was known, in that they will not change often. further to this the data used with those rules is minimal - do I really need a rule engine for this? The obvious answer is no, I can hand crank some minimal Java code to process this DSL.

Java retrieved the list OperationCommands based on the requested operation the following java snippet shows how to iterate over the list of commands - note actually it processes the status and the previousStatus fields, but this is just for status:
        for ( OperationCommand command : commands ) {
// first find out if we have a matching status
if ( command.getStatus() != null ) {
for ( Status status : command.getStatus() ) {
if ( taskData.getStatus() == status ) {
statusMatched = true;

// next find out if the user can execute this operation
if ( !isAllowed( command,
task,
user,
targetEntity ) ) {
return new TaskError( "User '" + user + "' does not have permissions to execution operation '" + operation + "' on task id " + task.getId() );

}

commands( command,
task,
user,
targetEntity );
return null;
}
}
}
...

So in the case of "Start" we would have two OperationCommands and it would find the one that matches the current status of the Task. It then checks if the user has the correct permissions, which is encapsulated in the isAllowed(...) method. If the user has permissions it will then execute the commands, a java snippet for the commands method is below:
    private void commands(OperationCommand command,
Task task,
User user,
OrganizationalEntity targetEntity) {
PeopleAssignments people = task.getPeopleAssignments();
TaskData taskData = task.getTaskData();

if ( command.getNewStatus() != null ) {
taskData.setStatus( command.getNewStatus() );
} else if ( command.isSetToPreviousStatus() ) {
taskData.setStatus( taskData.getPreviousStatus() );
}

if ( command.isAddTargetEntityToPotentialOwners() && !people.getPotentialOwners().contains( targetEntity ) ) {
people.getPotentialOwners().add( targetEntity );
}

if ( command.isRemoveUserFromPotentialOwners() ) {
people.getPotentialOwners().remove( user );
}

if ( command.isSetNewOwnerToUser() ) {
taskData.setActualOwner( (User) user );
}

if ( command.isSetNewOwnerToNull() ) {
taskData.setActualOwner( null );
}
...

The full DSL processing code can be found in the TaskServiceSession source code.

So what can we take away from this?
  1. When the problem domain is well defined and known, try and design a self documenting DSL to represent the problem domain.
  2. We don't always have to use XML, you do have a choice. Typically the dsl is going to be authored through a custom GUI, or just edited by the develop by hand. So why not keep it simple with a nice compact syntax, which will help with the "self documenting" benefits of the DSL.
  3. If the Flow is simple, we don't have to use BPM software. Some times a DSL can be less verbose and provide more upfront visual information. Further to that a DSL encapsulates the flow and the rules in a single format.
  4. If their are a small number of non-complex rules that don't change often and don't require dynamic deployment with a small data set that won't benefit from indexing or from optimising of data changes over time, maybe we should just write a few hundred lines of java code with good unit testing.

Thursday, September 18, 2008

Drools Research Network

The number of research projects around Drools is growing, however I've noticed that institutes are generally not aware of each others efforts. For this reason I'm launching the Drools Research Network. This consists of a web page (synced from a wiki page so anyone can edit) that shows all the University stakeholders in Drools and the various research projects. Please do help keep this page up to date and comprehensive. There is also a mailing list that is strictly for institute staff to discuss existing and potential projects, fundings, students and general collaboration to increase and improve Drools related research - if you fit in this category please subscribe.

Drools Research Network home page
The Wiki Page
The Mailing List

Wednesday, September 17, 2008

Improving the Naval engineering process using Drools ( Michael Zimmermann )

Michael Zimmermann, from Rostock University, has emailed us an overview with findings for his Phd thesis on helping to improve the navel engineering process using Drools. The thesis title is "Knowledge based design patterns for detailed steel structural design".

The Scope: Naval Engineering,

In naval engineering vessels consists of thousands of parts (read: way more than an aircraft has today). Focusing on the steel structure, most of these parts are 2-dimensional plates of a certain size, thickness, grade and contour. For most fields of applications detailed regulations
from classification societies etc. exist.

Currently, the design of such objects is done using specialized CAD systems. Here the following issues are present in all cad systems and the design process today:
  • Design time is 6 - 18 months (and not 10 - 15 years as for an air plane)
  • This means concurrent design, i. e. different people are working on parts or features that are closely related (strength, fatigue, cost, functional aspects or just being in the same room) on different levels of granularity (changing the hull 6 weeks before building happens!).
  • No connection between design intent (specification on paper), design conditions (regulations, by the customer, results of calculations) and design solution chosen.
Result:
  • We just have the geometrical data, nothing more in the CAD-model
  • No automatic checks if a certain component is changed; no automatic tests if a chosen design solutions really satisfies the conditions at lowest cost today
  • Therefore, changes (which you can't avoid) are cost intensive and error prone. Also, no one really knows why a certain solutions was chosen
Enter Logic & Drools

The objective of our research is to make the design process context aware. Example: If I design a "door" in a watertight "wall" the cad system should check whether the selected door is a watertight model.

So, using one of the most popular commercial cad systems for naval engineering the approach is to define the standards (currently paper-based) electronically using DROOLS. Also, context-information like watertight, stress level=x ... is added to the model and reused in the design process. For standard design tasks (in a part of the field of detailed steel structural design) we use drools to completely automate the design process, i. e.
  • Find a design problem in the model
  • Select a suitable solution adhering to all known boundary conditions
  • Design the solution
  • And assign a assurance level for the solution (how good is it?)
Lessons Learnt from an Engineering POV
  1. Extracting the knowledge is hard
  2. Formulating it logically sound even harder (even official regulations are vague on a regular basis)
  3. Defining the context a solution is valid for is even more difficult.
  4. Current CAD systems in naval engineering are not really capable to store meta information and to interface with other applications.
Lessons Learnt from a Drools POV
  1. Drools is quite a nice system :-)
  2. With DSL even engineers can use it (once they are trained how to "Think Rules". And that is next to impossible)
  3. What's missing is some solution to easily define classes, class hierarchies and (!) instance data. We use OWL for now. eCore might be usable yet is terrible from a UI usability perspective
  4. Not drools is the problem but getting the data in and out!
    1. The Smooks binding could be a godsend for this
    2. Fact templates sound really promising if you think dynamically generated classes via web services...
What's missing in Drools?
  • An OWL/RDF binding would be really great (we use OWL to define, edit, store our standards. But encountered the clash of open world logic (DL) and closed world logic (CS) more than once.) I know there is quite a large interest for such a solution in the Ontology user base.
  • Better support for constraint programming (what we do here and there) for simple primitive cases (read: selection processes) would help. Drools solver is overkill; drools rules can not handle this if you think optional constraints. The custom operator "== (open world style)" we talked about helps, though.
University of Rostock, http://www.uni-rostock.de
Chair of Naval Architecture, http://www.schiffbauforschung.de
Contact: michael.zimmermann at uni-rostock.de

Drools and WS-HumanTask

I've previously blogged about how Drools is becoming a Business Logic integration Platform that unifies rules, work flow and event processing. An important aspect of work flow is human task management, which to date has been missing in our efforts. Kris Verlaenen, the Drools Flow lead and general evil genius, did a review of the various specs and other non-standard implementations. As it turns out the WS-Humantask (WSHT) spec is pretty decent and comprehensive, the pdf can be found here, so when thinking about implementing this feature for Drools it made sense to base it on WSHT rather than a proprietary implementation such as the one found in jBPM. WSHT has become an oasis standard, which InfoQ covered back in january "BPEL4People and WS-HumanTask Head To OASIS".

Kris has been busy working away on a implementation of WSHT and it's almost at a usable stage, for now he has taken a more practical approach to this to deliver something that we can use for Drools, rather than just aiming for WSHT compliance. Although we do hope to eventually make this fully WSHT compliant, hopefully someone from the community can help us from that side.

The class model, which is made persistable through EJB3, is close to complete and able to represent the whole of WSHT - except for the presentation elements, which I have left off for now, these can be easily added later but we don't have much use for them yet.

For now we have chosen to do ignore the WS aspect and focus on a apache mina based client/server architecture this allows us to create a simpler and lower latency implementation for integration with our runtime and tooling, easily supports p2p and is more easily embeddable as mina is just a small JAR. The last one is important as the WSHT server needs to message events to running clients, who are typically in a wait state.

The spec does not specify anything about iCalendar notifications, so kris has added this anyway. So now when someone claims a task they get two iCalendar emails one for the first start date and one for the last start date. iCalendar VEvents was chosen over the more symantically correct VTodo as there doesn't seem to be much support for the later - neither gmail or zimbra can detect a VTodo sent via an email. Maybe over time we can make this configurable and users can specify whether they want events or todos.

Typically a Task has a start by date and an end by date, WSHT allows for multiple start deadlines and multiple end deadlines. Each deadline can have zero or more escalations that result in a re-assignment or a notification. WSHT doesn't specificy what form the notification takes place, this is one of their extension points. We have hooked up the notification system to integrate with our existing "work items" framework, initially with the email work item. Work items are pre made units of re-usable code, typically with GUI configuration in the flow editor, for executing actions. Later we could include a JMS or WS notification, leveraging any pre-made work items we have made.

A Task can be in one of the following states:
Created, Ready, Reserved, In Progress, Completed

And supports the following main actions:
Create, Claim, Start, Stop, Release, Suspend, Skip, Resume, Delegate, Forward, Complete, Fail.

WSHT supports the following role types, which it refers to as People Assignments:
Task Initiator, Task Owner, Potential Owners, Business Administrators, Excluded Owners, Recipients, Task Stakeholders.

To get an understanding of how the WSHT life cycle works with the various allowed operations the spec pdf provides this state transition diagram which hopefully makes it all clear.

WSHT Lifecycle from spec PDF

The Drools Task code currently lives here, while the WSHT client/server implementation is close to complete the tooling integration will be minimal for 5.0 due to time constriants. We hope to quickly crank that up to make the tooling in eclipse and the Guvnor BRMS feature full. This is a great project for anyone wanting to get involved as it's relatively self contained and thus straight forward and no complex algorithms :) Things to do include full WSHT compliance, improved tooling including various extensions like inbox style views that support task labelling and also "read" status.


For now here is a simple screenshot showing some of the minimal Task tooling integration into Eclipse.

Tuesday, September 16, 2008

Rule Engines and Performance Misconceptions - Followup

My previous posting "Rule Engines and Performance Misconceptions" proved very popular with a lot of page hits and interesting comments, as the comments aren't syndicated or shown on the main page I thought I'd repaste them as a blog entry for more exposure.

Adam Warski said...
The same argument applies in many places. For example: C vs. Java - the same programs written in C (or directly in ASM ;)) can run faster than Java programs.

But the layer of abstraction Java provides makes it possible to write more maintainable code, in a reasonable amount of time etc.

So when somebody asks you again why he should choose Drools instead of a custom Java solution, ask him why he uses Java instead of C - after all, C is faster :)

Edson Tirelli said...
That is the problem with micro benchmarks. They usually don't take into account volumes that is also a very important non-functional requirement. Getting a fast hand coded algorithm for like 10 rules that will not change over time is feasible, but try to do the same for 100 rules, or rules that change frequently, and things will start to look very different.

Adam, BTW, :) your analogy is very informative.

old-skool rule hacker said...
Mark is right that rules can make it easier to deal with evolving complexity. This is true where the declarative aspects of the rule base provide a clear and succinct formalism for capturing constraints in the application domain. Of course, how well they cope and will continue to cope over time depends upon the nature of the problem and the way in which it changes. You will always have to pick and choose your problem if you have already decided that rules are the solution and you may not always get that right. But the same is true for any programming language or toolkit.

What Mark did not mention was the F-word, "flexibility". So, even where the complexity of an application does not grow over time, rules can still be of great use when changes to requirements or even just changes to the data model alter the constraints which govern the application's behaviour. Rewriting a large body of code is rarely a problem-free task. When a rule formalism models the application domain well then rewriting is often a lot quicker and simpler than it would be for, say, rewriting a C or Java program.

It's also often a lot easier for someone who is an application domain expert to check and verify the assumptions coded into a rule base than it is to verify the assumptions coded into a C or Java program. This is critical when an application's remit is subject to change, especially if a quick response is required. In many cases rules can *directly* encode aspects of an application. It is not always possible for an application domain expert to directly verify a rule base. But it's often a lot easier for an expert's understanding to be correctly and *verifiably* translated into declarative rules than into algorithms. That's what declarative programming is all about -- providing a clear and minimal specification of what needs to be done in terms which directly display the application requirements.

Algorithmic solutions, by contrast, tend to cloak invariants which need to be maintained during execution for the program to correctly model the application's requirements. Different choices of algorithms can impose arbitrary and often unnecessary sequential orderings on operations; yet, at the same time this fails to make clear the true order dependencies required by the application.

Algorithmic languages are also mostly tied to a Von-Neumann state model (aka a poke and peek data model). This opens up all sorts of possibilities for data model dependencies to be coded in to programs in obscure ways -- and then accidentally get coded out when the code base is changed. It is easy for the effect of a change in one piece of code only to be felt at a distance in some other piece of code. Scoping models such as APIs and object oriented programming help deal with this problem but do not eliminate it.

Greg Barton said...
There are some instances where efficient algorithms are easier to code up in rules. One example I give is dynamic programming. This is because rule based systems are particularly adept at both 1) complex recursion on subproblems, and 2) storage of subproblems in a generic yet accessible (i.e. change indexed) manner.

Anonymous said...
Good praises for rule engines.
But speaking of performance of rule engines, which use cases do you know where hand crafted java code is a better way than rule engines? Or, which main business use cases do you know where performance of rule engines is not “good enough”?
(I have seen some “old” rule engines that were not good enough, but I can change my mind if I know the limits).

woolfel said...
There's no simple answer to the question of "when a rule engine is better".

The reality is you have to try both and weigh the trade-offs. Obviously, that's prohibitive from a cost perspective, so the best a developer can do is take an educated guess.

For me, if the rules are simple and capture a linear process, using Java is probably going to be a better choice.

On the other hand, if you have complex rules that modify facts, create new facts, derive partial results and produce a final result, using rule engine "could" be a better solution. Really, only way to tell is to do a proof of concept and put it in front of the business user.

Greg Barton said...
Anonymous, I can give you a concrete example of a time when rules were ill advised for performance reasons. It's along the lines of woolfel's thoughts.

I worked on a project for a large American telco which shall remain nameless. They were moving into the long distance business and were using rules (ILOG JRules) to process the several hundred gigs of data they gathered each night. The thing is, by design, they did not want to process the data in a non-monotonic manner and also had few joins between working memory objects. In other words, data was processed (mainly validated) once, then never touched again by this particular system.

Using a rules based system was unnecessary in that case. The system, by initializing the rete network, was effectively preparing for nonmonotonic work, but it was never being done. CPU cycles were being burned in preparation for work that, by design, would never happen: the very definition of waste.

However, I'm pretty sure this was before JRules had a sequential mode, so it might have made the system perform better. But barring that, rules were not the best solution. They scuttled the project, firing upwards of 60 developers, and shelving two years of work. I hear they eventually switched to PL/SQL. The efficiency of the private sector at it's best. :) It wasn't the fault of the rules based approach, though. It was just bad management and a lack of technical due diligence.

nheron said...
Hello,
We use drools in many projects for company in the retail area . When the java class are well-defined (real business classes and not just database tables badly representing the business) and the rete algorithm well understood, there are no performance issues. The main problem is to understand what facts are and how to use them well. Most people when starting programming rules use the facts as a "if" and put most business code in the "then" place. But when using the facts to get the good rule to apply, performance is there.
In one project with complex business rules for a purchase department that was previously using an excell file with macros, the new software using drools was 3 times quickler and really implemented all business rules.

woolfel said...
Here is a concrete example where using a rule engine is "better" in my mind.

In the past, I worked on an order management system and had to support complex compliance rules. The rules had to check the "risk" of the portfolio and determine if a trade would violate any of the rules. The tricky part is this. The authors of the rules are portfolio managers and compliance officers. The other big requirement is deploying new rules without goind through a full development cycle. By that I mean write a bunch of code, compile, unit test, UAT, sign-off and then deploy.

I know many olders compliance systems commercial and home grown go with the code approach. The problem is those systems can't handle real-time pre-trade compliance validation, nor could they add new rules on the fly. Large institutions like fidelity, putnam, wellington and state street normally schedule new releases on a quarterly basis. This means writing compliance rules with code isn't feasible. I know many compliance system use an interpreted approach, but they can only handle post-trade compliance in overnight batch processes.

In the case of compliance, it's difficult for a developer to implement those rules correctly because they're largely arbitrary, and complex. Using a normal development process, it would take a month or more to implement those rules. Using an business rule engine like jrules, jess, clips and drools means the rule engine can handle the complexity and make life easier for the developers. That doesn't mean a BRE can solve all the problems, but it can make it much easier, as well as provide real-time capability that would be difficult to implement in a custom rule engine. For the record, if a developer was an expert on rule engines and pattern matching, they could build a custom engine that out performs jrules, jess, clips or drools, but those individuals are few and hard to find.

Friday, September 12, 2008

Rule Engines and Performance Misconceptions

I'm starting to see a misconception between users on rule engine performance. They seem to struggle on why you can hand craft some java code and get better performance than a rule engine, and then wonder what the value of a rule engine is. For most veterans this is obvious, it's always possible for an experience developer to write faster code if you write a custom algorithm with all the correct data structures and indexes, some times order of magnitudes, as you know exactly which bits to optimise. The same is also true for work flow. At the more basic level a "for" loop in java is always going to outperform a ReteOO network representation of a "for" loop (although ReteOO bytecode generation and flattening will help here).

ReteOO, our enhanced implementation of Rete, is a generic algorithm with generic optimisations further to that it is interpreted to easily support dynamic rules at runtime. Hand crafted algorithms will always outperform generic algorithms. The point of a rule engine is it provides "good enough" performance while giving you the added benefits of declarative programming and the various authoring metaphors, improved maintenance, ability to grow in complexity in a sane way, enterprise management. You may also find that your hand crafted engine might start to scale poorly, compared to a rule engine, after 3 different developers have slapped on their enhancements over the years.

Simply put, you aren't going to write a video codec in a rule engine. But if you are going to write a business or a monitoring problem who's complexity is going to grow over time then the idea of maintaining spaghetti java code over the years might not appeal to you, and the rule engine performance will be "good enough" that the performance side is no longer the main consideration in your solution.

Drools performance at the moment is "good enough" in most cases where users have performance issues the matter can be dealt with by a change in approach. That said there is still a lot we can do to get increased performance, and we will continue to improve over time. For example we can flatten a ReteOO network down into bytecode methods to get native level java performance (for those that don't care about dynamic capabilities), we can add indexing for literals and variables on !=, <, >, <= and >=. Those two alone, when we do them, should provide a robust increase in performance. But we also need to start targeting memory usage, looking at disk paging solutions to assist.

Something else I always tell people is that if you have a large application written in Drools that you can make public we can look over it and see what optimisations that can be done to Drools to improve it's performance. We'd rather have user driven performance enhancements than blindly adding benchmark/academic driven enhancements.

Thursday, September 04, 2008

Drools Boot Camp and Texas Rules Fest goes bigger and better (Oct 22-24)

Due to large number of registrations the Texas Rules Fest has had to move location from the Addison to the Sheraton Dallas:
http://www.rulesfest.org/OctoberRulesFest/Location.html

The organisers have asked me to remind everyone thinking of going to register sooner, rather than later, as they may actually sell out if the rate of registrations continues.

As a reminder entire Drools team will be at the Dallas Rules Fest, this October 22nd to 24th. This is a must attend non-profit event, with $150 registration fee, for hands on rule training on a variety of products and disciplines with some top industry speakers and a fantastic agenda - just stuff, not fluff ;) Gary Riley will be there, author of the rules "bible" Expert Systems, talking about his rule engine the venerable Clips. Charles Forgy, the inventor of Rete and "father" of rule engines, will be there talking about his break through research in parallel executing rule engines. Daniel Selman from Ilog will be there giving tutorials on the ilog engine and tooling.

Come Join the Week Long Drools Boot Camp

The Drools team will actually be arriving the week before, on the 15th, for our team meeting which is open to all and starts official on the the 16th in the Best Western hotel conference room. So if you want to chat rules in general or specifically on your problems, or just get involved in some hard core programming, please do come along. We will be staying at the Best Western hotel which has rooms from just $69USD per night :)

So far we have Franklin American and German Aerospace Center joining us for the team meeting and hard core Drools coding, still room for more :)

Tuesday, September 02, 2008

Open source for underwriting

Suncorp's CIO, (Suncorp are a large Australian insurance and finance company), has committed to moving to open source platforms. This is being publicised as a cost cutting measure (which no doubt it is), but I (obviously) think that its just a sensible decision.

Lets make the assumption that all software, once its deployed, becomes legacy. Isn't it nicer to have that legacy not beholden to the whims of whatever company owns the source code? For the software to have a chance to evolve (open source development models encourage a more evolutionary approach) and also to be supported (regardless of any commercial organisations providing support, long term open source has a fighting chance that someone else can support it, regardless of what happens).

Its interesting that this sort of shift to open source is now happening at the CIO level (I guess it is inevitable over time - stuff that was bleeding edge and "risky" often becomes mainstream). Obviously people are seeing that at least for horizontal software, open source is the best bet to avoid black-box legacy applications which at some point no one will know how to maintain.

An interesting quote mentioning drools:

"Even for our underwriting service, we are going to be using Drools." ... "It is simpler, it is cheaper and it is actually easier to manage if we standardise it on one platform."