Friday, September 25, 2009

Drools Flow: Variable Persistence Strategies

As community contributors, we would like to introduce one of the new features that will be part of the upcoming Drools Flow 5.1 release: pluggable variable persistance. This new feature allows you to define how to persist your process data (aka the process variables), in a relational manner. The main idea of this feature is to allow you to customize data persistence, especially if the data your process is using is for example already stored in an external system, for example, when using JPA entities, external documents or other relational information. This way we can avoid duplicating this information inside the binary snapshot that Drools Flow automatically generates when one of our processes reaches a wait state (storing the runtime state of your processes to allow safe recovery if necessary). The approach is shown in the following figure:



Variable instance data is not stored as part of the runtime state of a process instance, but stored separately as VariableInstanceInfo objects. Different strategies can be used to generate a VariableInstanceInfo that will maintain different information to be able to store and retrieve the external variable information.

How can we get this working in our project? First of all, we need to configure our custom variable persisters. These persisters will be in charge of knowing all the environmental details to persist and restore each external variable type. Drools Flow provides a few out-of-the-box like for example, one for JPA entities and one for serializable variables. We can do that with the following lines:

VariablePersistenceStrategyFactory.getVariablePersistenceStrategy()
.setPersister("javax.persistence.Entity",
"org.drools.persistence.processinstance.persisters.JPAVariablePersister");
VariablePersistenceStrategyFactory.getVariablePersistenceStrategy()
.setPersister("java.io.Serializable",
"org.drools.persistence.processinstance.persisters.SerializableVariablePersister");

With these two lines we are configuring two different variable persisters, one for JPA entities and another for any object that implements the java.io.Serializable interface. This new feature will introspect each variable and decide which persister to use to store and retrieve each variable. Now we can start our processes with different variables and they will be handled using the configured persisters.

Map parameters = new HashMap();
// just a String
parameters.put("x", "SomeString");
// MyEntity is a JPA mapped entity
parameters.put("y", new MyEntity("This is a test Entity"));
// MyVariableSerializable is just a class that implement the java.io.Serializable
parameters.put("z", new MyVariableSerializable("This is a test SerializableObject"));
ProcessInstance processInstance =
ksession.startProcess("org.plugtree.labs.droolsflow", parameters);

As you can see, three variables are used to start the process, each of these variables is of a different type, so, based on the configured persisters, each of them will be treated differently. Take a look at the following snapshot that shows us the status of the database table called VariableInstanceInfo that contains our wrapped variables:



We can see the JPAPersistedVariable "y" that contains the reference to the MyEntity instance, which stores the fully qualified name of the MyEntity class and the ID that can be used to retrieve it from the external database.



The important thing to know here is that we can create custom persisters that will know how to store a particular type of variable. All the persisters need to implement the interface called VariablePersister. This interface will force us to implement the following two methods:

public interface VariablePersister {

VariableInstanceInfo persistExternalVariable(
String name,
Object o,
VariableInstanceInfo oldValue,
Environment env);

Object getExternalPersistedVariable(
VariableInstanceInfo variableInstanceInfo,
Environment env);

}

Once we create our custom persister that implements this interface, we are ready to register it, binding it with the variable type that it is ready to handle it:

VariablePersistenceStrategyFactory.getVariablePersistenceStrategy()
.setPersister("org.plugtree.labs.CustomType",
"org.drools.persistence.processinstance.persisters.MyCustomTypeVariablePersister");

Now all our variables inserted inside our processes that are instances of the class CustomType will be persisted using the MyCustomTypeVariablePersister.

There is only one thing left, we need to create the structure that will contain the specific data for our CustomType variable. This is a very easy step, we just need to create a new VariableInstanceInfo subclass to store only the specific information of our CustomType variable. This will be a new Entity that needs to be mapped into our persistence.xml file.

Here I have included an example about how this works that includes configurations for H2 and MySQL to test the behavior and see how the process persists and handles your process information--I encourage you to debug the test--and take a look at the database to see how the variables are persisted. I've also included a simple variable persister to show you a very basic example about how you can wrap a String variable in a relational way to see its content in a VARCHAR field.

Remember to take a look at the persistence.xml file and confirm that your new variable instance info type is mapped as a JPA entity:

<class>org.drools.persistence.processinstance.variabletypes.JPAPersistedVariable</class>
<class>org.drools.persistence.processinstance.variabletypes.SerializablePersistedVariable</class>
<class>org.plugtree.labs.variablepersistence.StringPersistedVariable</class>

Please let us know if this has been useful to you and in what way. As always, we welcome you questions, criticism, and (hopefully) compliments.

Project File: DroolsFlowVariablePersistenceStrategies.zip


5 comments:

  1. Hello salaboy, i have noticed your commit to drools-persistence-jpa related to this is something what does not allow run drools flow. I propose to create some integration tests. Such integration tests should help to find out problems with production metadata like persistence.xml. cheers

    ReplyDelete
  2. a nice example is given by u salaboy but it has some problem b'coz it does no run with me.......it gives some
    exception saying :
    Unable to load dialect 'org.drools.rule.builder.dialect.java.JavaDialectConfiguration

    please help if u can

    ReplyDelete
  3. Ask those Qs on the USER mailing list:
    http://www.jboss.org/drools/lists.html

    ReplyDelete
  4. Dear all,
    sorry for the previous post in the wrong place, now actually i got the example correct and now i understood the code ( as i m a fresher at drools as well as JPA)
    but still the code shows that how variables and state of process are persisted but i want to know in case a sever fails then how the recovery will take place....

    i had an idea that the restart will trigger and load all sessions present in the DB and so i tried it
    and restarted the session but i could not load the workitems and the variables thus got an null pointer exception

    any help is welcome
    thanks in advance.....

    ReplyDelete
  5. and yes.....sorry but forgot to mention that the sample code uses the existing variable in code for processInstance and workItem to get their ID(xyz.getId()) which wouldnt work in case of server crash.....thanks

    ReplyDelete