Wednesday, August 28, 2013

KIE Press #1: SNOMED CT Ontology Quality Assurance

Welcome to the first article of KIE (Knowledge Is Everything) Press. Under the KIE Press title we will be publishing  a set of articles and blog post to share with the community where the Drools, jBPM, Opta Planner and other related projects are being used by our community members.

In this situation we wanted to share a really interesting use case of Drools in the Health Care Industry. The International Health Terminology Standards Development Organization (IHTSDO - www.ihtsdo.org) is in charge of maintain the SNOMED CT which is an ontology composed of 400,000 concepts and uses Description Logic based definitions. SNOMED CT is a standard for representing clinical knowledge in electronic medical records, widely adopted as a national standard in many countries (see http://www.ihtsdo.org/members/).

A new version of SNOMED CT is published every 6 month, in a time oriented, relational database structure. The IHTSDO manages an authoring team that makes any necessary changes in the ontology for each release, adding new concepts, descriptions, relationships, etc. It’s important to notice that every change must be coherent and a set of checks must be done in order to guarantee that the editor is not leaving the ontology in an inconsistent state. That’s where Drools kicks in.

The IHTSDO is using Drools to do real time validation of SNOMED CT. These validations are based on rules which are defined using Guvnor and exposed via the Knowledge Repository. These validations operates on the changes that are being introduced in the authoring process before saving the changes. If the validation process succeed the changes are applied, if not the user is notified.

The IHTSDO has developed the IHTSDO Terminology Workbench (terminology IDE), which is the tool used by the organization members to update and maintain the SNOMED ontology.  This tool has an integration with the Drools based QA System, so it can submit content for a real time check using the rules stored in Guvnor. The IHTSDO Workbench is the first example of an integration with the Drools Knowledge Bases and models represented in the Guvnor server, and potentially this can be extended to any other tool, as SNOMED CT content is represented in a generic way in the knowledge repository, independent of the tooling. The Drools Knowledge Bases are versioned in Maven, so they are readily accessible as dependencies for any tool development environment.

The IHTSDO is also running checkings on the full ontology in batch processes everyday, which guarantees that the changes introduced by one person doesn’t conflict with the changes introduced by another. In this case the batch process operates on the full ontology that is 400,000 concepts and 1.5 million relationships between them.

The IHTSDO Workbench integration

The Workbench is a desktop application (Swing), which is downloaded by each of the organization members with the rights of updating the ontology. The application allows the users to inspect the ontology, make queries to it and propose changes.


Realtime errors detection in the IHTSDO Workbench
The previous figure shows how the tool notifies the user about the validation errors found by the execution of the rules that verify the changes consistency. Notice that the validation in this case failed because there are two Fully Specified Names for the same concept, and only one active FSN is allowed at a given time. The Desktop application runs the rules locally, but they are defined and compiled inside Guvnor and its Knowledge Repository.
BRL Rule in Guvnor
Here we can see a rule that was defined using the Business Rule Editor, inside Guvnor, which is also used to compile the Rules Packages and provide the Desktop clients the rules to perform the validations.

Finally, because the IHTSDO requires to release the ontology every 6 months, a report is generated based on the daily batch validations executed on the full ontology. This report is verified by all the authors and is used to measure the quality of the released ontology.
Report of Batch QA Runs results

This Drools integration has been in production for 2 years now, and it has provided great benefits and flexibility, the check of one concept, including model conversion is around 20 milliseconds for a random check, and it can be as low as 3 milliseconds when is run in batch over a collection iterated in a natural order, reducing data access delays.

One of the main benefits has been the simplified maintenance of the business rules base compared to the previous environment, that was based on hard coded Java if-then conditions. Drools also allows for thinking on how to write each of the rules independently, leaving the complexity of deciding how the rules run to the rules engine. And also has provided easy updates of the knowledge bases without the need of updating actual software libraries or the version of the application itself, when a rules managed in the IHTSDO updates a rule, the author only needs to run a rules "refresh" function in the workbench and the latest updates are immediately effective.

Join KIE Press

If you want to share your own implementations and use cases of Drools, jBPM, Opta Planner or other related projects feel free to contact us, and we can help you to share your experiences with the community. If you don’t have time to write an article about what you are doing, we can also help you out on that. I’ve wrote this article because I personally know one of the maintainers of the tool (Alejandro Lopez Osornio, working for termMed), but if you are interested in sharing your experiences here, we can do a Google Hangout to define how to share your story.

If your use case is confidential, you can a more generic version of what you are doing, the problems that you found or more generic architectural patterns that you have used in your implementations.

Some of the benefits of sharing what you are doing with the KIE community are:
  • Keep everyone else informed about what you are doing, share experiences and improve your implementations based on the community feedback
  • Save research time in your implementations by staying in contact with people that is implementing similar tools
  • Most of the time, similar solutions can be implemented for different industries
  • Serve as inspirations for new implementations
  • Build confidence on the tools provided by the projects and be part of the community members that are actively creating tools using these technologies

Original post: http://salaboy.com/2013/08/28/kie-press-1-snomed-ct/


Wednesday, August 14, 2013

Dublin : JBoss BRMS (Drools) event Wed 28th August

Making an updated post. Thanks to Sabeo and IrishDev I'll be doing two presentations now, one in the morning, and one in the evening. Both events are free, but you must register, and spaces are limited. So please help spread the word.

Both events are sponsored by Sabeo, and the evening event is hosted by Dublin Java User Group (aka DubJUG).

Morning Event
Date : 28 August
Start: 09:30
Location: The Westin Dublin Hotel, College Green, Westmorland Street, Dublin 2
URL: http://tinyurl.com/mfralbu

Evening Event
Date : 28 August
Start: 18:30
Location TCube, Pheonix House, 32-34, Castle St, Dunblin 2
URL: http://irishdev.com/Home/Events/896-Dublin-Java-User-Group-.html


Friday, August 09, 2013

Maven BOM's (Bill of Materials)

Starting from version 6.0.0.CR2, we 'll release BOM (Bill of Materials) files:
  • org.drools:drools-bom
  • org.jbpm:jbpm-bom
  • org.optaplanner:optaplanner-bom
A BOM file makes it easier to use Drools, jBPM or OptaPlanner in your maven project, because you need to specify the <version> only once:


  <dependencyManagement>

    <dependencies>

      <dependency>

        <groupId>org.drools</groupId>

        <artifactId>drools-bom</artifactId>

        <type>pom</type>

        <version>6.0.0.CR2</version>

        <scope>import</scope>

      </dependency>

      ...

    </dependencies>

  </dependencyManagement>

  <dependencies>

    <dependency>

      <groupId>org.kie</groupId>

      <artifactId>kie-api</artifactId>

    </dependency>

    <dependency>

      <groupId>org.drools</groupId>

      <artifactId>drools-compiler</artifactId>

      <scope>runtime</scope>

    </dependency>

    ...

  <dependencies>

Wednesday, August 07, 2013

Dublin : JBoss BRMS event Wed 28th August

I'll be giving a "JBoss BRMS 6.0" talk in Dublin this month, it's free, so please help spread the word ;)


---
JBoss® BRMS Technical Workshop
In this FREE Technical Workshop on Wednesday 28th August at the Westin Dublin Hotel, you will discover how the JBoss Business Rules Management System (BRMS) allows you to automate, manage and improve your critical business processes and policies.

Learn how other organisations are benefiting from maintaining business processes and policies outside of application code; and explore emerging developments and the technology roadmap withMark Proctor, Red Hat's Worldwide Technical Lead for BRMS.

Places are limited, so register early to avoid disappointment.


Agenda.
09.30:Registration - tea and coffee
10.00:Welcome and Introduction
10.15:JBoss BRMS - The Business Case
11.00:JBoss BRMS - Development Update and Roadmap
12.30:Questions and Answers Session
12.45:Lunch and Networking

Tuesday, August 06, 2013

False assumptions for the Vehicle Routing Problem

Many companies are faced with the vehicle routing problem, when they need to:
  • Deliver/pick up items at multiple locations
  • Or execute repairs/maintenance at multiple locations
These companies want to minimize their fuel and time usage, to reduce their costs and ecological footprint.
Sounds easy, right? Just take the shortest route. Unfortunately it’s not that simple... Let’s take a closer look.

Minimize the distance

In Vehicle Routing Problem (VRP), we need to transport items from the warehouse to the customers:


In this case, we have 7 customers across the region and 2 available vehicles stationed at the warehouse.
The shortest route to visit all these customers is this:
This optimal solution requires 210 fuel (which includes each vehicle driving back to the warehouse).

Notice that we only use 1 vehicle. Let’s continue from that assumption.
Assumption: An optimal VRP route uses only 1 vehicle. (false)

Vehicle capacity

In a real-world delivery/pick up scenario, each customer needs a number of items, but a vehicle’s capacity to transport items is limited.

In this case, all 7 customers need 20 items and a vehicle can transport 100 items. So a single vehicle cannot transport the 140 items of all customers.
We need to use 2 vehicles now:

This optimal solution requires 224 fuel, which is - of course - more than the 210 fuel of the previous solution. The yellow truck transports 60 items and the green one 80 items.

Notice that none of the lines cross. Let’s assume that’s always the case.
Assumption: An optimal VRP route has no crossing lines. (false)

Let’s see what happens when some of the customers require more items than other customers.

In this case, 2 customers need 50 items and the other 5 still need 20 items. So the previous solution is infeasible because the yellow truck would need to transport 120 items.
Now the lines do need to cross:

The optimal solution now requires even more fuel: 284. We found a feasible solution with 2 vehicles.

So we don’t seem to need any more vehicles.
Assumption: An optimal, feasible VRP route with n vehicles is still optimal for n+1 vehicles. (false)

Let’s add a 3th vehicle to disprove that:
By adding an extra vehicle, the optimal solution now uses less fuel (274). This is a paradox: buying more vehicles can reduce expenses.

Notice that in both solutions above, no vehicle crosses its own line.
Assumption: An optimal VRP route has no crossing lines of the same color. (false)

Time Windows

In any real-world scenario, time is of the essence. Items need to be delivered on time, within the time window of each customer.

In the case above, a vehicle needs to arrive at the top left customer between 3 and 4 o'clock. Different customers have different time windows. For example, all 4 customers on the right are flexible: they are available between 1 and 6 o’clock. Additionally, each delivery/pick up/repair at a customer takes 1 hour to complete.

In the optimal solution, the yellow truck does cross its own line now:


In the optimal solution, the yellow truck arrives at the most left customer at 1 o’clock. An hour later it leaves for the bottom left customer at which it arrives at 2:20 (because driving takes some 20 minutes). Again an hour later it departs and arrives at its 3th customer at 3:40.

Notice how the time windows pretty much dictate the route, especially on the left side.
Assumption: We can focus on time windows before focusing on capacity (or vica versa). (false)

Let’s see what happens if the time windowed customers also need a number of items:

Given these requirements, we need to focus on the capacity and the time windows in parallel:

The optimal solution now puts the bottom right customer in the yellow truck, because there was no more room in the green truck.

Conclusion

In a real-world vehicle routing problem, many assumptions fail. Finding a good solution is hard: there are no short-cuts. We need to be able to optimize without making assumptions. Yet, we cannot iterate through all possible states in a brute force manner either - even on relatively small problems - because of hardware limitations. So we need good, flexible algorithms - such as the heuristics and metaheuristics implemented in OptaPlanner (Open Source, Java) - to solve bigger cases:

All screenshots are taken from the OptaPlanner vehicle routing example.

Monday, August 05, 2013

What's new in Drools 6.0 - London JBUG 2013

A while back I gave a "What's new in Drools 6.0 talk" in London, I posted the slides before, but the Video is now available too. I have re-uploaded the slides to slideshare too, with the demo's embedded from youtube links.
https://www.youtube.com/watch?v=_fyxXDyNRRU
http://www.slideshare.net/MarkProctor/whats-new-in-drools-6-london-jbug-2013

Note this was done during the beta3 time. The look and feel of the UI has been improved, and the spring and camel xml has also changed.