Wednesday, March 04, 2009

A new testing tool

This is a new experimental library looking for early adopters and feedback before we include it in the main part of the project. Previously there was the Fit For Rules library which worked well, but for whatever reason fit and documents haven't proven super popular (also, it is GPL licenced).

So, based on some work from James Kerridge (Qantas) I have worked on a replacement that uses spreadsheets to set up test data:


You can run it and get pretty printed output like the following:

(and much more of course).

This is just a first cut, am looking for feedback and suggestions - the basic idea is that that is another way to capture test data (columns == scenarios).

The project is currently on github for the moment. You can read a bit more about it in the README and download it.

Most of the documentation is in the test/sample spreadsheet.

As a small curiosity, this is written entirely in scala (but when using the library people wouldn't notice as its just a java api).

Would love to hear about peoples experiences with it (michael dot neale at gmail dot com).

2 comments:

  1. Hi,

    we're using Drools to perform mortgage scoring. For testing, we developed a somewhat similar, but already more sophisticated testing framework. Currently, our test set comprises of roughly 800 test scenarios/test cases.

    Similarities:
    * also uses spreadsheet tables to define test cases.
    * each column resembles a test case
    * leftmost column used to name elements of input facts or define expected conditions (see below)

    Differences:
    * concept of "expected values" not expressed as mere values, but as drools conditions which are expected to hold after rules all under test finished execution. These conditions are automagically converted into a drools query which is run against the working memory after rule execution.

    Enhancements:
    * conditions support simple text replacement, like a macro preprocessor. Wildcards in drools condition can be replaced by test case-specific text
    * allows skipping of certain condition checks on a per-test-case basis
    * event logging of the drools working memory embedded into fit output file for failed checks. visibility of log can be toggled using html-embedded javascript. allows for simple copy/paste into "audit view" window in eclipse. This eases debugging rules a lot!
    * possibility to define a subset of rules contained in a *.drl file to be tested

    Open issues:
    * Syntax of initial facts specification is probably a bit cumbersome. We're using a self-brewed language to do this. Already thought of replacing this by MVEL, but not done yet. For us, the current solution works ok.
    * Up to now, requires explicit import of all used Java classes. This is rather a limitation of drools - we did not manage to get package imports in *.drl files to work. Could probably be solved.

    I see some chances that this code could be donated to the community, but cannot assure this until I checked this with my employer. If you are interested, let me know.

    ReplyDelete
  2. @Ansgar
    Would love to take a look at that stuff, that all sounds like very reasonable features. I will put that comment on a JIRA for future enhancement (either by new code or donation), but all great ideas and very very useful, thanks for that !

    ReplyDelete