Benchmarking drools-solver configurations

One of the little known gems in Drools Solver is the Benchmarker utility. Until now, it wasn't documented in the manual and few people knew about it.

The Benchmarker allows you to play out different solver configurations against each other, so you can determine the best one for your problem domain. It's pretty easy to use:

XmlSolverBenchmarker benchmarker = new XmlSolverBenchmarker();
benchmarker.configure(".../nqueensSolverBenchmarkConfig.xml");
benchmarker.benchmark();
benchmarker.writeResults(resultFile);

This benchmark configuration will run 3 different solvers on 2 datasets, so it will do 6 solver runs:

<?xml version="1.0" encoding="UTF-8"?>
<solverBenchmarkSuite>
    <solvedSolutionFilesDirectory>local/data/nqueens/solved</solvedSolutionFilesDirectory>

    <inheritedUnsolvedSolutionFile>data/nqueens/unsolved/unsolvedNQueens32.xml</inheritedUnsolvedSolutionFile>
    <inheritedUnsolvedSolutionFile>data/nqueens/unsolved/unsolvedNQueens64.xml</inheritedUnsolvedSolutionFile>
    <inheritedLocalSearchSolver>
        <scoreDrl>/org/drools/solver/examples/nqueens/solver/nQueensScoreRules.drl</scoreDrl>
        <scoreDefinition>
            <scoreDefinitionType>SIMPLE</scoreDefinitionType>
        </scoreDefinition>
        <finish>
            <maximumSecondsSpend>20</maximumSecondsSpend>
        </finish>
        <selector>
            <moveFactoryClass>org.drools.solver.examples.nqueens.solver.move.factory.NQueensMoveFactory</moveFactoryClass>
        </selector>
        <forager>
            <foragerType>MAX_SCORE_OF_ALL</foragerType>
        </forager>
    </inheritedLocalSearchSolver>

    <solverBenchmark>
        <localSearchSolver>
            <accepter>
                <completeSolutionTabuSize>1000</completeSolutionTabuSize>
            </accepter>
        </localSearchSolver>
    </solverBenchmark>
    <solverBenchmark>
        <localSearchSolver>
            <accepter>
                <completeMoveTabuSize>7</completeMoveTabuSize>
            </accepter>
        </localSearchSolver>
    </solverBenchmark>
    <solverBenchmark>
        <localSearchSolver>
            <accepter>
                <completeSolutionTabuSize>1000</completeSolutionTabuSize>
                <completeMoveTabuSize>7</completeMoveTabuSize>
            </accepter>
        </localSearchSolver>
    </solverBenchmark>
</solverBenchmarkSuite>

Afterwards, it will sort the solvers and write their results to the resultFile.

Benchmarking drools-solver configurations