Sunday, September 23, 2007

Drools Solver

Geoffrey De Smet has been busy working on the Drools Solver module, which will hopefully be part of the next major Drools release. Drools solver aims to efficiently solve search based problems finding a valid solution from large search areas. It currently provides implementations for Tabu, simulated annealing and Local search. I personally hope to expand the system for Genetic Algorithms in the future, I'll have a good look at jgap too see if we can provide true value over existing systems.

Geoffrey has started to flesh out the manual and is looking for feedback, it focuses on explaining the engine via solving the N-Queens problems.

Drools Solver Manual

Drools Solver provides a framework for searching and uses Drools to provide the scoring, moves themselves are still done in Java code. I know that Geoffrey long term is hoping to have the move instructions also as part of the DRL.

22 comments:

  1. You're jumping the gun here mate :) I haven't moved solver into trunk yet so the build instructions at the beginning of the manual won't work...
    at least not till tomorrow when I 'll move it :)

    ReplyDelete
  2. heh, good job I only asked people to review the manual :)

    ReplyDelete
  3. ok, it's all been moved and the examples should work fine now :)

    The beginning of the manual describes how to try out one of the examples in no time.

    ReplyDelete
  4. I've made a blog entry of our abstract too btw.

    ReplyDelete
  5. solver sounds great :-) when I finished testing all the normal drools stuff, I'll test solver

    ReplyDelete
  6. Very interesting stuff :-)
    However, the maven build seems to be broken again.

    After entering: "mvn -Psolver clean install", I'm getting "Missing: org.drools:drools-analytics:jar:4.0.2.SNAPSHOT"

    ReplyDelete
  7. The rule analysis stuff was just moved top level, which broke the build temporarily, this is now fixed.

    ReplyDelete
  8. The build appears to be getting further now and the errors are different now.
    I checked out revision 15412 and ran "mvn -Psolver clean install". This time I am getting a "Build Failure" with 104 tests run and 77 Errors.

    If I then just go ahead and try to run any of the tests anyway, I get a missing org.drools.solver:drools-solver-core:jar:4.0.2.SNAPSHOT

    Would it at all be possible to compile a version of Drools Solver against Drools 4.0.1? This way, it would be much easier for us to experiment with the drools-solver component (without the moving target of drools 4.0.2 main), and share feedback with other Drools-Solver users.

    Thanks in advance.

    P.S. the Tests in error:
    testListPackages(org.drools.repository.PackageItemTest)
    testRulePackageItem(org.drools.repository.PackageItemTest)
    testPackageCopy(org.drools.repository.PackageItemTest)
    testPackageSnapshot(org.drools.repository.PackageItemTest)
    testLoadRulePackageItem(org.drools.repository.PackageItemTest)
    testPackageRuleVersionExtraction(org.drools.repository.PackageItemTest)
    testIgnoreState(org.drools.repository.PackageItemTest)
    testDuplicatePackageName(org.drools.repository.PackageItemTest)
    testPackageInstanceWrongNodeType(org.drools.repository.PackageItemTest)
    testLoadRulePackageItemByUUID(org.drools.repository.PackageItemTest)
    testAddRuleRuleItem(org.drools.repository.PackageItemTest)
    testGetRules(org.drools.repository.PackageItemTest)
    testToString(org.drools.repository.PackageItemTest)
    testRemoveRule(org.drools.repository.PackageItemTest)
    testSearchByFormat(org.drools.repository.PackageItemTest)
    testListArchivedAssets(org.drools.repository.PackageItemTest)
    testMiscProperties(org.drools.repository.PackageItemTest)
    testGetFormat(org.drools.repository.PackageItemTest)
    testPackageCheckinConfig(org.drools.repository.PackageItemTest)
    testTagItem(org.drools.repository.CategoryItemTest)
    testCreateCateories(org.drools.repository.CategoryItemTest)
    testGetChildTags(org.drools.repository.CategoryItemTest)
    testGetChildTag(org.drools.repository.CategoryItemTest)
    testGetFullPath(org.drools.repository.CategoryItemTest)
    testRemoveCategoryUneeded(org.drools.repository.CategoryItemTest)
    testFindArchivedAssets(org.drools.repository.ArchiveItemTest)
    testArchiveBooleanFlag(org.drools.repository.ArchiveItemTest)
    testLoadArchivedAsset(org.drools.repository.ArchiveItemTest)
    testFindArchivedAssetsByCategory(org.drools.repository.ArchiveItemTest)
    testLoadPackages(org.drools.repository.ArchiveItemTest)
    testFindRulesByNameArchived(org.drools.repository.ArchiveItemTest)
    testToString(org.drools.repository.AssetItemTest)
    testGetFormat(org.drools.repository.AssetItemTest)
    testAssetItemCreation(org.drools.repository.AssetItemTest)
    testGetPackageItem(org.drools.repository.AssetItemTest)
    testGetContent(org.drools.repository.AssetItemTest)
    testUpdateContent(org.drools.repository.AssetItemTest)
    testCategories(org.drools.repository.AssetItemTest)
    testUpdateCategories(org.drools.repository.AssetItemTest)
    testFindRulesByCategory(org.drools.repository.AssetItemTest)
    testRemoveTag(org.drools.repository.AssetItemTest)
    testGetTags(org.drools.repository.AssetItemTest)
    testSetStateString(org.drools.repository.AssetItemTest)
    testStatusStuff(org.drools.repository.AssetItemTest)
    testGetLastModifiedOnCheckin(org.drools.repository.AssetItemTest)
    testGetDateEffective(org.drools.repository.AssetItemTest)
    testGetDateExpired(org.drools.repository.AssetItemTest)
    testSaveAndCheckinDescriptionAndTitle(org.drools.repository.AssetItemTest)
    testGetPrecedingVersionAndRestore(org.drools.repository.AssetItemTest)
    testGetSucceedingVersion(org.drools.repository.AssetItemTest)
    testGetSuccessorVersionsIterator(org.drools.repository.AssetItemTest)
    testGetPredecessorVersionsIterator(org.drools.repository.AssetItemTest)
    testHistoryIterator(org.drools.repository.AssetItemTest)
    testGetTitle(org.drools.repository.AssetItemTest)
    testDublinCoreProperties(org.drools.repository.AssetItemTest)
    testAnonymousProperties(org.drools.repository.AssetItemTest)
    testBinaryAsset(org.drools.repository.AssetItemTest)
    testAddDirectories(org.drools.scm.jcr.JcrActionFactoryTest)
    testAddFiles(org.drools.scm.jcr.JcrActionFactoryTest)
    testUpdateFiles(org.drools.scm.jcr.JcrActionFactoryTest)
    testDefaultPackage(org.drools.repository.RulesRepositoryTest)
    testAddVersionARule(org.drools.repository.RulesRepositoryTest)
    testFindRulesByName(org.drools.repository.RulesRepositoryTest)
    testLoadRuleByUUIDWithConcurrentSessions(org.drools.repository.RulesRepositoryTest)
    testAddRuleCalendarWithDates(org.drools.repository.RulesRepositoryTest)
    testGetState(org.drools.repository.RulesRepositoryTest)
    testGetTag(org.drools.repository.RulesRepositoryTest)
    testListPackages(org.drools.repository.RulesRepositoryTest)
    testCategoriesAndSnapshots(org.drools.repository.RulesRepositoryTest)
    testMoveRulePackage(org.drools.repository.RulesRepositoryTest)
    testCopyAsset(org.drools.repository.RulesRepositoryTest)
    testRenameAsset(org.drools.repository.RulesRepositoryTest)
    testRenamePackage(org.drools.repository.RulesRepositoryTest)
    testCopyPackage(org.drools.repository.RulesRepositoryTest)
    testListStates(org.drools.repository.RulesRepositoryTest)
    testImportExport(org.drools.repository.RulesRepositoryTest)
    testExportZippedRepository(org.drools.repository.RulesRepositoryTest)

    ReplyDelete
  9. For the reader's information: JGAP is a Genetic Algorithms and Genetic Programming Package.
    More information about JGAP on http://jgaf.sf.net

    ReplyDelete
  10. Martin,

    try adding -Dmaven.test.skip:

    mvn clean install -Psolver -Dmaven.test.skip

    it will ignore the testcases.
    I 'll update the solver manual accordingly.
    Thanks for the feed-back.

    ReplyDelete
  11. After just skipping the tests, the drools-solver-examples work fine for SVN version 15447. I used Geoffry's command line options (mvn clean install -Psolver -Dmaven.test.skip) and threw in MAVEN_OPTS=-Xmx1024m just foor good measure. According to a readme.txt somewhere, MAVEN_OPTS=-Xmx521m is neccesary to build successfully on windows. I'm using Linux so I'didn't know the ideal, but this works for me.
    Excellent work :) Especially with the NQueens the intent and solution are also very clear. And can be demonstrated to less technical people as well.

    ReplyDelete
  12. I must admit I am concerned that you are having issues with compiling drools, could you give us any insight into the problems? We have hudson running constantly, and it's been blue now for a while. You can check hudson here:
    https://hudson.jboss.org/hudson/job/drools/

    Btw hudson does not yet set the profile to compile solver, but it does everything else.

    ReplyDelete
  13. ... makes me wonder, given the manual says most of the time is spent evaluating relatively static situations, how difficult it would be to run this on multiple cores, or even multiple machines concurrently.

    ReplyDelete
  14. I don't know the exact details, but Geoffrey has said he doesn't believe that it would be that hard at all.

    ReplyDelete
  15. Mark,just checked out trunk revision 15448
    I then did a "cd drools" and a "mvn -Psolver clean install" as per Geoffry's initial instructions. I have also set MAVEN_OPTS=-Xmx1024m according to a readme.txt which encouraged me to increase that memory size. Which tests failed differed with different builds. In this particular case "testThreadSafetyEclipse(org.drools.integrationtests.PackageBuilderThreadSafetyTest)"
    was the one I got. I am using maven 2.0.7, Java 1.6.0_02 on a single core XP3000+ Ubuntu Feisty Linux machine with 2GB of RAM. Build time was approx. 2 minutes.

    ReplyDelete
  16. ok yes that one particularly test did break, was broken on hudson for a bit too. Think you just got unlucky. If that was your only failing test, you should be fine now.

    ReplyDelete
  17. testThreadSafetyEclipse(org.drools.integrationtests
    .PackageBuilderThreadSafetyTest)

    Was the whole error description (the second part was eaten by the blgging filter

    ReplyDelete
  18. Mark, if I do "mvn -Psolver clean install", I still get the "Tests run: 104, Failures: 0, Errors: 77, Skipped: 0" message. If I skip the tests by using "mvn clean install -Psolver -Dmaven.test.skip", the Solver examples still seem to work fine.

    ReplyDelete
  19. All I can say is hudson works, as you can see from the url I gave you. You don't just have test failing there, but lots. If you don't feedback to use what the problems are, they will not get fixed for this coming release - because as far as we can tell now, everything is ok.

    I suggest you open a jira with details of what's breaking and how.

    ReplyDelete
  20. Multi-threading the local search solver isn't implemented yet, but in theory it shouldn't be too hard:
    I 've got a few other priorities first, but it's on my todo list:
    http://jira.jboss.com/jira/browse/JBRULES-681

    I am expecting a more interesting scalability boost from the selector refactor.
    http://jira.jboss.com/jira/browse/JBRULES-1232


    As for the memory: I haven't increased maven2's memory yet, and I can run build without running the tests.

    As for the test failures:
    Open drools-*/target/surefire-reports/...
    and copy paste one of those report txt's in the jira :)

    ReplyDelete
  21. Geoffry, for those of us who want to experiment with drools-solver in combination with the stable 4.0.2.GA drools release: what do you recommend? Drools-solver is not part of the 4.0.2 binary release. However, both svn checkout http://anonsvn.labs.jboss.com/labs/jbossrules
    /tags/4.0.2.15666.GA
    drools-4.0.2.GA
    as well as the trunk version result in working examples. Have you diverged from the 4.0.2 path at all just yet? Or when are you planning to?

    I get the impression that the combination of solutions generated by drools-solver and a stable drools release would be usable if the stable drools component is the part that does (additional?) sanity checks

    ReplyDelete
  22. drools-solver-core should work with any 4.0+ release, I am not even using multipattern accumulate yet due to performance reasons (which would require drools-core/compiler 4.0.2+).
    But I am only testing with the trunk on my side. If you notice any bugs, feel free to make me a jira.

    If you're using maven2, you can easily tie it to a stable release, by overwriting declaring a direct dependency on drools-core AND drools-compiler with a stable version, which you'll probably already have done anyway. It will ignore the SNAPSHOT drools-core/compiler version received by the transitive dependencies of drools-solver.

    If you're not using maven2: use the drools release and add drools-solver-core-???.jar

    ReplyDelete