Sunday, April 06, 2008

Revisiting ANTLR

I believe most of you users that had a look at the Drools source code, in special at the parser, knows that we use ANTLR as our parser generator. We started using ANTLR v2, with our first grammar written by Bob, and then we moved to ANTLR v3 right after it was available, still in beta phase.

We always used a regular single step ANTLR generated parser, specially because of this heritage from ANTLR v2, but DRL is becoming a complex language to parse as we make it more user friendly and add features to it.

So, this weekend I decided to learn about the next step in flexibility provided by ANTLR, and that takes the form of multi-step AST/Tree grammars.

Have in mind that I'm a beginner in the parser/compiler black magic arts, but I must say I'm astonished by the simplicity with which ANTLR allows us to handle complex scenarios. AST and Tree grammars are an amazing tool! Add template processing to that and you get something really unique!

I will not repeat here what is in ANTLR docs and specially in Terence's book, but I can safely tell you: if you need to write a parser, ANTLR is the one stop for you and The Definitive ANTLR Reference book is your must read bible. Thank you twice Terence!

So, soon we shall move our main grammar to a multi-step AST/Tree parser, and hopefully get even better user feedback on DRL rules, ranging from better error reporting to more intelligent context assistance. Stay tuned.

My thanks also to Alexandre Porcelli, that provided real-time help with some questions I had during the weekend.