Creating DSLs in Java, Part 4: Where metaprogramming matters

In this final article in the Creating DSLs in Java series, Venkat Subramaniam lets you see for yourself why JVM-compatible languages such as Scala, Groovy, and JRuby are better suited to creating internal DSLs than the Java language. As you’ll learn, dynamic typing has very little to do with why these languages are ideal for internal DSLs. So what’s the special ingredient in the secret sauce? Read on to find out.

You learned in the last article in this series why the Java language is a better fit for creating external DSLs than internal ones, which are more dependent on the host language syntax. As I demonstrated, Java lacks key characteristics found in newer Java platform languages such as Groovy and JRuby. While both of these languages feature dynamic typing, it isn’t key to creating internal DSLs. In fact, Scala, which is statically typed, is also a popular choice for creating internal DSLs.

So, what do these three languages have in common? Two characteristics you’ve probably heard about lately are essence over ceremony and metaprogramming.

All three of these languages allow programmers to accomplish goals with less ceremony than you’ll find in older languages like Java and C++. Moreover, they all allow you to add and call methods dynamically, which is one of the hallmarks of metaprogramming. Let’s look at some examples that demonstrate the power of essence over ceremony and metaprogramming.

Essence over ceremony

The Java language syntax is heavy on ceremony, which is one of the characteristics that lends itself to the writing of tomorrow’s legacy code. Ceremony in this case means excess verbosity and irrelevant detail — the Olde English of the Java platform. For example, in order to define an ArrayList of Integer in Java, you would type

ArrayList<Integer> list = new ArrayList<Integer>();

Further, even though you repeated the type Integer, the program still might send something other than Integer to the list and fail at runtime.

Some might blame static typing for this particular example of verbosity. In fact, Scala (which is statically typed) gets around the ArrayList example just fine by using type inference. For instance, when you write

var lst = new ArrayList[int]

Scala knows what type lst is.

Similarly, the following Java code

//Java
String str = "hello";
int length = str.length();
System.out.println(length);

can be more concisely written in Scala as

//Scala with essence
val str = "hello"
val length = str length
Console println length

Scala is statically typed, but infers the type of str as String and length as Int. Of course, if you want to add ceremony you can:

//Scala with ceremony
val str : String = "hello"
val length : Int = str.length()
Console.println(length)

Metaprogramming

When building internal DSLs you need to be able to add methods dynamically and also to invoke methods dynamically with ease. This sort of metaprogramming is a common characteristic of newer Java-platform languages like Groovy and JRuby, but not of the Java language.

Consider the following Groovy example:

// Groovy
str = 'hello'
println str.class
println str.encrypt()

If you run the above code, you will get a MissingMethodException because java.lang.String does not have an encrypt method. Fortunately, Groovy makes it easy to add the encrypt() to the String class:

String.metaClass.encrypt = {-> '^%&$*' } 
str = 'hello'
println str.class
println str.encrypt()

My implementation of the encrypt method simply returns the String '^%&$*'. When you run the above code you will get the output

class java.lang.String
^%&$*

Note that I added the encrypt method to the metaclass of String. Because Groovy maintains a metaclass for each Java class you can easily add methods to classes at runtime. (See the Resources section to learn more about metaprogramming in Groovy.)

Method synthesis

In another application of metaprogramming, Groovy also makes it easy to synthesize methods at runtime. Listing 1 is an example of a Groovy Person class that will accept any method that starts with the name play:

Listing 1. Method synthesis in Groovy

class Person
{
  def methodMissing(String name, args)
  {
    if (name.startsWith('play'))
    {
      println "I like to play ${name.split('play')[1]}"
    }
    else
    {
      throw new MissingMethodException(name, Person, args)
    }
  }
}

peter = new Person()

peter.playTennis()
peter.playPiano()
peter.sing()

I call methodMissing the mother of all methods: it says, “if you don’t find the method you’re looking for, come to Mama.” In that method, I check if the method name starts with play; if it does, I simply print that I like the activity mentioned (the word that follows play in the method name). Otherwise, I throw an exception that I don’t recognize the method as valid. The output from the above code is shown below:

I like to play Tennis
I like to play Piano
Caught: groovy.lang.MissingMethodException: No signature of method:  
Person.sing() is applicable for argument types: () values: {} ...

As I’ve mentioned, metaprogramming in Groovy is a huge topic and well worth exploring. Here I’ve covered just enough to get you started with writing internal DSLs in Groovy.

The Groovy-based internal DSL

In the next sections, we’ll incrementally write an internal DSL based on a simple input file. When processed, the final DSL will provide the scores of our three players and tell us who won. If the first iteration of our DSL below looks familiar, it’s because it also served as the basis of the external DSL in Creating DSLs in Java, Part 3.

Listing 2. The initial internal DSL

//version 1 of our DSL input file
players 'James', 'John', 'Jake'
James 12
John 14
Jake 9
result()

What you see in Listing 2 is a DSL made of pure, executable Groovy code. It currently looks more like a data input file than code, however, except for the parentheses after “result” in the last line. We’ll eventually find a way to get rid of the parentheses, as well as the single quotes on the first line of code.

Before we can execute this internal DSL we need to write some code to process it. We will create all the code and the DSL in one file, and eventually split it into multiple files. In the end we’ll have a good working solution.

Processing the DSL

We can readily process the first and last lines of the above DSL. So, let’s comment out the rest of the lines and get started with them, as shown in Listing 3.

Listing 3. Processing the DSL

playersAndScores = [:]

def players(String[] names)
{
  names.each { playersAndScores[it] = 0 } 
}

def result()
{
  def max = -1
  def winner = ''
  playersAndScores.each { name, score ->
    if (score > max)
    {
      max = score
      winner = name
    }
  }

  println "Winner is $winner with a score of $max"
}

players 'James', 'John', 'Jake'
//James 12
//John 14
//Jake 9
result()

We start by defining a HashMap named playersAndScores. In the players method, we populate the map with the names as keys and 0 as value for scores. In the result method, we iterate over the map looking for the player with maximum score and print the result. Given all scores are zero at this point (the scores are commented out), the result simply shows one of the players as winner with a score of 0, as shown below:

Winner is James with a score of 0

Now, we need to get down to the business of providing the scores. James 12 is simply James(12), so we need a method named James, and also methods named John and Jake. However, writing methods with these names is not going to help us if the names change. So, we will make use of the all-encompassing methodMissing method.

Listing 4. Use methodMissing for dynamic methods

playersAndScores = [:]

def players(String[] names)
{
  names.each { playersAndScores[it] = 0 } 
}

def result()
{
  def max = -1
  def winner = ''
  playersAndScores.each { name, score ->
    if (score > max)
    {
      max = score
      winner = name
    }
  }

  println "Winner is $winner with a score of $max"
}

def methodMissing(String name, args)
{
  if (playersAndScores.containsKey(name) && args.length == 1 && args[0]
instanceof Integer)
  {
    playersAndScores[name] = args[0]
  }
  else
  {
    throw new MissingMethodException(name, this.class, args)
  }
}

players 'James', 'John', 'Jake'
James 12
John 14
Jake 9
result()

In the methodMissing method we check to see if the method name is a valid key in the playersAndScores map. We also check whether the number of arguments presented is 1, and whether the argument is of type integer. You also can add checks that the score is greater than zero, if you like. If all the checks are true, we assign the score for the given player name. Otherwise, we generate an exception that the method called is not valid. The output from the above code is here:

Winner is John with a score of 14

The result above is correct for the given input file, but we’re not done improving this DSL yet! First, we want to separate the processing of the DSL into a separate class. Then, we want to further refine the DSL using “essence over ceremony” as our guide. We’ll first get rid of the parentheses, and then the single quotes. Let’s see how we go about these tasks.

Separating the processing code

First, we’ll move the code to process the DSL into a separate class, as shown in Listing 5.

Listing 5. GameDSL

class GameDSL
{
  def playersAndScores = [:]

  def players(String[] names)
  {
    names.each { playersAndScores[it] = 0 }
  }

  def result()
  {
    def max = -1
    def winner = ''
    playersAndScores.each { name, score ->
      if (score > max)
      {
        max = score
        winner = name
      }
    }

    println "Winner is $winner with a score of $max"
  }

  def methodMissing(String name, args)
  {
    if (playersAndScores.containsKey(name) && args.length == 1 &&
args[0] instanceof Integer)
    {
      playersAndScores[name] = args[0]
    }
    else
    {
      throw new MissingMethodException(name, this.class, args)
    }
  }

  def static process(dsl)
  {
    def closure = (Closure) new GroovyShell().evaluate("{->" + dsl +
"}")
    closure.delegate = new GameDSL()
    closure()
  }
}

Most of the code is quite obvious. The only new method is process. It accepts the dsl as a String, forms a closure around it, and executes the closure in the context of an instance of GameDSL (set by the delegate property of closure).

Now, let’s move the DSL into a separate file, game.dsl:

players 'James', 'John', 'Jake'
James 12
John 14
Jake 9
result()

To process this DSL, we need to send the content of game.dsl to the process method of GameDSL. Here’s the code to do that:

GameDSL.process(new File('game.dsl').text)

Processing the DSL in Java

Processing the DSL was easy to do in Groovy, and it’s just as easy to do in Java. Listing 6 shows how you can invoke the Groovy process method using Java code.

Listing 6. Processing the Groovy DSL in Java

import java.util.Scanner;
import java.io.File;

public class UseDSL
{
  public static void main(String[] args) throws Exception
  {
    String dsl = new Scanner(new File("game.dsl")).useDelimiter("
Z").next();
    GameDSL.process(dsl);
  }
}

To run this, type

java -classpath $GROOVYCLASSPATH:. UseDSL

where GROOVYCLASSPATH is an environment variable pointing to groovy-all-1.6-beta-1.jar (or to the appropriate version of Groovy you’re using). Note: on Windows use %GROOVYCLASSPATH% instead of $GROOVYCLASSPATH.

Refine the DSL

We’ve roughed out our DSL and it’s proven functional. Now we focus on refinement — making sure that the DSL is as free of ceremony and as clear in essence as we can make it. First, we’ll remove the parentheses from the last line of the DSL, so that it reads:

players 'James', 'John', 'Jake'
James 12
John 14
Jake 9
result

Go ahead and run the code and see what happens:

Caught: groovy.lang.MissingPropertyException: No such property: result
for class: ...

Uh oh, Groovy is complaining that result is not a valid property name. So, when you strip out the parentheses, it assumes that the given name is a property. In Groovy, the parentheses are almost optional. If a method takes parameters (like the players method), then you can lose the parentheses. However, if the method does not take any parameters, Groovy insists that you provide the parentheses.

You may not like this rule, but you don’t need to fight with it. If Groovy is expecting a property, why not simply give it just that? Instead of writing a method named “result,” write one named “getResult.” Groovy will then apply the Java beans convention and recognize this as a result property. Listing 7 shows the modified version of the code.

Listing 7. Getting around Groovy

class GameDSL
{
  def playersAndScores = [:]

  def players(String[] names)
  {
    names.each { playersAndScores[it] = 0 }
  }

  def getResult()
  {
    def max = -1
    def winner = ''
    playersAndScores.each { name, score ->
      if (score > max)
      {
        max = score
        winner = name
      }
    }

    println "Winner is $winner with a score of $max"
  }

  def methodMissing(String name, args)
  {
    if (playersAndScores.containsKey(name) && args.length == 1 &&  
args[0] instanceof Integer)
    {
      playersAndScores[name] = args[0]
    }
    else
    {
      throw new MissingMethodException(name, this.class, args)
    }
  }

  def static process(dsl)
  {
    def closure = (Closure) new GroovyShell().evaluate("{->" + dsl +  
"}")
    closure.delegate = new GameDSL()
    closure()
  }
}

Is it loud in here?

Sometimes quotes are just so much noise in your code, and that is definitely the case in a DSL. Our next step is to get rid of the remaining static in our code, but cutting out the single quotes around player names. Here’s the result:

players James, John, Jake
James 12
John 14
Jake 9
result

If you run it, however, you will get the following error:

Caught: groovy.lang.MissingPropertyException: No such property: James  
for class: ...

Groovy is complaining that these names without quotes are not valid property names. When we presented 'James' in single quotes (or double quotes), Groovy took it the name as a String parameter. Without the quotes, Groovy thinks we’re referring to a property.

We can get around this by providing a propertyMissing method, just like the methodMissing method we used for dynamic methods. Here’s the refined code:

def propertyMissing(String name) { name }

If you re-run the code with the above method added to the GameDSL class, you will get the following result:

Winner is John with a score of 14

Ah, the sweet sound of success!

Complete code example

Let’s put our DSL together and run the program. Listing 8 shows the GameDSL class.

Let’s put our DSL together and run the program. Listing 8 shows the GameDSL class.

Listing 8. GameDSL.groovy

class GameDSL
{
  def playersAndScores = [:]

  def players(String[] names)
  {
    names.each { playersAndScores[it] = 0 }
  }

  def getResult()
  {
    def max = -1
    def winner = ''
    playersAndScores.each { name, score ->
      if (score > max)
      {
        max = score
        winner = name
      }
    }

    println "Winner is $winner with a score of $max"
  }

  def methodMissing(String name, args)
  {
    if (playersAndScores.containsKey(name) && args.length == 1 &&  
args[0] instanceof Integer)
    {
      playersAndScores[name] = args[0]
    }
    else
    {
      throw new MissingMethodException(name, this.class, args)
    }
  }

  def propertyMissing(String name) { name }

  def static process(dsl)
  {
    def closure = (Closure) new GroovyShell().evaluate("{->" + dsl +  
"}")
    closure.delegate = new GameDSL()
    closure()
  }
}

Here is the game.dsl file:

players James, John, Jake
James 12
John 14
Jake 9
result

If you’d like to process this DSL from within Groovy code, you could do so like this:

GameDSL.process(new File('game.dsl').text)

If you’d like to process the DSL from within your Java code, type this instead (yes, it’s much longer!):

import java.util.Scanner;
import java.io.File;

public class UseDSL
{
  public static void main(String[] args) throws Exception
  {
    String dsl = new Scanner(new File("game.dsl")).useDelimiter(" 
Z").next();
    GameDSL.process(dsl);
  }
}

To use the Java example, compile GameDSL.groovy using groovc and UseDSL.java using javac. if you’d prefer, you could use groovyc’s joint compilation facility.

So long, farewell …

This article concludes the Creating DSLs in Java series, where I’ve introduced you to the fundamentals of domain-specific languages and the various ways to create them on the Java platform. You’ve learned about the characteristics of internal and external DSLs and where to use each type in your everyday Java-based development. You’ve also learned why the Java language isn’t such a good choice for writing internal DSLs (although it works well for external ones) and seen how you can leverage the strengths of newer Java-platform languages such as Groovy, Scala, and JRuby for this purpose.

See the Resources section to learn more about creating DSLs using the languages discussed in this series.

Dr. Venkat Subramaniam has trained and mentored thousands of software developers in the U.S., Canada, Europe, and Asia. He helps his clients succeed with agile development. He’s author of the book .NET Gotchas (O’Reilly), and co-author of the 2007 Jolt productivity award-winning book Practices of an Agile Developer (Pragmatic Bookshelf). Venkat is a frequently invited speaker at international conferences. His latest book is Programming Groovy: Dynamic Productivity for the Java Developer (Pragmatic Bookshelf).