Groovy recipes for sweeter Ant builds

If you want to incorporate Groovy into an existing Java application development process, then you’ll need to convince your company’s leadership that it’s a good idea. One path to adoption is to demonstrate the benefits of using Groovy for an existing process such as nightly builds. Once your team has experienced Groovy’s efficiency, they’ll likely be more willing to integrate it into new application code. In this Java tip, Groovy developer Chris Bedford presents working recipes for integrating Groovy into your Ant-based Java build process.

Groovy is increasingly popular among Java developers seeking to build more functionality with less code. If you are among the recently hooked, you would probably like to use your new programming language for paid projects. But there are hurdles to adopting any new technology, and most decision makers will not want to take on the overhead unless they’re sure it will pay off. Leveraging Groovy for your release engineering processes is one way to demonstrate its strengths without pushing too hard on the way things have always been done. Once your team has experienced the efficiencies of Groovy, they’re more likely to support widespread adoption. In fact, they probably won’t want to go back to building without it!

This article presents a series of practical build recipes that you can use to integrate Groovy into your release engineering processes. Altogether, they map out an incremental approach to Groovy adoption. First I’ll introduce a Groovy build.xml script that invokes methods in Java classes, which is a must for writing tests in Groovy. After that we’ll look at a few scripts for handling more complex build scenarios, including direct embeds of Groovy code and callouts to external Groovy scripts. Finally, I’ll show you how to specify inter-task dependencies with Gant, which is a Groovy-based variant of Ant.

Setting up the Groovy integration project: Ant and Java

Ant is the “old reliable” of build technologies, and it’s useful to know even if your shop uses Maven or something more exotic like Gradle. You can always drop into Ant to procedurally specify tasks that are tricky to do within your framework of choice. (For instance, Maven provides the maven-antrun-plugin for this purpose.)

The sample code for this article includes a series of test cases. In order to work with the code, you will need to have Java 1.5 or greater installed in your development environment. You’ll also need a non-ancient version of Ant (at least version 1.6 or higher). Make sure that both your JAVA_HOME and ANT_HOME are set, and that ant and javac are in your classpath. These setup steps should be familiar to anyone who regularly uses Ant. (See the Resources section for an Ant tutorial with further setup instructions.)

Download and run the project code

To start out, download the article source code, unzip it, then cd to the basic.groovy.test.project directory. You’ll see the following directory structure:

The test class in Figure 1 invokes methods on the class under test, Sample.java, which is defined in the project’s src/java subdirectory. Now, invoke the build script for the project (see Listing 3) via the command, ant clean test.

You should see output similar to Listing 1, which indicates that the test has run but is initially failing:

Listing 1. Test failure

[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.247 sec
    [junit] Test com.lackey.SampleTest FAILED
        ...
    [echo] JUnit Report: output.dir/test/reports/index.<WBR>html

Ant tasks for Groovy compilation and integration with Java

We’ll deal with the test failure later — it’s a setup for another build recipe. For now, let’s explore how our build.xml script gets Groovy to play nicely with Java. The abbreviated code snippet in Listing 2 shows how build.xml‘s test target runs Ant’s junit task, followed by the junitreport task. The latter task generates the HTML report at the path shown by the last output line.

Listing 2. build.xml runs a junit task

<target name="test" depends="compile-test">
    ...
    <junit fork="yes" printsummary="on" haltonfailure="false">
        ...
        <batchtest ...=""><fileset dir="${test.classes.dir}">
                  <include name="**/*Test*.class">
        ...
        <classpath refid="test.run.classpath">

    </junit>


    <junitreport todir="${test.reports.dir}">
      <fileset dir="${test.reports.dir}">
         ...
      </fileset>
    </junitreport>

The batchtest subtag of the junit task is configured to look for all test classes matching the pattern **/*Test*.class. The compiled result of the one test we have defined, SampleTest.class, is picked up because it matches this pattern. The junit task is then able to resolve SampleTest‘s references to our Java class by virtue of the reference to test.run.classpath. The annotated listing of Basic Groovy Test Project’s build.xml, shown in Listing 3, provides the details of how test.run.classpath is constructed, in Line 51.

Listing 3. Partial Build.xml illustrating how Groovy test code accesses methods in Java classes (click to view complete source)

1:<project default="test" basedir=".">
2:    <property name="root.dir" location="${basedir}/../.."/>
3:    <property name="proj.dir" value="${basedir}"/>
4:
5:  <property name="src.dir"                  value="${proj.dir}/src"/>
6:  <property name="javasrc.dir"              value="${src.dir}/java"/>
7:
8:  <property name="output.dir"               value="output.dir"/>
9:  <property name="build.dir"                value="${output.dir}/build"/>
10:  <property name="classes.dir"              value="${build.dir}/classes"/>
11:  <property name="lib.dir"  value="lib"/>
12:  <property name="test.dir"                 value="${proj.dir}/test"/>
13:  <property name="test.src.dir"             value="${test.dir}/src"/>
14:  <property name="test.javasrc.dir"         value="${test.src.dir}/java"/>
15:  <property name="test.build.dir"           value="${output.dir}/test/<WBR>build"/>
16:  <property name="test.classes.dir"         value="${test.build.dir}/<WBR>classes"/>
17:  <property name="test.reports.dir"         value="${output.dir}/test/<WBR>reports"/>

...

How Ant generates JVM byte code from Groovy source

When you first downloaded and ran Basic Groovy Test Project, you may have noted that there were no *.class (that is, Java byte code) files available in the expanded directory hierarchy. But they do show up in Listing 3.

They got there because the project’s build script defines a chain of dependencies via the depend attribute on the various targets in build.xml (see Line 84 of Listing 3 for an example). When the tasks along this chain of dependencies execute, our Groovy and Java source files get compiled into .class files, which are then made available on the classpath when the tests run.

Working backwards in the dependency chain from the test target, we find a dependency to the compile target (Line 67), which first runs the javac task to invoke the Java compiler on all source files under ${javasrc.dir} (Line 70). javac then stores the resulting class files under ${classes.dir} (Line 72). This directory will become part of ${test.compile.classpath} (Line 41), the classpath which enables the Groovy compiler to resolve SampleTest.groovy‘s reference to the Java class whose methods it invokes.

Next, compile invokes the groovy-compilation task (via antcall, Line 80), to get the Groovy test files compiled to byte code. The groovy-compilation task runs as follows:

• First, it uses taskdef on Line 123 to define the <groovyc> task that compiles the Groovy classes. This extension to the “stock” Ant tasks lives in the groovy-all.jar under Basic Groovy Test Project’s lib directory.
• Next, it runs the newly defined groovyc task on all source files under ${test.groovysrc.dir}. The Groovy compilation phase runs with ${test.compile.classpath}, which includes the byte code for the Java application class Sample.java. The Groovy compiler then stores the resulting class files under ${test.classes.dir} (Line 131), which will become part of the classpath that is referenced by the junit task when running the tests (Line 91).

Parameterizing Groovy tests in builds

Thus far our tests have run but none has yet passed. Listing 4 shows the failing test code.

Listing 4. SampleTest extends GroovyTestCase

1.
2.              package com.lackey
3.
4.              class SampleTest extends GroovyTestCase{
5.
6.                public void testSample() {
7.                  String animal = System.properties.'com.lackey.<WBR>animal'
8.                  assert new Sample().getString().contains(<WBR>animal)  // note see junit report, Fig. 2
9.                }
10.             }

The code for the class being tested is shown in Listing 5.

Listing 5. public class Sample

package com.lackey;

                public class Sample {
                    public String getString() {
                        return "Monkey";
                    }
                }

We can fix the failing test by adding an additional parameter to the ant command we ran in Listing 1. This parameter will provide the correct value for the Java system property that defines the expected output of the Sample.getString() method. If you go back to look at the output initially obtained from ant clean test (in Listing 1) you will find that the last line provides the path of an HTML report detailing the test results (output.dir/test/reports/index.html). Drilling down will eventually get you to a page that indicates the line number of the assertion that caused the test to fail. You can see this line highlighted in Figure 2, below.

Two ways to re-set the system property

Our failing test extends GroovyTestCase (see Listing 4), which itself is an extension of Junit’s TestCase class. The test invokes methods on Sample.java, and uses Junit’s assert directive to ensure that the method yields the expected result. The return value of Sample().getString() is compared to the value of the Java system property com.lackey.animal.

For the test to pass, we need to invoke Ant with the correct value for the com.lackey.animal system property. Our build script allows us to define com.lackey.animal in either of two ways: via a -D directive on the command line, or via an environment variable.

So, you could invoke the project’s build.xml file like this:

ant -Dtest_input_argument=Monkey test

or you could just define an environment variable before invoking ant test. The following command would do the trick, or whatever is appropriate for your platform:

export test_input_argument=Monkey

Note that environment variables are evil, and usually best avoided. But sometimes you need them, which is why I’ve included this method.

Explicitly pass system properties

Lines 20 and 21 of Listing 3 show how to connect your environment with your Ant properties. The -D command switch always overrides whatever environment variable setting you may have established before invoking the build script. Whether you are setting your property via the -D directive from the command line, or via an environment variable, you need to make sure that your chosen setting is visible to your test code (which is invoked by the junit task on Line 87). By default, code invoked by the junit task will not “see” any system property settings defined on the command line. Ant’s documentation claims that Java system properties will be inherited by your test classes if you invoke the junit task with fork="true" and clonevm="true". I was never able to get this to work, however. Listing 3 illustrates the technique that does work for me: explicitly passing along all system properties that the test code needs to inherit, using the jvmarg tag, as shown on Line 88.

The project should run cleanly once you pass in the correct system property setting. Note that you could use this Ant technique to parameterize any code written in either Java or Groovy, not just tests.

More complex build logic with embedded Groovy

So far we’ve looked at Basic Groovy Test Project’s build.xml, which really just illustrates Ant’s support for using Groovy to test your Java code. Next you’ll see how Groovy can help streamline the logic of more complex build scripts. If you’re using Ant, you might have already implemented some of your more involved logic in XML. Typically this is done with a set of extension tasks from the ant-contrib project (see Resources). However, it quickly becomes painful to implement anything beyond one or two simple if/elses with these tasks. Don’t believe me? Just try looking at Listing 6 without wincing.

Listing 6. Conditional logic in Ant using XML

<if>
             <equals arg1="${param}" arg2="bar">
             <then>
               <echo message="time for bar">
             </echo>
             <else>
               <echo message="not time for bar">
             </echo>
            </else>
            </if>

Luckily, we can get some help from Apache’s Bean Scripting Framework (BSF). BSF enables Ant to call out to Groovy scriptlets, as well as scriptlets written in Python, Ruby, and a number of other third-generation programming languages (3GLs). The simplest way to implement Groovy’s logic is to embed it directly in your Ant script by using the script tag, as shown in Listing 7 on Line 6:

Listing 7. Embedding a Groovy scriptlet

1:<project name="embeded.groovy" default="main" basedir=".">
2:  <property name="lib.dir"   value="lib"/>
3:
4:  <property name="param"   value="define-me-with-dash-D-<WBR>on-command-line"/>
5:  <target name="main">
6:    <script language="groovy">
7:        <classpath>
8:          <fileset dir="${lib.dir}">
9:            <include name="*.jar"/>
10:          </fileset>
11:        </classpath>
12:
13:        "$param" == 'bar'  ?  println("time for bar") :  println("not time for bar" )   // one liner logic
14:
15:    </script>
16:  </target>
17:</project>

You will find the script from Listing 7 in the same directory as the Basic Groovy Test Project build.xml (just look for embedded-groovy.xml). Run it via the command ant -f embedded-groovy.xml and you will see the following output:

not time for bar

Running the command ant -Dparam=bar -f embedded.groovy.xml should produce the following change:

time for bar

In short, the nine lines of XML in Listing 6 have been replaced by one line in Listing 7. Admittedly, I am glossing over the overhead of setting up Groovy scripting support on Lines 6 through 11. But this is one-time overhead that becomes increasingly trivial as the number of XML-based logic lines grows (and then shrinks as you replace the XML with Groovy).

Calling out from Ant to an external Groovy script

Embedding scriptlets within your build.xml works out for simple tasks, but as things get more complex you’ll want to externalize your Groovy code. Then you can break out your logic across several classes, as well as edit your code in an IDE without surrounding XML clutter. However, once your Groovy code resides outside your build.xml, you’ll need to invoke it from Ant, as well as pass variables back. I’ll show you how to do this with a script called Build Project With Externalized Groovy, also found in the sample code packet.

To keep things lively, let’s assume we’re responsible for builds at the Internet arm of an animation studio. The build script in our sample project pretends to gather a variety of artifacts like wire frames from source control, preprocesses the artifacts, then passes back a list of files containing the modified artifacts to build.xml, which iterates through the list and invokes some kind of cloud rendering service to get the images nicely textured. To run the script, download it, cd to groovy.test.project.with.externalized.groovy.script then, invoke the command: ant -DpretendSourceCodeRepoUrl=http://javaworld.com and deploy. You should see the output shown in Listing 8.

Listing 8. Output of invoking Groovy from Ant

deploy:
     [echo] groovy script passed back: /tmp/fakefile.1,/tmp/fakefile.<WBR>2
     [echo] Invoking some other script to send file to rendering cloud:  /tmp/fakefile.1
     [echo] Invoking some other script to send file to rendering cloud:  /tmp/fakefile.2

The Ant code for the new build.xml‘s deploy task is given in Listing 9, and the code for a sample artifact gatherer is provided in Listing 10. The first thing the deploy task does is define the groovy task (see Line 2 in Listing 9). The deploy task then invokes gatherArtifacts.groovy, passing in the value of pretendSourceReporUrl as the first parameter (Line 10). The Groovy script then grabs the URL string (Line 12 of Listing 10) from the zero-th element of the args array (which provides access to command-line parameters), and then passes that value to obtainAndProcessArtifacts. This method doesn’t actually obtain artifacts from a real repository. It just grabs the text content of the URL provided and saves it twice into two files, which become our pretend artifacts. The list of file names is assigned to the properties hashmap, which is how variable bindings are communicated back to Ant (Line 12). Ant then tokenizes that returned value so it can iterate through the list of files on Line 21 of Listing 9.

Listing 9. Main target from build.xml for invoking Groovy from Ant

1:  <target name="deploy">
2:     <taskdef name="groovy"
3:               classname="org.codehaus.<WBR>groovy.ant.Groovy">
4:              <classpath>
5:                  <pathelement location="lib/groovy-all-1.7.<WBR>3.jar"/>
6:              </classpath>
7:        </taskdef>
8:
9:        <groovy src="${script.dir}/<WBR>gatherArtifacts.groovy">
10:         <arg value="${<WBR>pretendSourceCodeRepoUrl}"/> (line 10),
11:        </groovy>
12:
13:       <taskdef name="for"
14:          classname="net.sf.antcontrib.<WBR>logic.ForTask">
15:              <classpath>
16:                  <pathelement location="lib/ant-contrib.jar"<WBR>/>
17:              </classpath>
18:        </taskdef>
19:
20:         <echo  message="groovy script passed back: ${files}"/>
21:        <for list="${files}" param="file">
22:          <sequential>
23:            <echo>Invoking some other script to send file to rendering cloud:  @{file}</echo>
24:          </sequential>
25:        </for>
26:  </target>

Listing 10. Source for the invoked Groovy script

1:    String USAGE = "USAGE: groovy gatherArtifacts.groovy <sourceRepoUrl> "
2:
3:    if (args.length != 1) {
4:        System.err.println USAGE
5:        System.exit(1)
6:    }
7:
8:
9:    // The list of wire frame files we pretend to obtain from our source code repo is passed back
10:    // to Ant using the 'files' property
11:    //
12:    properties['files']  = obtainAndProcessArtifacts(<WBR>this.args[0])
13:
14:
15:    String obtainAndProcessArtifacts(<WBR>String sourceCodeRepoUrl) {
16:        // our demo does not really expect a URL to a source code repo. we will just get the
17:        // string contents of whatever URL we are passed and make believe that is one of the files
18:        // we are getting from the repo.
19:        String  contents = new URL(sourceCodeRepoUrl).text
20:
21:        String path1 = preProcessContentsAndSaveToFil<WBR>e(contents, "fakefile.1")
22:        String path2 = preProcessContentsAndSaveToFil<WBR>e(contents, "fakefile.2")
23:
24:        return "$path1,$path2"
25:    }
26:
27:    private String preProcessContentsAndSaveToFil<WBR>e(String contents, String fileName) {
28:        def preprocessedContents = "<!-- modified contents -->$contents "
29:        String tempDir = System.properties.'java.io.<WBR>tmpdir'
30:        File artifact = new File(tempDir, fileName)
31:        artifact.text = preprocessedContents
32:        return artifact.path
33:    }

Calling Ant from Groovy

The above technique for getting Ant to invoke Groovy code works, but it’s still an XML/Groovy hodgepodge. If you want to avoid XML completely, then you can drive your build process from your Groovy script and invoke Ant tasks as needed from that script. Listing 11 is an example that uses Ant’s fileScanner task to find all *.groovy and *.java files under the current working directory, then print out the first few bytes of each file.

Listing 11. Ant tasks invoked from Groovy

1:    def ant = new AntBuilder()
2:
3:    def scanner = ant.fileScanner {
4:        fileset(dir:".") {
5:            include(name:"**/*.groovy")
6:            include(name:"**/*.java")
7:        }
8:    }
9:
10:    for (f in scanner) {
11:        println    "Found file $f  and first few bytes are: " + f.text[0..30]
12:    }

Specifying inter-task dependencies with Gant

AntBuilder is useful for many types of build-related tasks, but one thing you lose by using it is Ant’s ability to declaratively specify your build process as a series of interdependent tasks.

Fortunately, you can have it both ways if you use Gant, which is basically Ant, but using Groovy syntax rather than XML to define task behaviors. Listing 12 shows a very simple Gant script which demonstrates how arbitrary Groovy code can be executed within a closure block that defines a task. (See the Gant project website for easy-to-follow installation instructions.) Once you have Gant set up, you can run the code in Listing 12 by copying it to a file such as /tmp/gant, and then running the command gant -f /tmp/gant. You’ll see output similar to what’s shown in Listing 13.

Listing 12. A minimal Gant script

target ( compile : 'compiles your code.' ) {
  echo ( message : 'running your code.' )
  println " We can invoke any Groovy code we want within a closure block " + moreOutput()
}

target ( run : 'runs your code' ) {
  depends ( compile )
  echo ( message : 'running your code.' )
}

def moreOutput() {
    return new String(" even call methods ")
}

setDefaultTarget ( run  )

Listing 13. Gant script output

run:
compile:
     [echo] running your code.
 We can invoke any Groovy code we want within a closure block  even call methods
------ compile
     [echo] running your code.
------ run

BUILD SUCCESSFUL

In conclusion

This tour of incremental Groovy build techniques for Ant started with a fully functional build.xml example. Next, I introduced several techniques for implementing complex build logic using Ant without XML’s sharp corners, including calling Ant from Groovy and specifying inter-task dependencies with Gant. Altogether, these examples address the fundamental tasks of a production build cycle. Hopefully seeing them has convinced you that integrating Groovy into your current Ant builds is not just doable but desirable, too.

If you want to take things a step further, then I suggest you check out Gradle, a very interesting project that has grown out of work done on Gant. Gradle provides Maven/Ivy-style dependency management and repository structure, as well as a Maven-like convention-over-configuration approach to specifying build behavior. One of Gradle’s design objectives appears to be deep integration with Groovy. It’s also worth noting that the latest (3.x) version of Maven allows you to use Groovy rather than XML when specifying your build’s dependencies and processing. Clearly, Groovy is catching on with the leading open source build automation projects. In fact, you might want to mention that argument as a foot-in-the-door for getting your workplace colleagues to “go dynamic.”