Dec 2, 2013

Open source releases: a piece of Gradle

Yennick Trevels wrote a great blog on how to set up a Gradle build to deploy to the Sonatype OSS Repository (and from there to maven central).
Here I will show how the Gradle release plugin can be configured to
  • streamline the release process
  • automatically provide download links to you latest release on Github

The standard release process

The principles of the Gradle release plugin originate from the Maven release plugin. The release steps are:
  • check release preconditions (no uncommitted files and no snapshot dependencies)
  • get the release version number
  • build (and test) the release
  • tag the release
  • update the version number and commit it

Configuration

Add the build dependency and apply the plugin:
buildscript {
    dependencies {
        classpath 'com.github.townsfolk:gradle-release:1.2'
  ...
    }
 ...
}

apply plugin: 'release'
The release plugin is not fully documented, but fortunately it is small and the code speaks for itself (its Gradle :)
In ReleasePluginConvention.groovy you find all the properties that can be specified. I changed the default versionPropertyFile (gradle.properties) to versions.txt, to make it obvious to find the project version. You can also see how to use complex version schemes.
release {
    versionPropertyFile = 'version.txt'

    //a map of [regular expression : increment closure],
    //if the version matches one of the regexps, the corresponding
    // closure is used to auto-increment the version
    versionPatterns = [
            // 1.2.3-groovy-1.8.4 -> next version: 1.2.4-groovy-1.8.4
            /(\d+)-groovy-(.+$)/: {matcher, project ->
                matcher.replaceAll("${(matcher[0][1] as int) + 1}-less-${matcher[0][2]}")
            }
    ]
}
The version needs to be read from the file:
//the release plugin expects the content to be 'version=xyz',
//without whitespace around the =
version = file('version.txt').text.split('=')[1].trim()

Build hooks

One of the most powerful features of Gradle is the ability to hook custom tasks about anywhere in the build cycle.
gradle tasks --all shows all the steps in the release task that we can hook into:

checkCommitNeeded - Checks to see if there are any added, modified, removed, or un-versioned files.
checkSnapshotDependencies - Checks to see if your project has any SNAPSHOT dependencies.
checkUpdateNeeded - Checks to see if there are any incoming or outgoing changes that haven't been applied locally.
commitNewVersion - Commits the version update to your SCM
confirmReleaseVersion - Prompts user for this release version. Allows for alpha or pre releases.
createReleaseTag - Creates a tag in SCM for the current (un-snapshotted) version. [uploadArchives]
initScmPlugin - Initializes the SCM plugin (based on hidden directories in your project's directory)
preTagCommit - Commits any changes made by the Release plugin - eg. If the unSnapshotVersion tas was executed
release - Verify project, release, and update version to next. [clean]
unSnapshotVersion - Removes "-SNAPSHOT" from your project's current version.
updateVersion - Prompts user for the next version. Does it's best to supply a smart default.

Hooking in is straightforward:
//always clean before building a release
release.dependsOn clean
//upload to sonatype before tagging the VCS
createReleaseTag.dependsOn uploadArchives
We should only upload release builds to the sonatype staging repository, so we tweak the 'uploadArchives' a little bit:
task uploadSnapshot(dependsOn: uploadArchives)

// the release task spawns a new GradleBuild that doesn't contain
// release itself, but it contains createReleaseTag
def sonatypeRelease = gradle.startParameter.taskNames.contains('createReleaseTag')
def sonatypeSnapshot = gradle.startParameter.taskNames.contains('uploadSnapshot')
if (sonatypeRelease) {
    //only sign releases
    signing {
        sign configurations.archives
    }
}
def sonatypeRepositoryUrl = sonatypeRelease ?
    'https://oss.sonatype.org/service/local/staging/deploy/maven2/' :
    'https://oss.sonatype.org/content/repositories/snapshots/'

uploadArchives {
    repositories {
        if (!(sonatypeRelease || sonatypeSnapshot)) {
            mavenLocal()
        } else {
            mavenDeployer {
                if (sonatypeRelease) {
                    beforeDeployment { deployment -> signing.signPom(deployment) }
                }

                repository(url: sonatypeRepositoryUrl) {
                    authentication(userName: sonatypeUsername,
                    password: sonatypePassword)
                }

                pom.project {
...

Provide download links

In the README.md we can include links to our artifacts on maven central:
You can download coolapp from the [maven central repository]
 (http://central.maven.org/maven2/org/awesome/coolapp/1.2.3-groovy-1.8.4/coolapp-1.2.3-groovy-1.8.4.zip)
With a few extra lines of Groovy, we can make the links point to the new release. The modified README can be committed to Github together with the new snapshot version:
task updateReadme << {
    File readme = file('README.md')
    //the release version is the version before the release minus 'SNAPSHOT'
    def releaseVersion = "${project['release.oldVersion']}".replaceAll('-SNAPSHOT', '')
    //replace all occurrences of x.y.z-groovy-a.b.c with the new release version
    readme.text = (readme.text =~ /\d\.\d(\.\d)?-groovy-\d\.\d(\.\d)?(-SNAPSHOT)?/)
                                                     .replaceAll("${releaseVersion}")
}

commitNewVersion.dependsOn updateReadme
And that's all there is to it. A working example can be found in the lesscss project.

Oct 22, 2013

Spock framework: writing tests at warp speed

Groovy  is a superb language for automated tests: it is far more powerful than Java and it allows to skip lots of boilerplate code. The Spock framework adds a nice test specification DSL on top of that. In this post, you will find out more about my favorite Spock feature: data driven testing.

Write one test and get 40 for free

I build a Java port of the LESS CSS compiler, originally written in JavaScript.
The LESS compiler already has more than 40 test cases that compare a compiled LESS file with the expected CSS result. I only had to create one unit test that loops over all LESS files, compiles them and checks the result.

def "less compatibility tests"() {
    expect:
    compiler.compile(lessFile) == getCss(lessFile)

    where:
    lessFile << new File('test/less').listFiles().findAll { it.name.endsWith('.less') }
}

def getCss(File lessFile) {
   return new File('test/css' + lessFile.getName().replace('.less', '.css')).text
}

Notice how you can tell Spock that it has to create one test for each variable in a collection:

where:
variable << collection

Also notice the absence of assertEquals in the expect block: we only need to write boolean conditions. As programmers (and even non-programmers), we are trained to recognize these comparisons, so they are easier to read, understand and maintain then the classical assertXxx method calls.

Test isolation

We could also write a plain JUnit test that loops over the files to test them, but the problem would then be test isolation: if one test fails, the test method is aborted and we won't know the status of the remaining tests.
Spock, instead, runs an isolated test for each iteration. All failures are reported separately, but if all tests pass, you will see only one method in the test report.
In the above example, it would be even better to have one test method with a descriptive name for each LESS file. This is what Spock's Unroll annotation does:

@Unroll
def "#lessFile.name compatibility test"() {...}

By using #lessFile.name in the method name, we get clear test reports:


Data tables

Spock where blocks are not limited to one variable. If you want to populate more than one variable in each iteration, you can use the collection approach as shown above, or you can provide a table with values:

when:
def reader = new FileSystemResourceReader()

then:
reader.canRead(location) == canRead

where:
location       | canRead
'file1.txt'    | true
'../file2.txt' | true
'/no file'     | false

This example will run 3 tests: one for each data row.
Data tables are especially useful for testing the typical use cases and corner cases in one go.
Syntactically, a data table is a bunch of unrelated or statements, so it may look strange at first sight. But if well formatted, data tables look like tables, and that's how Spock interprets them.
Intellij IDEA users get an extra here: it knows about Spock data tables and can format them automatically.

Links

The full examples are part of the Lesscss project at GitHub. Feel free to post any comments or suggestions.

Jun 17, 2013

JavaScript on the JVM: experimenting with Nashorn

I recently experimented with the upcoming Nashorn JavaScript run-time to compile Less style sheets.
I hoped to boost the performance in comparison with existing Less compilers for the JVM, but the results where not what I expected...

Motivation

Although I am a fan of Twitter Bootstrap and Less CSS in general, I found that the Less compilation in Java projects could take several seconds and thus be really bad for development turnaround.
During Devoxx 2012, Marcus Lagergren explained how much faster Nashorn was compared with Rhino (being already included in the standard Java distribution). So I took it for a test drive.

Can't wait for Java 8

 I didn't have JDK8 installed yet, so I forked a backport on GitHub and build the nashorn.jar that provides a standard javax.script.ScriptEngine implementation.
This made it easy to support both Rhino and Nashorn: by specifying -Djava.ext.dirs=/path/to/nashorn, Nashorn was used by default with a fallback to Rhino.

ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine scriptEngine = factory.getEngineByName("nashorn");
if (scriptEngine == null) {
    scriptEngine = factory.getEngineByName("nashorn");
}
if (scriptEngine == null) {
    throw new RuntimeException("No JavaScript script engine found");
}

The compiler

I didn't use the Rhino specific less script, but instead used the standard less-1.3.3.js. I only had to add a few lines of JavaScript in front, to stub a little browser functionality, and add a compile function at the back to be called from Java.
The Java code was rather simple:
scriptEngine.eval(getLessScriptReader());
String css =
   ((Invocable) scriptEngine).invokeFunction("compile", lessString);

The Bootstrap test

The Twitter Bootstrap style sheets compiled successfully with both Rhino and Nashorn
Compilation with Rhino took 10 seconds on my Macbook, while Nashorn did in in 8 seconds. Not exactly impressive...
To get an idea of the JavaScript compilation / execution ratio , I invoked the Less compile function multiple times, hereby reusing the JavaScript context.

Time needed to compile Twitter Bootstrap (seconds)
  The first row includes JavaScript compilation
# Rhino Rhino
(optimized)
Nashorn
JDK7
Nashorn
JDK8
1 9.8 5.5 8.0 15.0
2 7.5 2.1 2.5 5.8
3 7.3 1.5 / 3.7
4 7.4 1.1 / 3.4
5 7.3 1.0 / 3.4

The first column is for Rhino invoked as described above using the javax.script interfaces. By default Rhino always runs in interpreted mode. Compare this with the second column, where the Rhino Context is used directly which allows to set the optimization level to 9 (highest level).
The third column is for Nashorn on Java 7. The Nashorn backport apparently contains some bugs, causing the third Less compilation within the same JavaScript context to fail.
The fourth column contains the results of the Less compilation using a JDK8 snapshot. The engine now works correctly, but the performance still needs to improve a lot.

Conclusions

I realize that the Nashorn project is still in alpha, but until the team comes up with an API that allows compiler optimizations like Rhino does, the latter still seems to be the best bet.