Follow-up: JavaScript on the JVM, experimenting with Nashorn

The release of JDK8 was a good opportunity to test again how fast the Nashorn JavaScript engine is in compiling Less style sheets.
I hoped that performance improved since I last tried an early access build, but alas...

Setup

I recently released a new version of lesscss that supports 3 execution engines:

• rhino 1.7R4: it runs a pre-compiled version of less.js in the highest optimization level (9)
• nashorn: the JDK8 built-in version, which does not (yet) support pre-compilation nor optimization
• node.js lessc: prepare a commandline string and execute it with Runtime.exec

I compiled the Twitter Bootstrap style sheets using different engines.
I used my mac book (2.66GHz core i7 with 8GB memory and JVM args -Xms512m -Xmx1500m -XX:MaxPermSize=320m).
For each run I instantiated one engine for 15 consecutive compilations.

As you can see, both rhino and nashorn perform better when the JVM warmed up, but the differences in performance are huge:

Time needed to compile Twitter Bootstrap (seconds)
  The first row includes JavaScript compilation in case of Nashorn

#	Rhino (optimized)	Nashorn	node.js
1	3.9	18.4	0.9
2	1.9	7.9	0.6
3	1.7	6.1	0.6
4	1.6	5.8	0.6
5	1.4	5.0	0.6
6	1.4	4.9	0.6
7	1.5	4.5	0.6
8	1.4	5.3	0.6
9	1.3	5.8	0.6
10	1.3	4.4	0.6
11	1.0	4.0	0.6
12	1.0	3.8	0.6
13	1.0	3.5	0.6
14	0.9	3.1	0.6
15	0.9	2.8	0.6

Conclusions

Although this is not a real benchmark, the figures differed less then 20% between runs and the trends are clear (at least for this use case):

• Rhino with optimizations is still a lot faster then nashorn
• While rhino got faster on JDK8, nashorn seems to got slower (see previous blog)
• There is a big gap with node.js, despite the overhead of spawning a new process and writing its output to disk