| The roads I take... | Plugin Crash Protection in SeaMonkey! >>

The Speed of JavaScript in SeaMonkey

In the last days, I did some JavaScript performance tests on different SeaMonkey versions on my machine, just to find out where we are heading for SeaMonkey 2.1 in comparison to what we shipped in the past:

Image No. 22441

I ran 3 tests there: my personal Mandelbrot demo 1.0 with default settings (blue), the pretty common SunSpider benchmark (red), and the new Kraken benchmark (yellow). Note that the scale of Kraken results is different to the scale of the other two.

(Update/Note: I was told that my Mandelbrot demo uses JavaScript 1.7 and HTML5 features that Mozilla 1.8 supported but other browsers still don't, so there's a more cross-browser Mandelbrot demo 2.0 now - that one performs even a tad better on Mozilla trunk than the results here.)

The builds I tested were current pre-2.1 trunk builds, which already have the first stage of the new JaegerMonkey JavaScript engine, 2.0 branch builds without and with the venkman JavaScript Debugger add-on enabled (without JSD means that TraceMonkey can fully work, as activating venkman makes JSD put JavaScript into a non-tracing debug mode) - and finally, the 1.1.19 release build.

Unfortunately, I can't get data for Kraken on 1.1.19, as after a very very long time of running (given the "slow JavaScript" warning is deactivated), it errors out with not finding any JSON support. Comparing to the other tests, it suffices to say that SeaMonkey 1.1.x, coming from the Mozilla 1.8 tree, is very slow - but then, that's the baseline we started from.

SeaMonkey 2.0.x - even with venkman on and JSD knocking tracing off - is 90% or more faster than 1.1.x in those tests we can run in the older version. Suffice to say that Mozilla's JS team has made tremendous improvements between the 1.8 and 1.9.1 branches of the platform. One might think it's hard to go even further after such a jump, and indeed it has become much harder - but things could still get better. First, turning the debugger off and tracing on wins 19% in SunSpider, 13% for Mandelbrot and 2% for Kraken (the latter either doesn't have a lot of tight loops, which is what tracing help most with, or we are bad at tracing them in this version). At this level, SeaMonkey 2.0 looks already nice - but that's still 1-2 years old code, and development continued.

The current pre-2.1 builds of SeaMonkey bring another 61% win on Mandelbrot, 66% on SunSpider and 72% on Kraken - over 2.0 with full tracing! While the basic Mandelbrot set image took over 25 seconds to calculate and display on 1.1 and significantly over a second on 2.0, we're under half a second now for pre-2.1 code. And, as I already mentioned, JaegerMonkey work isn't even finished yet, there's a few more things to come there.

Of course, the question is if what those tests run is what Web sites and applications really need - we know a few things in SunSpider are non-realistic workloads. Kraken tries to look into CPU-intense workloads that could be useful for actual web applications, esp. if they want to to things like image and audio processing. My Mandelbrot set demo may look far-fetched, but then, I can surely see websites or even more web games dynamically generating fractal-based shades and textures based on variable parameters, and there you might have very similar code.

For those people who want the hard facts, here's the numbers for my graph with all details:
  • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.13pre) Gecko/20100913 Lightning/1.0b2pre SeaMonkey/2.0.9pre
    • Mandelbrot: 1421.8ms +/- 5.0% (1.336, 1.401, 1.497, 1.383, 1.492)
    • SunSpider: 1738.0ms +/- 2.6%
    • Kraken: 40428.1ms +/- 1.6%
    • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.13pre) Gecko/20100913 Lightning/1.0b2pre SeaMonkey/2.0.9pre (venkman disabled)
      • Mandelbrot: 1237.6ms +/- 3.3% (1.220, 1.216, 1.301, 1.197, 1.254)
      • SunSpider: 1402.2ms +/- 3.6%
      • Kraken: 39558.9ms +/- 1.5%
      • Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:1.8.1.24) Gecko/20100301 SeaMonkey/1.1.19
        • Mandelbrot: 25214.2ms +/- 1.0% (25.176, 24.907, 25.180, 25.216, 25.592)
        • SunSpider: 17501.2ms +/- 1.2%
        • Kraken: Runs slowly for a while, but does break for not finding JSON handling.
        • All in all, the Mozilla JavaScript team is really enabling a lot of new possibilities here, as now we can do things client-side we needed to do server-side up to now, and that makes completely new dynamics possible. And SeaMonkey fully features those improvements made by the Mozilla community. :)

          Beitrag geschrieben von KaiRo und gepostet am 22. September 2010 18:10 | Tags: JaegerMonkey, JavaScript, Mandelbrot, Mozilla, SeaMonkey, SunSpider | 4 Kommentare | TrackBack

          Kommentare

          AutorBeitrag

          Scott Baker

          zitieren
          Mandelbrot Demo only FF?
          It seems that your mandelbrot demo only works on Firefox? I can't make it work on Opera or Chrome, clicking the start button does nothing. Any ideas?
          22.09.2010 18:54

          KaiRo

          Webmaster

          zitieren
          That demo seems to use JavaScript 1.7 features, like let instead of var, and I'm told that Chrome also has a bug in that it doesn't allow putImageData on a non-imagedata object (which I'm told the canvas spec allows), so they break on that as well.

          If I'd be mean, I'd say that those browsers not being able to run it just show that they don't support ECMAScript and HTML5 well (actually, what scares me is that SeaMonkey 1.1 / Firefox 2 already supported those things and they don't) but my Mozilla friends are currently figuring out how to make this work cross-browser.
          22.09.2010 20:23

          njn

          zitieren
          Kraken traces very well
          Kraken has lots of tight loops and traces very well. Indeed, that's the main reason why SpiderMonkey performs so well on it.
          23.09.2010 04:55

          KaiRo

          Webmaster

          zitieren
          njn:
          Ah, OK, then we are just bad at tracing those loops in 1.9.1, I guess, and much better at it on trunk. :)
          23.09.2010 16:17

          || Kommentar hinzufügen