The roads I take...
KaiRo's weBlog
| Zeige Beiträge veröffentlicht im September 2010 und mit "Mandelbrot" gekennzeichnet an. Zurück zu allen aktuellen Beiträgen | |||||||||||||||||||||||||||||||||||||||||||
22. September 2010
The Speed of JavaScript in SeaMonkey
In the last days, I did some JavaScript performance tests on different SeaMonkey versions on my machine, just to find out where we are heading for SeaMonkey 2.1 in comparison to what we shipped in the past:
I ran 3 tests there: my personal Mandelbrot demo 1.0 with default settings (blue), the pretty common SunSpider benchmark (red), and the new Kraken benchmark (yellow). Note that the scale of Kraken results is different to the scale of the other two.
(Update/Note: I was told that my Mandelbrot demo uses JavaScript 1.7 and HTML5 features that Mozilla 1.8 supported but other browsers still don't, so there's a more cross-browser Mandelbrot demo 2.0 now - that one performs even a tad better on Mozilla trunk than the results here.)
The builds I tested were current pre-2.1 trunk builds, which already have the first stage of the new JaegerMonkey JavaScript engine, 2.0 branch builds without and with the venkman JavaScript Debugger add-on enabled (without JSD means that TraceMonkey can fully work, as activating venkman makes JSD put JavaScript into a non-tracing debug mode) - and finally, the 1.1.19 release build.
Unfortunately, I can't get data for Kraken on 1.1.19, as after a very very long time of running (given the "slow JavaScript" warning is deactivated), it errors out with not finding any JSON support. Comparing to the other tests, it suffices to say that SeaMonkey 1.1.x, coming from the Mozilla 1.8 tree, is very slow - but then, that's the baseline we started from.
SeaMonkey 2.0.x - even with venkman on and JSD knocking tracing off - is 90% or more faster than 1.1.x in those tests we can run in the older version. Suffice to say that Mozilla's JS team has made tremendous improvements between the 1.8 and 1.9.1 branches of the platform. One might think it's hard to go even further after such a jump, and indeed it has become much harder - but things could still get better. First, turning the debugger off and tracing on wins 19% in SunSpider, 13% for Mandelbrot and 2% for Kraken (the latter either doesn't have a lot of tight loops, which is what tracing help most with, or we are bad at tracing them in this version). At this level, SeaMonkey 2.0 looks already nice - but that's still 1-2 years old code, and development continued.
The current pre-2.1 builds of SeaMonkey bring another 61% win on Mandelbrot, 66% on SunSpider and 72% on Kraken - over 2.0 with full tracing! While the basic Mandelbrot set image took over 25 seconds to calculate and display on 1.1 and significantly over a second on 2.0, we're under half a second now for pre-2.1 code. And, as I already mentioned, JaegerMonkey work isn't even finished yet, there's a few more things to come there.
Of course, the question is if what those tests run is what Web sites and applications really need - we know a few things in SunSpider are non-realistic workloads. Kraken tries to look into CPU-intense workloads that could be useful for actual web applications, esp. if they want to to things like image and audio processing. My Mandelbrot set demo may look far-fetched, but then, I can surely see websites or even more web games dynamically generating fractal-based shades and textures based on variable parameters, and there you might have very similar code.
For those people who want the hard facts, here's the numbers for my graph with all details:Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.13pre) Gecko/20100913 Lightning/1.0b2pre SeaMonkey/2.0.9pre Mandelbrot: 1421.8ms +/- 5.0% (1.336, 1.401, 1.497, 1.383, 1.492) SunSpider: 1738.0ms +/- 2.6% Kraken: 40428.1ms +/- 1.6% Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.13pre) Gecko/20100913 Lightning/1.0b2pre SeaMonkey/2.0.9pre (venkman disabled) Mandelbrot: 1237.6ms +/- 3.3% (1.220, 1.216, 1.301, 1.197, 1.254) SunSpider: 1402.2ms +/- 3.6% Kraken: 39558.9ms +/- 1.5% Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:1.8.1.24) Gecko/20100301 SeaMonkey/1.1.19 Mandelbrot: 25214.2ms +/- 1.0% (25.176, 24.907, 25.180, 25.216, 25.592) SunSpider: 17501.2ms +/- 1.2% Kraken: Runs slowly for a while, but does break for not finding JSON handling. All in all, the Mozilla JavaScript team is really enabling a lot of new possibilities here, as now we can do things client-side we needed to do server-side up to now, and that makes completely new dynamics possible. And SeaMonkey fully features those improvements made by the Mozilla community. 
I ran 3 tests there: my personal Mandelbrot demo 1.0 with default settings (blue), the pretty common SunSpider benchmark (red), and the new Kraken benchmark (yellow). Note that the scale of Kraken results is different to the scale of the other two.
(Update/Note: I was told that my Mandelbrot demo uses JavaScript 1.7 and HTML5 features that Mozilla 1.8 supported but other browsers still don't, so there's a more cross-browser Mandelbrot demo 2.0 now - that one performs even a tad better on Mozilla trunk than the results here.)
The builds I tested were current pre-2.1 trunk builds, which already have the first stage of the new JaegerMonkey JavaScript engine, 2.0 branch builds without and with the venkman JavaScript Debugger add-on enabled (without JSD means that TraceMonkey can fully work, as activating venkman makes JSD put JavaScript into a non-tracing debug mode) - and finally, the 1.1.19 release build.
Unfortunately, I can't get data for Kraken on 1.1.19, as after a very very long time of running (given the "slow JavaScript" warning is deactivated), it errors out with not finding any JSON support. Comparing to the other tests, it suffices to say that SeaMonkey 1.1.x, coming from the Mozilla 1.8 tree, is very slow - but then, that's the baseline we started from.
SeaMonkey 2.0.x - even with venkman on and JSD knocking tracing off - is 90% or more faster than 1.1.x in those tests we can run in the older version. Suffice to say that Mozilla's JS team has made tremendous improvements between the 1.8 and 1.9.1 branches of the platform. One might think it's hard to go even further after such a jump, and indeed it has become much harder - but things could still get better. First, turning the debugger off and tracing on wins 19% in SunSpider, 13% for Mandelbrot and 2% for Kraken (the latter either doesn't have a lot of tight loops, which is what tracing help most with, or we are bad at tracing them in this version). At this level, SeaMonkey 2.0 looks already nice - but that's still 1-2 years old code, and development continued.
The current pre-2.1 builds of SeaMonkey bring another 61% win on Mandelbrot, 66% on SunSpider and 72% on Kraken - over 2.0 with full tracing! While the basic Mandelbrot set image took over 25 seconds to calculate and display on 1.1 and significantly over a second on 2.0, we're under half a second now for pre-2.1 code. And, as I already mentioned, JaegerMonkey work isn't even finished yet, there's a few more things to come there.
Of course, the question is if what those tests run is what Web sites and applications really need - we know a few things in SunSpider are non-realistic workloads. Kraken tries to look into CPU-intense workloads that could be useful for actual web applications, esp. if they want to to things like image and audio processing. My Mandelbrot set demo may look far-fetched, but then, I can surely see websites or even more web games dynamically generating fractal-based shades and textures based on variable parameters, and there you might have very similar code.
For those people who want the hard facts, here's the numbers for my graph with all details:
- Mozilla/5.0 (X11; Linux i686; rv:2.0b7pre) Gecko/20100917 Firefox/4.0b7pre SeaMonkey/2.1b1pre
- Mandelbrot: 483.0ms +/- 1.4% (0.481, 0.492, 0.474, 0.487, 0.481)
- SunSpider: 477.4ms +/- 1.1%
- Kraken: 11151.6ms +/- 0.5%
Von KaiRo, um 18:10 | Tags: JaegerMonkey, JavaScript, Mandelbrot, Mozilla, SeaMonkey, SunSpider | 4 Kommentare | TrackBack: 0
