Browser wars: the non-evil empire strikes back

By Sterling “Chip” Camden
Contributing Writer, [GAS]

Since we last left our heroes back in July, several new browser versions have been released:  Chrome 3, Opera 10, Lunascape 5.1.5, and incremental updates to Firefox, Safari, and even Internet Explorer 8.  So it’s time for us to revisit our benchmarks.

Dromaeo benchmark

The Dromaeo test suite measures JavaScript performance in computing a variety of algorithms, many of which are common in web applications.  I ran the recommended suite on Google Chrome 3.1.195.21, Mozilla Firefox 3.5.3, Internet Explorer 8.0.6001.18813, Apple Safari 4.0 (531.9.1), Opera 10.00, and Lunascape 5.1.5 (testing each of its three rendering engines).  The test machine is an Intel Centrino Duo 2Ghz with 2GB of RAM running Windows Vista – not top of the line, but pretty typical.

First the raw data.  The winner in each test is colored green, and any runner-up that is within 10% of the winner is colored yellow.  As usual, not all tests could be run on Internet Explorer (or Lunascape with the Trident Engine) due to JavaScript errors – thus, I can provide no permalink to those results on dromaeo.com.

image

Next, a new visualization.  Since the individual tests are not comparable in terms of run time, I’ve plotted the results in terms of percentage of the best score, so you can more easily compare the performances in each category.

Legend:

image

image

As you can see, Chrome 3 has regained dominant performance in many categories – winning in 24 of the 49 tests (up from 14 with version 2).  Safari wins in 13 of the tests.  The new version of Lunascape holds its own by winning 4 of the tests when using WebKit, and 7 when using Gecko (stealing every win from Firefox).  Opera 10 takes one important test (DOM Traversal) away from Safari, winning a category for the first time ever.  Opera 10 is now faster than Internet Explorer in all but three tests, though it still lags far behind the other browsers in most categories.  Internet Explorer is, well, it’s Internet Explorer – what do you expect?

Peacekeeper Benchmark

The Peacekeeper benchmark, which also tests rendering speed, gave the following results:

image

I was unable to run the Lunascape:Trident test on the same system, apparently because I couldn’t figure out how to clear the correct cookie.

In Peacekeeper’s overall score, Safari edges out Chrome overall.  Drilling down to details, it appears that Safari’s main strength lies in data manipulation, while Chrome wins easily on complex graphics.  Which one better serves the future of web apps?  It’s interesting that Lunascape with the WebKit engine scored so much lower on this test than either Chrome or Safari, when it ran comparably to both of them in most of the Dromaeo tests.

Acid3 test

Standards compatibility has always been bothersome for Internet Explorer, as the lack of results from some of the Dromaeo tests above still shows.  But there’s another test specifically designed to gauge compatibility with web standards:  the Acid3 test, which can be run here.  These were my results for each of the above-named browsers:

Chrome:  100%

Firefox:  93%

Internet Explorer:  20%, and “LINKTEST FAILED”

Safari:  100%

Opera:  100%

Lunascape WebKit:  100%, but “LINKTEST FAILED”

Lunascape Gecko:  93%

Lunascape Trident:  13%, and “LINKTEST FAILED”

It looks like WebKit (Chrome, Safari) and Opera have these standards nailed.  Gecko (Firefox) still needs work, and Trident (Internet Explorer) doesn’t seem to be trying very hard.

Conclusions

Of course, there’s a lot I didn’t test:  load time for the browser, page load speed, and memory footprint all come to mind.

What do these results tell you about your choice of browser?  Personally, I’m still quite happy with Chrome – but it’s good to see healthy competition in this space.