Google has decided that the existing JavaScript benchmarks just don't do the job of a testing a modern language in a modern web page. The solution is Octane a new benchmark which Google has designed to be more representative of how we use browsers today. Of course we all know what is really wrong with benchmarks...

They generally prove whatever point of view the designers of the benchmark happened to hold.  What activities you include in a benchmark and how you weight them alters the result and this can be made to show your own favorite in a good light. Even if you accept that a benchmark is fair, its conclusions are only relevant to you and any projects you may be working on if the mix of activities used in the benchmark have some similarity to what you are trying to do.

In the case of Octane the mix is very important. Rather than build special benchmark tasks, Octane uses some real world apps and runs them as a set to obtain an overall performance figure. There are five new tasks added to the original V8 benchmark that Google used to test its JavaScript engine:

• Box2DWeb runs a JavaScript port of a popular 2D physics engine that is behind many well-known simulations and web games. 
• Mandreel puts a JavaScript port of the 3D Bullet Engine to the test with a twist: The original C++ source code for the engine is translated to JavaScript by Onan Games’ Mandreel compiler, which is also used in countless web-based games. 
• Pdf.js is based on Mozilla’s PDF reader and shows how JavaScript applications can replace complex native browser plug-ins. It measures how fast the browser decodes a sample PDF document. 
• GB Emulator is derived from an open source emulator of a famous game console running a 3D demo. 
• CodeLoad measures how quickly a JavaScript engine can bootstrap commonly used JavaScript libraries and start executing code in them. The source for this test is derived from open source libraries (Closure, jQuery)

You can see that three of the tests are heavily graphic oriented and there are some features from WebGL that stop the benchmark running in IE9 - but it does work in IE10.  The original tasks in the V8 benchmark are:

• Richards
  OS kernel simulation benchmark, originally written in BCPL by Martin Richards.
  
• DeltaBlue
  One-way constraint solver, originally written in Smalltalk by John Maloney and Mario Wolczko.
  
• Crypto
  Encryption and decryption benchmark based on code by Tom Wu.
  
• RayTrace
  Ray tracer benchmark based on code by Adam Burmister.
  
• EarleyBoyer
  Classic Scheme benchmarks, translated to JavaScript by Florian Loitsch's Scheme2Js compiler.
  
• RegExp
  Regular expression benchmark generated by extracting regular expression operations from 50 of the most popular web pages.
  
• Splay
  Data manipulation benchmark that deals with splay trees and exercises the automatic memory management subsystem.
  
• NavierStokes
  Solves NavierStokes equations in 2D, heavily manipulating double precision arrays. Based on Oliver Hunt's code.

This is a fairly intensive set of tests, but one obvious missing aspect is the interaction of JavaScript with the DOM. If you are writing a program that is going to be dynamically updating the structure of the page then the benchmark isn't going to tell you much.

Octane can be run on a web browser of your choice simply by visiting the web site and clicking Start. It also has a nice user interface and will work on some mobile browsers.