Running a Worker Without a Separate File

Sometimes it is desirable to include the code for a Worker thread in the same file as say the UI Thread code or even the HTML page itself.

This can be done as a special case of using a Blob and a Data URL. It is a special case because generally a Blob can represent anything that you can put in a file, and a Data URL can provide a URL to the Blob.

So all we have to do is put the code that would have gone into the file into a Blob, generate a Data URL for the Blob, and pass this to the worker's constructor.

If you have a string, workerCodeString say, which has all of the code you want to execute then implementing this is easy:

var blob = new Blob([workerCodeString],
          {type: 'application/javascript'});
var url = URL.createObjectURL(blob);
var worker = new Worker(url);

The Blob constructor takes an array of strings that it concatenates to make the "file". The second parameter, the options object, is used to specify the file type using the same MIME type you would specify in a full <Script> tag loading the JavaScript. The blob can now be considered to be an in-memory version of the file holding the JavaScript. We now generate a Data URL for the blob and use it to create the worker.

The only part of this procedure that is difficult is getting the code into the workerCodeString String. Getting a multi-line string complete with any escape characters into a JavaScript variable isn't easy. Fortunately there is an easier way.

If you take all of the code that was in the separate JavaScript file and surround it by a function declaration:

function workerCode(){
  all the code that would have gone in the file
  completely unmodified
}

Notice that you put all of the code into the function ignoring the fact it is within a function. Next you can get a string version of the code using the workerCode function's toString() method. So we can get the String we need using:

var workerCodeString = workerCode.toString();

Almost, but not quite. We don't want the workerCode string in the Worker thread we want all of the code that is defined within it so we use the usual trick of immediately executing the workerCode function by wrapping it in () and following it with () to execute it:

var workerCodeString = "(" + workerCode.toString()
                                + ")()";

Now when the thread receives the code it will immediately execute the function and all of the code it contains will be either executed or added to the global environment. Notice that top level variables declared in the code aren't global, but are accessible to all of the functions due to closure.

This is a completely general procedure.

1. Create a function workerCode or whatever you want to call it.

2. Write your thread code within the workerCode function as if it was executing at the global level.

3. Get the code into a string using workerCode.toString(), but wrap it in () and follow it by (). That is "("+workerCode.toString()+")();".

4. Convert the String to a Blob.

5. Get the Data URL of the Blob.

6. Pass the Data URL to the Worker constructor.

For example the previous pi computation can be loaded directly from the same file as the UI thread using:

var workerCodeString = "(" + workerCode.toString()
      + ")()";
var blob = new Blob([workerCodeString],
              {type: 'application/javascript'});
var url = URL.createObjectURL(blob);


var worker = new Worker(url);

worker.addEventListener("message",
  function (event) {
    result.innerHTML = event.data.value;
  } );

  worker.postMessage("start");
    setInterval(function(){
       worker.postMessage("update");
    },100);}

  function workerCode() {
   var pi = 0;
   this.addEventListener("message",
     function (event) {
       switch (event.data) {
         case "start":
           computePiAsync();
           break;
         case "update":
           postMessage(pi);
           break;
         case "stop":
           close();
           break;
         }
   });

   function computePiAsync() {
     var computePi = genComputePi();
     function resume() {
       pi = computePi.next();
       if (!pi.done)
         setTimeout(resume, 0);
       if (pi.done)
         postMessage(pi);
    return;
   }
   setTimeout(resume, 0);
   return;
  }


  function* genComputePi() {
    var k;
    var pi = 0;
    for (k = 1; k <= 1000000; k++) {
     pi += 4 * Math.pow(-1, k + 1) / (2 * k - 1);
    if (Math.trunc(k / 1000) * 1000 === k)
         yield pi;
    }
    return pi;
  }
}

Transferable Objects

The way that Worker threads avoid many of the problems of multi-threading is by limiting the way threads can interact. They can only communicate by sending messages and, in particular, any data that is included in the message isn't shared, but copied. This means that each thread is always working on its own data and no object can be used by more than one thread at a time.

Copying data is safe, but if the data is large it can be slow. An alternative is to use transferable objects. These are not copied, but only one thread can access them at any given time. The idea is that the object belongs to just one thread and only that thread can access the object. Ownership of the thread can be passed to another thread by the object being included in a message sent from the current owner to another thread.

Transferable objects are only supported in the latest browsers and to a limited extent in IE 10 so they need to be used with caution. The mechanism only applies to three types of object - ArrayBuffer, MessagePort and ImageBitMap. This makes sense because these are all potentially large objects which are best not passed by copy.

To send a transferable object you simply follow the usual data object in postMessage by an array of transferable objects – IE10 is limited to a single object.

The way that transferable objects work is simple, but it can be confusing.

You pass data using the message object as usual which becomes the data property of the event object passed to the event handler. The data that you want to transfer has to be included either as the message object or a property of the message object. To be passed by transfer rather then clone the object also has to be listed in the transfer array.

So for example:

sendMessage(object,[object]);

will transfer object and the event handler will receive the objects as event.data.

Alternatively you could use:

sendMessage({mydata: object},[object]);

which will also transfer the object, but the event handler will receive it as event.data.mydata. Obviously you could pass additional properties and if these were not included in the transfer list they would be passed by cloning.

Once an object has been transferred it is no longer usable in the original thread. For example in the case of an ArrayBuffer its size is reduced to zero. Notice that once transferred its ownership cannot be simply transferred back because the reference that is passed is in event.data or event.data.myobject and not the original reference. That is, it is not a simple turning off and on of the original reference. You can arrange for this to happen, however.

We need a very simple example and to do this we need to work with an ArrayBuffer. An ArrayBuffer is a raw collection of bytes and you cannot access it directly. It has to be converted into a typed array before you can access its data.

However, we can simply create a raw buffer and pass it back and forth between the main thread and the worker thread as an example without worrying about what data it contains:

var worker = new Worker("transfer.js");
var arrayBuf = new ArrayBuffer(8);
console.log("UI before transfer " +  
                       arrayBuf.byteLength);
worker.postMessage(arrayBuf, [arrayBuf]);
console.log("UI after transfer" +
                       arrayBuf.byteLength);

This simply creates an eight byte ArrayBuffer and transfers it to the Worker thread. When the program is run you see:

UI before transfer 8
UI after transfer 0

which indicates that the ArrayBuffer is no longer available in the UI thread.

<ASIN:1871962560>

<ASIN:1871962579>

 <ASIN:1871962528>

<ASIN:1871962501>