It’s not unusual for a Firefox test to have to wait for various things such as a tab loading. But recently I needed to write a test that loaded a content tab with a web worker and wait for that before observing the result in a different tab. I am writing this for my own reference in the future, and if it helps someone else, that’s extra good. But I don’t think it will be of much interest if you don’t work on Firefox as the problem I’m solving won’t be relevant and the APIs won’t be familiar.
I don’t think of myself as a JavaScript programmer - I’m learning what I need to know when I need to know it, but mainly to write tests. So I’m not sure I’ll pitch this article at any particular level of JS knowledge, sorry.
Web Workers
Web Workers provide web pages a way to execute long-running JavaScript tasks in a separate thread, where it won’t block the main event loop. They solve the same problem, allowing a page to use concurrency. However their programming model is more like processes because they don’t share state (global variables or even functions) and communicate by sending and receiving messages.
I realise this is a tangent but it’s a topic I like and you may have the
same questions I did:
So if workers are supposed to solve the same problems as threads do in other
languages, why are they more like processes?
Furthermore, at least in Firefox, each worker instantiates another copy of
the JavaScript engine (the JSRuntime class) with its own instantiation of
JIT, GC etc.
Isn’t this fairly heavy just to add concurrency?
It is, but there are benefits:
-
I’m not certain, but I think this was the easiest way to retrofit concurrency to JavaScript (the language standard) without breaking backwards compatibility with existing web sites.
-
Message-passing concurrency makes the boundary between threads very clear. This makes it a simpler programming model, especially if you’re working on some code that is isolated from the concurrency happening elsewhere.
-
It worked for Erlang, although Erlang likely shares bytecode caches and some other systems. But not garbage collection.
Anyway, the point is that Web Workers are concurrent "process like" things that communicate through message-passing.
about:performance
Firefox has a number of about: pages, used for diagnostics and tweaking.
about:config is probably the most infamous (if you touch those settings
you can break your browser or make it insecure).
about:support is interesting too it contains diagnostic information about
Firefox on your computer.
Today we’re looking at about:performance, which is useful when you are
thinking "Firefox seems slow, I wonder why..". about:performance will
show your busiest tabs, how much CPU time/power and memory they’re using.
Measuring memory usage can be tricky at the best of times
(more on this in an upcoming article).
We can’t afford to count every allocation since that is too slow for a page
like about:performance. Although about:memory comes closer to doing
this.
For about:performance we can ask major subsystems how much
memory they’re using and rely on their counters.
This isn’t accurate but it’s good enough.
I noticed two major things that weren’t counted:
-
Malloc memory used by JS objects was not counted.
-
Web workers were not counted.
I fixed them in Bug 1760920.
So I wanted to write a test that would verify that we are indeed counting memory belonging to web workers.
My web worker
To make it easier to see if we’re counting a component’s memory, it’s great of our test causes that component to use a lot of memory then we can test for that.
Here’s a Web Worker that uses about 40MB of memory using an array with 4 million elements.
var big_array = [];
var n = 0;
onmessage = function(e) {
var sum = 0;
if (n == 0) {
for (let i = 0; i < 4 * 1024 * 1024; i++) {
big_array[i] = i * i;
}
} else {
for (let i = 0; i < 4 * 1024 * 1024; i++) {
sum += big_array[i];
big_array[i] += 1;
}
}
self.postMessage(`Iter: ${n}, sum: ${sum}`);
n++;
};
It registers an onmessage event hander. When the page sends it a message it
will execute the anonymous function.
The first time this happens the function will create the array, the next
time it will manipulate the array.
Since the array is a global and is also
captured by the handler I doubt the GC would free it.
But I also don’t want an optimiser (now or in the future) from reducing the
whole program to a large summation, or caching an answer.
Which is why the array is manipulated each time the event handler is called.
It doesn’t matter that it’s ridiculous - it’s a test - just that it uses
"enough" memory.
From the main page it can be started like this:
var worker = new Worker("workers_memory_script.js");
worker.postMessage(n);
But that’s not enough to make a working test.
The test
Our test needs to open this page in one tab, and in another tab look at
about:performance and observe that the memory is being used.
Opening and managing multiple tabs and is standard faire for a browser test,
but what we need is for our test to wait for the tab with the worker to be
/ready/.
Waiting for a tab to be loaded is also very easy, which means that the tab
will have executed worker.postMessage(n) by the time the test code checks.
But that doesn’t mean that the worker has received the message.
So we need to make our test wait for the worker to start and complete one iteration (creating its array).
In the test we can add code such as:
let tabContent = BrowserTestUtils.addTab(gBrowser, url);
// Wait for the browser to load the tab.
await BrowserTestUtils.browserLoaded(tabContent.linkedBrowser);
// For some of these tests we have to wait for the test to consume some
// computation or memory.
await SpecialPowers.spawn(tabContent.linkedBrowser, [], async () => {
await content.wrappedJSObject.waitForTestReady();
});
The last three lines here are the interesting ones. SpecialPowers.spawn
allows us to execute code in the context of the tab. In which we wait on a
promise that the test is ready.
Now we need to add this promise to the page that owns the worker:
var result = document.querySelector('#result');
var worker = new Worker("workers_memory_script.js");
var n = 1;
var waitPromise = new Promise(ready => {
worker.onmessage = function(event) {
result.textContent = event.data;
ready();
// We seem to need to keep the worker doing something to keep the
// memory usage up.
setTimeout(() => {
n++;
worker.postMessage(n);
}, 1000);
};
});
worker.postMessage(n);
window.waitForTestReady = async () => {
await waitPromise;
};
Starting at the bottom.
For some reason I had to wrap the promise up in a function, I can’t remember
why!
I’m tempted to complain about JavaScript and it’s inconsistent rules here,
but it could also be my limited understanding preventing me from getting it.
What I do know is that this function must be in the window object so that
the test code above can find it in wrappedJSObject.
The promise wrapped here (waitPromise I could have picked a better name)
is resolved when ready() is called, which happens after we receive the
worker’s response.
Finally we use setTimeout() to post another message to keep memory usage
up.
I don’t know why this was necessary either. Was the worker completely
terminated without it?
One more thing
Our test almost works.
For whatever reason when the test accesses the right part of the
about:performance page there’s no value for how much memory is being used.
Waiting for a single update fixes this:
if (!memCell.innerText) {
info("There's no text yet, wait for an update");
await new Promise(resolve => {
let observer = new row.ownerDocument.ownerGlobal.MutationObserver(() => {
observer.disconnect();
resolve();
});
observer.observe(memCell, { childList: true });
});
}
let text = memCell.innerText;
For the complete code for this test checkout Bug 1760920 and toolkit/components/aboutperformance/tests/browser.
There’s things I don’t know
There’s three places here where I’ve said "it needs this code, I don’t know why". I hate programming like this, and I feel shameful writing it in a blog post and calling myself an engineer. I don’t want to spin it as a joke on JavaScript, or myself "lol, that’s programming! AMIRITE?!" There’s obviously some further subtleties I don’t know the rules for, and JavaScript does have some pretty inconsistent rules, throw in a browser, two tabs and a web worker and feeling like you don’t know how something works is relatable.
Do I wish I knew? Sure, I’m uncomfortable not knowing, but I’ve already spent enough time on this. But this is also why I wrote down what I do know. Next time I’ll be able to find this much and solve my problem quicker.