How We Made Cloud Browsers 3x Cheaper and Faster

← All Posts

Our cloud browsers need to do three things at once: start quickly, remain isolated, and be cheap. That is why we rebuilt Browser Use Cloud, so a new session starts in under a second and costs $0.02 per browser hour, down from $0.06.

This is harder than it sounds. A browser has Chromium, a filesystem, cookies, cache, proxy settings, downloads, and sometimes a logged-in customer session. If one browser can read another browser's state, it creates a security problem.

The normal answer is a virtual machine, or VM. A VM is a computer inside a computer: it gets its own CPU, memory, disk, and network devices. It is separate from everything else on its host, and if the browser breaks, leaks information, or gets attacked, the damage stays within the VM.

Normal VMs, however, are too heavy for cloud browsers. We need to create them constantly, sometimes thousands at a time, and throw them away as soon as sessions end. If each browser needs a slow, expensive VM, the product becomes slow and expensive, too.

The question for us is whether we could give every browser its own VM without making users wait or pay for it. We now do that with Firecracker, a lightweight VM system.

Every Browser Use Cloud session runs in its own, tiny VM. These VMs run on EC2, Amazon's rented cloud servers.

That is the unusual part. Firecracker is normally run on bare-metal servers, where you rent the whole physical machine. To reduce customers' cost, we run it on regular EC2, where AWS has already put your server inside a VM.

This should be slow. Nested VMs make memory and CPU operations more expensive, and Chromium takes time to start. This post is about how we made this setup fast and efficient.

But first, why did we rebuild our infrastructure?

It is difficult to be fast, isolated, and cheap all at once.<br>Why we left unikernels behind

We used to run cloud browsers with Unikraft , which builds small, VM-like containers called unikernels . Unikernels, instead of booting a full Linux system, load a small image built for your purposes. Unikernels start quickly and are cheap when idle because you can shut them down when no one is using them.

Unikraft was good for turning browsers off when they were not in use, but bad at adding more browsers quickly when traffic spiked. If more users suddenly asked for browsers at once, you would need to scale browser capacity rapidly. Unikraft does not have good built-in autoscaling, so an engineer had to change a variable, manually adding more instances.

During a burst in traffic, the system, instead of reacting on its own, required humans to adjust it. This caused problems: one load test brought down production for 45 minutes. So we rebuilt our setup on Firecracker.

Unikraft needed an engineer to add capacity by hand, so it lagged behind a traffic spike and collapsed. After the rebuild, capacity tracks demand automatically.<br>Firecracker provides a layer through which you can create, monitor, and run VMs. It gives each VM CPU, memory, disk, and network devices, and it keeps it isolated from the host and from other VMs.

Teaching browsers to scale themselves

Firecracker gave each browser its own VM. But it did not inherently solve the problem that broke the old system: deciding how many VMs to run, where to put them, and when to add more.

So we built our own control plane . The control plane monitors our fleet of browsers and decides whether we should scale up or down.

When a user asks for a browser, the control plane picks a machine with room. When traffic rises, it starts more machines. When traffic falls, it stops sending new browsers to machines we want to remove.

It checks the fleet in real time. That is much faster than waiting on CloudWatch, AWS's monitoring service, which usually reacts on one-minute windows. It also knows things generic metrics do not: browsers that are still starting, machines we are trying to remove, and machines that should not receive new sessions.

A request flows from user code through stateless edge routers; the control plane picks an EC2 host with room and keeps draining hosts out.<br>Why we run VMs inside VMs

Once we had a control plane, the next question was what kind of machines it should add.

The usual way to run Firecracker on AWS is a .metal instance. This means you rent the whole physical server, and Firecracker runs directly on it.

We chose regular EC2 instead. Regular EC2 machines are faster to get and cheaper to keep around. Our hosts boot from a pre-built image and start serving browsers about 30 seconds after launch. The faster we can add a host, the less idle capacity we need to pay for, and the lower the cost we pass on to our customers.

The catch is that regular EC2 is already a VM. AWS runs our host inside its own isolation layer, and then we run browser VMs inside that host. In other words, every browser is a VM inside a VM.

This is not the normal way of using Firecracker. When a...