What we learned from our Inference Experiment
Engineering
Inference Experiment: What we learned after one week<br>August 17, 202610 min read
Back to Overview
T - 4 months: introducing Hetzner Experiments Platform with OpenClaw<br>T - 1 month: getting specialized hardware<br>T - 2 weeks: benchmarking and picking the right models<br>T - 1 week: final preparations<br>T - 0 hours: launch<br>T + 6 hours: hitting capacity limits<br>T + 1 day: coping with the demand<br>T + 2 days: more adjustments, more hardware<br>T + 7 days: taking stock and scaling down the experiment
In this article
T - 4 months: introducing Hetzner Experiments Platform with OpenClaw
T - 1 month: getting specialized hardware
T - 2 weeks: benchmarking and picking the right models
T - 1 week: final preparations
T - 0 hours: launch
T + 6 hours: hitting capacity limits
T + 1 day: coping with the demand
T + 2 days: more adjustments, more hardware
T + 7 days: taking stock and scaling down the experiment
Close
TL;DR<br>The experiment proved to be a great success.<br>The design of our inference API and the hardware behind it were generally sound, and provided very good results.<br>The demand for inference proved much greater than we expected. To meet the high demand for large models, we will need further improvements.<br>While we make those improvements, the experiment will continue on a smaller scale, with smaller models.
We recently launched our Inference Experiment: an inference API that allows you to use Large Language Models (LLMs) which we host on our infrastructure. You can connect any agent, harness, or coding tool of your choice to it, and use our LLM free of charge. This way, you can try out LLMs running on powerful hardware, while we gain experience running such models.<br>Now that the Inference Experiment is one week old, we would like to give you some insights into how it works, some of the things we learned, and what the future for it might look like. To do this, we’d like to show you a timeline of the Hetzner Experiment Platform and all experiments on it so far.
T - 4 months: introducing Hetzner Experiments Platform with OpenClaw<br>Ever since we’ve been offering GPU servers to our customers, we have tested ways of serving LLMs internally. With many years of Kubernetes experience, we were able to build an internal inference platform, which we used to test different server configurations and LLM-serving software.<br>We eventually settled on a stack consisting of a few GEX131 servers, vLLM and Open WebUI. This combination allowed us to run a few smaller LLMs, which we could use for some internal production-grade applications. It also allowed us to trial and benchmark larger models internally.<br>At the start of 2026, OpenClaw became a much-hyped topic in the AI space. Many enthusiasts were using Hetzner Cloud VMs to try out OpenClaw, but struggled with safely configuring the server for it. To help our customers with this challenge, we decided to offer OpenClaw as a preconfigured instance.<br>We quickly built a new platform, which we could use to offer not only OpenClaw, but also other services and products we might want to experiment with in the future. It would allow rapid iteration and experimentation, while coming with fewer guarantees than our more mature products. One core feature would be the pricing: we would initially offer all Experiments completely for free.<br>OpenClaw was also an opportunity to gain experience with self-hosted inference. Our OpenClaw came preconfigured with our own inference API. The API was not publicly available but was limited to OpenClaw, so we could control the request volume to a degree. By offering inference only via OpenClaw, we could try out some smaller open-weight models, in various configurations. And our customers would get an AI product that was completely EU-based.<br>To show how unique the Hetzner Experiments Platform is, we built a separate interface panel for it at experiments.hetzner.com.<br>This limited OpenClaw Experiment gave us lots of insights:<br>Small models like Qwen 3.6 proved sufficient for most OpenClaw-based use cases.<br>GEX131 with its single RTX PRO™ 6000 Blackwell GPU was sufficient for small LLMs, but couldn’t run large models in a meaningful way.<br>Lots of users praised the fact that this was entirely EU-based.<br>OpenClaw’s code quality was a constant problem: Each new OpenClaw release introduced new challenges, as the volume of upstream bugs was quite high.
T - 1 month: getting specialized hardware<br>At Hetzner, our hardware teams are constantly working on new server models. One such model was a new custom server, which houses eight RTX PRO™ 6000 Blackwell GPUs in a single server, instead of the single GPU of a GEX131. This would be much better suited for running larger models.<br>With some of these 8-GPU servers, we could offer larger models to our customers. The OpenClaw Experiment had already shown that our customers are interested in AI topics and inference. However, the software quality of OpenClaw remained a challenge.<br>To...