GLM-5.2: Considerations for enterprise teams starting out with open-weight models
Artificial Intelligence
·<br>08 July 2026<br>·
5 min read
GLM-5.2: Considerations for enterprise teams starting out with open-weight models
Robat Williams,
Chris Price
Setting yourself up to try out open-weight models for agentic development isn’t difficult, but it isn’t as straightforward as downloading a coding agent from one of the handful of well-known AI vendors. In preparation for the latest round of our AI productivity experiments, we’ve recently been through this process. Read on for the choices we made, the considerations at play, and what made our situation unusual.
Our AI experiments
Our small team is undertaking a series of A/B tests in which developers use different AI development models/tools to complete tasks. These are pre-selected issues from a different open source project for each round of the experiment. Through quantitative and qualitative measures, we seek to capture a well rounded picture of using these tools on real work - not benchmarks nor ad-hoc trials.
The open-weight contestant
With the eagerly awaited open-weight vs. Claude Code round of the experiment nearing, we set out to pick the things we needed for an open-weight agentic developer setup.
The key parts we knew we needed from the outset were:
The model
How and where to run the model
The coding agent software
Model
Going into this selection, we identified two universal considerations:
Likely competitiveness against frontier commercial models
Practicalities of running the model for the experiment - locally, or hosted
“Open-weight” models are sometimes conflated with “local” models - which to be correct and precise, are open-weight models that are small enough to run on a developer’s own high specification computer. We quickly ruled those out due to benchmark results showing them to be less capable than even GPT-5 Mini (which we’ve previously found to be poor).
We shortlisted the top open-weight models according to benchmarks, and spent a couple of hours ad-hoc trialling them: DeepSeek V4 Flash & Pro, GLM-5.2, and Kimi K-2.6. All appeared to be adequately capable. Our final choice of GLM-5.2 was swayed by:
Seen to be more capable than DeepSeek V4 Pro on challenging tasks we were familiar with from previous rounds of the experiment.
Seen to ask pertinent questions during task planning, where other models would either not ask, or not even realise there was an ambiguity or a choice to be made.
Better performance on benchmarks, indicating higher capability.
All the shortlisted models were from top Chinese labs. Those available from US/EU labs were not near competitive on benchmarks - Google’s Gemma, OpenAI’s gpt-oss, Meta’s Llama, nor Mistral (France).
Model hosting
As we would be working on open-source projects for the experiment, we had considerable freedom on this aspect that many real use cases wouldn’t have. The only thing we ruled out up front was buying or renting our own hardware to run the model - we wanted a convenient “model as a service”, just like is provided by the well-known AI labs via their APIs.
None of the usual big name providers (e.g. AWS, Azure, Cloudflare) offered our chosen model, although some do offer older versions of various shortlisted ones (e.g. GLM-5, DeepSeek V3.2). For organisations who typically prefer major suppliers they already have relationships with, and who can provision in preferred countries to meet data sovereignty and legal requirements, this will likely be a sticking point. However, for our experiment, it’s of no concern, and something we have to accept to work with a competitive model.
Whilst we expect this situation to improve in future, for now, GLM-5.2 is currently only hosted by relatively small companies, based in the US, China, and Singapore. Rather than creating accounts directly with each of these smaller companies, you can access models hosted by each of these companies via centralised marketplaces instead.
OpenRouter is one such marketplace for hosted models, where hosting companies (providers) offer many different models. The “router” part routes your model calls to a suitable provider based on some criteria, which you can customise. You pay OpenRouter, which takes a small cut, and pays the underlying hosting company for the actual model calls. Having used it for experimenting with different models, we decided it was sufficient for use in the actual experiment too - purely out of practical convenience. We created an organisation, topped up our credits, and invited our team colleagues.
We’re less sure that a model routing service/marketplace will form part of most organisations’ settled approach to open-weight models, so we decided to partially align to that direction by selecting a single underlying provider on OpenRouter. For similar reasons, we also opted for a provider which did not make use of submitted data for model training.
To make sure that we...