Measuring the Trustworthiness of Open-Source-Derived Models | Cognition<br>Menu

Close

Key Takeaways#

We built an evaluation suite to assess model trustworthiness. This suite combines direct questioning to check for propaganda output, as well as realistic coding scenarios to ensure that model behavior remains constant across users and contexts.

We ran these evaluations on a range of models, including Kimi K2.7 Code, the open-source base model from which SWE-1.7 was developed. These evaluations allowed us to assess model trustworthiness, identify faults, and track improvements.

The model we developed, SWE-1.7, performs as well or better on our trustworthiness evaluation suite than models from leading U.S.-based frontier labs, despite having taken an open-source model as our starting point.

We are still actively developing and building our trustworthiness evaluation suite. But our initial results indicate that models developed from open-source models can be trusted, provided that sufficient thought and care is put into their development.

Motivation#

The low cost and broad availability of open-source models are key for innovation, and these models power everything from research prototypes to production coding agents. Developing new models from open-source starting points, however, presents a distinct challenge. Many of the strongest open-source models, such as Kimi K2.7 Code, DeepSeek-V4, and GLM 5.2, come from AI labs based in mainland China. Models from these labs raise two concerns. First, studies have found these models often repeat Chinese Communist Party ("CCP") aligned narratives considerably more than their American counterparts2,7. This is perhaps unsurprising, because these labs are subject to regulations that require the content they generate to "adhere to core socialist values."6 Second, in agentic settings, at least two studies claim that some of these labs’ models may produce less secure code depending on the user and use context1,3. Thus, any decision to use an open source model—even when developing a new model—requires careful consideration.<br>At Cognition, we build autonomous software engineers that major financial institutions, Fortune 500 companies, government agencies, and other critical institutions trust with their codebases. Thus, when developing our latest model, SWE-1.7, by applying large-scale reinforcement learning on top of an open-source model, we were determined to ensure that it did not exhibit the negative behaviors similar to the above. In other words, we needed to be sure this new model could be trusted.<br>To this end, we made several deliberate choices. As our starting point, we selected Kimi K2.7 Code because, among the open-source models we assessed, it showed both strong coding ability and strong neutrality. We then took additional measures during SWE-1.7’s development to improve neutrality further, beyond the K2.7 baseline. Finally, we tested the resulting model in several different settings, measuring both what it says under direct questioning and how it behaves when writing code in realistic scenarios inside our production harness, the only environment in which SWE-1.7 is deployed. This second type of test is especially important for avoiding "evaluation awareness"; a model may answer more carefully in obvious test scenarios, but reveal its true behavior in a more ordinary setting.<br>Our trust evaluations come in two parts capturing both of these settings:<br>Propaganda and censorship. Following Pan and Xu (2026)2, we probe models with 145 politically sensitive questions (five samples each, in English, Simplified Chinese, and Traditional Chinese) and grade every response along six axes, such as active propaganda rate and factual accuracy.

Security and vulnerabilities. We measure whether models comply with politically motivated requests (for example, building an unauthorized mass-surveillance system), and whether their capabilities change depending on who they appear to be working for.

As the results below show, SWE-1.7 performs comparably or even favorably to models from U.S. frontier labs on these evaluations, and improves substantially over the base Kimi K2.7 Code model. This suggests that open-source models are not inherently unsafe: with targeted post-training and other techniques, they can be made at least as safe as leading closed models. These experiments provide an initial framework for measuring a model’s neutrality and trustworthiness, and we plan to develop these benchmarks further in the coming months.

Propaganda and Censorship Eval#

First, we considered propaganda. Chinese open-source models often echo CCP-aligned talking points. Audits of DeepSeek give a sense of what this looks like in practice. NIST’s Center for AI Standards and Innovation found that DeepSeek models echoed four times as many inaccurate and misleading CCP narratives as U.S. reference models7, and Promptfoo found that DeepSeek-R1 answered roughly 85% of 1,360 prompts on CCP-sensitive topics...