Chain of trust with AI generated code

../cot

2026-06-22

Chain of trust with AI generated code

Introduction

Chain of trust has long served as a foundational security principle in computing systems. From secure boot architectures to software supply-chain[1] security, trust is established at a known root and propagated through a sequence of verifiable components. As with the increased influence of Agentic AI based code generation, a fundamental question emerges: can traditional Chain-of-Trust concepts be extended?<br>This document argues that while provenance, auditability and attestation can be established for AI-generated artifacts, constructing a traditional chain of trust for them is fundamentally difficult.<br>An LLM introduces probabilistic reasoning, non-deterministic outputs, opaque decision-making, and gaps between human intent and generated implementation that weaken the assumptions upon which traditional trust propagation depends.

The core principle of establishing chain of trust is based on the fact that "trust" as a measurable entity can be propagated through a system in the case of Computer Engineering, software and hardware components from the origin which is the "root of trust". In the chain of trust model each stage has to verify the integrity and authenticity of the next before transferring control.

Traditional software engineering concepts such as,

Secure boot processes.<br>Trusted platform module.<br>Code signing.<br>Reproducible builds.<br>Deployment pipelines.

rely on the principle that trust can be established at known root and propagated through a sequence of verified components.

The effectiveness of these architectures depends on the following assumptions,

Components can be uniquely identified.<br>Integrity can be cryptographically verified.<br>Behavior is deterministic and predictable.<br>Trust can be inherited from previously verified components.<br>Verification outcomes are objective and reproducible.

The above assumptions have largely been true for traditional hardware and software as the components of the system have remained deterministic and their integrity can be verified through cryptographic mechanisms.

This raises an important question:<br>Can trust be propagated through AI-generated artifacts in the same way that it is propagated through traditional software systems?

Idea of trust

In computing systems the idea of chain-of-trust originates from a root of trust.<br>A simplified representation for a secure boot system is as below.

"Hardware Root of trust -> Bootloader -> Operating Systems -> Application"

Each component verifies the integrity and authenticity of the next component before transferring control.

In this architecture, trust assertions become viable because<br>Components are deterministic.<br>Verification criteria are objective.<br>Trust can be inherited.

Therefore trust is established through lineage. i.e., if the root is trusted and every component successfully verifies the next component, the resulting system can be trusted.

Extending the chain of trust to AI systems, a hypothetical chain-of-trust for AI-generated software may appear as:

"Human Intent -> Prompt -> Planning agent -> Research Agent -> Coding Agent -> Testing Agent -> Deployment Pipeline -> Production System"

The fundamental difference in computing with an AI-system is it doesn't implement a pre-determined logic like a bootloader, there are non-deterministic layers such as<br>interpretation, reasoning, transformation and generation.

With a fundamental shift in the approach to developing software, the trust baseline changes from

"Can we verify this component to establish trust?" to

"Can we verify that this component correctly interpreted and implemented the intent of the previous component?"<br>This distinction introduces significant challenges.

AI-systems possible marriage?

An AI system can provide increasing levels of provenance like,

Model version

Prompt history.

Agent actions.

Tool invocations.

Generated artifacts.

Test results.

Security scans.

The above information creates an auditable record of how an artifact was produced.

A future AI deployment could include:

Signed prompts.

Verified models.

Attested inference environments.

Encrypted signed outputs.

Immutable audit logs.

The above mechanisms create a verifiable chain of custody, using which we get answers to questions such as.

Which model was used?

Which prompts were used?

Which tests were executed?

Which deployment process approved the artifact?

which are necessary for governance and compliance, however provenance is not trust.<br>Provenance answers where an artifact came from.<br>Trust answers whether the artifact deserves confidence.

Towards Divorce?

AI systems violate several assumptions upon which trust propagation depends.

The Intent gap

Consider the example of using a prompt like "Implement secure authentication.", with the Skills insertion and agentic workflows setup.<br>When a prompt as such is met with the workflow of different agents each...