Hardening STB libraries using AI agents | Mr.UNIX<br>Hardening STB libraries using AI agents<br>stb is a collection of single-header C libraries used for a variety of tasks, such as decoding images, fonts, audio, and many other things. However, these libraries have a history of security issues, and many of them are not addressed quickly, making them unsuitable for environments where untrusted input is expected.<br>Meanwhile, in the last few months, there has been a lot of discussion around the use of LLMs for security research, especially after the introduction of Claude Mythos by Anthropic, followed by the short-lived Claude Mythos 5.<br>And while I don&rsquo;t have access to Mythos, I still believe currently available LLMs can be useful for security research and hardening, so here I&rsquo;m gonna use them to harden stb libraries.<br>The agent flow<br>After some experiments, I was able to come up with an agentic setup that allows us to discover and fix security vulnerabilities, you can check out the AGENTS.md used for this.<br>The main advantage of this flow is that it ensure that the bugs being found are real security issues, and not just false positives, by validating the bugs before patching.<br>This flow goes through 3 different phases:<br>Discovery<br>Validation<br>Patching<br>Phase 1: Discovery<br>In this phase, the agent searches for potential security issues and documents them in BUGS.md.<br>For the discovery process, there are three different techniques our agent can use:<br>Web search : The agent can look up known bugs and vulnerabilities online from different sources, such as GitHub issues and CVEs.<br>Fuzzing : The agent can use or implement tools and scripts for fuzzing to test the target against different sets of malicious inputs.<br>Static analysis : The LLM can read the source code and flag suspicious sections that may contain bugs or security issues.<br>Phase 2: Validation<br>Since LLMs are well known for hallucinations and for reporting security vulnerabilities that do not exist, it&rsquo;s generally a good idea to validate the bugs that have been flagged earlier, since we do not want to patch bugs that do not exist.<br>For that, we let the agent implement some automated tests (unit tests or E2E tests) that would let us validate both the vulnerability and the patch for it.<br>Phase 3: Patching<br>After we validate the bugs we discovered earlier, it&rsquo;s time to fix them.<br>Here, the agent can simply patch these bugs with the minimal amount of changes, then re-run the previously implemented tests to validate the fix.<br>The results<br>I have tested multiple AI agents, models and subscriptions, including Opencode, Codex, GitHub Copilot and Hermes.<br>Suprisingly, all of them gave impressive results, including the free models from Opencode.<br>At the time of writing, this is the number of bugs that have been found and fixed in stb:<br>LibraryTotal bugs foundweb-searchfuzzingstatic-analysisValid bugsstb_c_lexer.h1272312stb_easy_font.h50145stb_hexwave.h1254312stb_image.h30215421stb_image_resize2.h17131313stb_image_write.h94237stb_truetype.h361315834stb_vorbis.c1382311Total 134 59 34 41 115 Performance Impact<br>So I did some benchmarks to measure the performance impact of the patches that have been done, the performance regressions for these libraries was negligible, except for stb_truetype.h .<br>WorkloadOriginal meanHardened meanChangestb_image.h PNG decode, 512x512 RGB1.930 ms1.894 ms-1.9%JPEG decode, 512x512 RGB1.999 ms1.988 ms-0.6%PNG decode, 1024x1024 RGB7.041 ms6.930 ms-1.6%PNG decode, 512x512 RGBA conversion1.643 ms1.606 ms-2.3%16-bit PNG decode, 512x512 RGB1.503 ms1.431 ms-4.8%HDR decode, 512x512 RGB2.259 ms2.438 ms+7.9%stb_truetype.h Kerning advances, 95 glyph pairs0.459 ms0.472 ms+2.8%Glyph shapes, 95 glyphs0.010 ms0.011 ms+10.0%Bitmap rendering, 16px, 95 glyphs0.093 ms0.103 ms+10.8%Bitmap rendering, 64px, 95 glyphs0.378 ms0.500 ms+32.3%Bitmap rendering, 128px, 95 glyphs0.880 ms1.162 ms+32.0%SDF rendering, 16px, 94 glyphs6.348 ms6.219 ms-2.0%SDF rendering, 64px, 94 glyphs36.872 ms36.096 ms-2.1%Atlas packing, 16px, 94 characters0.110 ms0.124 ms+12.7%Atlas packing, 48px, 94 characters0.343 ms0.429 ms+25.1%stb_image_resize2.h uint8 sRGB upscale, 256x256 to 512x5120.837 ms0.835 ms-0.2%uint8 sRGB downscale, 1024x1024 to 512x5122.732 ms2.671 ms-2.2%float linear downscale, 512x512 to 256x2560.407 ms0.402 ms-1.2%BOX filter, 512x512 to 256x2560.192 ms0.190 ms-1.0%MITCHELL filter, 512x512 to 256x2560.422 ms0.418 ms-0.9%CLAMP edge mode, 256x256 to 512x5120.332 ms0.329 ms-0.9%4x upscale, uint8 sRGB0.794 ms0.791 ms-0.4%4x downscale, uint8 sRGB0.494 ms0.486 ms-1.6%stb_image_write.h PNG encode, 512x512 RGB (format suite)6.052 ms5.946 ms-1.8%JPEG quality 80 encode, 512x512 RGB3.293 ms3.296 ms+0.1%HDR encode, 512x512 RGB1.718 ms1.785 ms+3.9%PNG encode, 1024x1024 RGB20.641 ms20.613 ms-0.1%JPEG quality 80 encode, 1024x1024 RGB13.197 ms13.628 ms+3.3%HDR encode, 1024x1024 RGB6.672 ms7.216 ms+8.2%PNG compression level 8, 512x512 RGB6.130 ms6.237 ms+1.7%JPEG quality 90, 512x512 RGB3.971...