CS SYD - Announcing Mutation Testing in Haskell
Announcing Mutation Testing in Haskell
ERT<br>5 min
Date<br>2026-06-03
mutation testing
testing
haskell
Mutation testing is now generally available in sydtest. This is a major step towards a saner development workflow in the age of AI-generated code.
What is mutation testing?<br>Mutation testing aims to improve a test suite by automatically mutating code and asserting that the tests start failing.
Alternatively:<br>Mutation testing is like a type-system for your tests. It asserts that the tests test the code thoroughly.
Example<br>Consider this simple function:<br>code.sourceCode { white-space: pre; position: relative; }<br>pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }<br>pre > code.sourceCode > span:empty { height: 1.2em; }<br>.sourceCode { overflow: visible; }<br>code.sourceCode > span { color: inherit; text-decoration: inherit; }<br>div.sourceCode { margin: 1em 0; }<br>pre.sourceCode { margin: 0; }<br>@media screen {<br>div.sourceCode { overflow: auto; }<br>@media print {<br>pre > code.sourceCode { white-space: pre-wrap; }<br>pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }<br>pre.numberSource code<br>{ counter-reset: source-line 0; }<br>pre.numberSource code > span<br>{ position: relative; left: -4em; counter-increment: source-line; }<br>pre.numberSource code > span > a:first-child::before<br>{ content: counter(source-line);<br>position: relative; left: -1em; text-align: right; vertical-align: baseline;<br>border: none; display: inline-block;<br>-webkit-touch-callout: none; -webkit-user-select: none;<br>-khtml-user-select: none; -moz-user-select: none;<br>-ms-user-select: none; user-select: none;<br>padding: 0 4px; width: 4em;<br>background-color: #2a211c;<br>color: #bdae9d;<br>pre.numberSource { margin-left: 3em; border-left: 1px solid #bdae9d; padding-left: 4px; }<br>div.sourceCode<br>{ color: #bdae9d; background-color: #2a211c; }<br>@media screen {<br>pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }<br>code span.al { color: #ffff00; } /* Alert */<br>code span.an { color: #0066ff; font-weight: bold; font-style: italic; } /* Annotation */<br>code span.at { } /* Attribute */<br>code span.bn { color: #44aa43; } /* BaseN */<br>code span.bu { } /* BuiltIn */<br>code span.cf { color: #43a8ed; font-weight: bold; } /* ControlFlow */<br>code span.ch { color: #049b0a; } /* Char */<br>code span.cn { } /* Constant */<br>code span.co { color: #0066ff; font-weight: bold; font-style: italic; } /* Comment */<br>code span.do { color: #0066ff; font-style: italic; } /* Documentation */<br>code span.dt { text-decoration: underline; } /* DataType */<br>code span.dv { color: #44aa43; } /* DecVal */<br>code span.er { color: #ffff00; font-weight: bold; } /* Error */<br>code span.ex { } /* Extension */<br>code span.fl { color: #44aa43; } /* Float */<br>code span.fu { color: #ff9358; font-weight: bold; } /* Function */<br>code span.im { } /* Import */<br>code span.in { color: #0066ff; font-weight: bold; font-style: italic; } /* Information */<br>code span.kw { color: #43a8ed; font-weight: bold; } /* Keyword */<br>code span.op { } /* Operator */<br>code span.pp { font-weight: bold; } /* Preprocessor */<br>code span.sc { color: #049b0a; } /* SpecialChar */<br>code span.ss { color: #049b0a; } /* SpecialString */<br>code span.st { color: #049b0a; } /* String */<br>code span.va { } /* Variable */<br>code span.vs { color: #049b0a; } /* VerbatimString */<br>code span.wa { color: #ffff00; font-weight: bold; } /* Warning */<br>">canCastFireball :: Int -> Int -> Bool<br>canCastFireball level mana =<br>level >= 5<br>&& mana >= 10
with a corresponding test suite:<br>code.sourceCode { white-space: pre; position: relative; }<br>pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }<br>pre > code.sourceCode > span:empty { height: 1.2em; }<br>.sourceCode { overflow: visible; }<br>code.sourceCode > span { color: inherit; text-decoration: inherit; }<br>div.sourceCode { margin: 1em 0; }<br>pre.sourceCode { margin: 0; }<br>@media screen {<br>div.sourceCode { overflow: auto; }<br>@media print {<br>pre > code.sourceCode { white-space: pre-wrap; }<br>pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }<br>pre.numberSource code<br>{ counter-reset: source-line 0; }<br>pre.numberSource code > span<br>{ position: relative; left: -4em; counter-increment: source-line; }<br>pre.numberSource code > span > a:first-child::before<br>{ content: counter(source-line);<br>position: relative; left: -1em; text-align: right; vertical-align: baseline;<br>border: none; display: inline-block;<br>-webkit-touch-callout: none; -webkit-user-select: none;<br>-khtml-user-select: none; -moz-user-select: none;<br>-ms-user-select: none; user-select: none;<br>padding: 0 4px; width: 4em;<br>background-color: #2a211c;<br>color: #bdae9d;<br>pre.numberSource { margin-left: 3em; border-left: 1px solid #bdae9d; padding-left: 4px; }<br>div.sourceCode<br>{ color: #bdae9d; background-color: #2a211c; }<br>@media screen {<br>pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }<br>code span.al { color: #ffff00; } /* Alert */<br>code span.an { color: #0066ff; font-weight:...