Mutation Testing Resources

Core introductions

• Mutation Testing (Google Testing Blog, 2021) – short, practical explanation of why a large engineering org uses mutation testing and what it changes in day-to-day development.
• The Power of Mutation Testing (2024) – modern, readable overview of concepts and benefits.
• Publications section of Awesome Mutation Testing – if you want the full academic background, start here.

Rust

Our main language focus. These are the tools we consider most relevant if you write or study Rust code.

• cargo-mutants – source-level mutation testing for Rust. Mutates your code and re-runs cargo test. Great reference for CLI design, operator choices, and JSON outputs.
• mutagen – experimental mutation testing for Rust using attributes. Interesting for exploring alternative operator and integration strategies.
• Rust section in Awesome Mutation Testing – for more context and related links.

Solidity & smart contracts

Smart contracts are a natural fit for mutation testing: small programs, high stakes. These are the key tools to know.

• Certora/gambit – a mutation system for Solidity used to evaluate both tests and formal verification specifications.
• RareSkills/vertigo-rs – actively maintained Rust implementation of Vertigo with Foundry support. Mutates contracts and runs forge test to produce a mutation score.
• JoranHonig/vertigo – original Vertigo tool for mutation testing of smart contracts.
• Solidity Mutation Testing (RareSkills) – practical overview of how Solidity mutation tools work and where they fit into a security workflow.

Key papers (short list)

For a deeper understanding, these are good entry points from the much longer bibliography in Awesome Mutation Testing.

• A general survey such as Mutation Testing Advances: An Analysis and Survey – covers core ideas, operators, and research directions.
• An empirical study like Are Mutants a Valid Substitute for Real Faults in Software Testing? – addresses how well mutants approximate real bugs.
• A smart-contract–focused work such as RegularMutator: A Mutation Testing Tool for Solidity Smart Contracts – shows how mutation operators look in the Solidity domain.
• For more, see the Publications section and follow the citation chains.

Talks & videos

If you prefer to see mutation testing in action, these are good starting talks.

• GOTO 2019 – Making Mutants Work for You (Henry Coles) – live demos and a clear explanation of why mutation testing improves test suites.
• GOTO 2015 – Mutation Testing in Python (Austin Bingham) – language-specific, but great for understanding general concepts and tooling.
• More links are under Blogs/Posts and Videos in the Awesome list.

From Mutorium Labs

These are our own resources that build on the tools and papers above, especially in the context of Web3 and ZK.

• What Is Mutation Testing? A Practical Introduction – our long-form explanation of the concept, aimed at practitioners.
• Swiss Cheese Model of Web3 Security – how mutation testing fits into a layered defence model for smart contracts and ZK systems.
• The Mutorium Labs Book of Mutation Testing – a living, programmer-focused book; chapters are published as blog posts first.
• noir-metrics (GitHub) – metrics for Noir/Nargo projects, forming the basis for future coverage and mutation testing work on ZK circuits.