QA dashboard monitoring smart contract\u00a0state<\/p>\n
The previous post<\/a> walked through an end-to-end implementation: a minimal token contract, off-chain state reconstruction, and a React frontend\u200a\u2014\u200aall the way from `mint()<\/em>` to MetaMask. This post picks up where that left off: how do you QA something like\u00a0this?<\/p>\n I\u2019m not a blockchain engineer (yet), but QA patterns port well across domains, and borrowing what already works elsewhere is how I learn\u00a0fastest.<\/p>\n The contract only does three things: `mint<\/em>`, `transfer<\/em>`, and `burn<\/em>`, but even that is enough to practice the full QA toolchain: static analysis, mutation testing, gas profiling, formal verification.<\/p>\n The code is in `egpivo\/ethereum-account-state<\/a>`.<\/p>\n Blockchain QA Pyramid: from static analysis at the base to formal verification at the\u00a0top<\/p>\n Before adding anything new, the project already\u00a0had:<\/p>\n 21 Foundry unit tests<\/strong> covering each state transition (success, revert on illegal input, event emission)3 invariant tests<\/strong> via a `TokenHandler<\/em>` that runs random sequences of `mint`\/`transfer`\/`burn` on 10 actors (128k calls\u00a0each)Fuzz tests<\/strong> checking `sum(balances) == totalSupply<\/em>` for random\u00a0amountsTypeScript domain tests<\/strong> (Vitest<\/em>) mirroring the on-chain state\u00a0machineCI<\/strong>: compile, test, lint (Prettier<\/em> +\u00a0solhint<\/em>)<\/p>\n All tests passed. Coverage looked fine. So why bother with\u00a0more?<\/p>\n Because \u201call tests pass\u201d does not mean \u201call bugs are caught.\u201d 100% line coverage can still miss a real bug if no assertion checks the right\u00a0thing.<\/p>\n Slither<\/a>(Trail of Bits) catches issues that are invisible to tests: reentrancy, unchecked return values, interface mismatches.<\/p>\n .\/scripts\/run-qa.sh slither<\/p>\n Result: 1 Medium finding: <\/strong>`erc20-interface<\/em>`: `transfer()<\/em>` doesn\u2019t return\u00a0`bool<\/em>`.<\/p>\n This is expected. The contract is intentionally not a full ERC20: it is an educational state machine. But the finding is not academic:<\/p>\n USDT\u2019s `transfer()<\/em>` famously does not return `bool` either, and that non-compliance has caused real integration failures<\/a> in DeFi protocols that assumed standard ERC20 behavior.<\/p>\n If someone later imports this token into a protocol expecting ERC20, the interface mismatch would silently fail. Slither flags it now so the decision is conscious.<\/p>\n .\/scripts\/run-qa.sh coverageCoverage result.<\/p>\n One uncovered function: `BalanceLib.gt()<\/em>`. We will come back to\u00a0this.<\/p>\n forge coverage output: 24 tests passed, Token.sol coverage\u00a0table<\/p>\n .\/scripts\/run-qa.sh gas<\/p>\n Baseline gas costs for the three operations:<\/p>\n Gas in terms of operations<\/p>\n On subsequent runs, `forge snapshot\u200a\u2014\u200adiff<\/em>` compares against the baseline. A 20% gas regression in `transfer()<\/em>` is a real cost to every user\u200a\u2014\u200acatching it before merge is\u00a0cheap.<\/p>\n This is where things got interesting. Gambit<\/a>(Certora) generates mutants<\/em><\/strong>: <\/em>copies of `Token.sol<\/em>` with small deliberate bugs (`+=<\/em>` to `-=<\/em>`, `>=<\/em>` to `><\/em>`, conditions negated). The pipeline runs the full test suite against each mutant. If a mutant survives (all tests still pass), that is a concrete test\u00a0gap.<\/p>\n .\/scripts\/run-qa.sh mutation<\/p>\n Result: 97.0% mutation score\u200a\u2014\u200a<\/strong>32 killed, 1 survived out of 33\u00a0mutants.<\/p>\n Gambit\u2019s output log shows each mutant and what it changed. A few examples:<\/p>\n Generated mutant #7: BinaryOpMutation \u2014 Token.sol:168 Generated mutant #19: RelationalOpMutation \u2014 Token.sol:196 Generated mutant #28: SwapArgumentsMutation \u2014 Token.sol:81 The surviving mutant swapped `a > b<\/em>` to `b > a<\/em>` in `BalanceLib.gt()<\/em>`. No test caught it because `gt()<\/em>` is dead code<\/strong>. It is never called anywhere in `Token.sol<\/em>`.<\/p>\n Coverage flagged 91.67% functions but could not explain the gap. Mutation testing did: `gt()<\/em>` is dead code, nothing calls it, and nobody would notice if it were\u00a0wrong.<\/p>\n Dead or unprotected code in smart contracts has real precedent.<\/p>\n In 2017, an unprotected `initWallet()<\/em>` function in the Parity multisig library was called by an outside user, who then triggered `kill()<\/em>`\u200a\u2014\u200apermanently freezing over $150M<\/a> across 500+\u00a0wallets.<\/p>\n The function was not intended to be callable, but nobody tested that assumption. Our `gt()<\/em>` is harmless by comparison, but the pattern is the same: code that exists but is never exercised is code that nobody is watching.<\/p>\n Halmos<\/a>(a16z) reasons about all possible inputs<\/em> symbolically. Where fuzz tests sample random values and hope to hit edge cases, Halmos proves properties exhaustively.<\/p>\n .\/scripts\/run-qa.sh halmos<\/p>\n Result: 9\/9 symbolic tests pass<\/strong>\u200a\u2014\u200aall properties proven for all\u00a0inputs.<\/p>\n Properties verified:<\/p>\n Verified properties<\/p>\n One practical note: Halmos 0.3.3 does not support `vm.expectRevert()<\/em>`, so I could not write revert tests the normal Foundry way. The workaround is a try\/catch pattern\u200a\u2014\u200aif the call succeeds when it should revert, `assert(false)<\/em>` fails the\u00a0proof:<\/p>\n function check_mint_reverts_on_zero_address(uint256 amount) public { Not the prettiest, but it works\u200a\u2014\u200aHalmos still proves the property for all inputs. This is the kind of thing you only find out by actually running the\u00a0tool.<\/p>\n For context on why formal verification matters:<\/p>\n the 2016 DAO hack exploited a reentrancy pattern that drained ~$60M and led to Ethereum\u2019s hard\u00a0fork<\/a>.<\/p>\n The vulnerability was in the code, reviewable by anyone, but no tool or test caught it before deployment. Symbolic provers like Halmos exist precisely to close that gap\u200a\u2014\u200athey do not sample; they exhaust the input\u00a0space.<\/p>\n Halmos output: 9 tests passed, 0 failed, symbolic test\u00a0results<\/p>\n The test file is `contracts\/test\/Token.halmos.t.sol<\/a>`.<\/p>\n The first post\u2019s architecture has a TypeScript domain layer that mirrors the on-chain state machine. This phase tests whether the two actually\u00a0agree.<\/p>\n I added fast-check<\/a> property tests for the TypeScript domain layer, mirroring what Foundry\u2019s fuzzer does for Solidity:<\/p>\n npm test – tests\/unit\/property.test.ts<\/p>\n Result: 9\/9 property tests pass<\/strong> after fixing a real\u00a0bug.<\/p>\n Properties tested:<\/p>\n `Balance<\/em>`: commutativity, associativity, identity, inverse, comparison consistency`Token<\/em>`: invariant `sum(balances) == totalSupply<\/em>` under random operation sequences (200 runs, 50 ops\u00a0each)`Token<\/em>`: `totalSupply<\/em>` non-negative after random sequences`mint<\/em>` always succeeds for valid\u00a0inputs`transfer<\/em>` preserves `totalSupply<\/em>`<\/p>\n fast-check found a real cross-layer consistency bug in `Token.ts<\/em>` `transfer()<\/em>`. The shrunk counterexample was immediately clear:<\/p>\n Property failed after 3 tests Self-transfer (`from == to<\/em>`) broke the `sum(balances) == totalSupply<\/em>` invariant. `toBalance<\/em>` was read before<\/em> `fromBalance<\/em>` was updated, so when `from == to<\/em>`, the stale value overwrote the deduction:<\/p>\n \/\/ Before (buggy) Fix: read `toBalance<\/em>` after writing `fromBalance<\/em>`, matching Solidity\u2019s storage semantics:<\/p>\n \/\/ After (fixed) The Solidity contract was not<\/strong> affected: it re-reads storage after each write. But the TypeScript mirror had a subtle ordering dependency that no existing unit test\u00a0covered.<\/p>\n Cross-layer mismatches at larger scale have been catastrophic.<\/p>\n The 2022 Wormhole bridge hack ($320M)<\/a> exploited a gap between off-chain guardian validation and on-chain verification\u200a\u2014\u200athe two layers disagreed on what constituted a valid signature, and the attacker walked through the\u00a0gap.<\/p>\n Our self-transfer bug would not have lost anyone money, but the failure mode is structurally the same: two layers that are supposed to agree,\u00a0don\u2019t.<\/p>\n Running QA tools on an existing project is never just \u201cinstall and run.\u201d A few things broke before they\u00a0worked:<\/p>\n 0% coverage because `foundry.toml<\/em>` had no test path<\/strong>: The first `forge coverage<\/em>` run returned 0% across the board. Turns out `foundry.toml<\/em>` did not specify `test = \u201ccontracts\/test<\/em>\u201d` or `script = \u201ccontracts\/script<\/em>\u201d`, so Forge was not discovering any tests. The coverage command succeeded silently\u200a\u2014\u200ait just had nothing to cover. This was the most misleading failure: a green run with no useful\u00a0output.`InvariantTest<\/em>` import gone in forge-std v1.14.0<\/strong>: `Invariant.t.sol<\/em>` imported `InvariantTest<\/em>` from `forge-std<\/em>`, which was removed in a recent release. Compilation failed with an opaque \u201csymbol not found\u201d error. The fix was to drop the import\u200a\u2014\u200a`Test` alone is sufficient for Foundry\u2019s invariant testing\u00a0now.`uint256(token.totalSupply())<\/em>` vs `Balance.unwrap()<\/em>`<\/strong>: Tests were using an explicit cast to extract the underlying `uint256<\/em>` from the user-defined `Balance<\/em>` type. It compiled, but it is the wrong idiom\u200a\u2014\u200a`Balance.unwrap(token.totalSupply())<\/em>` is what the UDVT system is designed for. Applied across `Token.t.sol<\/em>`, `Invariant.t.sol<\/em>`, and `DeploySepolia.s.sol<\/em>`.<\/p>\n Everything runs through two\u00a0scripts:<\/p>\n scripts\/setup-qa-tools.sh<\/a>`: installs Slither, Halmos, Gambit (idempotent)`scripts\/run-qa.sh<\/a>`: runs checks, saves timestamped results to `qa-results\/<\/em>`.\/scripts\/run-qa.sh slither gas # just static analysis + gas Not every check is fast. Slither and coverage run on every commit. Mutation testing and Halmos are slower\u200a\u2014\u200abetter suited for weekly or pre-release runs.<\/p>\n Blockchain QA Toolchain: what each layer catches\u200a\u2014\u200afrom static analysis to cross-layer property\u00a0testing<\/p>\n Five QA layers, each catching a different class of\u00a0problem.<\/p>\n Layer explanation<\/p>\n Gambit and fast-check gave the most actionable results in this\u00a0round.<\/p>\n The QA checks are now wired into GitHub Actions as a six-stage pipeline:<\/p>\n CI Pipeline: Build & Lint fans out to Test, Coverage, Gas, Slither, and Audit\u00a0stages<\/p>\n GitHub Actions pipeline: Build & Lint gates all downstream stages.<\/p>\n Stage explanation<\/p>\n Ethereum Account State source: [github.com\/egpivo\/ethereum-account-state](https:\/\/github.com\/egpivo\/ethereum-account-state)Previous post: Ethereum Account\u00a0State<\/a>Slither: github.com\/crytic\/slither<\/a>Gambit: github.com\/Certora\/gambit<\/a>Halmos: github.com\/a16z\/halmos<\/a>fast-check: github.com\/dubzzz\/fast-check<\/a>Foundry: getfoundry.sh<\/a><\/p>\n This post is adapted from my original blog\u00a0post<\/a>.<\/p>\n Ethereum Account State: QA Pipeline for a Minimal Token<\/a> was originally published in Coinmonks<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>","protected":false},"excerpt":{"rendered":" QA dashboard monitoring smart contract\u00a0state The previous post walked through an end-to-end implementation: a minimal token contract, off-chain state reconstruction, and a React frontend\u200a\u2014\u200aall the way from `mint()` to MetaMask. This post picks up where that left off: how do you QA something like\u00a0this? I\u2019m not a blockchain engineer (yet), but QA patterns port well […]<\/p>\n","protected":false},"author":0,"featured_media":148719,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-148718","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-interesting"],"_links":{"self":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts\/148718"}],"collection":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=148718"}],"version-history":[{"count":0,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts\/148718\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/media\/148719"}],"wp:attachment":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=148718"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=148718"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=148718"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}What we started\u00a0with<\/strong><\/h3>\n
Phase 1: Smart contract static analysis and\u00a0coverage<\/strong><\/h3>\n
Slither<\/strong><\/h4>\n
Coverage<\/strong><\/h4>\n
Gas snapshots<\/strong><\/h4>\n
Phase 2: Mutation testing and formal verification<\/strong><\/h3>\n
Mutation testing\u00a0(Gambit)<\/strong><\/h4>\n
totalSupply = totalSupply.add(amountBalance) \u2192 totalSupply = totalSupply.sub(amountBalance)
KILLED by test_Mint_Success<\/p>\n
if (!fromBalance.gte(amountBalance)) \u2192 if (fromBalance.gte(amountBalance))
KILLED by test_Transfer_Success<\/p>\n
return Balance.unwrap(a) > Balance.unwrap(b) \u2192 return Balance.unwrap(b) > Balance.unwrap(a)
SURVIVED \u2190 no test caught thisGambit mutation testing: 32 killed, 1 survived, mutation score\u00a097.0%<\/p>\nFormal verification (Halmos)<\/strong><\/h4>\n
vm.assume(amount > 0);
try token.mint(address(0), amount) {
assert(false); \/\/ should not reach here
} catch {
\/\/ expected revert – Halmos proves this path is always taken
}
}<\/p>\nPhase 3: Cross-layer property\u00a0testing<\/strong><\/h3>\n
Property-based testing with fast-check<\/strong><\/h4>\n
The bug fast-check found<\/strong><\/h4>\n
Shrunk 2 time(s)
Counterexample: transfer(from=0xaaa\u2026, to=0xaaa\u2026, amount=1n)
\u2192 from == to (self-transfer)
\u2192 verifyInvariant() returned false<\/p>\n
const fromBalance = this.getBalance(from);
const toBalance = this.getBalance(to); \/\/ \u2190 stale when from == to
this.accounts.set(from.getValue(), fromBalance.subtract(amount));
this.accounts.set(to.getValue(), toBalance.add(amount)); \/\/ \u2190 overwrites the subtraction<\/p>\n
const fromBalance = this.getBalance(from);
this.accounts.set(from.getValue(), fromBalance.subtract(amount));
const toBalance = this.getBalance(to); \/\/ \u2190 now reads updated value
this.accounts.set(to.getValue(), toBalance.add(amount));<\/p>\nPitfalls hit along the\u00a0way<\/strong><\/h3>\n
Pipeline design<\/strong><\/h3>\n
.\/scripts\/run-qa.sh mutation # just mutation testing
.\/scripts\/run-qa.sh all # everything<\/p>\nSummary<\/strong><\/h3>\n
CI pipeline<\/strong><\/h3>\n
References<\/strong><\/h3>\n
Notes<\/h3>\n