{"id":211936,"date":"2026-08-11T15:52:42","date_gmt":"2026-08-11T15:52:42","guid":{"rendered":"https:\/\/mycryptomania.com\/?p=211936"},"modified":"2026-08-11T15:52:42","modified_gmt":"2026-08-11T15:52:42","slug":"before-the-result-could-count-the-benchmark-had-to-freeze","status":"publish","type":"post","link":"https:\/\/mycryptomania.com\/?p=211936","title":{"rendered":"Before the Result Could Count, the Benchmark Had to Freeze"},"content":{"rendered":"

The previous post<\/a> reported the execution result. This companion documents the evaluation contract behind\u00a0it.Overview.<\/strong> The single closed-loop Rust environment shared by every policy in the execution study. Schedule baselines, tuned lookahead, planners, and the DQN all act through the same state, action space, pool mechanics, market-response operators, and completion accounting. Within each step: agent trade \u2192 noise routing \u2192 arbitrage \u2192 oracle advance \u2192 fee update \u2192 next state. The schematic uses agent-first ordering; the headline block uses agent-last (noise and arbitrage before the agent trade). The fee-closure block produces the next-state fees after oracle advance. Schematic only\u200a\u2014\u200anot live-chain telemetry.<\/p>\n

Beating time-weighted average price (TWAP) was not enough. The two-hidden-layer DQN, which maps sixteen market-state features to eight execution actions, also had to beat tuned one-step control without future leakage, unfinished inventory, or feedback from the final\u00a0seeds.<\/p>\n

That required one transition engine, common completion accounting, tuned baselines, role-separated seeds, and frozen checkpoints. The implementation, manifest, and per-seed outputs live in egpivo\/amm-lab<\/a> under data\/rl_equilibrium\/<\/a>. The simulator and result are in the previous post<\/a>; this post covers the five checks behind\u00a0them.<\/p>\n

One transition engine<\/h3>\n

The learner and baselines must face the same transition law, and the observation cannot contain future information.<\/p>\n

Market physics live in one Rust execution environment (). Python never reimplements the step. The DQN calls the same binary over a JSON-lines bridge; every heuristic and planner calls the same step function.<\/p>\n

Decision-time fees are snapshotted before any leg runs. Fees update only after the oracle advances. Intra-step priority<\/strong> -whether the agent\u2019s trade runs before or after noise flow and arbitrage within the step-is a declared parameter, not an implementation accident.<\/p>\n

A cargo test whitelists every field on the observation vector. Add a field without review and the test fails. Same seed \u2192 identical episode outcome. Drift + slippage-ex-fee + fee + gas + terminal term must equal reported implementation shortfall to\u00a01e-6.<\/p>\n

The learner also sees less<\/em> than the tuned baseline: heuristics read the full observation struct; the DQN reads sixteen scaled features. The information asymmetry runs against the\u00a0learner.<\/p>\n

Why Rust and\u00a0Python<\/h3>\n

Rust world model:<\/strong> pools, noise flow, arbitrage, fee updates, completion accounting, baselines, planners, and tabular Monte\u00a0Carlo.Python learner:<\/strong> PyTorch DQN, replay buffer, hyperparameter sweeps, and checkpoint selection.<\/p>\n

Rust keeps simulation deterministic and semantically single. Python keeps network, optimizer, reward-scaling, and exploration changes\u00a0cheap.<\/p>\n

The PyTorch path is rl_equilibrium_bridge<\/a> \u2192 \u2192 dqn_train.py<\/a>. Tabular Monte Carlo stays in Rust, where thousands of episodes finish in seconds. Rewriting or in Python would duplicate the transition law and create another place for train\/evaluation drift.<\/p>\n

rl_equilibrium_bridge<\/a> is a thin stdin\/stdout JSON layer. Rust holds state and computes reward; Python sends reset and step, then reads observation, reward, and completion. Tabular runs never cross the\u00a0bridge.<\/p>\n

Fig. 1.<\/strong> Development diagnostic. Waiting varies with the oracle gap; routing varies with the buy-fee gap. Not the headline\u00a0block.<\/p>\n

The behavior is economically legible, though this diagnostic does not establish optimality.<\/p>\n

A baseline worth\u00a0beating<\/h3>\n

TWAP, myopic routing, and an untuned heuristic are weak comparison points.<\/p>\n

The bar is tuned one-step lookahead<\/strong>. It rebuilds exact quote curves and scores each action using immediate execution premium plus a carry for unfinished inventory. The carry multiplier is selected on validation seeds; the frozen value is 16. Immediate liquidation and myopic routing are still reported.<\/p>\n

On the final block under agent-first ordering, paired differences relative to lookahead are +10.8 bps<\/strong> for two-step expectimax, +1.6<\/strong> for three-step expectimax, +0.1<\/strong> for stochastic rollout, and \u221214.9<\/strong> for the DQN (final_ladder.csv<\/a>, m3r_stochastic_planner_final.csv<\/a>). Positive values are worse; negative values are better. Stochastic rollout ties lookahead, while the DQN reduces shortfall. Realized fee and quote variation may contribute, but the experiment cannot separate that mechanism from shallow search and heuristic continuation.<\/p>\n

The learner rung also includes model-free tabular Monte Carlo control at two discretizations. The fine table ties tuned lookahead; the coarse table does not. That comparison is consistent with a discretization bottleneck in the coarse learner rather than an absence of sequential structure.<\/p>\n

Hindsight coordinate descent over full action sequences provides an achieved reference only. It consumes realized future shocks and is never a deployable policy.<\/p>\n

Strict completion<\/h3>\n

An execution policy should not win by leaving inventory unfinished.<\/p>\n

Under forced terminal completion-the headline setting-any inventory left after the agent\u2019s final action is executed at the terminal state. Completion is 1.0<\/strong> for every policy. No controller wins by leaving the book\u00a0open.<\/p>\n

Reward is the negative normalized execution premium: maximizing undiscounted return is identical to minimizing reported implementation shortfall. The terminal term is part of the same accounting identity: a penalty under the standard rule, and actual forced execution cost under the headline rule. It is never removed from reported shortfall or added as a post-hoc correction.<\/p>\n

Freeze the seed\u00a0blocks<\/h3>\n

Seeds used for training, selection, diagnostics, and the headline need distinct\u00a0roles.<\/p>\n

Training, development, fresh-check, and final headline blocks are isolated by seed range. Learner validation ( 20,000-20,049) is declared upfront as a nested subset of the broader baseline-validation block ( 20,000-20,199):<\/p>\n

Train: <\/strong>episode index 1,000,000 + ep (fresh path per\u00a0episode)Baseline validation: <\/strong>seeds 20,000-20,199 (lookahead urgency weight; planner\u00a0grid)Learner validation: <\/strong>seeds 20,000-20,049 (DQN checkpoint pick at episode 11,000-not the final\u00a0epoch)Development test:<\/strong>seeds 30,000-30,499 (ladder figures, behavior diagnostics)Fresh check: <\/strong>seeds 40,000-40,499 (one-shot reproduction)Final headline: <\/strong>seeds 90,000-90,999 (all frozen policies; paired confidence intervals vs lookahead)Fig. 2.<\/strong> Seed roles from training through final evaluation. The headline block is read only after design and checkpoint freeze.<\/p>\n

The completed manifest at data\/rl_equilibrium\/m3r_run_manifest.json<\/a> records Python\/torch\/rustc versions, hash, git commit, command list, the pre-final design choices, and sha256 of every checkpoint and result CSV. Because it hashes final-result CSVs, the manifest is assembled after those files are produced. The policies and evaluation choices it records were frozen before the final block. make -C scripts\/rl_equilibrium verify<\/a> runs verify_paper_artifacts.py<\/a> against the recorded contract.<\/p>\n

Headline (final block only):<\/strong> forced completion, agent trade after noise and arbitrage, n = 1,000\u200a\u2014\u200aDQN 100.3 bps<\/strong> vs lookahead 113.6 bps<\/strong>, paired \u221213.29 bps<\/strong> [\u221214.22, \u221212.32]. Agent trade before noise and arbitrage: \u221214.9 bps<\/strong>. Randomized intra-step ordering is the smallest edge: \u22125.62 bps<\/strong> [\u22127.03,\u00a0\u22124.18].<\/p>\n

That ordering spread has a loose analogue in recent centralized-exchange-DEX (CEX-DEX) work on priority fees and stochastic settlement delays ( Bergault, Hafsi & S\u00e1nchez-Betancourt, arXiv:2602.10798<\/a>): execution speed, uncertainty, and cost trade off. It is not the same model as the simulator\u2019s trade-ordering parameter; the basis-point numbers are ours. The paper helps motivate treating intra-step priority as a finance parameter rather than a bookkeeping detail.<\/p>\n

Transfer is not re-solving<\/h3>\n

Retraining under a perturbation does not show that the original checkpoint generalized.<\/p>\n

Perturbations change the transition law. The protocol separates:<\/p>\n

Frozen transfer<\/strong>\u200a\u2014\u200aevaluate the frozen checkpoint under the perturbed environment without retrainingRe-solving<\/strong>\u200a\u2014\u200aretrain under the perturbed environment on the pre-specified set (fee mode, intra-step priority)<\/p>\n

On development seeds, moving an agent-first checkpoint to agent-last execution shrinks its paired edge from \u221215.5<\/strong> to \u22126.3 bps<\/strong> (m3r_priority.csv<\/a>). After retraining under agent-last ordering, the final-block edge is \u221213.3 bps<\/strong> against matched lookahead (m3r_final_paper_seeds.csv<\/a>). The first comparison measures transfer; the second re-solves the changed environment.<\/p>\n

Scalar nuisance perturbations-gas level, arbitrage speed, noise scale, fee coefficients-are transfer-only. None flip the ranking on the frozen checkpoint.<\/p>\n

Fee-mode re-solving (a 300-seed subset of the development-test block, default bridge ordering-agent trade before noise and arbitrage):<\/strong> when trained and evaluated under constant fees, the paired DQN-lookahead estimate is +1.2 bps<\/strong> (means round to 116.1 and 115.0 bps), a near tie rather than evidence of an advantage. Dynamic monopoly: \u221223.2 bps<\/strong>. Dynamic duopoly: \u221214.7 bps<\/strong>. Each row pairs the retrained DQN against tuned lookahead in the same fee environment (m3r_dynamic_fee_ablation.csv<\/a>). The edge concentrates in dynamic-fee environments. These are ablation results, not final-block headline estimates.<\/p>\n

Fig. 3.<\/strong> Priority transfer and retraining on development seeds. Diagonal cells use matched train\/evaluation ordering; off-diagonal cells test transfer. Source: m3r_priority.csv<\/a>.<\/p>\n

Two optional sensitivity layers default off in the headline freeze. On separate 500-seed blocks, neither layer flips the DQN-lookahead ranking at baseline or weak stress (m4_lp_adaptation.csv<\/a>, m4_jit_mev.csv<\/a>). Aggressive threshold-based sandwich-searcher stress raises shortfall for both policies but leaves the gap\u00a0intact.<\/p>\n

Fig. 4.<\/strong> Optional appendix layers on 500-seed sensitivity blocks (LP: 95,000\u201395,499; JIT: 96,000\u201396,499), not the 90,000\u201390,999 headline block. Left: liquidity-provider depth adaptation. Right: threshold-based sandwich-searcher stress. Sources: m4_lp_adaptation.csv<\/a>, m4_jit_mev.csv<\/a>. Both are explicit stress models, not claims about live LP or searcher behavior.<\/p>\n

What the contract does not\u00a0certify<\/h3>\n

The contract supports a model-conditioned comparison with tuned one-step routing under forced completion. It does not establish live profitability, actual mempool ordering, adaptive liquidity-provider behavior, or equilibrium of the fee\u00a0rule.<\/p>\n

If I rebuilt this study next, resolving intra-step priority and adaptive liquidity-provider response would take priority over network\u00a0width.<\/p>\n

Closing<\/h3>\n

The result counted only after the transition law, completion rule, baseline, checkpoint, and final seed block stopped moving. The next test is a richer transition law with mempool ordering and adaptive liquidity-provider repositioning before adding network capacity.<\/p>\n

Appendix: sources and reproducibility<\/h3>\n

Paper:<\/strong> Wang, Wen-Ting (2026), Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator<\/a>.Execution study:<\/strong> A Large DEX Order Is a Sequential Control\u00a0Problem<\/a>.Code, manifest, and artifacts:<\/strong> egpivo\/amm-lab<\/a>. Simulator: src\/sim\/<\/a>. Training and evaluation: scripts\/rl_equilibrium\/<\/a> (gym_env.py<\/a>, dqn_train.py<\/a>, verify_paper_artifacts.py<\/a>). Frozen outputs: data\/rl_equilibrium\/<\/a> (m3r_run_manifest.json<\/a>).External reference:<\/strong> Bergault, Hafsi & S\u00e1nchez-Betancourt (2026), Trading in CEXs and DEXs with Priority Fees and Stochastic Delays<\/a><\/p>\n

This post was originally published on my personal blog: <\/em>https:\/\/egpivo.github.io\/2026\/08\/11\/before-the-result-could-count-the-benchmark-had-to-freeze.html<\/em><\/a>.<\/em><\/p>\n

Before the Result Could Count, the Benchmark Had to Freeze<\/a> was originally published in Coinmonks<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>","protected":false},"excerpt":{"rendered":"

The previous post reported the execution result. This companion documents the evaluation contract behind\u00a0it.Overview. The single closed-loop Rust environment shared by every policy in the execution study. Schedule baselines, tuned lookahead, planners, and the DQN all act through the same state, action space, pool mechanics, market-response operators, and completion accounting. Within each step: agent trade […]<\/p>\n","protected":false},"author":0,"featured_media":211937,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-211936","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-interesting"],"_links":{"self":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts\/211936"}],"collection":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=211936"}],"version-history":[{"count":0,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts\/211936\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/media\/211937"}],"wp:attachment":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=211936"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=211936"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=211936"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}