Mint State Machine Hardening Changelog

Date: June 10, 2026 Status: 🚧 In progress Plan: planning/mint-state-machine-hardening/{adr,impl}.md

Overview

Hardening pass over the emprops-api mint_request pipeline following the on-chain reconciliation work (39f90dc95). Fixes three audit findings in the reconciliation commit itself, then closes the structural trust gaps: test coverage for the money-moving cron branches, double-mint protection via a transaction-submission journal, axes-based metrics, and visibility for refund-owed rows.

Key Changes

Phase 0: Audit fixes to the reconciliation commit

File: apps/emprops-api/src/crons/mint-request-tick.ts

• Free mints no longer misclassified by the admin-retry truth check. Free mints (creator's first NFT, usdc_tx_hash IS NULL) have no on-chain Payment record, so getPayment returns none — the truth check read that as "never tagged" and downgraded retryable free mints to detected, a shape recovery cannot complete without a USDC transfer. The on-chain truth check is now skipped for free mints, which go straight to the attempt-cap + mintTo(recipient) retry.
• Never-tagged downgrades get a real recovery window. The downgrade to detected now bumps expires_at by 7 days. Without it, the detected-TTL sweep saw the stuck row's long-past expires_at and abandoned it on the very next tick — before anyone could run recover-payment.
• The tagged + not_started + not_started dead-end shape is swept. Rows that paid and tagged on-chain but stalled before generation (e.g. a transient RPC failure in executePaidMint's verification) previously sat invisible forever: no cron sweep, no user badge, chain-consistent to the reconciler. They now promote to the user-refundable shape after TTL + grace.
• Tick reentrancy guard. A tick now performs per-row on-chain reads and can exceed the 20s interval; overlapping ticks could double-submit mintTo (free mints have no contract-side double-mint protection). Overlapping ticks are skipped with a warning.
• Honest heal metrics. On-chain heals now emit dedicated healed_consumed / healed_refunded outcomes instead of inflating succeeded, and heal counts feed the tick log.

Phase 1: Cron test coverage

File: apps/emprops-api/src/crons/mint-request-tick.test.ts (new)

20 tests covering every on-chain truth-check branch (consumed/refunded heals, never-tagged downgrade with its recovery window, no-write-on-read-failure, tagged fall-through), the retry mechanics (free vs paid mintTo overloads, attempt counting, exhaustion), all five expiry sweeps, and generation finalization dispatch. Mocked at the I/O boundary only; state-machine writes are asserted end-to-end.

Phase 2: Reconciler fairness + terminal-row exclusion bug

File: apps/emprops-api/src/lib/mint-reconciler.ts

• Bug found: {...ACTIVE_PENDING_WHERE, NOT: {...}} silently replaced the spread's NOT array with the admin-retry exclusion, so the reconciler scanned terminal rows — and ordered oldest-first, the same ancient completed rows would monopolize every 25-row batch. In production the reconciler would likely never have corrected anything. Fixed with AND composition.
• Starvation fix: batches now walk a wrap-around keyset cursor on payment_id, so every non-terminal row is visited within ⌈N/25⌉ passes; a short batch wraps the sweep back to the top.

Phase 3: mintTo submission journal (double-mint protection)

Files: apps/emprops-api/src/lib/nft/submit-mint-journaled.ts (new), src/clients/tx-queue-client.ts, src/lib/mint-finalizer.ts, src/crons/mint-request-tick.ts

If a mintTo mined but the process died before the DB write, the retry path resubmitted — the contract blocks paid double-mints, but free mintTo(recipient) would mint a duplicate NFT. Now every mintTo journals its Engine queueId to step_data.mint_submission before waiting, and every retry path first asks the engine what became of any prior attempt: mined → heal to completed with its txHash (no resubmission, bypasses the attempt cap — nothing is being submitted), in-flight → skip the pass, errored/cancelled/stale → resubmit. A status read failure on a fresh journal is always treated as in-flight: a flaky read must never cause a double-submit.

Also fixed: the generating-TTL and awaiting-mint-TTL sweeps clobbered step_data wholesale, destroying recovery breadcrumbs (generation job_ids, payment evidence) — and they sat on exactly the path a crashed-mid-mint row takes, so the journal would not have survived without the fix.

Phase 4: Metrics on the axes

Files: apps/emprops-api/src/lib/mint-request-predicates.ts, src/crons/mint-request-tick.ts

The Dash0 gauges and state logs grouped by the legacy status column, which the axis-native write paths no longer maintain — dashboards were drifting toward fiction. New shapeOf() maps every axis triple to a named shape (names reuse the legacy vocabulary for label-for-label alert migration), feeding:

• mint_request.shape_count{shape=…} — the partition dashboards/alerts should read; shape="unmapped" is a canary for writer-invented shapes
• mint_request.refund_required_count — abandoned rows holding verified user funds (kept separate so shape_count stays a clean partition)

Legacy state_count/stale_count keep emitting, marked DEPRECATED, and are removed after the Dash0 alert migration proves out.

Phase 5: refund_required visibility

Files: apps/emprops-api/src/routes/nft/mint-request-admin.ts, src/index.ts, apps/docs/src/guides/operations/MINT_REFUND_REQUIRED_RUNBOOK.md

refund_required was write-only: set by the detected-TTL sweep, read by nothing — money owed to users was flagged and then invisible. Now:

• GET /nft/admin/mint-pipeline/refund-required — oldest-debt-first work queue with amounts and a totalOwed rollup
• mint_request.refund_required_count gauge (Phase 4) for the Dash0 check
• Operator runbook covering verification, the on-chain decision tree (consumed → don't refund; tagged → prefer re-mint), store credit vs treasury transfer, and close-out

Money movement stays human-approved by design (ADR D4): never-tagged payments have no on-chain Payment record, so refundPayment cannot target them.

Also: the admin force-retry endpoints (single + bulk) had bypassed the Phase 3 submission journal — both now resolve prior submissions first (409 on in-flight, heal on already-mined), and the single force-retry gained the free-mint mintTo(recipient) overload it was missing.

North Star Alignment

Reliable mint completion is a prerequisite for elastic generation workloads: every stuck mint_request is a paid generation the platform owes and re-runs waste pool capacity. Bulletproof state convergence keeps the job pipeline's inputs trustworthy.

Upcoming Phases

• Phase 1: unit tests for retryAwaitingMint / expirePending / finalizeGeneratingRows
• Phase 2: reconciler keyset cursor (fairness)
• Phase 3: mintTo submission journal (lost-confirmation / double-mint protection)
• Phase 4: mint_request.shape_count axes metrics, legacy status gauges deprecated
• Phase 5: refund_required metric + admin endpoint + runbook
• Phase 6: staging verification against replicated stuck-cohort shapes