CodeGateway Billing Precision & Metering Whitepaper

CodeGateway Billing Precision & Metering Whitepaper

This whitepaper documents CodeGateway's complete billing calibers: the units we keep books in, where rounding happens, how streaming requests are metered, how aborted requests settle, how tier markup is computed, and how top-ups and refunds post to your balance. Every caliber described here is the actual production behavior, with code anchors (file:symbol; line numbers as of 2026-10) and migration numbers as evidence. Past precision incidents are disclosed as they are — fixed, refunded, or still being tracked.

We only describe capabilities that are already live, never planned ones.

1. Units: an integer micro-dollar, two-layer system

CodeGateway never keeps books in floating point. The billing path has two unit layers, with a single source of truth (SoT) for conversion in backend/src/balance-units.ts:

Why 10⁸ and not 10⁴: one cent = 100,000,000 micro. After two incidents where the conversion factor was written as 10_000 (off by 10,000×) — see §8 — every crossing of the cents↔micro boundary must go through the constants and helpers in balance-units.ts; git grep MICRO_PER_CENT lists every conversion site. Large-amount arithmetic uses bigint to stay exact above 2⁵³ (balance-units.ts:27-29).

Three rounding rules, three jobs

The most important caliber in this paper (balance-units.ts:66-149):

Deductions no longer round through cents

After #2818, per-request deduction passes the raw costMicro and performs an atomic deduction against the micro ledger directly (workspace-balance.ts:210, deductWorkspaceBalanceMicro: a single SQL UPDATE guarded by WHERE balance >= cost). Sub-cent precision is never lost. The "per-request ceil to cents" step that once caused systematic over-charging no longer exists on the main path (workspace-balance.ts:31-32 comment).

2. Price sources and the cost formula

Two sources, kept consistent: the source of truth for model prices is D1 model_catalog / model_costs (managed via the admin portal); the hardcoded MODEL_PRICES in backend/src/pricing.ts is the billing fallback. CI (model-prices-parity.yml) enforces that every public model id exists and matches in both.

Cost formula (pricing.ts:1186-1201, calculateCostMicro):

usd       = (inputTokens × rate.input
           + outputTokens × rate.output
           + cacheCreationTokens × rate.cacheCreate
           + cacheReadTokens × rate.cacheRead) / 1,000,000
costMicro = Math.round(usd × markup × 10¹⁰)

Four token classes are priced separately (input / output / cache-write / cache-read), rates in USD per million tokens. The final rounding is Math.round — not a ceil; the code comment reads "no ceil overcharge". The legacy cents entry calculateCostCents uses Math.ceil and is kept only for old tests; production does not use it (pricing.ts:1204-1219).

Long-context surcharge (pricing.ts:1158-1183, applyLongContextSurcharge): when a request's total prompt tokens (input + cacheRead + cacheCreation) exceed the model's thresholdTokens, a triplet (thresholdTokens / inputMultiplier / outputMultiplier) multiplies the input, output, cache-read and cache-create rates (WHI-1387). Per-model thresholds share the same source as the published price table and D1 rates.

3. Tier markup and the 90-day rolling recalculation

Current tier table (tier-config.ts:73-117, DEFAULT_CONFIG; must reproduce the live operational_config row):

Since the "Plan 3" change (2026-10-05), token-side markup is flattened to 1.07× and monetization moved to a 2% top-up service fee (§7); tiers now primarily carry rate-limit ceilings (tier-config.ts:61-65). The source of truth is the D1 operational_config row (admin portal), cached in KV for 300s; the in-code default table is only a fallback for when the config plane is unreachable, and it must match the live row (tier-config.ts:56-60, the #2514 lesson).

The 90-day caliber: a rolling windowDays=90; spend = SUM(request_logs.cost_micro) — composed only of actual request costs. Top-ups do not move it (get90DaySpend in tier-config.ts:537-545, aggregating SUM(cost_micro); tier-writeback.ts:151 carries the same "top-ups do not change 90-day spend" note).

Two recalculation paths (tier-writeback.ts header):

• Recompute on top-up (recomputeTierOnTopup, triggered by Stripe webhooks): upgrade-only. If the should-be tier is higher, upgrade immediately; this path never demotes.

• Daily 02:00 sweep (dailyTierSweep): upgrades apply immediately; demotions are staged and only take effect after a 7-day grace (TIER_DEMOTION_GRACE_DAYS=7). If the target drops further during grace, the target updates but the clock does not reset (anti-abuse). If spend recovers during grace, the pending demotion clears.

Discounts (#940): per-tier and global promotional discounts are supported; when both are active the more favorable one to the user wins (tier-config.ts:20-42).

Config drift monitoring (tier-config-drift.ts, #2516): reads D1 directly, bypassing the admin worker's 200-wrapping, to distinguish "row exists but diverges from the fallback" (row_diverges) from "row missing" (row_missing). The module is read-only and never auto-repairs. A separate workspace pool-ledger drift item (a different class — not tier-config drift) is tracked as GitHub #3472 and disclosed in §8.

4. Streaming vs non-streaming metering

Streaming (SSE): createMeteringStream in metering/stream.ts parses upstream SSE events one by one and accumulates token usage incrementally:

• every message_delta fires onProgress — used for incremental reservation release (§5);

• message_stop fires onUsage for final settlement;

• a heartbeat comment is injected every 15 seconds to keep clients from timing out, and onHeartbeat keeps the reservation alive (#1356, preventing the Durable Object from reaping an in-flight stream's reservation).

Non-streaming: settles once from the response body's usage field, using the same calculateCostMicro formula. The two paths differ only in how usage is obtained; the cost formula and units are identical.

5. Reservation, abort paths, and error paths

The settlement lifecycle of a request:

1. Pre-deduction (preDeduct): before the pipeline runs, an estimated cost is reserved (computePreDeductEstimate in backend/src/preDeduct.ts:254: input priced from measured tokens, remaining headroom converted to max output tokens). Insufficient balance returns 402 with a "lower max_tokens to N" hint — computed with the same formula and the same measured input tokens as the reservation (#2536), so following the hint never re-trips the same 402.

2. Incremental release: each message_delta during streaming releases part of the reservation.

3. Settle exactly once (#2501): no matter how the stream ends — clean message_stop, a truncated upstream close, a pipe error, or a client abort — it settles exactly once, billing the usage accumulated so far (metering/stream.ts:23-50, metering/stream-settle.ts). Client disconnects are caught via the request signal, the only reliable disconnect net.

Honest disclosure: before #2501, aborted or errored streams posted no entries at all (neither balance_events nor request_logs) — we had already paid the upstream for those tokens but never billed the user. After the fix they settle normally on the tokens already streamed.

Error paths: the cost formula is usage-driven; an error response carries no usage and is therefore not billed (cost = 0). For example, 402s from our upstream wholesale credit exhaustion (CF AI Gateway code 2021) are all billed 0 in production (forensics recorded in upstream-error-rate-alert.ts:14). A failed stream also releases its rate-limit TPM reservation, leaving no phantom occupant (#1668).

6. Pool wallet and the ledger mirror

• Ledger (fact layer): balance_events on the user layer and workspace_balance_events on the workspace layer, plus the micro balance caches (users.balance_cache_micro, workspaces.balance_usd_micro). All integer micro; since D2 (#2860) the workspace micro column is the write-side authority.

• Wallet (workspace layer): workspaces.balance_usd_micro is authoritative; workspaces.balance_usd_cents is the customer-visible cents mirror written in the same UPDATE (balances are isolated per workspace since Sprint 9 multi-tenancy).

• Mirror rule: cents = floor(micro / 10⁸) (balance-units.ts:115-138). Migration 0172:217 added the balance_usd_micro column and 0172:222-227 backfilled it from balance_usd_cents; from then on cents is the floor mirror, not the source of truth (workspace-balance.ts:46-48). Principle: the displayed value is never higher than the real balance — conservative by up to one cent, never optimistic.

• Atomicity: deduction is a single guarded UPDATE … WHERE balance >= cost; concurrent requests cannot over-draw the balance.

• Reconciliation backstop: a daily reconciliation cron reviews rounding and deduction anomalies (the backstop promised in the balance-units.ts:90-92 comment).

7. Top-ups, bonuses, and refunds

Three top-up channels — Stripe one-time payments, auto top-up (SetupIntent binding + cron trigger), and bank transfer — all credit workspaces.balance_usd_micro (the write-side authority since D2/#2860); the cents column is the floor mirror written in the same UPDATE.

2% top-up service fee (Plan 3, since 2026-10-05): with token-side markup flattened to 1.07×, monetization moved to the top-up side, platformFeeRate = 0.02 (dashboard/lib/payment-config.ts:20, backend/src/payment-config.ts:33). The charged amount = face value + service fee + Stripe fee gross-up; the balance is credited with face value + prepay bonus — the platform service fee and the Stripe fee are not credited (stripe-handlers/manual-checkout.ts:231-235, :598-601 passing amountMicro; auto-topup-charge.ts:661).

Prepay bonuses: higher prepay tiers earn bonus credit — $100 → 10%, $200 → 10%, $500 → 12% (dashboard/lib/topup-bonus.ts:39-43). Bonuses post through the same channel as the top-up itself.

First top-up bonus (live): a user's first completed top-up earns extra credit on top of the face value — $10 → $2, $20 → $5, $50 → $12 (FIRST_TOPUP_BONUS_TIERS, dashboard/lib/topup-bonus.ts:71-75). These amounts do not overlap with the prepay tiers above.

Auto top-up:

• the webhook is the source of truth for card binding (SetupIntent state lands via POST /api/webhooks/stripe/auto-topup; the browser-side save is an optimistic path — see the auto-topup-webhook.ts header);

• charge crediting is idempotent: deduplicated on auto_topup_events.balance_event_id IS NULL, so duplicate webhooks never double-credit;

• consecutive failures past a threshold auto-suspend the config and notify by email; a 3DS challenge suspends it (suspended_3ds) until the user completes SCA in the dashboard.

Refunds and corrections:

• platform-side corrective credits write directly to the pool ledger: migration 0197_pool_ceil_overcharge_credit.sql recomputes the #2818 over-charge per request (ceil(cost_micro/1e8)×1e8 − cost_micro, 0197:43) and credits it per workspace (0197:213-220, also rewriting the cents floor mirror). It is a pool-ledger settlement, not an outward payment; the migration header discloses that class-(a) rows may overstate the amount when the pool was drained and guarded deductions were silently dropped (§8);

• refunds and debits can produce negative sub-cent residuals (balance-units.ts:69-71) — where truncate and floor differ (−0.5¢ truncates to 0¢, floors to −1¢); the ledger keeps the exact micro value and the display layer converts per §1;

• frontend balance rendering uses microToUsdDisplay (truncated to cents, signed for negatives, balance-units.ts:159-166).

8. Incident disclosure and known items (recorded as they are)

Transparency is not "nothing ever went wrong" — it is what went wrong, how it was fixed, and how the money was returned, all verifiable:

Appendix: how to reproduce the calibers

The body of this paper states mechanisms and constants only; it cites no live business numbers. To reproduce any spend-side data:

-- 90-day spend (USD): aggregate request_logs per workspace/user
SELECT SUM(cost_micro) / 1e10 AS spend_usd FROM request_logs WHERE ...;

-- Verify the rounding mirror: derived cents must equal floor(micro/1e8).
-- Divide by 100000000.0 (float), not an integer: SQLite integer division truncates toward zero before FLOOR.
SELECT balance_usd_cents, FLOOR(balance_usd_micro / 100000000.0) FROM workspaces WHERE ...;

Any quantified claim published outward (e.g., incident-affected request counts, total refunded) must be independently recomputed before publication, with the data-extraction caliber attached to the published page.

Calibers change with the code: every claim above has the code/migration anchors given; if the code changes, this document must be updated in the same PR. Code anchors refer to our internal repository and the issue numbers are internal references. Source: internal issue GH#2294.