How to Allocate LLM Cost by Team, Product, and Feature

Identity must precede the provider call

A virtual gateway key can map a request to an organization and team even when application code supplies no labels. Add application, environment, route, and cost-center metadata through centrally managed key records or policy, not arbitrary user text.

This prevents the unattributed bucket from becoming the largest line item and makes ownership stable when teams rename services.

Use an append-only request ledger

For each call, store a request identifier, timestamp, tenant, team, application, route, requested and selected model, provider, token counts, cache state, retries, latency, and integer monetary amounts. Preserve both actual cost and baseline cost if savings are reported.

Integer nanoUSD or another fixed smallest unit avoids floating-point drift during aggregation. Conversion belongs at display and export boundaries.

Choose showback before chargeback

Show teams their usage and unit costs first. Validate mappings and explain shared costs before money is transferred between budgets. Chargeback built on disputed identity data creates organizational resistance instead of accountability.

When chargeback begins, publish rules for shared gateway infrastructure, cache cost, failed calls, credits, and provider commitments. The FOCUS specification offers a useful normalization model for billing data, even when request-level fields remain AI-specific.

• Allocated provider spend percentage
• Cost by feature and environment
• Cost per successful business outcome
• Baseline versus routed cost
• Cache and retry contribution
• Unattributed direct-provider spend

Reconcile instead of assuming

Gateway estimates and provider invoices can diverge because of rounding, cached-token pricing, credits, asynchronous jobs, or calls made outside the gateway. Month close should ingest provider totals, identify gaps, and keep the variance visible until it has an owner.