How to Monitor LLM Costs Without Changing Application Code

Path 1: read the provider's cost API

Both major providers expose organization-level usage and cost endpoints that need no change to the application. Anthropic's Usage and Cost Admin API takes an Admin API key and supports 1 minute, 1 hour, and 1 day buckets for usage, with filtering and grouping by API key, workspace, model, service tier, context window, data residency, and speed. Anthropic says data typically appears within 5 minutes of request completion and says the API supports polling once per minute for sustained use.

OpenAI's organization Costs endpoint supports a bucket width of 1d only. It can group by project_id, user_id, line_item, api_key_id, and api_source, and the line_item value carries the model name or token type. Anthropic's cost report is also a daily view, and its docs note that Priority Tier costs are not available in the cost endpoint, so for that tier you read the usage endpoint and group by service_tier.

The limit of this path is the grouping. The dimensions are the ones the provider knows. If your application sends every customer through one API key, the cost report cannot split them, and nothing you configure on the provider side will add a customer dimension after the fact.

Path 2: put a proxy behind the base URL

An OpenAI-compatible proxy changes where requests go, not what the code sends. LiteLLM's docs show the standard OpenAI Python client working against its proxy by setting base_url to the proxy address, and the same page lists LangChain, LlamaIndex, Instructor, and curl examples. In many codebases that is one argument or one environment setting per client, which is why vendors describe it as no code change. It is still a change you must roll out to every service that calls a model.

What you get in return is attribution the provider does not have. LiteLLM supports budgets per virtual key, per team member, and per end customer when the request carries a user field, and a customer budget is keyed to the customer id alone, so it is shared across every key and team. Its docs also note that a customer budget object is cached for 60 seconds, so edits take up to a minute to apply.

The costs of this path are operational. The proxy is in the request path, so it becomes something you run, scale, and secure, and a request that bypasses the base URL is invisible to it. Treat any dashboard it produces as covering only the traffic that went through it.

Path 3: reconcile the two

The two paths answer different questions and fail differently, so teams that need both customer attribution and a trustworthy total run both and compare. The provider report is the authoritative bill. The proxy log is the attribution layer. If the proxy total for a day is lower than the provider total, some traffic is bypassing the proxy: a script with a hard-coded key, a notebook, or a service nobody migrated.

A hypothetical example: the provider report shows $412.00 for a day and the proxy log shows $371.00. The $41.00 gap, about 10 percent of the bill, is unattributed spend. Group the provider report by api_key_id or workspace to find which key produced it, which tells you which service skipped the proxy without reading any application code.

Choosing between them

Start with the provider API if your question is how much did we spend, which project or key spent it, and which model. It is free of rollout work and you can have a daily number today. Add a proxy when the question becomes which customer or team spent it, or when you need a budget enforced in the request path rather than reported afterward.

• Provider cost API: no app change, authoritative totals, grouped by provider-side dimensions only, daily grain for cost on OpenAI.
• Proxy on a base URL: a config change per client, adds customer and team attribution, adds a component you operate.
• Both: the provider total is the check on the proxy total.