FinOps for AI: How to Track and Control Token Spend

Learn how FinOps for AI tracks token spend per request, attributes it to teams and features, and sets budgets and alerts.

Coralogix Team10 min read

FinOps for AI: How to Track and Control Token Spend

FinOps for AI gives your engineering and finance teams a single view of which models, features, and teams consume tokens before the monthly invoice arrives. Without that view, LLM spend lands as a single provider line item that is hard to trace to a deployment or an owner. Per-request telemetry and named ownership come first, and cost cutting comes after both are in place.

This guide covers how to capture token spend on every request, attribute it to teams and features, and set budgets and alerts that stop a runaway agent before the invoice does.

What Is FinOps for AI?

FinOps for AI is the practice of applying FinOps visibility, cost allocation, and governance to AI spend, covering LLM tokens, GPU training, and inference. Unlike cloud costs, which track resource-hours that teams tag when they provision infrastructure, AI spend bills per token and moves with every prompt, context window, and agent decision.

The FinOps for AI technology category frames the work around cost complexity, faster development cycles, unpredictable spend, and policy that still lets teams innovate. AI for FinOps, which uses AI to run the FinOps practice, is a separate topic.

Why AI Spend Breaks Cloud FinOps Assumptions

AI spend breaks cloud FinOps assumptions because tags applied at provisioning never reach a per-token bill, and token costs follow usage instead of capacity. The mismatch now reaches almost every practice, since practitioners managing AI spend rose from 31 percent in 2024 to 98 percent of respondents in 2026.

1. Tokens Are a New Unit of Cost

Providers bill per token, with separate token rates for input, output, and cached tokens, and output tokens typically cost three to five times the input price. Billed tokens also exceed what a user types, because your application adds system prompts, history, and retrieved documents to every call. Provider consoles aggregate spend by account, project, or key, so the cost of one feature or conversation appears only in request-level telemetry.

2. Agents Multiply Calls

Agent architectures turn one user action into many model calls. In one provider's internal data, agents use about four times more tokens than chat interactions, and multi-agent systems use about 15 times more. Each agent turn resends the growing conversation as input, so cost per turn climbs as a session runs.

3. Many Providers and Models Fragment the Bill

Teams using OpenAI, Anthropic, and a cloud-hosted model manage three invoices, three consoles, and three usage APIs. A model version bump can move cost with no change in traffic, and self-hosted models add GPU hours that bill the same whether the server answers requests or sits idle. Your platform team has to join these costs on a shared identifier captured with each request.

How to Track Token Spend per Request

Track token spend per request by writing one record per model call before aggregating spend, so the same record supports showback, unit economics, and anomaly alerts.

The Fields to Capture on Every LLM Call

Record these fields on every model call, using the GenAI attribute names from OpenTelemetry for AI tracing where one exists:

  • Model and provider: gen_ai.request.model and gen_ai.provider.name show which model served the call and whose price list applies.
  • Token counts: gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and gen_ai.usage.cache_read.input_tokens hold the billed units.
  • Cost: Estimated cost, computed from the token counts and the model's current rate card.
  • Team and feature: The owning team and the product feature that triggered the call.
  • User and environment: A hashed user identifier, plus labels that keep test spend out of production unit costs.

The GenAI semantic conventions still carry development status, and they direct instrumentation to report the billed count when a provider returns both billed and consumed tokens.

Coralogix, the telemetry lake for observability, ingests this data through OpenTelemetry, including spans that follow the GenAI conventions. LLM cost tracking in AI Center attributes cost per message, per session, and per agent, and you can drill from the organization-wide view into a single interaction.

Unit Economics That Show Efficiency

Raw spend gives you the size of the bill, and unit metrics show whether it buys anything. Teams track these per use case:

  • Cost per request: Total inference cost divided by model requests in the period.
  • Cost per feature: Cost tagged to a feature divided by that feature's invocations.
  • Cost per successful outcome: Total cost of all runs divided by runs that met the quality bar, such as a resolved ticket.
  • Token budget burn rate: Tokens consumed so far divided by tokens budgeted, against the share of the period elapsed.

Each metric divides spend by something a team owns, and reporting it as a median and a high percentile keeps run-to-run variation visible.

How to Attribute AI Costs to Teams, Features, and Models

You attribute AI costs by tagging every request with team, feature, and environment, then rolling tagged spend into showback or chargeback. Provider invoices group usage only by account, project, or API key, so business context must come from your own metadata.

Tagging by Team, Feature, and Environment

A short metadata convention should cover team, feature, env, and a hashed user_id, with values from a controlled list. An LLM gateway should enforce the convention so application code can't skip it, and LiteLLM, for example, supports key-level tags that every request made with a tagged key inherits. Untagged requests go to an explicit bucket with a named owner, and a shrinking share there shows the convention is reaching the teams that spend.

Showback vs. Chargeback for AI Spend

Showback reports each team's AI costs for awareness, and chargeback posts those costs to P&L budgets. The FinOps invoicing and chargeback capability treats showback as required in every practice, and chargeback as dependent on accounting policy. Shared platforms such as a central gateway need a split rule, and the three common split methods are even, fixed proportional, and variable proportional by usage.

Calculating a Showback Line for One Feature

Take a support team's ticket-summary feature running on Claude Sonnet 5.5, listed at $2 input and $10 output per million tokens as of November 2026. Each call sends 2,000 input tokens and returns 400 output tokens, across 300,000 calls a month:

LineCalculationMonthly figure
Input tokens2,000 × 300,000600M tokens
Input cost600 × $2$1,200
Output tokens400 × 300,000120M tokens
Output cost120 × $10$1,200
Showback total$1,200 + $1,200$2,400
Cost per request$2,400 ÷ 300,000$0.008

Output is 17 percent of token volume and half the cost, which is why input and output belong in separate fields. Prompt caching or batch processing changes these rates, so record which rate applied to each call.

How to Set Budgets and Catch Runaway Spend Early

You catch runaway spend early by setting budgets per team and feature, deciding in advance whether overspend warns or blocks, and alerting on usage before the invoice arrives. Forecasts then run within the same ownership boundaries.

Token Budgets and Spend Caps

A soft limit notifies, and a hard limit blocks. OpenAI's hard spend limit returns 429 errors once tracked spend reaches the cap. Azure OpenAI budgets send alerts without a hard dollar cap. Gateways fill that hole, and LiteLLM separates a soft budget email alert from a max_budget that rejects requests. Hard caps belong on experiments and developer keys, and customer-facing features get soft limits with an owner authorized to raise them.

Alerts and Forecasts on Inference Cost

Billing data arrives too late to stop a runaway loop, so alert on tokens per minute, cost per session, and burn rate against the period budget. In Coralogix, AI Center's Alerts extension ships prebuilt threshold alerts on cost spikes, error rate, and latency. Forecasts come from the same drivers, requests, tokens per request, and price per token, and a weekly or monthly rolling forecast catches surprise spend faster than a quarterly one.

How to Report AI Spend to Finance and Leadership

AI spend reports work when they show unit costs tied to outcomes. A report line such as "AI-assisted ticket resolution costs $0.18 per ticket, down from $0.31 last quarter" gives a CFO a decision, and a monthly token total gives them only a number. Name a single accountable owner for AI spend, since organizations with defined ownership of AI economics were 3.7 times more likely to show value to the CFO. A monthly review then splits responsibilities this way:

  • Engineering: Owns instrumentation and model cost-quality trade-offs.
  • Finance: Owns budgets, forecasts, and chargeback rules.
  • Leadership: Sets outcome expectations and funds or retires projects.

A cost spike then has a clear path, from the engineer who explains it to the finance owner who adjusts the budget. For multi-provider reporting, the FOCUS 1.5 release scope covers labels for model, token type, and the person or agent behind a charge.

How to Introduce FinOps for AI Without Losing Visibility

Introducing FinOps for AI starts with per-request attribution and anomaly alerts, with hard caps as a backstop, because a cap stops the spend but not the cause. An agent that keeps retrying a failing tool call resends its growing context on every attempt. If the provider console is your only cost signal, that spend shows up only when the invoice is reconciled.

A hard cap on the agent's API key stops the spend, but it also blocks every legitimate request on that key until someone raises the limit. With agent, session, and feature recorded on every span, a tokens-per-minute alert catches the retry loop early, and the trace shows where it started. Three decisions settle how the practice scales:

  • Ownership: Every feature that calls a model has a named cost owner before it reaches production.
  • Tag coverage: The gateway quarantines untagged requests, and the untagged bucket has a target share.
  • Cap policy: Experiments get hard caps, and production features get soft limits with paging.

Together they keep attribution and alerts working as new features and teams start calling models.

Why Attribution Has to Come Before Cost Cutting

Cutting cost before attribution leaves you unable to tell whether a saving came from a cheaper model, a shorter prompt, or a quieter week. Coralogix, recognized as a Market Shaper in the October 2026 Gartner® Emerging Market Quadrant for AI FinOps Tools – Startup Vendors, starts from that order. Its LLM cost tracking records AI cost on the same request telemetry as latency, errors, and evaluation results, so engineers answer cost questions from the traces they already read.

Send your LLM traces to the Coralogix platform and see cost by team, feature, and agent on your own traffic.

Frequently Asked Questions About FinOps for AI

Do reserved capacity and commitment discounts apply to AI workloads?

Yes, through provisioned throughput and GPU reservations, and both bill whether you use them or not. Normalize a commitment to cost per million tokens at the utilization you can sustain, and compare it with pay-as-you-go before you buy.

How do you track coding agent spend per developer?

Collect each coding agent's usage data, from its native OpenTelemetry output or its vendor's usage API, and attach developer identity to every session. Coralogix Code Agent Observability covers Claude Code, Claude Cowork, Codex CLI, GitHub Copilot, and Cursor, and breaks spend down by model, user, and session.

Where should a team start with FinOps for AI?

Start with one high-volume feature, instrumented with model, token, team, and feature fields. Route its traffic through a gateway that enforces tags, and set one budget alert on its burn rate before you expand.

Coralogix

One lake.
Every signal.

Logs, metrics, traces and security data in one queryable place, ready for your agents and your people.