Tracking Token Usage Across an Organization

Token usage tracking is the practice of recording, attributing, and analyzing every token an organization sends to and receives from an LLM provider, broken down by team, service, user, and feature so that cost and behavior can be explained instead of guessed at. Most teams start with nothing more than a provider dashboard showing a single monthly total. That works until three teams share one API key, a agentic workflow starts looping, or finance asks which product line is actually profitable. This article walks through how to build token usage tracking that survives contact with a real organization: what to capture, how to attribute it, where to store it, and how to turn it into alerts and reports people trust.

Why the provider dashboard is not enough

Every major LLM provider gives you an aggregate usage view: total tokens, total spend, maybe a per-model breakdown. That is a fine starting point and a poor ending point. The dashboard cannot tell you:

• Which internal service or team generated a given request
• Whether a spike came from a genuine traffic increase or a bug in a retry loop
• The cost of a single feature (e.g., "AI summarization" vs "AI search") when both share a backend
• Per-customer cost in a B2B SaaS product, which you need for margin analysis
• Whether prompt caching or context truncation is actually saving money

All of that requires you to own the attribution layer yourself. The provider only sees a request and a response; only your application knows the business context behind it. This is the same reason cloud cost tools eventually need tagging discipline: the raw bill is honest but meaningless without labels.

What to capture on every LLM call

Treat every LLM call like a billable event and log it with enough metadata to slice later. At minimum, capture:

• Timestamp (UTC, millisecond precision if you can afford it)
• Model identifier (exact model string, not a friendly name, since pricing differs by model and sometimes by dated snapshot)
• Input tokens and output tokens separately (they are priced differently for almost every provider)
• Cached tokens, if the provider supports prompt caching, since cached reads are typically billed at a fraction of the input rate
• Request purpose or feature tag (e.g., chat-completion, doc-summarize, code-review-agent)
• Team or service owner (which internal repo, service, or squad issued the call)
• End user or tenant ID, if the product is multi-tenant
• Latency (useful for correlating cost spikes with performance regressions)
• Status (success, error, timeout, rate-limited)
• Trace or request ID that ties the call back to the originating user action, for debugging runaway loops

Here is a minimal shape for that record, in TypeScript, that most teams converge on eventually:

interface LlmUsageEvent {
  timestamp: string;        // ISO 8601 UTC
  requestId: string;        // ties back to your app trace
  model: string;            // e.g. "claude-sonnet-5-20260101"
  provider: string;         // "anthropic" | "openai" | "google" | ...
  inputTokens: number;
  outputTokens: number;
  cachedInputTokens: number;
  feature: string;          // logical feature, not endpoint
  team: string;             // owning team/squad
  tenantId: string | null;  // customer or workspace, if applicable
  userId: string | null;    // end user, if applicable
  status: "success" | "error" | "timeout" | "rate_limited";
  latencyMs: number;
  costUsd: number;          // computed at write time from a pricing table
}

Compute costUsd at write time using a pricing table you control, not by trusting a provider's later reconciliation. Providers change prices, and you need the number that was true at the moment the call happened, not today's rate applied retroactively.

Centralize the write path, not the read path

The single biggest structural decision is where the usage event gets written. There are two common patterns:

1. Client-side logging: every service that calls an LLM emits its own usage event to a shared sink (Kafka topic, log pipeline, or database table).
2. Gateway/proxy logging: all LLM traffic routes through a single internal proxy that talks to the provider and logs usage centrally.

The proxy pattern wins almost every time at organizational scale. If ten services each independently log usage, you get ten slightly different schemas, ten places where someone forgets to log errors, and no single point to enforce rate limits or catch a runaway agent loop before the bill arrives. A thin internal gateway that all services call instead of the provider directly gives you:

• One place to normalize the usage schema
• One place to enforce per-team budgets or rate limits
• One place to add caching, retries, and fallback models without touching every caller
• Consistent cost computation, since pricing logic lives in one codebase

A basic Node/Express gateway wrapping an Anthropic call looks like this:

import { Anthropic } from "@anthropic-ai/sdk";
import { recordUsage } from "./usageStore";
import { priceFor } from "./pricingTable";

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

export async function callModel(req, res) {
  const { model, messages, feature, team, tenantId, userId } = req.body;
  const start = Date.now();

  try {
    const response = await anthropic.messages.create({
      model,
      max_tokens: 1024,
      messages,
    });

    const usage = response.usage;
    const cost = priceFor(model, usage.input_tokens, usage.output_tokens);

    await recordUsage({
      timestamp: new Date().toISOString(),
      requestId: req.headers["x-request-id"],
      model,
      provider: "anthropic",
      inputTokens: usage.input_tokens,
      outputTokens: usage.output_tokens,
      cachedInputTokens: usage.cache_read_input_tokens ?? 0,
      feature,
      team,
      tenantId: tenantId ?? null,
      userId: userId ?? null,
      status: "success",
      latencyMs: Date.now() - start,
      costUsd: cost,
    });

    res.json(response);
  } catch (err) {
    await recordUsage({
      timestamp: new Date().toISOString(),
      requestId: req.headers["x-request-id"],
      model,
      provider: "anthropic",
      inputTokens: 0,
      outputTokens: 0,
      cachedInputTokens: 0,
      feature,
      team,
      tenantId: tenantId ?? null,
      userId: userId ?? null,
      status: err.status === 429 ? "rate_limited" : "error",
      latencyMs: Date.now() - start,
      costUsd: 0,
    });
    res.status(500).json({ error: "model call failed" });
  }
}

Note the error path also writes a usage record with zero cost. Teams that only log successful calls lose the ability to diagnose why a feature's cost dropped: was it optimized, or is it silently failing?

Where to store usage events

For most organizations, a columnar or time-series-friendly store beats a plain relational table once volume grows past a few million events a month. Options that work well in practice:

• ClickHouse: purpose-built for exactly this kind of append-only, high-cardinality analytical workload; aggregation queries over billions of rows stay fast.
• Postgres with monthly partitioning: fine up to moderate volume, especially if the rest of your stack is already Postgres. Partition by month on timestamp and index team, feature, tenantId.
• A managed observability platform that already has LLM tracing support: convenient, but confirm it lets you export raw events, since you will eventually want to join usage data with billing or product analytics.

Whatever you choose, do not let this table live only in application logs. Logs get sampled, rotated, or dropped in ways that quietly corrupt cost reporting. Usage events are financial data; treat them with the durability guarantees you would give billing records.

A simple Postgres schema to start with:

CREATE TABLE llm_usage_events (
    id BIGSERIAL PRIMARY KEY,
    ts TIMESTAMPTZ NOT NULL,
    request_id TEXT NOT NULL,
    provider TEXT NOT NULL,
    model TEXT NOT NULL,
    input_tokens INTEGER NOT NULL,
    output_tokens INTEGER NOT NULL,
    cached_input_tokens INTEGER NOT NULL DEFAULT 0,
    feature TEXT NOT NULL,
    team TEXT NOT NULL,
    tenant_id TEXT,
    user_id TEXT,
    status TEXT NOT NULL,
    latency_ms INTEGER NOT NULL,
    cost_usd NUMERIC(12, 6) NOT NULL
) PARTITION BY RANGE (ts);

CREATE INDEX idx_llm_usage_team_ts ON llm_usage_events (team, ts);
CREATE INDEX idx_llm_usage_feature_ts ON llm_usage_events (feature, ts);
CREATE INDEX idx_llm_usage_tenant_ts ON llm_usage_events (tenant_id, ts);

Partition by month and drop or archive partitions older than your retention policy requires. You rarely need per-request granularity older than 90 days once daily rollups exist.

Build rollups, do not query raw events for dashboards

Raw event tables are for debugging and audits. Dashboards should read from pre-aggregated rollups, computed on a schedule (hourly or daily), grouped by the dimensions people actually ask about:

CREATE TABLE llm_usage_daily_rollup AS
SELECT
    date_trunc('day', ts) AS day,
    team,
    feature,
    model,
    SUM(input_tokens) AS total_input_tokens,
    SUM(output_tokens) AS total_output_tokens,
    SUM(cached_input_tokens) AS total_cached_tokens,
    COUNT(*) AS request_count,
    COUNT(*) FILTER (WHERE status != 'success') AS error_count,
    SUM(cost_usd) AS total_cost_usd
FROM llm_usage_events
GROUP BY 1, 2, 3, 4;

This single rollup answers most of the recurring questions: cost per team per day, cost per feature, model mix over time, and error rate trends. Refresh it with a cron job or a materialized view refresh, and keep the raw table around for the rare case where someone needs to drill into a specific request.

Attribution for agentic and multi-step workflows

Token usage tracking gets harder once you introduce agents that make multiple LLM calls per user action: a planning call, several tool-use calls, and a final synthesis call. If you log each call independently without a shared trace ID, a single user action can look like ten unrelated events, and nobody can answer "how much does one agent run cost end to end."

Fix this by generating a traceId at the top of the agent loop and threading it through every LLM call within that run:

const traceId = crypto.randomUUID();

async function runAgent(userQuery: string) {
  const plan = await callModel({ ...planningPrompt(userQuery), traceId });
  const toolResults = await Promise.all(
    plan.toolCalls.map((call) => executeToolAndLog(call, traceId))
  );
  const final = await callModel({ ...synthesisPrompt(toolResults), traceId });
  return final;
}

Add traceId as a column in the usage event schema, and you can now group by trace to compute per-agent-run cost, which is often the number that actually matters to a product team deciding whether an agentic feature is economically viable.

Alerting on anomalies, not just totals

A monthly cost report tells you what already happened. Alerting catches problems while they are still cheap to fix. Set up threshold and anomaly alerts on the rollup table:

• Budget threshold per team: alert when a team's daily spend crosses a configured limit, before the monthly invoice arrives.
• Rate-of-change alert: alert when a feature's hourly token volume is more than, say, three times its trailing seven-day average at the same hour. This catches retry loops and runaway agents fast.
• Error rate alert: a spike in error_count alongside flat request_count often means a provider outage or a broken prompt template, not a cost problem, but it is usually caught by the same pipeline.
• Cache hit rate alert: if you rely on prompt caching to control cost, alert when the cached-token ratio drops sharply, since that is a silent cost regression.

A simple anomaly check you can run as a scheduled job:

async function checkForAnomalies() {
  const rows = await db.query(`
    SELECT team, feature, SUM(total_cost_usd) AS today_cost
    FROM llm_usage_daily_rollup
    WHERE day = CURRENT_DATE
    GROUP BY team, feature
  `);

  for (const row of rows) {
    const baseline = await getSevenDayAverage(row.team, row.feature);
    if (row.today_cost > baseline * 3) {
      await sendAlert({
        channel: "#llm-cost-alerts",
        message: `${row.team}/${row.feature} spend is ${row.today_cost.toFixed(2)} today, ` +
                  `vs a 7-day average of ${baseline.toFixed(2)}.`,
      });
    }
  }
}

Run this hourly against a same-day partial rollup, not just once a day, so the alert fires while the loop is still running instead of after it has already finished burning budget.

Reconciling with the provider bill

Self-reported usage will drift from the provider's invoice over time: pricing table lag, retried requests that were not logged correctly, or a service that bypassed the gateway. Build a monthly reconciliation step:

1. Pull the provider's official usage export for the billing period.
2. Sum your internal rollup for the same period, same model breakdown.
3. Diff the two. A gap under a few percent is normal rounding and timing noise. A gap larger than that usually means a service is calling the provider directly instead of through your gateway, or your pricing table is stale.
4. Fix the source of drift, do not just adjust the report to match.

This reconciliation step is also your enforcement mechanism: once leadership expects the internal dashboard to match the invoice within a small tolerance, "call the provider directly, skip the gateway" stops being a viable shortcut for any team, because it gets caught within a month.

Making the data actionable for teams, not just finance

Token usage tracking pays for itself fastest when individual engineers can see their own team or feature's numbers without filing a request. A lightweight internal dashboard, even a simple internal page backed by the rollup table, that shows:

• This week's spend vs last week, by feature
• Top five most expensive request types
• Cache hit rate trend
• Error rate trend

...turns cost from an abstract finance concern into something engineers optimize the same way they optimize latency or memory. Teams that can see "your feature is 40% of org spend and cache hit rate dropped from 60% to 12% last Tuesday" fix it themselves, fast, without a meeting.

FAQ

Do I need this if I only use one LLM provider and one API key? Yes, if more than one team or feature shares that key. The provider dashboard shows you a single number; it cannot tell you which feature is responsible for it. The moment two teams share billing, you need internal attribution.

Should I track tokens or dollars? Track both. Tokens are the stable unit for spotting behavioral changes (a prompt got longer, a loop is retrying), since token counts do not move when the provider changes pricing. Dollars are what finance and leadership care about. Store token counts as the source of truth and compute cost at write time from a pricing table, so you can re-derive historical dollar figures if pricing changes.

How long should I retain raw usage events? Keep raw per-request events for 30 to 90 days for debugging, and keep daily rollups indefinitely, since rollups are a fraction of the storage cost and answer almost every historical question you will actually ask.

What is the simplest way to start if I have nothing today? Add a single logging call after every LLM SDK call in your codebase that writes model, input tokens, output tokens, and a feature tag to a database table. That alone, even without a gateway, gets you 80% of the value. Build the gateway and rollups once you have more than two or three services calling LLMs directly.

Does prompt caching change how I should track usage? Yes. Log cached and non-cached input tokens as separate fields, since they are billed differently and often at a large discount. If you only log total input tokens, your cost calculations will be wrong and you will not be able to measure whether your caching strategy is actually working.

How do I attribute cost per customer in a multi-tenant product? Pass a tenantId through every LLM call from the request context down to the gateway, and include it as a column in the usage event schema. This lets you compute gross margin per customer, which matters a lot once usage-based LLM costs become a meaningful fraction of your cost of goods sold.

Should alerts go to engineering or finance? Both, but on different cadences. Engineering needs real-time or hourly anomaly alerts to catch runaway loops and bugs. Finance needs weekly or monthly summaries tied to budget lines. Route the two use cases into different channels so engineers are not desensitized by alerts meant for finance and vice versa.