Skip to content
AI & finance · Intermediate

AI model economics explained: training versus inference

What it costs to train and run AI models, how per-token pricing is falling, and what reported lab margins and run-rates show.

BlockWest Editorial Desk·Updated November 2, 2026·6 min read·Educational, not investment advice

Key takeaways

  • Training is the large, mostly upfront cost of building an AI model; inference is the ongoing cost of running it each time a user or application sends a request.
  • List prices per token keep falling. Anthropic’s top Opus tier cost $15 per million input tokens and $75 per million output tokens at the Claude Opus 4 generation, versus $4 and $20 for Claude Opus 5.5 on its pricing page as of November 2, 2026.
  • Usage is growing faster than prices fall. Google said in May 2026 it was processing more than 3.2 quadrillion tokens a month, about seven times the level a year earlier.
  • Reported revenue run-rates reached nearly $70 billion at OpenAI (Axios, September 29, 2026; the Financial Times later put it at roughly $50 billion on a different accounting basis, per Bloomberg on October 8, 2026) and more than $65 billion at Anthropic (end of July 2026, per Bloomberg and the Financial Times).
  • Because inference demand compounds, it is now a main driver of data center capex, which also makes the economics sensitive to price deflation and efficiency gains.

Training versus inference

Training is the process of building a model: feeding it very large amounts of data on thousands of chips for weeks or months until it learns useful patterns. It is a large, lumpy expense incurred before a model earns anything. Inference is using the finished model: every chat reply, code suggestion or agent step runs the model once more on a chip. It is a variable cost that scales with usage.

Epoch AI, a research group, estimated in June 2024 that the cost of the largest training runs had grown about 2.4 times per year since 2016 and that the largest runs would cost more than $1 billion by 2027. In its breakdown, hardware (chips, servers, networking) made up 47% to 67% of development cost, research staff 29% to 49%, and energy only 2% to 6%.

The two costs behave differently on a lab’s books. Training spend is usually treated as research and development, while inference cost sits in cost of revenue. That is why labs can report healthy gross margins and still lose money overall.

Tokens, GPU-hours and per-token pricing

A token is a chunk of text, roughly three quarters of an English word on average. AI APIs (application programming interfaces, the way developers buy model access) charge per million tokens, with separate prices for input (what you send) and output (what the model writes). Output costs more because the model generates it one token at a time.

Underneath, providers pay for GPU-hours: one graphics processing unit running for one hour. Silicon Data’s H100 rental index, which tracks Nvidia’s previous-generation flagship chip across cloud providers, stood at $2.93 per GPU-hour on November 1, 2026. A lab’s inference margin depends on how many tokens it can serve per GPU-hour, which improves with newer chips, better software and smaller models.

Published list prices show the trend. All figures below are per million tokens, standard tier, taken from each company’s pricing page on November 2, 2026 unless noted.

Provider and modelInputOutputNote
OpenAI GPT-4 (8K context)$30$60Launch price, March 2023
OpenAI GPT-5.6 Sol$4$20Current flagship tier; promotional price OpenAI says is available at least through November 21, 2026
OpenAI GPT-5.6 Luna$0.20$1.20Current low-cost tier
Anthropic Claude Opus 4 and 4.1$15$75Retired in 2026
Anthropic Claude Opus 5.5$4$20Current Opus tier
Anthropic Claude Haiku 5.5$0.10$0.50Current small model; prompts up to 100K tokens ($0.50 and $2.50 above that)
Google Gemini 4 Argon$2$10New flagship; introductory price through November 30, 2026, then $4 and $20
Google Gemini 3.1 Pro Preview$2$12Prompts up to 200K tokens
Google Gemini 3.8 Flash$0.75$3.75Price listed through December 31, 2026

All three providers also offer roughly 50% discounts for batch (non-urgent) processing and steep discounts for cached, repeated inputs. Measured by capability rather than model name, the decline is faster still: Epoch AI found in March 2025 that the price of reaching GPT-4-level performance on a set of PhD-level science questions fell about 40 times per year, with declines ranging from 9 to 900 times per year depending on the task.

Margins and revenue at the labs

Neither OpenAI nor Anthropic is public, so figures come from company statements or press reports of investor materials.

  • OpenAI. CFO Sarah Friar wrote in January 2026 that annualized revenue passed $20 billion in 2025, up from $6 billion in 2024, alongside compute capacity growing from 0.6 to 1.9 gigawatts. Axios reported on September 29, 2026 that the run-rate was nearing $70 billion. The Financial Times later reported a figure of roughly $50 billion, with the gap attributed to accounting choices, including how sales through cloud providers are counted (per Bloomberg, October 8, 2026). The Information reported in December 2025 that OpenAI’s compute margin (revenue minus the cost of running models for paying users) reached about 70% in October 2025, up from about 52% at the end of 2024.
  • Anthropic. The Information reported in January 2026 that Anthropic expected a 2025 gross margin of about 40%, after inference costs ran 23% above plan. By September 2026, the Financial Times reported that Anthropic told investors Q2 2026 revenue was $11.5 billion, its run-rate passed $65 billion by the end of July (versus about $9 billion at the end of 2025), and gross margins exceeded 80% before revenue sharing with cloud partners and before training costs. It also said adjusted operating income would be positive for a second consecutive quarter.

A revenue run-rate annualizes a recent month or quarter. It is useful for fast-growing companies but is not the same as audited annual revenue, and definitions vary between companies.

Why inference demand drives capex

Training a frontier model is a one-off project per generation. Inference, by contrast, grows with every new user, product and AI agent, and agents in particular consume many more tokens per task than a single chat reply. Google’s jump from about 480 trillion monthly tokens in May 2025 to more than 3.2 quadrillion in May 2026 illustrates how quickly volume can compound.

Serving that volume with acceptable speed requires chips sitting in data centers near users, running continuously. That is a large part of why the largest cloud companies keep raising capital spending (covered in our AI infrastructure guide) and why labs sign multi-year compute contracts with clouds and GPU rental firms. OpenAI’s own January 2026 statement tied its revenue growth directly to added compute capacity.

Risks to the economics

  • Price deflation. If prices per unit of capability keep falling 10 times or more per year, revenue per token shrinks and growth depends on volume rising even faster.
  • Efficiency gains that cut both ways. Cheaper inference helps lab margins, but it can also reduce the number of chips needed per unit of demand, which matters for the companies and lenders financing data centers.
  • Competition and commoditization. Many capable models now sit at similar price points, including open-weight models that anyone can host, which limits pricing power.
  • Training costs outside gross margin. Headline gross margins exclude training, which remains large and rising. OpenAI, per Financial Times reporting cited by Reuters, projected about $278 billion of negative free cash flow across 2026 to 2030.
  • Disclosure quality. Most lab figures are leaked or selectively shared investor data, not audited filings. Treat them as indicative until public filings exist.

What to watch next

  • Any public filings from OpenAI or Anthropic, which would replace reported figures with audited ones.
  • Price-page changes at OpenAI, Anthropic and Google, especially for flagship models.
  • GPU rental indices, as a signal of whether compute supply is catching up with demand.
  • Token volume disclosures on cloud and platform earnings calls.

Sources and further reading

Frequently asked questions

What is the difference between training and inference costs?

Training is the large upfront cost of building a model on thousands of chips. Inference is the ongoing cost of running the model every time it answers a request, so it scales with usage.

How much do AI APIs cost per token?

Prices are quoted per million tokens. As of November 2, 2026, Anthropic's Claude Opus 5.5 listed $4 input and $20 output, OpenAI's GPT-5.6 Sol $4 and $20 (a promotional price available at least through November 21, 2026), and Google's Gemini 3.1 Pro Preview $2 and $12 for prompts up to 200K tokens.

Are AI labs profitable?

Reported gross margins have improved: the Financial Times reported in September 2026 that Anthropic's exceeded 80% before revenue sharing and training costs. But training and expansion costs are large, and OpenAI projected about $278 billion of negative free cash flow across 2026 to 2030, per FT reporting cited by Reuters.

Why does falling token pricing not reduce AI capex?

Usage is growing faster than prices fall. Google reported more than 3.2 quadrillion tokens a month in May 2026, about seven times a year earlier, and serving that volume requires more chips and data centers.

This explainer is reviewed and updated as the rules and the market change. Last reviewed November 2, 2026. It is educational content and not financial, legal or tax advice.

Keep learning