Talk to an Expert

Foundations · AI line

Inference cost and the margin it moves

5 min on the light depth

The frame

In classic software, gross margin, the revenue left after the cost of delivering the product, barely moved with use and never shaped a price. Generative AI changes that. Each time a model produces an output the vendor pays for LLM inference, metered in tokens, and the bill recurs with use. That inference cost is the first real marginal cost software has had in decades, and a supplier sets it on a schedule you do not control. It turns gross margin from a constant into a distribution and runs under every pricing decision for the AI line.

Finance will show you one blended margin for that line. The average can hide a Customer Group running at a loss, often the one growing fastest, because usage is never evenly spread across a book. Under a grow-without-adding-staff mandate, it is the cost you cannot hire your way around. Ask first where the cost enters your pricing. Does the price attach to a value metric, the unit of customer value you count on an invoice, or to the supplier’s unit with a markup on top? The application layer prices its own value; the model vendor’s rate card is the floor under your list price and never its source.

Inference cost argues that the cost moves when you change nothing, and the value metric decides whether revenue runs ahead of it. The subsidy cliff argues that application prices were set against an input the model layer was holding down, and the exposure is an unbounded metric. AI gross margin argues that the value metric, the packaging boundary and the discount surface decide the blended number before finance reports it. Cost-plus pricing for software argues that a supplier’s token rate passed through with a markup is cost-plus, whatever the deck calls it.

If your model supplier repriced inference tomorrow, what moves first: your gross margin, your customer's invoice, or what your meter counts?

Three things to say in the room this week

  • Our AI line has a cost on every use, so the margin question comes before the price question.
  • Show me the AI margin by Customer Group, because the blended number is an average over a distribution.
  • If our unit tracks the supplier's token rate, every efficiency gain flows to the buyer, and we are pricing cost-plus.

Core reading

Go deeper: the heavy depth

Read the core in this order.

  • Inference cost: what it does to software pricing. Separates the vendor’s cost from inference pricing and token-based pricing, and ends on tests against your own exposure. Start here.
  • The subsidy cliff. Why application prices were set against a model-layer cost held below its real level, and why the fix is a bounded value metric.
  • AI gross margin in software. Where the blended number is decided before finance reports it, and why cost engineering only buys time.
  • Cost-plus pricing for software. The pass-through trap: the unit came from the supplier and the efficiency gains go to the buyer. Cost is the floor; the value metric decides the unit.

Then the shelf, lighter.

  • AI price cuts and who captures the deflation. A model-layer cut goes to your margin, the customer’s invoice or the meter; a unit linked to cost donates it.
  • Thin wrapper AI product margin. A launch margin measures a launch condition; falling API prices do not rescue a cost-anchored price.
  • AI software pricing. The three decisions for an AI product, in order, and why inference cost makes a bounded metric mandatory.
  • Generative AI pricing challenges. The older piece on AI features added for perception and the dot-com pattern they repeat.
  • OpenAI vs Anthropic pricing. The posted token rate is the wrong number; choose the API against the value metric you intend to charge on.

The research hubs on AI and LLM economics and on cloud and resource-based pricing hold this module’s peer-reviewed work, each study read in full with a written ceiling on what it supports.

Next on the AI track: Credits and surrogate units, what a credit denominates and where the price lives when the vendor mints it.

The shelf

Which depth did you take?