Today was a bad day so Iâve decided to rage against this index. Shockingly, people donât know that you canât reliably track tokens. Nobody outside the labs / hypers sees the actual token flows.
Before I even get to the flaws in the data, did anyone consider that as capability rises and cost falls, task complexity goes up? Agentic work has a long tail of simple steps for every hard one, and the easy stuff scales disproportionately. This index skews toward API middlemen running harnesses that route subtasks to the cheapest capable model (including using sonnet > opus!!). So a falling blended price may say very little about whoâs using what, and a lot about total usage and task complexity going up. Now to the data.
This index in particular is a blended avg price of itself a subset of incomplete data. For ex, one big synthetic data run on a cheap open model could impact it while frontier demand remains the same.
The sample also liens on aggregators (api middlemen man). So youâre not seeing direct enterprise contracts or first party consumption (using the app). This is like looking at online retail and not having prime or Walmart.
And it has no idea what anyone actually pays. 1) All tokens are not the same, 2) list prices do not equal token prices.
On 2, important cost reduction techniques like batching / caching are offered at huge discounts (50-90%) and so mix shift here adds to the noise.
Free tokens / expanded limit promos (liek what anthro is doing right now) also show up as falling prices⌠so the marketing budget is bleeding into the âdemandâ chart.
And if you needed another reason: if the conclusion is people are using frontier models less bc the chart is rolling, how would you explain the chart also rolling at the start of the year when everyone was having an orgasm about frontier lab (Claude) ARR absolutely ripping during this period?
Which is all to say that model usage is assuredly evolving (see NBIS token factory) and that may be the right conclusion, but i would not be using this data to âtrackâ that.
This data is not what people think it is.