AI Predictions

Submitted by gil on

I'm putting these down on paper, let's see how I do:

Model intelligence has plateaued. The scaling hypothesis was rejected years ago, but labs and true believers keep it around like a ghost to keep people hopeful. We've mostly seen gains from improved tooling and chain of thought techniques. Chain of thought is useful but it shouldn't be counted as improved model intelligence: if you had the ability to make a model that performed as well as a CoT one without spending the extra tokens you'd just do that.

For industry finances we need to wait for the GAAP numbers. Believe no one, trust nothing until we get something audited (or at least something that carries some liability) out to the public. This could be a big moment: it probably won't deflate the bubble but it could be a serious shock of cold water. I'm sure all the labs are hurtling along to IPO as soon as they can but I also wonder if there's a bit of a game of chicken around who has to be the first to bare it all.

I don't think labs are profitable on marginal inference. At the least they're just barely breaking even.

Model training depreciation is going to be steep, I think the realistic way to model it is to depreciate it in just a year[1] or year and a half. With time this race will slow down and that'll stretch to two years, maybe three.

Anthropic may be compute constrained, but they're also probably unprofitable: you don't look at killing the $20 plan for code use unless you're not getting the profit you need out of it. If you were profitable on compute you'd welcome signups even when you're compute constrained: your profit only increases if users pay but can't use your system! I'm aware you need to treat profitability on these monthly plans in a more complex way: paying but infrequent users wind up subsidizing users who saturate their usage. Your customer growth can happen with either cohort too: it's great for your business if your rate of gain of subsidizers beats the rate of gain of maximizers. But decisions like dropping features from the $20 plan, killing Sora, and other comments made by Altman suggests that labs have seen growth in revenue but it's from the smaller margin customers.

OpenAI is doing everything sleazy and uncompetitive that they can to win the market. Right now they've been raining down compute on codex users in hopes of starving out competition so they can extract monopoly rents later. Altman is burning as much cash as they can to get any edge they can. Some of this might even be uncompetitive: I am sure that they have internal predictions to justify their agreements to buy compute and drive up its price across the market (see RAM prices) but I am also thinking that if OpenAI trips up at all along the way there they'll breach any agreements and free up a lot of capacity very quickly and I think it is more likely that we'll see those breaches than not. Once again, we have to wait to see the GAAP numbers and cashflow.

[1] I'm fully aware how silly it is to call it depreciation when your asset has a useful lifespan of one year but things are moving fast.