Internet and SMS used to be expensive and metered until they weren't thanks to technological advances and expanded use. I think LLMs will follow the same path, maybe on a shorter timespan.
Right, that's crucial to understand. In 1985 you could make a direct dial from England to the US but it was eye wateringly expensive. £2 per minute. An hour's call to your mum? That's over £100.
But the cost to Bell and British Telecom was not £2 per minute, or £1 per minute, or even 1p per minute, it was nothing at all. Their costs were not for the call, but for the infrastructure over which the call was delivered, a transatlantic cable. If there was one call for ten minutes, once a week essentially at random, that cable must still exist, but if there are 10 thousand call minutes per week, a thousand times more, it's the same cable.
So the big telcos all just picked a number and understood it as basically free income. If everybody agrees this call costs £2 then it costs £2 right, and those 10 thousand call minutes generate a Million pound annual income.
It's maybe easier for Americans to understand if you tell them that outside the US the local telephone calls cost money back then. Why were your calls free? Because why not, the decision to charge for the calls is arbitrary, the calls don't actually cost anything, but you will need to charge somehow to recoup the maintenance costs. In the US the long distance calls were more expensive to make up for this for a time, today it's all absorbed in a monthly access fee on most plans.
This analysis doesn't concern the limited bandwidth available for call delivery on plain old telephone networks (POTS). They did squeeze extra money out of the system with their networks as a monopoly, but the cost was zero only if you don't consider the cost of operating and maintaining the network, or the opportunity cost of having much less bandwidth than currently available. For the former, they still had to fix problems. For the latter if they had made calls pennies everyone would have had "all circuits are busy" all the time. A single line wasn't capable of carrying 10,000 calls back then. Pricing to limit usage to available bandwidth was as important as recouping infrastructure costs and ongoing maintenance. There's also a lemonade stand pricing effect. If you charge too little you don't get enough to cover costs. But if you charge too much, not enough people will do business and you won't cover costs. Also, ma bell was broken up in 1982, but regional monopolies lasted a lot longer (telecommunications act of 1996).
TAT-7 which was in operation in 1985 when I cited the £2 per minute price carried 4000 simultaneous calls, ie up to £8000 per minute
Its successor TAT-8 carried ten times as many calls a few years later, industry professionals opined that there was likely no demand for so many transatlantic calls and so it would never be full. Less than two years later TAT-8 capacity maxed out and TAT-9 was already being planned.
Today lots of people have home Internet service significantly faster than all three of these transatlantic cables put together.
To lay the cables required a huge amount of capital, to make that feasible its required financial engineering. That translates to high operating expenses.
SMS was originally piggybacking off unused bytes in packets already being sent to the tower, which was being paid for by existing phone bills. The only significant expenses involved transiting between networks. That was a separate surcharge in the early days.
Competition is the thing. Prices will drop as more AI code assistants get more adoption.
Prices will probably also drop if anyone ever works out how to feasibly compete with NVIDIA. Not an expert here, but I expect they're worried about competition regulators, who will be watching them very closely.
> Prices will drop as more AI code assistants get more adoption.
No, they won't. Because "AI assistants" are mostly wrapped around a very limited number of third-party providers.
And those providers are hemorrhaging money like crazy, and will raise the prices, limit available resources and cut off external access — all at the same time. Some of it is already happening.
> Prices will drop as more AI code assistants get more adoption.
What's the reasoning behind this? They are already doing the efficient "economies of scale" thing and they are already at full capacity (hence rate limiting).
The only way forward for this AI providers is to raise prices, not lower them.
The more AI assistants there are which are roughly equally competent, the more price becomes a factor. Mobility between providers is quick, it only takes one company willing to burn a lot of cash to win users or strategically hobble a competitor to start a price war. Maybe I'm wrong, but intuitively it feels like this will be the probable endgame.
It’s very expensive to create these models and serve them at scale.
Eventually the processing power required to create them will come down, but that’s going to be a while.
Even if there was a breakthrough GPU technology announced tomorrow, it would take several years before it could be put into production.
And pretty much only TSMC can produce cutting edge chips at scale and they have their hands full.
Between Anthropic, xAI and OpenAI, these companies have raised about $84 billion dollars in venture capital… VCs are going to want a return on their investment.
SMS was designed from the start to fit in the handul of unused bytes in the tower handshake that was happening anyway, hence the 160 char limit. Its marginal cost has always been free on the supply side.
SMS routing and billing systems did cost money.
Especially billing, as the standards had nothing for it, so it was done by 3rd party software for a very long time.
I think LLMs follow more of an Energy analogy: Gas or Electricity, or even water.
How much has any if these decreased over the last 5 decades? The problem is that as of right now, LLM cost is linearly (if not exponentially) related to the output. It's basically "transferring energy" converted into bytes. So unless we see some breakthrough in energy generation, or better use it, it will be difficult to scale.
This makes me wonder, would it be possible to pre-compue some kind of "rainbow tables" equivalent for LLMs? Either stored in the client or in the server; so as to reduce the computing needed for inference.
I don't think so. Yes, LLMs use electricity. But they use electricity in the data-center, not in your home. That's very different, because it's cheap to transfer tokens from the data-center to your home, but it's not cheap to transfer electricity from the data-center to your home. And that matters, because we can build a data-center in a place where there's lots of renewable and hence cheap energy (e.g. from solar or from water/wind).
If you think about it, LLMs are used mostly when people are awake, at least right now. And when is the sun shining? Right. So, build a data-center somewhere where land is cheap and lots of solar panels can be build right next to it. Sure, some other energy source will be used for stability etc., but it won't be as expensive as the energy price for your home.
> This makes me wonder, would it be possible to pre-compue some kind of "rainbow tables" equivalent for LLMs?
Already happening. Read up on how those companies do caching prompt-prefixes etc.
Isn't it the exact opposite? No one is making profit yet, it is a mad dash to monopolize the market, it has to get more expensive to ever turn profit, so the screws will turn
Yes! I agree completely. They’ve not even turned on the money faucets yet. These prices are likely just to hook users on the product, and will be more comparable to paying something that compares, but favourably, to minimum wage per hour in the future. Not implying a nefarious scheme, I just think that’s how the economics of it will pan out.
Just a small comment, encourage your son (or do it yourself) to archive his site and code so he can revisit it years later. I did and it is amazing to learn again the code I did 25 years ago!
Wrong. Lithium in Chile was nationalized in the 1980s by Pinochet. Lithium extraction and processing has been granted to privates in long term contracts that will continue to be valid but will not be renewed, and new mines will be owned directly by the state.
"Fun" fact about that, it was nationalized because it is a material for use in atomic bombs, in the 70 and 80 there wasn't another use for large scale lithium mining, except for that
>> Part of Oxford's 15 minutes city is to fine anyone that leaves their zone.
You are being dishonest. The fine is for driving through zones[1], not unlike fines for driving in city centers that are seen across the world.
[1]: "Under these new filters, residents will be able to drive freely around their own neighbourhoods but will be fined up to £70 for driving into other neighbourhoods through the filters." https://www.oxfordstudent.com/2023/01/25/15-minute-city-plan...
