I don’t think we should use current prices as landmarks for large scale demand. That Studio’s current prices is inflated because of a (presumably) short term supply crunch, not because the average user is willing to pay $24k for a home AI inference device.
It assumes that RAM remains supply constrained and that none of the existing RAM contracts are cut short.
But Meta and xAI putting A TON of AI compute onto the market. OpenAI and Anthropic are raising the costs of inference (by reducing how much inference users get via subscriptions). And we haven’t seen Oracle / CoreWeave struggle to pay their debts yet, but they will be selling assets once they get close to that point.
Funnily the current high end Mac Studio are not suited for current LLMs. M3 Ultra is "quite an old" chip for AI, despite its bandwidth. The issue for running local models (especially LLMs), you need few things to align really well: 1) compute power (affecting PP) 2) VRAM capacity (affecting model size you can load) 3) Bandwidth (somewhat affecting decoding speed).
The issue with the M3 chip is the compute performance, as it doesn't fit well the transformer architecture. This changed with the M5 (apple baked their own matmul into the chip), which would significantly speed up PP (and video/image generation btw), making the M5 Ultra significantly faster than the M3 Ultra and in practice much more usable. You can try to load Kimi or GLM on M3 Ultra, but it's not usable. Now the M5 Ultra is not out yet, but undoubtedly it will be a superior offering, and shilling 15k on 512GB version is actually reasonable (if it's every priced remotely around that tag).
By the way, they cut the supply because they run out of RAM. They still sell the 96 GB model. Everything else is gone, no stock, they cannot manufacture them. Even minis are 48GB max!
If demand doesn't fall down or current manufacturers supply go up, somebody (presumably in China) will spin up fabs. Apple wanted to use blacklisted Chinese RAM already.
DDR5 is still mostly made with DUV (remember Intel 14+++++++++?), and even though manufacturers have slowly been moving a few layers to EUV the advantage is at the margin. Lack of EUV at scale will not prevent China from ramping useful RAM into this market.
Chinese fabs might not be so tied with red tape and regulation upon regulation (which is a funny reversal, in terms of "communism vs capitalism" bureucracy/inefficiency cold war thinking)
>1.) China is not communist, even remotely so. China is fascist in every sense of the word.
Except in the actual historical sense. They appear to enjoy all sorts of freedoms, increased prosperity, even have elections at different levels but under a single party system. Which is not necessarily that different than a effectively two party system.
>2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.
Now that China is more effective, "it's easy because they're authoritarian". Before the argument was "authoritarianism can never be as effective as free-market democracy".
>3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."
It's real world capitalism, not some fantasy some guy imagined removing all warts.
I think the most ironic fact of the 21st century is that there are less than 20,000 naturalized citizens in China. Western leftists don't really have a good explanation for that one and it definitely leans into the fascist characterization.
Ebay and Amazon are flooded with it. Especially if you are looking for anything prior to DDR5. DDR2 and DDR3 are especially flooded with weird brands you've never heard of before.
Unfortunately its not so cheap anymore as everyone ramped prices up of course.
Last year I could still get 32GB of DDR4 for under $60 from chinese brands.
Some of us keep the worthwhile trash alive forever. This laptop is gonna be twenty years old soon and it's too comfortable of a machine to be tossed to ewaste.
If the increased demand is not short term, production capacity will eventually increase. In the meantime, the logistics disruptions and industrial material shortages and energy inflation will disappear as soon as the wars disrupting them stop, which should bring prices down.
If demand and prices keep rising without production capacity being built fast enough, there will likely eventually be a rush leading to overinvestment and price crashes, but there are too many other factors involved; state investment for security, international politics and trade relations, the possibility of an AI bubble burst, etc.
There are wars coming. The prices are not going down.
We are in a bubble which will be burst the moment the world starts retaliating against the US' 20+ year history of supporting genocide and committing war crimes unabated.
The underlying demand for massive RAM builds is coming from AI hyperscalers.
There are lots of signals that the sector has been overinvested and that corporate customers are pulling back on spending as the cost of the APIs is revealed.
Once the hyperscalers start struggling to bay their debts (it will happen, just a question of time), there will be a supply glut.
So the only question is: do we share the same definition of “short term”.
Unless the raw materials have an inherent limit on mining/production due to the amount present on the planet, why should or would companies not ramp up to eventually meet demand?
Edit: Okay, this doesn’t mean that that’s actually possible in the short-term, so I think you’re right. But that means as the silver lining, in the medium term horizon there’ll be enough supply again? :’)
For existing producers expanding capacity would be a risky move. But it's the perfect time for any newcomers to enter the market. Low yields and worse product don't matter as much right now, and by the time the market cools down you have everything dialed in and can compete on even ground
> it's the perfect time for any newcomers to enter the market
This is a good hypothesis. Curious if anyone has data on the failure rates of new entrants in semiconductors based on how frothy it was on founding.
On one hand, more demand makes selling easier. On the other hand, a shortage makes your input costs (consumable and capital) pricier.
EDIT: It seems like the 2 to 3 year lead time and a crowding effect from new entrants historically made booting up a fab into a boom a bad bet [1]. (The article argues, convincingly, that this time may be different.)
I heard that China was spinning up DDR5 (but not HBM?) production in the next couple of years, with the hope of outcompeting Korea and Taiwan in the mid to long term.
In two to four years, the Chinese will have at least half the memory market worldwide, and once in, they will continue forward and not look back.
I also believe there would be one or two tech companies that will get into memory by taking it in-house to make sure that they won’t have this problem again in the future.
Thanks for the link (and underlying thoughts), I really hadn’t considered that.
So essentially, due to technological progress and other factors inducing price collapses (or at least cycles), you can’t start stockpiling insane amounts of finished-product semiconductor, which means you can’t scale production at current technology levels to infinity either?
It assumes that RAM remains supply constrained and that none of the existing RAM contracts are cut short.
But Meta and xAI putting A TON of AI compute onto the market. OpenAI and Anthropic are raising the costs of inference (by reducing how much inference users get via subscriptions). And we haven’t seen Oracle / CoreWeave struggle to pay their debts yet, but they will be selling assets once they get close to that point.