Maybe I am blinded by my own use case, but I find the caching pricing and strategy (since different providers use a different implementation of caching as well as different pricing) to be a major factor rather than just the "raw" per token cost, and that is missing here, as well as on the Simon Willison site [1]. Do most people just not care / not use caching that much that it matters?
I just filled it out for store credit, and it didn't matter at all the condition of my phone (it's in horrible shape), I just put in the IMEI number and that was that. It could be different for cash, though.
Interesting quote from the venturebeat article linked:
> “There is also a reason why clinicians who deal with patients on the front line are trained to ask questions in a certain way and a certain repetitiveness,” Volkheimer goes on. Patients omit information because they don’t know what’s relevant, or at worst, lie because they’re embarrassed or ashamed.
In order for an LLM to really do this task the right way (comparable to a physician), they need to not only use what the human gives them but be effective at extracting the right information from the human, the human might not know what is important or they might be disinclined to share, and physicians can learn to overcome this. However, in this study, this isn't actually what happened - the participants were looking to diagnose a made-up scenario, where the symptoms were clearly presented to them, and they had no incentive to lie or withhold embarrassing symptoms since they weren't actually happening to them, it was all made up - and yet, it still seemed to happen, that the participants did not effectively communicate all the necessary information.
> In order for an LLM to really do this task the right way (comparable to a physician), they need to not only use what the human gives them but be effective at extracting the right information from the human
That's true for most use-case, especially for coding.
> Sure, but think of a good help desk tech: if they waited for users to accurately report useful information, nothing would ever get fixed.
I sometimes also have to do "help-desk-like" duties on the applications that I am responsible for (think like 3rd level technical support):
I can tell you that you can train your users to give more helpful useful information (but of course sometimes they don't don't know by themselves what is important and what is not).
Sure. But as a patient, you are also not expected to know what is or isn't important. Omitting unimportant information (to you) because your brain does a low pass filter is partially what the doctor is trying to bypass.
Its as if every single person had to be an expert in every field to be able to function, that's really not a thing and we expect the actual experts to know how to extract the needed information.
That's one of the main differences between mediocre and incredible engineers, being able to figure out what the problem that needs to be solved is and not work on whatever a stakeholder asks them to build.
Okay, so your code has been segfaulting at line 123 in complicated_func.cpp, and you want to know to which version of libc you have to roll back to as well as related packages if any.
What's the current processor temperature, EPS12V voltage, and ripple peaks if you have a oscilloscope? Could you paste cpuinfo? Have you added or removed RAM or PCIe device recently? Does the chassis smell and look normal, no billowing smoke, screeching noise, fire?
Good LLMs might start asking these questions soon, but you wouldn't supply these information at the beginning of interaction(and it's always the PSU).
Yeah, there's a lot of agency on both sides of the equation when it comes to any kind of consultant. You're less likely to have bad experiences with doctors if you're self aware and thoughtful about how you interact with them.
I think that the concept of a "foundation model" for time series is actually a bit flawed as presented in this blog post. A foundation model is interesting because it is capable of many tasks _beyond the target tasks_ that it was trained to do, whereas what the author is looking for is a time-series model that can make out-of-distribution predictions without re-training - which is, in my opinion, a problem that is pretty well solved by existing ARIMA and (especially) Prophet models (Yes, you have to re-fit the model on your distribution, but this is not at all akin to the task of training or fine-tuning an LLM, it's something you can do in seconds on a modern CPU, and yes, there are certain hyperparameters that may need to be selected, but they are actually fairly minimal).
But for a model to make out-of-distribution predictions does not make it a foundation model for time series, really that's just the basic task that all time series forecasting models do. A more interesting question is, does an LLM architecture seem to improve the task of univariate or multivariate time-series prediction? I don't think the answer is yes, although, depending on your domain, being able to use language inputs to your model may have a positive impact, and the best way to incorporate language inputs is certainly to use a transformer architecture, but that isn't what is addressed in this post.
A lot of people try to hedge this kind of sober insight along with their personal economic goals to say all manner of unfalsifiable statements of adequate application in some context, but it is refreshing to try to deal with the issues separately and I think a lot of people miss the insufficiency compared to traditional methods in all cases that I've heard of so far.
A lot of it is just fun and silly, but for me it was also an interesting way to develop my taste in something - to hear other people who are real heavy connoisseurs of something discuss it, and learning from them. Of course, you can get this from your friends and the people who are really around you in your life (or, just don't develop your taste at all because you just like what you like), and there's nothing wrong with that, I get why some people find it odd to watch people play video games.
You have to understand as well that Giant Bomb was the first of its kind in a lot of ways, this was an era where video game journalism began to loosen up from the corporate, PR-friendly, very stiff and consumer-focused era it had been in during the dominance of print media, and Giant Bomb was this novel thing where people who had been deeply involved in that era began to find their own voices. If you followed video games at the time online, Giant Bomb was this total breath of fresh air.
You might not know this if you don't actually play these games (Madden, 2K for NBA, MLB The Show), but the commentary is extremely high quality, sometimes comparable to the TV broadcast with riffs and tangents as well as describing the action. Over many years of producing these games they have continually refined the process. Of course, eventually you will hear repeating dialogue if you play the games enough, but I think the baseline quality is going to be _very_ hard to replicate with an LLM.
> Tech needs to keep innovating to keep investors happy and keep investing.
It's nice to think that this is just a "tech" problem but unfortunately this is a wider problem in the rich world - it just so happens that "tech" has been the answer for finding huge economic growth for the past few decades. The whole economy is addicted to tech growth at this point (including your 401k if you have one, those of your your friends and neighbors).
I think part of the author's point is that, specifically if you are _coding_ for fun, it is much harder to "turn off" that part of your brain that analyzes it from a business perspective. It's not as if you can close one IDE at 4 and open another IDE at 4:01 at put yourself in a different mindset.
I do just that by switching colour schemes. Light mode at work, dark mode at home. It takes a little bit, but the brain is easily tricked by flashy lights.
Of course you can. (I mean many of us can.) I've been doing it for years. I'm surprised that because you find it hard to "turn off" that part of your brain, it must be so for everyone else. It isn't. There are many of us who code for fun and do it with a totally different mindset with no "business thoughts" in mind.
Speaking for myself, the author's post resonated with me in two ways - both that it's hard to turn off the business side of the brain ("Could this side project be a startup? Should I build it this way just in case I decide to do that later?") but also that I find it hard to turn off the manager brain ("Is this really the right order to do things? Is this the most valuable thing I could be doing?"), too, other people in the thread are mentioning thinking about opportunity cost to be interfering with their ability to commit to side projects (and also to actually _enjoy_ doing them).
It's easy to switch for some of us because the last thing in the world we want is more work and a startup is a lot of work. When I'm making something for myself it never crosses my mind that I could commercialize my hobby project because I know that's the fastest way to ruin a hobby.
> So you want them to watch a few exclusive shows a year so they feel like they got their money's worth, while not actually costing netflix much.
No, that's not what the strategy is and they're quite open about it - the strategy is to maximize user consumption for every user, because that keeps them subscribed. I think a lot of people think that they use sophisticated analytics and machine learning etc to decide what to greenlight, but they don't. They use the judgment (and politics, and egos) of Hollywood studio executives (and often the same Hollywood execs that a few years ago were employed in "legacy" media). Although I will grant that they've been innovative in producing/distributing international content, this is really just globalization and labor arbitrage (it is cheaper produce content not in Hollywood, that's not news - they just spend the extra $$$ localizing international content to different global target distribution markets but again, this flow has happened forever, it's just typically been Hollywood -> localization -> foreign market rather than foreign production -> localization -> Anglophone market).
Where analytics and ML does come into play is deciding which things out of their enormous catalogue they push to individual users at any one time - that process is highly reactive, individualized, dynamic - that's why strange and seemingly random media become big hits on Netflix while being largely ignored by the commentariat, and vice versa, why series with dedicated fanbases don't get renewed (the analytics tell you that, despite the apparent success, further investment will not improve user engagement with the platform by enough to be worth the spend).
> Where analytics and ML does come into play is deciding which things out of their enormous catalogue they push to individual users at any one time […]
Except they don't. Only Netflix has a vague reminiscence of ML/analytics-driven recommendations. The rest of streaming platforms offer anything but personalisation, which is particularly bewildering considering the financial and engineering resources available to the streaming behemoths. I do not have subscriptions for each streaming platform out there, but out of the several ones I do, Disney+ and especially Prime are the worst offenders that throw random trash either into the home screen or into the «personalisation» section, e.g. «because you have watched The Expanse, we thought you would like an NBA season / rugby World Cup» and stuff like that. You would think that obsessively clicking the «Like» button after watching something you actually liked would influence the personalisation, except it does not. Disney+, again, fills up the home screen with garbage I would never fathom could even exist.
The thing is that with the currently available technology, building a capable (it does not have to be perfect) recommendation is not that hard. At work, we almost daily design and build solutions that employ semantic similarity search / something, and with the current crop of multimodal LLM's that can generate vector embeddings with ease, it is relatively easy to build out a recommendation engine or algorithm tailored for the needs of a specific streaming platform.
Granted, specific optimisations are required and there will be unique new challenges in there; however, crafting such a solution is well within the realm of possibility. And the amount of money required is not even that high considering that many building blocks are available as mature, managed services, or creating a bespoke and tailored in-house solution does not require starting off from the clean slate by leveraging the prior art. That was not the case, say, back in 2018, but in 2025 it is a reality. For a bizarre reason that is beyond my comprehension, almost no streaming platforms do that.
> […] that process is highly reactive, individualized, dynamic […]
That is the aspiration and the high ideal; however, something else is going on, and it is not entirely inconceivable that the marketing department is complicit in the foul play.
To be clear, I was talking about Netflix, not Disney+. That's a completely different company with a different model and conflating the two is your mistake, not mine.
Maximizing reduces the variations, but there's still quite a bit of variation because of different display resolution + scaling settings + OS configuration (eg. short or tall taskbars).
[1] https://llm-prices.com/