That’s likely exactly how I feel about it. In the end the product companies like OpenAI will harness the monetary benefits of the academic advances.
You integrate, you build the product, you win, you don’t need to understand anything in terms of academic disciplines, you need the connections and the business smarts. In the end the majority of the population will be much more familiar with the terms ChatGPT and Copilot than with the names behind it, even if the academic behemoths such as Ilya and Andrej, who are quite prominent in their public appearance.
For the major population, I believe it all began with search over knowledge graphs. Wikipedia presented a dynamic and vibrant corpus. Some NLP began to become more prominent. With OCR, more and more printed works had begun to get digitalized. The corpus had been growing. With opening the gates of scientific publishers, the quality might have also improved. All of it was part of the grunt work to make today’s LLMs capable. The growth of the Cloud DCs and compute advancements have been making deep nets more and more feasible. This is just an arbitrary observation on the surface of the pieces that fell into place. And LLMs are likely just another composite piece for something bigger yet to come.
To me, that’s the fascination of how scientific theory and business applications live in symbiosis.
I’ve come to believe that such comparisons usually come from people who don’t understand the trade-offs of AWS in production.
Each project has certainly its own requirements. If you have the manpower and a backup plan with blue/green for every infrastructure component, then absolutely harness that cost margin of yours. If it’s at a break even when you factor in specialist continuity - training folks so nothing’s down if your hardware breaks, then AWS wins.
If your project can tolerate downtime and your SREs may sleep at night, then you might profit less from the several niners HA SLOs that AWS guarantees.
It’s very hard and costly to replicate what AWS gives you if you have requirements close to enterprise levels. Also, the usual argument goes - when you’re a startup you’ll be happy to trade CAPEX for OPEX.
For an average hobby project maybe not the best option.
As for latency, you can get just as good. Major exchanges run their matching engines in AWS DCs, you can co-locate.
More often than not, I’d rather avoid the self-focused staff who rarely give it to you with hygiene in mind and at this time of the year in the northern hemisphere are likely to be sick, the mediocre coffee (price surge in coffee beans), and the dirty tables at a café, and the uncomfortable seating. And it’s rather 5€ for the croissant alone, in many places these days. Lidl’s croissants aren’t very good but they’re only marginally less good than what you can hope for at a café. McDonald’s croissants in Italy are quite ok by the way.
Me too, but it's worth remembering that's not the case for everyone. Some people want to have a little chat with the person at the counter, sit down for 5 mins in the corner of the cafe and eat their croissant. 5 euro can be a good price if that's what you want, and it doesn't matter if the lidl croissant is free, it will still be disappointing to the person who wants the extras.
Absolutely. I believe what I wanted to convey is that there are trade-offs in every decision that you can make. Maybe that’s even the point of a decision in the first place.
This in turn means that you always have several options, and more importantly you can invent a new way to enjoy the experience you hope to get from that interaction at a café in your mind, maybe a scene from your past or from a movie, which you’re no longer as likely to experience on average.
That said, I’ve got a favorite café where I used to spend time frequently. But their service deteriorated. And the magic is gone. So I moved on with my expectations.
Back to the analogy with the hyperscalers. I had bad experience with Azure and GCP, I’ve experienced the trade-offs of DigitalOcean and Linode and Hetzner, and of running on-premises clusters. It turned out, I’m the most comfortable with the trade-offs that AWS imposes.
I was hesitant to submit for a while since it’s on YouTube, but I’ve got to the point of frustration with the iOS keyboard mangling my typed words to the point I couldn’t even comprehend and had to edit a lot.
The video is worth sharing and watching, the author carefully captures the behavior of iOS keyboard, highlighting its quirks. I searched on HN, it didn’t pop up for “iOS keyboard” as a search query.
I experimented with turning off swipe mode, not much better. I switched to swiping but it got tedious and very context specific.
So I’m sharing this just out of frustration. In fact, it’s similar on macOS where it “corrects” your words that it’s unfamiliar with or that are simply rare. Sometimes a minor typo flips the entire meaning even though the distance to the right word is minimal.
Interestingly, right now it’s better than usual.
I hope to learn what can be done do to improve typing on iOS and maybe get rid of the autocorrect in Apple apps on macOS.
Thanks, it’s certainly an option for iOS but for me typing on the phone keyboards isn’t as ergonomic as on physical keyboards, so I’d still like to keep autocorrect on. For comparison I have a couple Android keyboards, they’re more robust in correcting typos.
Yeah, the blog post is a very confused write-up. I saw lots of similar posts on LinkedIn recently, with quite a lot of likes and echo chamber comments. It’s just hilarious how a narrative emerges that reinforces biases due to ignorance. There must be a name for that sort of fallacy.
I love to write in Rust precisely because I can express failures more explicitly, it’s the transparency that wins here.
I’d frame the issue Cloudflare had rather in the PR review and QA corner, maybe as some AI complacency. But it’s not a problem with Rust.
Yes this is definitely an extra layer of safety. Essentially, you get two attention systems to get you where you’re going. If the backup system fails and the driving system recognizes that attention deficit to stop and alert, you’re safer than if you only had a sleepy driving system.
Tesla auto steering disengages for the ride, and you get several strikes until it turns off for good without recourse, if you’re inattentive while the mode is engaged, and you need to touch the wheel quite frequently especially at night to confirm for the legal requirements, since the interior cam can’t see you clearly. I hate to have to pull the wheel slightly so frequently when it gets dark (when wearing glasses).
I used to do Go in production for several years, along with Java and TypeScript event-loop backends. It was a breeze of fresh air, especially for new projects, where the Java conventions could be put to rest. But in such an environment, with mostly Java legacy, people did tend to bend Go to the Java idioms rendering the PR reviews very cumbersome.
From what I’ve experienced, if you need any fine-grained control over your data or allocations, precision on the type level, expressing nontrivial algorithms, Go is just too clumsy.
The more I read about how people use Go today and what issues people still have, the more I’m happy I picked Rust for almost everything. I even find it much more productive to write scripts in Rust than in Python or shell script. You just get it right very quickly and you don’t need to care about the idiosyncrasies of ancient tech debt that would otherwise creep into your new projects. And of course the outcome is way more maintainable.
Not saying that Rust hadn’t had its own warts, but most of them are made explicit. This is perhaps what I appreciate the most.
Intuitively, however, I still notice myself creating a new Python or shell script file when I need something quick, but then something doesn’t really work well the moment the logic gets a bit more complex, and I need to backtrack and refactor. With Rust, this hasn’t been an issue in my experience.
And intuitively, I still tend to think in Java terms when designing. It’s funny how it sticks for so long. And when writing some Java, I miss Go’s use-site interfaces and TypeScript’s structural typing, while I miss nominal typing in TypeScript. It’s just maybe that you get used to workarounds and idiosyncrasies in some system and then carry them over to another paradigms.
I do like Go’s value propositions, and lots of its warts have been sorted out, but I’m just not as productive in it for my use cases as I am with Rust. It just checks way more boxes with me.
I had similar issues with it. What helps today are LLMs. It’s really a boon to configuring such things. You do it on ace and forget unless that’s your job. Did you try to do what you had wanted back then with a recent LLM?
It’s not about the languages only, but also about runtimes and libraries. The vert.x vertices are reactive. Java devrel folks push everyone from reactive to virtual threads now. You won’t see it perform in that ballpark. If you look at the bottom of the benchmark results table, you’ll find Spring Boot (servlets and a bit higher Reactor), together with Django (Python). So “Java” in practice is different from niche Java. And if you look inside at the codebase, you’ll see the JVM options. In addition, they don’t directly publish CPU and memory utilization. You can extract it from the raw results, but it’s inconclusive.
This stops short of actually validating the benchmark payloads and hardware against your specific scenario.
> So “Java” in practice is different from niche Java.
This is an odd take, especially when in the discussion of Rust. In practice when talking about projects using Rust as an http server backend is non-existent in comparison. Does that mean we just get to write off the Rust benchmarks?
I don’t understand what you’re saying. Typical Java is Spring Boot. Typical Rust is Axum and Actix. I don’t see why it would make sense to push the argument ad absurdum. Vert.x is not typical Java, its not easy to get it right. But Java the ecosystem profits from Netty in terms of performance, which does the best it can to avoid the JVM, the runtime system. And it’s not always about “HTTP servers” though that’s what that TechEmpower benchmark subject matter is about - frameworks, not just languages.
Your last sentence reads like an expression of faith. I’ll only remark that performance is relative to one’s project specs.
In some of those benchmarks, Quarkus (which is very much "typical Java") beats Axum, and there's far more software being written in "niche Java" than in "typical Rust". As for Netty, it's "avoiding the JVM" (standard library, really) less now, and to the extent that it still does, it might not be working in its favour. E.g. we've been able to get better results with plain blocking code and virtual threads than with Netty, except in situations where Netty's codecs have optimisations done over many years, and could have been equally applied to ordinary Java blocking code (as I'm sure they will be in due time).
Hey Ron, I’ve got deep respect for what you do and appreciate what you’re sharing, that’s definitely good to know. And I understand that many people take any benchmark as a validation for their beliefs. There are so many parameters that are glossed over at best. More interesting to me is the total cost of bringing that performance to production. If it’s some gibberish that takes a team of five a month to formulate and then costs extra CPU and RAM to execute, and then becomes another Perlesque incantation that no one can maintain, it’s not really a “typical” thing worth consideration, except where it’s necessary, scoped to a dedicated library, and the budget permits.
I don’t touch Quarkus anymore for a variety of issues. Yes, sometimes it’s Quarkus ahead, sometimes it’s Vert.x, from what I remember it’s usually bare Vert.x. It boils down to the benchmark iteration and runtime environment. In a gRPC benchmark, Akka took the crown in a multicore scenario - at a cost of two orders of magnitude more RAM and more CPU. Those are plausible baselines for a trivial payload.
By Netty avoiding the JVM I referred mostly to its off-heap memory management, not only the JDK APIs you guys deprecated.
I’m deeply ingrained in the Java world, but your internal benchmarks rarely translate well to my day-to-day observations. So I’m quite often a bit perplexed when I read your comments here and elsewhere or watch your talks. Without pretending I comprehended the JVM on a level comparable to yours, in my typical scenarios, I do quite often manage to get close to the throughput of my Rust and C++ implementations, albeit at a much higher CPU and memory cost. Latency & throughput at once is a different story though. I genuinely hope that one day Java will become a platform for more performance-oriented workloads, with less nondeterminism. I really appreciate your efforts toward introducing more consistency into JDK.
You integrate, you build the product, you win, you don’t need to understand anything in terms of academic disciplines, you need the connections and the business smarts. In the end the majority of the population will be much more familiar with the terms ChatGPT and Copilot than with the names behind it, even if the academic behemoths such as Ilya and Andrej, who are quite prominent in their public appearance.
For the major population, I believe it all began with search over knowledge graphs. Wikipedia presented a dynamic and vibrant corpus. Some NLP began to become more prominent. With OCR, more and more printed works had begun to get digitalized. The corpus had been growing. With opening the gates of scientific publishers, the quality might have also improved. All of it was part of the grunt work to make today’s LLMs capable. The growth of the Cloud DCs and compute advancements have been making deep nets more and more feasible. This is just an arbitrary observation on the surface of the pieces that fell into place. And LLMs are likely just another composite piece for something bigger yet to come.
To me, that’s the fascination of how scientific theory and business applications live in symbiosis.