It's wild that Sonnet 4.6 is roughly as capable as Opus 4.5 - at least according to Anthropic's benchmarks. It will be interesting to see if that's the case in real, practical, everyday use. The speed at which this stuff is improving is really remarkable; it feels like the breakneck pace of compute performance improvements of the 1990s.
The most exciting part isn't necessarily the ceiling raising though that's happening, but the floor rising while costs plummet. Getting Opus-level reasoning at Sonnet prices/latency is what actually unlocks agentic workflows. We are effectively getting the same intelligence unit for half the compute every 6-9 months.
My Dad used to make the same joke in the 1980s about how they'd told him in the 1950s that nuclear power would be "too cheap to meter" which I assume is probably where the trope originated.
This is what excited me about Sonnet 4.6. I've been running Opus 4.6, and switched over to Sonnet 4.6 today to see if I could notice a difference. So far, I can't detect much if any difference, but it doesn't hit my usage quota as hard.
The point of the penny-farthing is that you drive the front wheel directly with the pedals, but this seems to have the pedals in a spot where they would drive a chain, although there is no chain?
> Sonnet 4.6 is roughly as capable as Opus 4.5 - at least according to Anthropic's benchmarks
Yeah it's really not. Sonnet still struggles while Opus, even 4.5 succeeds (and some examples show Opus 4.6 is actually even worse than 4.5, all while being more expensive and taking longer to finish).
Flash models are nowhere near Pro models in daily use. Much higher hallucinations, and easy to get into a death sprawl of failed tool uses and never come out
You should always take those claim that smaller models are as capable as larger models with a grain of salt.
Flash model n is generally a slightly better Pro model (n-1), in other words you get to use the previously premium model as a cheaper/faster version. That has value.
They do have value, because they are much much cheaper.
But no, 3.0 flash is not as good as 2.5 pro, I use both of them extensively, especially in translation. 3.0 flash will confidently mistranslate some certain things, while 2.5 pro will not.
Totally fair. Translation is one of those specific domains where model size correlates directly with quality, and no amount of architectural efficiency can fully replace parameter count.
Given that users prefered it to Sonnet 4.5 "only" in 70% of the cases (according to their blog post) makes me highly doubt that this is representative of real-life usage. Benchmarks are just completely meaningless.
For cases where 4.5 already met the bar, I would expect 50% preference each way. This makes it kind of hard to make any sense of that number, without a bunch more details.
I sent Opus a photo of NYC at night satellite view and it was describing "blue skies and cliffs/shore line"... mistral did it better, specific use case but yeah. OpenAI was just like "you can't submit a photo by URL". Was going to try Gemini but kept bringing up vertexai. This is with Langchain
I can see this image shows an *aerial/satellite view of a coastline*. Here are the key features I can identify:
## Geographic Features
- *Ocean/Sea*: A large body of deep blue water dominates a significant portion of the image
- *Coastline*: A clearly defined boundary between land and water with what appears to be a rugged or natural shoreline
- *Beach/Shore*: Light-colored sandy or rocky coastal areas visible along the water's edge
## Terrain
- *Varied topography*: The land area shows a mix of greens and browns, suggesting:
- Vegetated areas (green patches)
- Arid or bare terrain (brown/tan areas)
- *Possible cliffs or elevated terrain* along portions of the coast
## Atmospheric Conditions
- *Cloud cover*: There appear to be some clouds or haze in parts of the image
- Generally clear conditions allowing good visibility of surface features
## Notable Observations
- The color contrast between the *turquoise/shallow nearshore waters* and the *deeper blue offshore waters* suggests varying ocean depths (bathymetry)
- The coastline geometry suggests this could be a *peninsula, island, or prominent headland*
- The landscape appears relatively *semi-arid* based on the vegetation patterns
---
Note: Without precise geolocation metadata, I'm providing a general analysis based on visible features. The image appears to capture a scenic coastal region, possibly in a Mediterranean, subtropical, or tropical climate zone.
Would you like me to focus on any specific aspect of this image?
Strangely enough, my first test with Sonnet 4.6 via the API for a relatively simple request was more expensive ($0.11) than my average request to Opus 4.6 (~$0.07), because it used way more tokens than what I would consider necessary for the prompt.
This is an interesting trend with recent models. The smarter ones get away with a lot less thinking tokens, partially to fully negating the speed/price advantage of the smaller models.
It's a perfectly reasonable choice: flexible, well specified, well supported, reasonably performant. I think the extreme level of hype 20 years ago was overdone and (just like with anything) there's good ways to adopt it and bad ways. But as a basic technology choice, it's fine. Particularly these days when you can have a coding agent write the parser boilerplate, etc. for you.
> It's a perfectly reasonable choice: flexible, well specified, well supported, reasonably performant. I think the extreme level of hype 20 years ago was overdone and (just like with anything) there's good ways to adopt it and bad ways. But as a basic technology choice, it's fine.
Absolutely with you up to here, but...
> Particularly these days when you can have a coding agent write the parser boilerplate, etc. for you.
Absolutely not. Having seen the infinite different ways a naive implementation of XML goes wrong, arguably being one of the main causes of death for XHTML because browsers rightfully rejected bad XML, "Don't roll your own XML implementation" should be right up there with "Don't roll your own crypto".
I don't feel like it's going out on a limb to say that if someone needs to defer to a LLM to implement XML they're not qualified to determine if it's done it right and/or catch what it got enthusiastically wrong.
Oh sorry, I don't at all intend to say you should write your own parser! Totally agree: "Don't roll your own XML implementation"
What I was addressing is, interfacing with an XML parser and converting that into whatever your internal representation is, can be a chore. LLMs are great at that stuff.
I'm going to stand by my position there, if you're writing an application that's primary technical purpose is to communicate via an XML based protocol and feel the need to outsource the XML part to the bullshit machine, IMO you probably shouldn't be writing that application.
There was a sockets API though (https://en.wikipedia.org/wiki/Winsock) and IIRC we all used Trumpet Winsock on Windows 3.1 with our dialup connections. But could have been 3.11 - my memory is a bit hazy.
There's also a corollary to this: if the organization does not recognize some work as needed or useful, you could well be actively wasting your time putting effort into it. There might be a good reason the company doesn't care that you just don't see, and leadership could be (at best) confused about why you would spend time on it.
Given enough soft skills, you can persuade your boss that what you are doing is important, and help him/her represent the department as uncovering and proactively addressing an important issue. Ideally it should align well with the boss's boss agenda.
For sure, but sometimes what you or I think should be important really isn't in the grand scheme of things. An example could be focusing on cost or efficiency - generally very reasonable things to care about - but if all a company cares about right now is growth at all costs, then that focus would be wrong. This can happen - the company leadership might see a market that they absolutely must enter and be dominant in no matter the cost. That may not filter down well 3-4 layers of management; so the soft skill in that instance would be in sussing out what several layers of management above you actually care about and surfacing to them things that align with those concerns.
reply