I've worked with a type of (anti?) developer in my career that seems to only be able to add code. Never change or take it away. It's bizarre. There's some calculation bug, then a few lines down some code which corrects it instead of just fixing the original lines.
It's bizzare, and as horrible as you might imagine.
And it's been more than one or two people I've seen do this.
People need to understand that code is a liability. LLMs hasn't changed that at all. You LLM will get every bit as confused when you have a bug somewhere in the backend and you then work around it with another line of code in the front end. line of code
I'm guessing they have a lot of shares in the AI companies they work(ed) for, and they would like to pump their value so they can buy an even nicer carribean island than they can already afford?
Kokotajlo in particular is notable for being the guy who quit OpenAI in 2024 in protest of their policy of requiring researchers to abide by a non-disparagement agreement to retain their equity. In the end OpenAI caved and changed their policy, but if he was lying all along to inflate the value of his shares, it would have been quite a 4d chess move of him to gamble the shares themselves on doing so.
Isn't it just that he left way before gpt-5, then? At that point a sufficiently naive person could have believed that scaling was going to lead to AGI, but that sort of optimism died after he was already an outsider.
I love the total lack of humility on that site.
"What if the METR study turns out not to capture anything relevant? We just add a constant gap to be conservative!".
But I guess these guys aren't really scientist, so it's probably a lot to ask that they relate critically to what they are doing and be honest about the limitations of their methods.
What if it turns out that the more you scale the more your LLM resembles a lobotomized human. It looks like it goes really well in the beginning, but you are just never going to get to Einstein. How does that affect everything?
What if it turned out that those AI companies were maybe having a whole bunch of humans solving the problems that are currently just below the 50% reliability threshold they set, and do fine tuning with those solutions. That will make their models perform better on the benchmark, but it's just training for the test... will the constant gap be a good approximation then?
Kokotajlo quit because he didn't think OpenAI would be good stewards of AGI (non-disparagement wasn't in the picture yet). As part of his exit OpenAI asked him to sign a non-disparagement as a condition of keeping his equity. He refused and gave up his equity.
To the best of my knowledge he lost that equity permanently and no longer has any stake in OpenAI (even if this episode later led to an outcry against OpenAI causing them to remove the non-disparagement agreement from future exits).
I enjoyed playing mastermind with LLMs where they pick the code and I have to guess it.
It's not aware that it doesn't know what the code is (it isn't in the context because it's supposed to be secret), but it just keeps giving clues. Initially it works, because most clues are possible in the beginning, but very quickly it starts to give inconsistent clues and eventually has to give up.
At no point does it "realise" that it doesn't even know what the secret code is itself. It makes it very clear that the AI isn't playing mastermind with you, it's trying to predict what a mastermind player in it's training set would say, and that doesn't include "wait a second, I'm an AI, I don't know the secret code because I didn't really pick one!" so it just merilly goes on predicting tokens, without any sort of awareness what it's saying or what it is.
It works if you allow it to output the code so it's in context, but probably just because there is enough data in the training set to match two 4 letter strings and know how many of them matches (there's not that many possibilities).
It really dispelled the illusion for me, but it's not that easy to find those examples, but the combinatorics of possible number of guesses is untractable enough that it can't learn a good set of clues for all possible guesses.
Right! I had to get up in the morning, at ten o'clock at night, half an
hour before I went to bed... sorry, wrong sketch. I had to set up my PIN number to display on the LCD display of an ATM machine with the instructions printed in PDF format telling me how to add VAT tax all before midday GMT time.
And you try and tell the young people of today about RAS syndrome, they won't believe you!
It’s a comment about who Gary Marcus is presenting himself as
My intention is for other people to think what I believe which is Gary Marcus is a hack and has no business being listened to with respect to technical evaluation of AI because he’s not technically competent enough to do. The existence of his polemics waste everybody’s time and generally waste resources like we’re wasting right now.
His entire schtick has been as the debunker in chief of claims of AI capabilities
If you actually look at his polemics they increasingly have nothing to do with his original argument because his original argument not only is flawed but is ignorant of the technical capabilities
The guy founded two AI companies and exited one, is a researcher who is an expert on how actual human babies learns things and who has done research comparing how LLMs learn and represent language compared to humans. How much more expertise does he need to have an opinion?
What "technical competence" do you need to provide a technical evaluation, o person who is posting a comment to the "here's how I vibe-coded a fantasy football analytics service" forum? You think everyone here has a technically deep background?
> This translates to an average throughput of 3.5 PRs per engineer per day, and surprisingly the throughput has increased as the team has grown to now seven engineers.
Several related reasons working at once. The nature of work changed. The boundary between accidental and incidental complexity shifted (and it’s unclear whether this distinction still exists). Niche specializations within the field emerged. The way to structure and decompose projects changed dramatically (agile and stuff).
One pathological example: if you’re running a server-based product, quite often what stands between you and a new feature launch is literally couple of thousands of lines of Kubernetes YAML. Would adding someone who’s proficient in Kubernetes slow you down? Of course not.
One may say, hey, this is just the server-side Kubernetes-based development being insane, and I’ll say, the whole modern business of software development is like this.
That’s a lovely comment, thank you. If you’re keen to think about it more, consider the fact that the existing members of the project that’s being late are actually in not as much of an advantage compared to the new joiners, as it’s common to think.
Yes, they know how the feature they work on relates to other features, but actually implementing that feature is very often mostly involves fighting with technology, wrangling the entire stack into the shape you need.
In Brooks’s times the stack was paper-thin, almost nonexistent. In modern times it’s not, and adding someone who knows the technology, but doesn’t have the domain knowledge related to your feature still helps you. It doesn’t slow you down.
One may argue that I’m again pointing to the difference between accidental and incidental complexity, and my argument is essentially “accidental complexity takes over”, but accidental complexity actually does influence your feature too, by defining what’s possible and what’s not.
I sort of agree that the surface area and incidental complexity of stacks give more space to plug more developers in than was true in the 70s and 80s. But I disagree strongly this invalidates Brooks Law. Certainly there are cases where adding people helps, especially if they are stronger engineers than the ones that are already there, but I’ve also seen way too many projects devolve into resourcing conversations when the real problem was over-complicated, poorly reasoned requirements, boil-the-ocean solutions promising a perfect end state without a clear plan to get there iteratively.
"We excluded reasoning models from our analysis of per-token prices. Reasoning models tend to generate a much larger number of tokens than other models, making these models cost more in total to evaluate on a benchmark. This makes it misleading to compare reasoning models to other models on price per token, at a given performance level."
It's just price per token. Token usage is exploding.
Absolutely, the belief in scientific circles is that the way forward to develop cures (or at least treatment that slows down the progression) is to treat it early. When you get to the point where you start showing clear symptoms, your brain is already mush. If you have a potential treatment that attacks the root cause, you would have to catch the very early, pre-clinical, stages of the disease, but without good diagnostics there is no way to do that (short of giving the disease to a wide swath of the population, like a vaccine... but that gets expensive very quickly, and side effects become a bigger worry.
reply