I've found Claude's usefulness is highly variable, though somewhat predictable. It can write `jq` filters flawlessly every time, whereas I would normally spend 30 minutes scanning docs because nobody memorizes `jq` syntax. And it can comb through server logs in every pod of my k8s clusters extremely fast. But it often struggles making quality code changes in a large codebase, or writing good documentation that isn't just an English translation of the code it's documenting.
The problem I had that the larger your project gets, the more mistakes Claude makes. I (not a parent commenter) started with a basic CRUD web app and was blown away by how detailed it was, new CSS, good error handling, good selection and use of libraries, it could even write the terminal commands for package management and building. As the project grew to something larger Claude started forgetting that some code already existed in the project and started repeating itself, and worse still when I asked for new features it would pick a copy at random leaving them out of sync with eachother. Moving forward I've been alternating between writing stuff with AI, then rewriting it myself.
> The problem I had that the larger your project gets, the more mistakes Claude makes
I think the reason for this is because these systems get all their coding and design expertise from training, and while there is lots of training data available for small scale software (individual functions, small projects), there is much less for large projects (mostly commercial and private, aside from a few large open source projects).
Designing large software systems, both to meet initial requirements, and to be maintainable and extensible over time, is a different skill than writing small software projects, which is why design of these systems is done by senior developers and systems architects. It's perhaps a bit like the difference between designing a city and designing a single building - there are different considerations and decisions being made. A city is not just a big building, or a collection of buildings, and large software system is not just a large function or collection of functions.
Here's mine fully deployed, https://hackernewsanalyzer.com/. I use it daily and have some users. ~99.7% LLM code. About 1 hour to first working prototype then another 40 hours to get it polished and complete to current state.
It shows, quite an interesting wrapper over GPT with unauthorized access to prompting it you assembled there ;) Very much liked the part where it makes 1000 requests pulling 1000 comments from the firebase to the client and then shoots them back to GPT via supabase
41 hours total of prompting, looking at code diffs, reverting, reprompting, and occasional direct code commits. I do review the full code changes nearly every step of the way and often iterate numerous times until I'm satisfied with the resulting code approach.
Have you tried to go back to the old way, maybe just as an experiment, to see how much time you are actually saving? You might be a little surprised! Significant "reprompting" time to me indicates maybe a little too much relying on it rather than leading by example. Things are much faster in general if you find the right loop of maybe using Claude for like 15%-20% of stuff instead of 99.7%. You wouldn't give your junior 99.7% ownership of the app unless they were your only person, right? I find spending time thinking through certain things by hand will make you so much more productive, and the code will generally be much better quality.
I get that like 3 years ago we were all just essentially proving points building apps completely with prompts, and they make good blog subjects maybe, but in practice they end up being either fragile novelties or bloated rat's nests that end up taking more time not less.
I’ve done things in days that in the before times would have took me months. I don’t see how you can make that time difference up.
I have at least one project where I can make that direct comparison - I spent three months writing something in the language I’ve done most of my professional career in, then as a weekend project I got ChatGPT to write it from scratch in a different language I had never used before. That was pre-agentic tools - it could probably be done in an afternoon now.
I'm not a fulltime developer, but manage a large dev team now. So, this project is basically beyond my abilities to code myself by hand. Pre llm, I would expect in neighborhood of 1.5-2 months for a capable dev on my team to produce this and replicate all the features.
If you haunt the pull requests of projects you use I bet you'll find there's a new species of PR:
> I'm not an expert in this language or this project but I used AI to add a feature and I think its pretty good. Do you want to use it?
I find myself writing these and bumping into others doing the same thing. It's exciting, projects that were stagnant are getting new attention.
I understand that a maintainer may not want to take responsibility for new features of this sort, but its easier than ever to fork the project and merge them yourself.
I noticed this most recently in https://github.com/andyk/ht/pulls which has two open (one draft) PRs of that sort, plus several closed ones.
Issues that have been stale for years are getting traction, and if you look at the commit messages, it's AI tooling doing the work.
People feel more capable to attempt contributions which they'd otherwise have to wait for a specialist for. We do need to be careful not to overwhelm the specialists with such things, as some of them are of low quality, but on the whole it's a really good thing.
If you're not noticing it, I suggests hanging out in places where people actually share code, rather than here where we often instead brag about unshared code.
> People feel more capable to attempt contributions
That does not mean that they are more capable, and that's the problem.
> We do need to be careful not to overwhelm the specialists with such things, as some of them are of low quality, but on the whole it's a really good thing.
That's not what the specialists who have to deal with this slop say. There have been articles about this discussed here already.
At this point my prior is that all these 300/ns projects are some kind of internal tools, with very narrow scope and many just for a one-off use.
Which is also fine and great and very useful and I am also making those, but it probably does not generalize to projects that require higher quality standards and actual maintenance.
Places that aren't software businesses are usually the inverse. The software is extremely sticky and will be around for ages, and will also bloat to 4x the features it was originally supposed to have.
I worked at an insurance company a decade ago and the majority of their software was ancient. There were a couple desktops in the datacenter lab running Windows NT for something that had never been ported. They'd spent the past decade trying to get off the mainframe and a majority of requests still hit the mainframe at some point. We kept versions of Java and IBM WebSphere on NFS shares because Oracle or IBM (or both) wouldn't even let us download versions that old and insecure.
Software businesses are way more willing to continually rebuild an app every year.
There's a massive incentive not to share them. If I wrote a project using AI I'd be reluctant to publish it at all because of the backlash I've seen people get for it.
People are and always were reluctant to share their own code just the same. There is nothing to be gained, the chances of getting positive reviews from fellow engineers are slim to none. We are a critical and somewhat hypocritical bunch on average.
Claude has taught me so much about how to use jq better. And really, way more efficient ways of using the command line in general. It's great. Ironically, the more I learn the less I want to ask it to do things.
Maybe the most depressing part of all this is if people start thinking they would not have been able to do things without the LLM. Of course they would have, it's not like LLMs can do anything that you cannot. Maybe it would have taken more time at least the first time and you would have learned a few things in the process.
Sure, I can write all of it. But I simply won’t. I have Claude generated Avalonia C# applications and there is no way I would have written the thousands of lines of xaml they needed for the layouts. I would just have done it as a console app with flags.
But reducing friction, eliminating the barrier to entry, is of fundamental importance. It's human psychology; putting running socks next to your bed at night makes it like 95% more likely you'll actually go for a run in the morning.
I understand the point, and to some degree agree. For myself, I really couldn't (not to say it wouldn't have been possible). I tried many many times over so many years and just didn't have the mental stamina for it, it would never "click" like infra/networking/hardware does etc and I would always end up frustrated.
I have learnt so much in this process, nowhere near as much as someone that wrote every line (which is why I think being a good developer will be a hot commodity) but I have had so much fun and enjoyment, alongside actually seeing tangible stuff get created, at the end of the day, that's what it's all about.
I have a finite amount of time to do things, I already want to do more than I can fit into that time, LLMs help me achieve some of them.
This is a "scratch an itch" project I initially started to write manually in the past, but never finishing. I then used claude to do it basically on the side while watching the world series http://nixpkgs-pr-explorer.s3-website-us-west-2.amazonaws.co...
It’s not just good for small code bases. In the last six months I’ve built a collaborative word processor with its own editor engine and canvas renderer using Claude, mostly Opus. It’s practically a mini Google Docs, but with better document history and an AI agent built in. I could never have built this in 6 months by myself without Claude Code.
I think if you stick with a project for a while, keep code organized well, and most importantly prioritize having an excellent test suite, you can go very far with these tools. I am still developing this at a high pace every single day using these tools. It’s night and day to me, and I say that as someone who solo founded and was acquired once before, 10 years ago.
yes, I am using my voice agent, my head tracker, my sql writer, my odbc client, my shopping list, my sharepoint file uploader, my Timberborn map generator, my wireguard routing, my oxygen not included launch scripts, my i3wm config, my rust ATA over Ethernet with Content Addressable storage
The former tasks are directly from the training material, directly embedded into the model. For the latter task, it needs a context window and intelligence.
It'll be a common paradigm. Some agents support the coding agent discover relevant context for a plan, others will help the agent stay on track and ensure no rules break.
They really should have been supplying at least a week worth of readymade "projects" to every freelance AI promoter out there to demonstrate x9000 AI productivity gains for the skeptics.
Because vibing the air about those gains without any evidence looks too shilly.
Pointing out the where the burden of proof lies is not an ad hominem. Calling it such is in fact a good example of poisoning the well. all the fan girls have to do is post links to code they have vibe coded. some people have even done that in this thread. it's not an unreasonable standard.
I've found Claude's usefulness is highly variable, though somewhat predictable. It can write `jq` filters flawlessly every time, whereas I would normally spend 30 minutes scanning docs because nobody memorizes `jq` syntax. And it can comb through server logs in every pod of my k8s clusters extremely fast. But it often struggles making quality code changes in a large codebase, or writing good documentation that isn't just an English translation of the code it's documenting.