With your analogy I would be the one saying that I'm still not convinced that skis are faster than snowshoes.
I still use ChatGPT/Claude/Llama daily for both code generation and other things. And while it sometimes does do exactly what I want it to, and I feel more productive, it still seems to waste my time an almost an equal amount of time, and I have to give up on it and rewrite it manually or do a google search/read the actual documentation. It's good to bounce things off, it's good as starting point to learn new stuff, gives you great direction to explore new things and test things out quickly. My guess on a "happy path" it gives me 1.3 speed up, which is great when that happens, but the caveat is that you are not on a "happy path" most the time, and if you listen to the evangelists it seems like it should be 2x-5x speed up (skis). So where's the disconnect?
I'm not here to disprove your experience, but with 2 years of almost daily usage of skis, how come I feel like I'm still barely breaking even compared with snowshoes? Am I that bad with my prompting skills?
I thoughtfully limit the context of what I'm coding (on 2 of 15 files).
I ./ask a few times to get the context setup. I let it speculate on the path ahead but rein it in with more conservative goals.
I then say "let's carefully and conservatively implement this" (this is really important with sonnet as its way too eager).
I get to compile by doing ./test a few times, there is sometimes a doom loop though so -
I reset the context with a better footing if things are going off track or I just think "its time".
I do not commit until I have a plausible building set of functions (it can probably handle touching 2-3 functions of configs or one complete function but don't get too much more elaborate without care and experience).
I either reset or use the remaining context to create some tests and validate.
I think saying 1.3x more productive is fair with only this loop BUT you have to keep a few things in perspective.
I wrote specs for everything I did, in other words I wrote out in english my goals and expectations of the code, that was highly valuable and something I probably wouldn't have done.
Automatic literate programming!
Sheep shearing is crazy fast with an LLM. Those tasks that would take you off in the weeds do feel 5x faster (with caveats).
I think the 2x-5x faster is true within certain bounds -
What are the things that you were psychologically avoiding /dragging or just skipping because they were too tedious to even think of?
Some people don't have that problem or maybe don't notice, to me its a real crazy benefit I love!
That's were the real speedups happens and its amazing.
Do you mind sharing how much experience you have with the tech stack that have generated code? What I found with LLM is the perspective for AI generated code is different depends on your own experience, and I would like to know whether it is only my experience.
I have more than 20 years with backend development and just some limited experience with frontend tech stacks.
I tried using LLM initially with for frontend in my personal project. I found that code generation by LLM are so good. It produces code that works immediately with my vague prompts. It happily fixes any issue that I found pretty quick and correct. I also have enough knowledge to tweak anything that I need so at the end of the day, I can see that my project work as expected. I feel really productive with it.
Then I slowly start using LLM for my backend projects at work. And I was so suprise that the experience was completely opposite. Both ChatGPT and Claude generated code that either bad practice or have flaw, or just ignore instructions in my prompt to come back to bad solutions after just a few questions. It also fails to apply common practices from architecture perspectives. So the effort to make it work is much more than when I do all coding myself.
At that point, I thought probably there are more frontend projects used to train those models than in backend projects, therefore quality of code in frontend tech is much better. But when using LLM with another language that I did not have much experience for another backend project, I found out why my experience is so much different as I can now observe more clearly on what is bad and good in the generated code.
In my previous backend project, as I have much more knowledge on languages/frameworks/practice, my criteria was also higher. It is not just the code that can run, it must be extensible, in right structure and in good architecture, use correct idiom ... Whereas my frontend experience is more limited, the generated code work as I expected but possibly it also violated all these NFRs that I do not know. It explains why using it with a new program language (something I don't have much experience) in a backend project (my well know domain) I found a mixed experience when it seems to provide me working code, but failed on following good practices.
My hypothesis is LLM can generate code at intemediate level, so if your experience is limited you see it as pure gold. But if your level is much better, those generated code are just garbage. I really want to hear more from other people to validate my hypothesis as it seems people also have opposite experiences with this.
Or you're using skis on gravel. I'm a firm believer that the utility varies greatly depending on the tech stack and what you're trying to do, ranging from negative value to way more than 5x.
I also think "prompting" is a misrepresentation of where the actual skill and experiences matter. It's about being efficient with the tooling. Prompting, waiting for a response and then manually copypasting line by line into multiple places is something else entirely than having two LLMs work in tandem, with one figuring out the solution and the other applying the diff.
Good tooling also means that there's no overhead trying out multiple solutions. It should be so frictionless that you sometimes redo a working solution just because you want to see a different approach.
Finally, you must be really active and can't just passively wait for the LLM to finish before you start analyzing the output. Terminate early, reprompt and retry. The first 5 seconds after submitting is crucial and being able to take a decision just from seeing a few lines of code is a completely new skill for me.
Huh, my recollection is the exact opposite. I remember the good old days when I could use inurl: link: and explore the website contents fully and drill down further if necessary, compared to now, where google seems to always think to know better than you what you are looking for. If you are not happy with the initial results it gave you, you are pretty much out of options, good luck trying to drill down to some specific thing.
But aren't you afraid that whenever you veer discussion from Wikipedia/stackoverflow type explanations it's likely lying to you? This was my general experience -- it's great at querying for stuff which already exists and is popular on the internet and for conversing on a surface level or broad level but as soon you delve into details it starts confidently lying and/or hallucinating things, which undermines my trust in it, which in turn means I need to verify what it says, which means it did not increase my productivity that much after all.
It routinely invents arguments, functions or concepts which don't exist in reality or don't apply to the current context, but look like they could, so you are even more likely to get caught by this.
Haha, yes, it indeed invents arguments that aren't part of specific APIs and would offer to do something that you'd like to do in a very easy way, but since they actually aren't part of the API, well, you're out of luck.
It's just taking the "I wish they'd thought of my use case when designing that API" on the next level by simply pretending in a very sincere and convincing way that your wish came true, then writing a usually-pretty-correct program around that assumption that would actually work _if that wish had come true_ - but unfortunately that API doesn't really accept this convenient parameter, so...it's not that easy in reality.
Not to mention that self sabotaging your good work habits is counter-productive. You may think that you are getting one over them, but it's just as likely that after a few months of doing bare minimum, it will become your new "default" behaviour and carry over to your new job/endeavor.
I think the GP is a bit confused and mixed Lithuania up with its neighbors. Lithuania, unlike its neighbors Estonia and Latvia, gave citizenship automatically to everyone residing in Lithuania right before the independence. While Estonia and Latvia chose to create a naturalization process aimed at excluding ethnic Russians.
In case you'll get an itch to play IF games, and don't know which one to choose.
Picking at random from this list of top 50 as voted by community, never failed me so far:
If you see a devlog post from Lucas Pope you know it's going to be a goldmine. No matter the topic. Dude has a real knack in writing these, clearly describing the problem and the thought process on possible solution. And making it all very interesting so you yourself start thinking how would you address it or what other cool thing could be built instead.
Here's some of his other huge devlogs on TIGSOURCE:
Yes, the guy is honestly some type of genius. To be this talented of a programmer, and across multiple platforms, languages, and toolsets just dazzles me. It's hard enough to just be competent with one platform. I can't imagine.
And on top of that he composes the music and makes the art and literally everything else.
I honestly just don't understand how a person can get that good at that many things.
He's also a fantastic game designer. I played through Return of the Obra Dinn last year and it was one of the coolest game experiences I've had. It really doesn't hold your hand, it's just using logic and clues to piece together a mystery and put names to faces. Keep a notepad handy and it's a real fun time!
This is what's truly mindblowing about Lucas Pope to me. Many people are technically talented, but good game designers are really, really hard to come by. Having both in one person just feels unfair.
Obra Dinn, especially, is one of my favourite games of all time, it's such a unique experience.
Interestingly Lucas Pope says he thinks Papers Please is a better game. As a method of telling a story I think it's as good as you can get - e.g. the gameplay in Obra Dinn doesn't thematically link to the story in the same way - but I'm just don't enjoy the game. I know: you're supposed to be bored, it's supposed to be unwieldy, you're supposed to be frustrated by the rules. But I just want to do something else.
His post on how exactly he came up with those really nice 2bit graphics in a way that it doesn't hurt the eyes and creates that very unique look was very enlightening.
Speaking as a game dev, I think you can learn it, but it takes time. I was interested since a kid to make games. And as a teenager programmed some for the web, and trying to learn C++ reading Quake2 source or whatever I could find, also bought some game dev books. Not much came out of it I thought. At around twenty, there were enough tutorials in Macromedia tools to sort of guide me toward something.
Then I struck luck getting into game programming at a tech institute. Finally I could make something, it took more than a decade to get simple games on the screen for me. But I get easily distracted, procrastinate and lack enough IQ. Those books and formulas were hard for a mediocre teenager.
Parallel to all this I was interested in music too. Fooling around in ReBirth and Fruity Loops. Many years later got a guitar and tried to learn that, you get a lot of music theory from that. That theory is much easier than programming, still takes a lot of time.
Drawing I'm not really good at, but got an eye for what looks decent and can play with colors. On a computer it can almost be like cheating because you can be "inspired" by others. But books and tutorials on art exists too of course, how far that rabbit hole do you want to go? And 3D is a bit of a mix between tech and art, you could draw a cute character on paper front and side with a pencil. And when you model, texture, rig and animate that; it can be really impressive.
What I am trying to say, it looks like genius but it is a lot of work over a period. But it is creative/craft work, and not boring. And you can learn the basics of all the areas and make something great. When making apps, all those areas of you are unused. No music, no art. At best some graphic design (UI).
If you want to really put it in a compressed environment where it is not (?) overwhelming, but still technical, try C64 programming. There are lots of resources and step-by-step guides, and you will get a taste of everything and can build from there.
We are many that have the desire to make our own games. Some are indeed geniuses, or very strong in multiple areas. Others just stumbled across Unity and are going through whatever guides there are with desire. These days you can choose your difficulty and dedication level.
For simpler games, getting your game to run on desktop, iOS and Android is not that hard. Because most platform code will be some glue, and then you are off inside your own game code. It takes weeks/months to figure that out, but then you are done and carry it with you for every game in the future. Then when you put it all together plus the uniqueness of your game (because it is unique right?!), you get a great and interesting blog post like this one!
Arstechnica did a war stories with him on Return of the Obra Dinn, combining one of my fave video series with one of my fave gamedevs. https://www.youtube.com/watch?v=OMi6xgdSbMA
His youtube[0] also has some very interesting behind the scenes content. If you haven't seen the Obra Dinn ship building timelapse[1] you're in for a treat! (Major spoilers - don't watch unless you've already beaten the game!)
From a practical standpoint, I can’t even understand how he writes these! Presumably, this process took him months (a year?) to work through. What was he doing that whole time? Taking notes and captures for an eventual blog post? It just seems he has an incredible ability for organization and foresight, one that I am deeply jealous of.
TL;DR -- it's a game. That was designed for computers with a mouse and not for a touch-oriented environment, and so a lot of hard work had to be done to port it to phones.
I'm using Orgzly. But to be honest my use case doesn't involve much use of org-mode on the phone. And I just a had more thorough look and it doesn't support a lot of important features, to the point I should probably retract that bulletpoint not be misleading:
- No codeblock support (execution not working is probably to be expected, but I would expect at least syntax highlighting, but it doesn't do that either)
I still use ChatGPT/Claude/Llama daily for both code generation and other things. And while it sometimes does do exactly what I want it to, and I feel more productive, it still seems to waste my time an almost an equal amount of time, and I have to give up on it and rewrite it manually or do a google search/read the actual documentation. It's good to bounce things off, it's good as starting point to learn new stuff, gives you great direction to explore new things and test things out quickly. My guess on a "happy path" it gives me 1.3 speed up, which is great when that happens, but the caveat is that you are not on a "happy path" most the time, and if you listen to the evangelists it seems like it should be 2x-5x speed up (skis). So where's the disconnect?
I'm not here to disprove your experience, but with 2 years of almost daily usage of skis, how come I feel like I'm still barely breaking even compared with snowshoes? Am I that bad with my prompting skills?