When working on a hard problem, I often had to visualize how the algorithm works, data moving there, changing this, replacing that. When I debug a tricky problem and I still have not figured it out yet, I have two options: either systematically note down what the code is going through, or keep the problem in the back of my head and sleep over it. More often than not, the solution will present itself after a while if I still bother with the problem. I reckon I can not solve all problems like that, but when my mind refuses to dig in, that is often the only effective way to solve the problem. Pushing harder only wastes time and risks saturation.
That really resonates with me! I’ve found that when I’m up against a tricky problem, the solution often pops into my head when I’m not actively thinking about it—whether I’m sleeping, going for a walk, or just doing something completely different. It’s like the answer appears out of nowhere after I’ve stepped away.
I’ve always wondered why this happens. Is it just how our brains work in problem-solving mode? Does anyone know of any articles or studies that dive into this? I imagine it’s not just a programming thing, but a general part of solving complex problems.
Julian Jaynes[1] makes a pretty compelling argument that our unconscious mind does pretty much all of the heavy lifting with regards to problem-solving. The conscious mind just steers us and lets us remember what we are doing.
If I recall correctly an example he gives is looking at a math problem. Looking at 2+2 as an adult you instinctively know it to be 4. As a child you may have had to count on your fingers or write out the problem until the abstraction of numbers was solidified for you, and now it is just a black box that your unconscious has added to a "tool belt" of sorts.
For me I would describe the process of learning/problem solving as my conscious mind navigating some solution/problem space and figuring out the general shape of it. Once I have figured out the shape of the problem, the answer either:
1. Immediately looks like the shape of another, solved problem. In which case I just use that solution.
2. Is not immediately apparent.
In the case of #2 I just play around with different strategies that mostly don't work, but it helps me build a mental model of the problem. Then I wake up in the middle of the night with a solution, or figure it out while I am showering/eating/driving/etc.
There must be a connection to how doctors advice to talk to (even unconscious) patients who survived a stroke: the brain of the patient is still actively receiving information and constantly rewires itself while trying to process the data over and over again. In case with a complex programming problem, the person itself is responsible for the 'feedback loop'. Like how you can't stop thinking about the problem under different angles. Thus no need for external stimulus. I'm guessing the best time for that kind of processing is when one is asleep because there is simply not much else for brain to do in that state. Plus, when asleep, some long-term memory mechanisms kick in which make the already processed data more accessible and easier to work with. If there are some studies explaining the why's, I bet the medical ones focusing on brain damage and long-term memory would be the most related.
I have that a lot. It seems my mind needs to try a lot of combinations and suddenly stumbles upon one that makes sense and a light lits up. Browsing through the code or reading the docs only tends to get in the way for that process.
There's the need for some noisy creativity to be able to try to match a lot of combinations including the non obvious ones, where the solution space is.
Yeah this is me too - which is why I enjoy turn based games while I’m coding, if I get stuck I distract my consciousness with 2-3 turns in Civilisation and the answer pops into my head while I’m playing.
Turn based is excellent because always I’m mid-turn when the answer comes so I alt-tab out and keep going.
My steam shows some ridiculous number of hours in game because it’s always on, nearly 24/7, in a window on my monitor
Very interesting. Any turn-based games you'd recommend for someone who's completely new to them? I have zero knowledge in that area but would love to try it out.
Try Civilisation, probably start with the current one - Civ 6
Then if you like it you can work your way backwards
If you like a more futuristic setting then Endless space 2. Or civilisation beyond earth is really fun to colonise another planet and research futuristic technology and alien biology.
If you like puzzle games with no time pressure SpaceChem.
