I'm genuinely scared about what society will look like in five years. I understand that outsourcing mentation to these LLMs is a bad things. But I'm a minority. Most people don't, and they don't want to. They slowly get taken over by a habit of letting the LLM do the thinking for them. Those mental muscles will atrophy and the result is going to be catastrophic.

It doesn't matter how accurate LLMs are. If people start bending their ears towards them whenever they encounter a problem, it'll become a point of easy leverage over ~everyone.