Neuroscientists who have attempted to compare what people think about how we think and how we ACTUALLY think have concluded that self-reports are entertaining to listen to, but not very informative. That is, we have more of an illusion of introspection than a reality.
The level at which we ARE self-aware of our own thinking does not seem to me to be more impressive than the ability of something like ChatGPT to generate training data that can in turn be used to help create much smaller models with similar capabilities, or future models with better ones. In other words LLMs are capable of some form of learning through introspection, and we do not appear to be close to the limits of how much self-improvement they are capable of.
Neuroscientists who have attempted to compare what people think about how we think and how we ACTUALLY think have concluded that self-reports are entertaining to listen to, but not very informative. That is, we have more of an illusion of introspection than a reality.
The level at which we ARE self-aware of our own thinking does not seem to me to be more impressive than the ability of something like ChatGPT to generate training data that can in turn be used to help create much smaller models with similar capabilities, or future models with better ones. In other words LLMs are capable of some form of learning through introspection, and we do not appear to be close to the limits of how much self-improvement they are capable of.
But https://simonwillison.net/2023/Mar/17/beat-chatgpt-in-a-brow... suggests that for less than the average developer's salary we could develop something equivalent to ChatGPT in a browser. And better systems are coming soon.