> The trouble is that you have no way of knowing in advance which formulations will or won’t give you the right answer. To an optimist, any hint of success means that there must be a pony in here somewhere.
Along with the examples given I think this is valid criticism.
It’s definitely true that a lot of the hype I’ve seen here is the result of careful tuning in the input prompt to get the desired output.
But it’s also true that criticism tends to also be curated examples which demonstrate failure. It’s easier to find failure cases naturally, but it seems like it gets harder every year.
Along with the examples given I think this is valid criticism.