Of course this fails. OpenAI continuously states that ChatGPT:
1) Doesn't generate facts.
2) Shouldn't be used for anything useful.
However, your point is still a good one. If Microsoft places ChatGPT somewhere alongside Bing, it'd be easy to mistake it for fact.
In my opinion, this new tool is going to be learned by the public just like Google and generic search was learned by the public. There's a certain way to prompt the machine and we all have certain expectations of what the machine should return.
With ChatGPT, those input/output expectations are vastly different than web search, but people will learn it.
> Of course this fails ... However, your point is still a good one
Exactly. ChatGPT is shamelessly awful on certain types of specifics. You might say it is only "statistically" accurate and my question was designed to exhibit this.
I noticed this possibility when it delivered a surprising, yet afaik correct, piece of information, and I asked it for cites in the form of webpages. Alongside a couple of actually existing sites, it manufactured a plausible sounding blog post from a domain expert which as far as I and the wayback machine could determine had never existed at the given URL.
It made me wonder if with sufficient ingenuity the engine could be induced to effectively libel authors by attributing statements to them which they never in fact made.
For fun, I asked my consciousness question again, but with a rider:
[Prompt:] Please give some recent citations for scientific work examining the nature of consciousness, but only include real papers that actually exist
And the bot was suitably contrite:
> I apologize for the error in my previous response. Here are some recent papers that explore the nature of consciousness:
> 1. "The neural basis of human consciousness: a focus on the thalamic reticular nucleus" by Christopher D. Schmitt, Leonid L. Moroz, and Laurence J. C. Mailly (2019)
> 2. "Consciousness and the brain: an overview of the experimental data" by Anil K. Seth (2018)
> 3. "A framework for consciousness" by Christof Koch and Naotsugu Tsuchiya (2016)
> 4. "Neural correlates of consciousness in humans: progress and problems" by Giulio Tononi and Christof Koch (2015)
Which is a better effort, though still at least partly fictional.
However, your point is still a good one. If Microsoft places ChatGPT somewhere alongside Bing, it'd be easy to mistake it for fact.
In my opinion, this new tool is going to be learned by the public just like Google and generic search was learned by the public. There's a certain way to prompt the machine and we all have certain expectations of what the machine should return.
With ChatGPT, those input/output expectations are vastly different than web search, but people will learn it.