I wonder if it would help if they could somehow expose the context and allow the user to modify it or apply weights to different parts. I've certainly noticed that chat GPT seems to sometimes simply forget what is going on, reply with the same answer that I've already rejected, etc.