I do something similar with reviewing code: I have one agent write the code and another reviews it, then they go back and forth for a bit improving the code. Seems to yield better results than one agent alone.
The difference is that in the code situation, you can run unit tests on the code, compile it, etc. Unless your LLMs are ordering diagnostics and reviewing the results, there is no further information that the LLMs have on the situation. Having a second LLM review the first is counterproductive, if the 2nd LLM is better, why not use it directly? If not, then what prevents it from sending the first on some incorrect tangent?
Also, there are multiple "correct" ways to code something, so imperfect code that solves the problem is still useful. A medical diagnosis is either correct or incorrect.
Different prompt approaches and training doctors to use LLMs can improve accuracy of LLM-assisted diagnosis. It’s pretty reasonable to hypothesize that LLM “peer review” could improve that as well.
I do something similar with reviewing code: I have one agent write the code and another reviews it, then they go back and forth for a bit improving the code. Seems to yield better results than one agent alone.
Seems like a similar principle.