I can see where Goodhart's Law applies to psychology and economics, pretty much any man-made domain without IDLH (immediate-danger-to-life-and-health) outcomes. But I think it's going to be hard to Goodhart a lot of medical AI safety. Biology doesn't give a shit.
However, identifying the right metrics and having the necessary test sets will, at times, be challenging.
However, identifying the right metrics and having the necessary test sets will, at times, be challenging.