This might sound unpopular but, when leaving a company I'm mentally telling everyone to f-off. Unless the the interview is with someone that could refer you, I wouldn't even waste any time or energy in that interview. It's not worth it. Miss it. Nobody will remember.
Totally not a problem. If you are in the f-off frame of mind then probably for the best!
You won't be pushed that hard for an exit interview if it was pretty clearly not a good fit. Some of these transitions are a relief for all concerned (employee AND manager).
If a manager has someone with leadership / significant upside potential I do think a discussion is worth doing if possible.