You could say the same thing about the "human right to live." Sure, some people might disagree that we have a right to live. But that is not really relevant to the scale of national policy.
If I said you don't have a right to a fair trial, that is not a reason to abolish fair trial laws.
If I said you don't have a right to a fair trial, that is not a reason to abolish fair trial laws.