Private citizens and companies do not have access to nuclear weapon technology and even the countries who do are being watched like hawks.

If equally or similarly dangerous, are you then saying AI technology should be taken out of the hands of companies and private citizens?

For the sake of argument, let's say yes, AI should be taken out of the hands of the private sector entirely.