That’s entirely rational but unless you build from source (and honestly even if you do) there’s an attack vector present. The only real alternative is to review and allow list outgoing network traffic
I use Little Snitch religiously and for third-party apps I use occasionally I'm very pedantic with allowing them 10 minutes of network access at a time. It sounds painful to manage but in practice only a few apps are super chatty. Apps that are my daily drivers get more generous network access rules.
