Yes, they can do that. But they can't select who gets the binary, so everybody gets it. Debian does reproducible builds on trusted machines so they would have to infect the source.
You can safely assume the source will be viewed by a lot of people over time, so the change will be discovered. The source is managed mostly by git, so there would be history about who introduced the change.
The reality is open source is so far ahead on proprietary code on transparency, there is almost no contest at this point. If a government wants to compromise proprietary code it's easy, cheap, and undetectable. Try the same with open source it's still cheap, but the social engineering ain't easy, and it will be detected - it's just a question of how long it takes.
You assume the binary can't just have a machine check in itself that activates only on the target's computer.