You should look into multistage docker builds, that lets you still use a full fat image for your build but then leave all the build tools out of your final image
Someone else suggested the same thing actually. Easy to get lazy when it "just works" and internet is 1gig home and office - you can see how bloat just builds up.
I liked jpetazzo's post on the subject but there are plenty to choose from https://www.ardanlabs.com/blog/2020/02/docker-images-part1-r...