I live within walking distance dozens of short-term rentals in a very small town.
The difference between having dozens of neighbors with families, children, and strong ties to the community and not having those neighbors and community connections is in no way artificial.
Some reasons I care: schools that decline with population loss, the lack of available/affordable rental homes for local residents, the effects on local retail and service businesses, the tendency for hurried visitors to drive at unsafe speeds while staring at their phone for directions.
Problems that you do not experience remain real problems.
So basically instead of thinking about how you can better accommodate visitor demand and respect property owners’ rights, you would prefer to regulate your “small town feel” back into existence.