The rules you're talking about were uniformly created as a way to replace de jure racial segregation, which was outlawed surprisingly early in the 20th century, with de facto racial segregation. People supply post-hoc rationalizations for density restrictions, and some of them might even be valid, especially as you get further away from urban cores, but just to be clear: the Chesterton analysis here says these fences were erected by morons; they do not inherit a presumption of reasonableness.
My confidence about this take is pretty high throughout the continental United States, but in Chicago it is something like 100%, since this is a local policy issue I work on here.
The blanket rejection of zoning, based on century-old racist transgressions, is pointless and tired. I lived in high-density areas my entire adult life until four years ago, when I finally bought a house that is a considerable distance away from those areas. I didn't do so for racist reasons. I did so because I required more space for my professional and personal pursuits. So I really don't give a shit about why the neighborhood was built originally, or feel the least bit guilty about opposing attempts to "Manhattanize" it.
The defense of zoning is not "post hoc," since we're talking about today and in the future. The "racism" attack on it is ad hominem.
I'm not against development, or density in areas that are designated for it. I'm against the vilification of entire neighborhoods under a false narrative that veils a craven profit motive, in combination with an entitled gimme-gimme-gimme delusion that cheap housing is going to rain down on people who could but don't work for it... if only we wipe out those horrible houses inhabited by rich racists.
Almost nobody is buying exclusively-zoned property for racist reasons. But the availability of those properties under those terms is an intended consequence of racist decisions, and so it is fair to re-evaluate whether it's reasonable for those terms to be enforced by the state's monopoly on violence. I say that in most exclusively-zoned areas in or near urban cores, the clear answer to that question is "no": most of what's single-family zoned today should be upzoned to fourplexes. That argument is easy to make on the merits: there's a real (and rippling) cost to maintaining single-family zones near cities, and the benefits accrue to a fortunate few.
I say this as someone who owns an extremely and multivalently single-family property, the only property I own, and is working actively and primarily on upzoning that specific area. I have neighbors that would say what you're saying: you bought into wherever you live based on the promise that the state would commit arms to preventing any of your neighbors from admitting too many new residents. My response, when we succeed: "we are altering the deal."
My confidence about this take is pretty high throughout the continental United States, but in Chicago it is something like 100%, since this is a local policy issue I work on here.