Suppose there are 1,000,000 tenants and 100,000 landlords whose land can each support 100 tenants if a tall building was built there. In the absence of collusion or zoning regulations to the contrary, each of the landlords has the incentive to add more apartments to their land until the net present value of rent of an apartment falls below the building cost, because the land isn't "scarce" in the sense that it can support ten times as many tenants as there actually are, and every landlord would prefer to make more money at the expense of the others.