there is no "we" - no overarching moral force that makes these decisions. Banks ruin people's lives for money just because they can. Those people could retaliate on banks, but they choose not to because they were taught morality, which puts them at a disadvantage.
i'm not convinced the people whose lives are ruined by banks are in a position to retaliate.
we is in theory the government chosen by the people to give direction to society and enforce it. In practice, i think the political systems in place fail to serve that purpose unfortunately (regulations too easily manipulated by lobbying/propaganda, fake choices to vote on, largely self serving to maintain a hierarchy of exploiters vs exploitees)