Punishment does work. What do you imagine instead? Nothing? Quoting [0]:
First, that punishment has three purposes – retribution, rehabilitation, and deterrence – does not entail that each of these purposes must be realized in a given act of punishment in order for that act to be morally legitimate. For example, we may justly imprison a recidivist thief even if we know from experience that he is extremely unlikely to change his ways as a result of his imprisonment and even if circumstances make it unlikely that his particular imprisonment will deter other thieves. Similarly, the fact that a given act of capital punishment may not fulfill all of the ends of punishment does not by itself suffice to make that act morally illegitimate.
Second, while there is obviously a sense in which capital punishment can prevent rehabilitation, there is also a sense in which it actually facilitates rehabilitation. How so? Consider first that a wrongdoer cannot truly be rehabilitated until he comes to acknowledge the gravity of his offense. But the gravity of an offense is more manifest when the punishments for that offense reflect its gravity – that is to say, when the principle of proportionality is respected. A society in which armed robbery was regularly punished with at most a small fine would be a society in which armed robbers would have greater difficulty coming to see the seriousness of their crimes, and in which they would for that reason be less likely to be rehabilitated. Similarly, a society in which even the most sadistic serial murderers are given the same punishments as bank robbers is going to be a society in which sadistic serial murderers will have greater difficulty in coming to see the seriousness of their crimes, and thus will be less likely to be rehabilitated.
I should clarify. I think punishment mostly makes the punishee angry with the punisher. I don't think that they generally feel like they are paying their penance. If they didn't already think that what they were doing is wrong, I really don't see how they are going to suddenly start just because they were punished.
I generally think that the way that Scandinavia does prisons is better. I think that punishing people specifically with the intent of making their life worse is viscerally satisfying (the retribution part of that quote), but I think it's not actually a good thing to organize a society around increasing suffering. The US has a higher murder rate than Western European countries, despite having stricter punishments, including the death penalty. There can be thousands of factors that influence that, obviously, but it doesn't seem to be massively deterring crime.
I think prisons are basically a necessary evil; there are certain people that are antithetical to a functioning society, and so it's probably better to separate them from most people. I think the point should be, though, to not view these things as "punishment" but more "a chance at reformation".
Prisons in the US used to have college education programs, and job training programs, so that when you left you had a means of supporting yourself that wasn't criminal. If I understand correctly, this is still the case in Sweden, and I think that's a good idea.
Now of course, there are humans that are so warped that really no amount of job training is going to help them (e.g. a Jeffrey Dahmer), and at that point you really do just need to treat it like punishment.