Alt-Left.

On Mac OS X, the standard shortcut is Command-[ instead (and Command-] is Forward).

I generally use three fingers swipe to the left.

As do I. Actually, if I have a mouse, I tend to right click, then hit back, it's the first option.

In general Command-Left/Right-arrow should work as well unless it's been taken out of Safari.

if you use vimium/vimperator/something else you can do shift+H to go backwards, shift+L to go forward.

Which would require me to press two keys instead of one.

Considering we're talking about Vim here, pressing keys should be second nature to you anyway...

yeah but not to move your hand if you're on the homerow