The Ergodox's thumb cluster is pretty widely considered suboptimal, I personally manage to use all three bottom keys but the other three are pretty unusable with the thumb (I tend to use them for media keys that I rarely use with the index and middle fingers). Hand size definitely matters, the Ergodox is pretty crap for people with smaller hands in particular.

Now I'm lucky enough to have large enough hands to use it comfortably and I really enjoy the experience.

I also don't use the wristrests at all, but that's true for any keyboard, not just the EZ. I just don't like them.