I often get really slow download speeds from the GitHub CDN, which my ISP must not peer with or something. My ISP has faster routes to most of the rest of the internet, including some VPN endpoints, so a VPN can be used to cut out the bottleneck and allow me to download large binaries off GitHub at 2 MB/s instead of 80 KB/s.
GithHub uses S3 for artifacts. If your typical S3 download speed is ~80KB/s, I suspect it would be a similar story for Cloudfront, in which case a huge part of the Internet would be painful to use...
Yeah I've heard some gaming folk say that their latency also goes down. I guess it all depends on the pipes your particular ISP has rented and their connecting places.