Yes. While I personally don’t like this practice, it is so widespread and there is so much demand for it that it’s not unusual given their privacy policy makes explicit mention of it.
The best argument you could make is that gem owners should be able to see “who” downloads their gems. If they were self-hosting the packages, they would have that data. Of course, charging for it is the ookier part.
Say you provide a service for free and are desperate for corporate sponsorship. Who wouldn't look at what companies are using your service and contact them with "Hey, I'm seeing you are using our service, can we have a chat"? You basically have no other means of contacting companies nowadays without getting into trouble for cold-calling/spamming.
The best argument you could make is that gem owners should be able to see “who” downloads their gems. If they were self-hosting the packages, they would have that data. Of course, charging for it is the ookier part.