> But here we're talking about a system where legitimate users (human browsers) and scrapers get the same value for every application of the work function. The cost:value ratio is unchanged; it's just that everything is more expensive for everybody. You're getting the worst of both worlds: user-visible costs and a system that favors large centralized well-capitalized clients.
Based on my own experience fighting these AI scrappers, I feel that the way they are actually implemented makes it that in practice there is asymmetry in the work scrappers have to do vs humans.
The pattern these scrappers follow is that they are highly distributed. I’ll see a given {ip, UA} pair make a request to /foo immediately followed by _hundreds_ of requests from completely different {ip, UA} pairs to all the links from that page (ie: /foo/a, /foo/b, /foo/c, etc..).
This is a big part of what makes these AI crawlers such a challenge for us admins. There isn’t a whole lot we can do to apply regular rate limiting techniques: the IPs are always changing and are no longer limited to corporate ASN (I’m now seeing IPs belonging to consumer ISPs and even cell phone companies), and the User Agents all look genuine. But when looking through the logs you can see the pattern that all these unrelated requests are actually working together to perform a BFS traversal of your site.
Given this pattern, I believe that’s what makes the Anubis approach actually work well in practice. For a given user, they will encounter the challenge once when accessing the site the first time, then they’ll be able to navigate through it without incurring any cost. While the AI scrappers would need to solve the challenge for every single one of their “nodes” (or whatever it is they would call their {ip, UA} pairs). From a site reliability perspective, I don’t even care if the crawlers manage to solve the challenge or not. That it manages to slow them down enough to rate limit them as a network is enough.
To be clear: I don’t disagree with you that the cost incurred by regular human users is still high. But I don’t think it’s fair to say that this is not a situation in which the cost to the adversary is not asymmetrical. It wouldn’t be if the AI crawlers hadn’t converged towards an implementation that behaves as a DDOS botnet.
I find it really frustrating that I am not able to avoid using whatsapp due to how popular it is to the point that it’s become the go-to communication channel for most things :/
Indeed. And the worse isn't that we'll have to deal with ads. The worse is that people will stick around despite the ads, which only goes to show how powerful the grip of Meta on our societies is.
... and once they have proof of they power because WhatsApp use keeps increasing despite all the enshittification done in the last few months, they'll feel encouraged to enshittify further in even bolder ways.
That blog post is very interesting in number of ways!
The principles they enlisted
> Advertising has us chasing cars and clothes, working jobs we hate so we can buy shit we don't need. – Tyler Durden, Fight Club
> Advertising isn't just the disruption of aesthetics, the insults to your intelligence and the interruption of your train of thought.
> Remember, when advertising is involved you the user are the product.
---
That said, building a product and selling it for 19 Billion dollars in 2012 was essentially a success of capitalism over those principles. There shouldn't be any complaints, since FB didn't kill it, and the number of users kept increasing.
Just wanted to call out that linkhut does have IFTTT integration [0]. Although you will need to pay for an IFTTT developer account if you want to enable that on your self hosted instance. If you need help setting that up let me know, I’ll be happy to walk you through the process (and that way I can write documentation on how to do it).
Edit: I also was under the impression that one could use the webhooks integration [1] to make bespoke integrations (as the documentation says they support passing arbitrary request headers) but haven’t tried it myself, have you tried it and run into any issues? I’d also be happy to help improving support for that workflow.
Do you have pointers to this? As a Spaniard I recall that even though I was originally told the spanish alphabet treated the LL as its own letter, it always felt quite inconsistent. And I always assumed it’s removal was more about simplifying things than having to do with computers
There is a comment from "adolph" parallel to yours with footnotes. But I remember the event well (though it was 30 years ago) as the difficulty for programmers was given as the justification for the law (this was a change in collation).
This particularly irked me as unicode was a few years old at this point, and while not really adopted yet, was clearly the future.
Just the typical thing: while adding a bookmark, when I start typing in the first letters of a tag (e.g. "hack" for "hackernews"), Pinboard is offering a drop-down menu with possible matches. Linkhut didn't have this last time I checked. It is maybe not a big issue, though -- possibly, relying on your own memory (not programmatically offered suggestions), your tagcloud may get less messy over time; you'll end up with a smaller amount of tags, which may be a good thing at times.
I think I like 'em both, Pinboard and Linkhut, though.
I’ve been using a Steel HR for a few years now. I’m extremely satisfied.
Long battery life is an understatement! Even when the battery drops to near 0 the watch is still useable as a plain analog watch for at least a week giving me enough time to find where I left the damn charger (given that I don’t get to use it often at all).
What I don’t get is how this model (analog watch with just enough smarts) hasn’t gained more traction. It seems to me there’s a lot of opportunities in this space, not everything requires a high resolution display.
I'm kind of disappointed that the Fossil hybrid watches don't do the trick where the watch keeps working when the smart has run out of juice. That said, I get four weeks of battery life, so it's never been an issue in practice.
> If we go further back in time, mobile phones were bricks used only by business people [...]
But they were widely used by business people, and I’m assuming that was an important aspect of permeating into the other demographics.
To me, what happened with cellphones is similar to what happened with personal computers. It first entered households of people that needed them for work, and as the tech evolved, so did their recreational capabilities making them more appealing to the mass market.
AR/VR devices are attempting this the other way around: their current use cases seem to be mostly recreational, and we are hoping they will eventually come up with some productivity use cases. Someone in this thread talked about how they are surprised that tablets have sort of flopped, and I feel it’s because of similar reasons. Sure there are now more and more ways to be productive on a tablet, but it’s still more of a consume-centric experience and I believe that’s the reason they haven’t replaced laptops. Time will tell.
> But they were widely used by business people, and I’m assuming that was an important aspect of permeating into the other demographics.
Similarly to how VR headsets are used by gamers and tech enthusiasts today. Just like VR for gaming, not everyone was willing to pay exhorbitant prices to lug around a briefcase of a phone, and use it only for poor quality phone calls.
> AR/VR devices are attempting this the other way around: their current use cases seem to be mostly recreational, and we are hoping they will eventually come up with some productivity use cases.
Sure, but the actual current use case doesn't matter. Once the capabilities, price and user experience improve, it will become more appealing to a mass audience.
Think of the iPhone: there were certainly devices that had the same functionality of "an iPod, a phone, and an internet mobile communicator" before it. But Apple saw the right opportunity when the tech got good and cheap enough to vastly improve the user experience and expand the capabilities beyond just a mobile phone. It's what they're good at.
> Someone in this thread talked about how they are surprised that tablets have sort of flopped
But tablets didn't flop. They got merged into 2-in-1s, foldables, "phablets", etc. They've certainly replaced laptops for many users. The only reason productivity is limited on devices like the iPad is because Apple wants to, for whatever reason (possibly to avoid cannibalizing sales of their other products?). But the iPad is still selling well, AFAIK, and many professionals use it. Google and Android never prioritized the tablet form factor, so it understandably lags behind. So productivity is restricted mostly because of software, and the touch interface makes it more suitable for consuming content. If you need to be productive and the software is not a problem, you add a mouse and keyboard, and there's your 2-in-1.
> Sure, but the actual current use case doesn't matter. Once the capabilities, price and user experience improve, it will become more appealing to a mass audience.
I'm not the poster you're replying to, but I think you've missed their point here.
Market penetration of mobile phones was (initially) driven by the advantages they provided as a work tool. This growing market that was willing to pay helped drive the cost vs. utility down to the point where non-business use cases became viable.
Goggles with screens in them have had a very different history. There are exceedingly few people/careers who have gotten a good ROI from buying and using a headset, so most sales are driven by entertainment (and a small number of enthusiasts).
Apple (and Meta) can definitely continue to subsidize the development of this technology to bring the price down, but expanding from a niche market for entertainment devices to a market of professional tools is the opposite of what's worked for new technologies in the past.
I get that, but my point is that it doesn't matter what the initial use case that drives adoption is. Whether the ROI is financial/business, or enjoyment from entertainment, there are early adopters willing to pay high prices and tolerate the poor UX for the tech to improve.
This is clearly Apple's angle with this product. They understand that this is not a mass market product, but are betting on their brand appeal for it to sell well enough to finance later generations.
You might be right that it won’t matter in the long run. But I’d argue that targeting gamers and tech enthusiasts is going to make it harder to reach mass market adoption.
For instance, with computers, laptops and cell phones being first and foremost business tools it was common in the early days for those to be provided by the employers. That really played a big role in normalizing their presence around the home before they became mainstream products.
I have a hard time seeing a similar thing happening with these goggles, but I’m far from being an expert in these things.
> could anyone have imagined walking into a living room 30 years ago and seeing everyone hunched over a phone?
True, but I don't recall smartphones ever being unveiled with promotional videos of people all being hunched over them at dinner.
Smartphones were promoted by enabling us to do things on the go: making and receiving phone calls, getting directions, finding restaurants in the area. This is why this feels different: the experience that is promoted is one that feels already slightly dystopian.
I’m confused. Are we judging a product category based on how it’s promoted, or how it’s used?
I don’t think my opinion of smartphones would be any different if early promotions had shown the downsides, or bigger upsides, or bizarre and unrealistic scenarios. Am I wrong to be anchored on how they’re actually used, which literally nobody foresaw?
I don't think OP is saying smartphones aren't dystopian in practice, just that no one set out to create the dystopia. That part of the Vision demo showed that the designers thought that this additional contribution to our dystopia was a positive thing, and that's concerning.
Do you judge Viagra based on how well it reduces blood pressure?
I agree that demo was dumb and dystopian. But it's silly to latch on to one imagined usage, ignore the others, and have no interest in what the reality will be.
It would be silly to latch onto it, if it were the vision of some random fan, but it is the vision of the company creating the product, so no, it's not silly to consider it critically.
Or driving. It would interesting to have directions overlaid on the view of the road. The exit I need to take could be outlined and highlighted for example.
I can’t imagine putting a CPU as complicated as the M2 in between my eyeballs and the road. It is too big and complicated.
I guess we don’t really know what the R1 does. It is hypothetically possible that there’s a failsafe road-eyeball path just goes through that chip, maybe that could be made as reliable as various other critical computers…
I guarantee you Mercedes or Lamborghini would _love_ to have a fully enclosed interior with screens instead of windows, but it's a critical safety feature and anything that makes it less likely to work is just not happening.
We’re basically converging on fully enclosed interiors anyway, just without the VR headset (the pillars for the windows are ridiculous nowadays). Maybe the argument could be made that it’d be safer for everyone (especially pedestrians waiting at crosswalks) if drivers had a headset system.
Getting rid of the windows and making the headset totally necessary seems like riskier proposition though. For the Apple thing on a normal car, the backup option of taking off the headset is always there (assuming sufficient warning can be given before a failure, which is a big assumption, but should be doable).
Drive-by-wire systems exist, so it isn’t as if replacing some normally mechanical steps with electronics is impossible. The electronics just need to be made sufficiently reliable.
Recon Jet made an HMD that is sorta similar to Google Glass, but designed more like ski goggles / sunglasses. One of its use cases when it came out around ten years ago was skiing, but I never used it for that. I mainly used mine biking with its ANT+ integration.
Based on my own experience fighting these AI scrappers, I feel that the way they are actually implemented makes it that in practice there is asymmetry in the work scrappers have to do vs humans.
The pattern these scrappers follow is that they are highly distributed. I’ll see a given {ip, UA} pair make a request to /foo immediately followed by _hundreds_ of requests from completely different {ip, UA} pairs to all the links from that page (ie: /foo/a, /foo/b, /foo/c, etc..).
This is a big part of what makes these AI crawlers such a challenge for us admins. There isn’t a whole lot we can do to apply regular rate limiting techniques: the IPs are always changing and are no longer limited to corporate ASN (I’m now seeing IPs belonging to consumer ISPs and even cell phone companies), and the User Agents all look genuine. But when looking through the logs you can see the pattern that all these unrelated requests are actually working together to perform a BFS traversal of your site.
Given this pattern, I believe that’s what makes the Anubis approach actually work well in practice. For a given user, they will encounter the challenge once when accessing the site the first time, then they’ll be able to navigate through it without incurring any cost. While the AI scrappers would need to solve the challenge for every single one of their “nodes” (or whatever it is they would call their {ip, UA} pairs). From a site reliability perspective, I don’t even care if the crawlers manage to solve the challenge or not. That it manages to slow them down enough to rate limit them as a network is enough.
To be clear: I don’t disagree with you that the cost incurred by regular human users is still high. But I don’t think it’s fair to say that this is not a situation in which the cost to the adversary is not asymmetrical. It wouldn’t be if the AI crawlers hadn’t converged towards an implementation that behaves as a DDOS botnet.