This is not going to be the popular opinion but at this point, if your project is not for profit (As I assume most fan-made wikis are), I've found it's best to just ignore Google. Leads from Google are typically not good anyway, especially if what you're trying to build is some type of community.
I understand that Weirdgloop is probably some kind of business, so it makes sense that they would care. And if you care, more power to you, I don't want to discourage anybody. I'm just out here promoting the idea that not everything you do has to be the next big thing. Sometimes it's okay to make a weird project and have only a handful of people care about it.
The article says this isn't an issue for subdomains of established domains, but the issues it describes match what we experienced setting up a game wiki for Cyber Knights: Flashpoint at https://cyberknightswiki.tresebrothers.com/ -- almost a year of not having any pages show up in search results, even though we were getting good links to it and there were no better alternatives that should have ranked (we never started the wiki for this game on any centralized site; at most all they had for it was a placeholder page).
Only after months of making sure our Steam news posts were consistently linking to pages within our wiki did we eventually see Google suddenly open the floodgates and start connecting searchers with the info they were searching for.
I've built out highly-ranked content sites before, and none of what I'd learned from getting those started seemed to work with this wiki, for no apparent reason. Google Search Console showed plenty of pages on the wiki were indexed; it just wasn't displaying them. Seeing this post I fully believe the premise that there's simply an arbitrary Google Jail for independent wikis. That sucks for search quality.
It used to be (late 00's) that wikis had a big advantage in search results because backlinks were weighted heavily by the ranking algorithm and the dense connectivity of wikis meant that there were always a zillion backlinks to pages on a wiki.
Entirely possible that this is caught in a malfunctioning webspam filter. A wiki is largely indistinguishable in structure from an SEO link farm; indeed, any abandoned wiki can be and usually is turned into a link farm by bots. Maybe Google decided that its default was going to be to ban all wikis until a sufficient mass of evidence comes that it's trusted. Which, again, sucks for search quality, but the web and Google Search have both been pretty much dead for the last ~5 years anyway, we just rely on them anyway.
The problem is that independent wikis don't pay the fandom tax to google for preferred search placement. Have you tried contacting your local corruption bureau to negotiate a rate tailored for your circumstances?
I was shocked to see Google's AI panel show my niece Bluey fan-fiction[1] as authoritative fact, sourced from an wiki (may have been a fandom wiki, IIRC).
Independent wikis have variable quality, pagerank isn't enough to gate the quality.
1. The infobox showed Mackenzie's full name as "Mackenzie Heeler", based on fanfic about grown-up Mackenzie marrying Bluey and taking up her last name. It also included names of their made-up kids.
There's some sort of reranking, maybe this is how they measure site reputation now? By linking from a large website and people in that website like it? Or SEO?
When I still payed Dota, there was a wiki hosted on fandom.
The experience was terrible. It wasn't always update-to-date with the latest patch, and on mobile there were so many ads rendering the whole thing unreadable.
The catch is that there was a better wiki, called Liquipedia. But Google never showed it first and I kept getting tricked to click that shitty fandom site.
(It was years ago. But I just tested it a bit and for most Dota-related keywords, fandom is still ranked higher than liquipedia. I don't know if the quality of fandom site has improved though.)
Yeah fandom has done this to several games. its taken a huge effort on the part of the Satisfactory community to make sure people know the official wiki is https://satisfactory.wiki.gg/ and not the garbage on fandom.
Using an extension to block fandom.com results is helpful because for other games I'm not really thinking about it when searching for something and find myself on that tumor of a website.
Google used to have a deny/block list. But such lists are obviously very advertising unfriendly, since the companies paying for placement and running the most ads are most likely to be blocked.
Back when I played WoW, at some point the people running the fandom wiki "forked" it to move to different hosting, and I think even now years later the fandom one (which stopped receiving updates when the fork happened) is still ranked higher in Google searches
WoW is now on wiki number three after the company hosting the second one was also bought by fandom. The original rotting corpse is now over a decade out of date and still often ranks highest.
Chrome/Firefox extension that redirects or alerts you when you open a Fandom or Fextralife wiki that has an independent alternative (many of which are migrations off those platforms): https://getindie.wiki/
Fandom is an abomination and filled with irrelevant or very basic content for most games. It wouldn’t exist if not for its phenomenal ranking on game-related searches…
Sometimes a fandom site I come across is legitimate and has tons of great information, and other times they completely make up fake games or features and just act like it was always real.
For what it's worth Liquipedia is sort of aimed at different people than the fandom wiki. Liquipedia has a very e-sports bend to its content so it may be that the ranks are skewed because the fandom wiki has more general purpose info about the game, lore, etc while Liquipedia has info about teams, matches, events, etc.
I think the fandom wiki for Dota is effectively dead, with everything being migrated to Liquipedia. Doesn't stop the fandom one from ranking highly though.
When Jimmy Wales launched Fandom, then called Wikia, a shit-ton of (actually encyclopaedic) content about fictional stuff was deleted from Wikipedia to be moved to Fandom, with the excuse that it was a "sister project", that it was the right place for things to be moved, etc.
Fandom was pure corruption even before the tenth round of enshittification.
>It wasn't always update-to-date with the latest patch
In case you didn't know Fandom is not the one who is writing the wiki. The pages are community contributed. If it's out of date it means no one has contributed information about newer patches.
Some of this might just be problems with the websites themselves. With the Path Of Exile 2 Wiki example, I'm looking at the sitemaps. Once you get through the first sitemap of nested sitemaps (ugh, not a fan of those), the very first actual sitemap returns:
XML Parsing Error: no root element found
Location: https://www.poe2wiki.net/sitemap/sitemap-poe_wiki-poe2_wiki--NS_0-0.xml
Line Number 18040, Column 1
(EDIT: Of course, after I post this, it's now working again...)
On to the Hytale Wiki example. I don't know if what I'm about to say applies to Google, but I'm approaching it from the perspective of my tiny dumb indieweb indexer for my personal search engine. It is much easier for me to index from a sitemap rather than try to crawl a website, so I basically look exclusively at sitemaps.
Looking at Hytale Wiki, my process:
* Site has a robots.txt file - good!
* Robots.txt mentions a sitemap - excellent!
* The sitemap is stored at /images/sitemaps/index.xml ... oh. I would usually not index anything from a /images/ folder, because I want to index pages only, not images. This would likely trip my exclusion filters. Let's ignore that and continue.
* The sitemap is a list of an additional 26 nested Gzipped sitemaps. Oh.
This is the point where my tiny dumb indexer would stop. Gzipped sitemaps are part of the standard, but they're relatively rare on the web for small sites. They typically only get used if a sitemap file exceeds the 50,000 pages-per-sitemap limit. In this case, 26 * 50,000 makes the HyTale Wiki look like a 1.3 Million page site. Do I really want to index 1.3 Million pages, an estimated 13GB of indexed text data, about a video game I'll probably never play?
My search index is storage constrained, and my indexer is very time constrained. The time I spend indexing your site is time not spent indexing another, possibly higher quality website. So at this point, I'd just grab the front page and disappear... like Google apparently does too.
Of course, the HyTale site isn't 1.3 Million pages, it's only 4,277 articles. That would all fit in the root sitemap file, and that might be the better approach for getting indexed.
Me again! I couldn't figure out why I was getting transient errors on that Poe2wiki sitemap, so I tried looking at other sitemaps on the site. I'm getting transient errors on all of them.
Then I realized - the sitemap files are being overwritten in real time. Every edit on the Wiki is causing the sitemap file to be edited in real time. That's why the sitemaps sometimes stop right in the middle of a filename when I access it - the sitemap file is in the process of being rewritten.
That's a behavior unique to a Wiki, and might explain the entire phenomenon.
Makes sense. Whatever generates those site maps should use the classic Unix trick of writing a separate file then renaming. Also maybe don’t update lastmods on each individual update, it’s a waste of resources since the granularity of lastmod is usually the day (though you can make it the second if you want).
Though weirdly it makes the subdomain result even more interesting: presumably those wikis are rewriting their sitemaps too, yet Google manages to index them normally
The sitemap issues are definitely worth fixing, but I don't think they explain the most interesting part: the same kind of wiki seems to behave very differently depending on whether it's on a fresh root domain or an established subdomain.
There's no need to speculate on whether the sitemap is received correctly by Google since Google Search Console tells you how many pages it can discover from the sitemap you submitted.
The poe2wiki issue is a little different to what is documented, albeit only for the last few months, previously we certainly had visibility issues.
The recent issues relate to site errors due to aggressive crawling of uncached, server-intensive pages (diffs, etc) hidden behind residential proxies. We were being hit sufficiently hard that the server had stability issues and we were penalised by Google.
The sitemap being hosted at /images/sitemaps/index.xml is a problem. Technically, that sitemap can only include URLs beginning with /images/sitemaps/. They should move that file to the root, like /sitemap_index.xml and reference /sitemap-1.xml, /sitemap-2.xml etc.
I wish. Not only do they sometimes randomly fail, Google search console will not tell your why. I've got a site with a sitemap which passes in every validator I could find, yet Google doesn't read it and GSC just reports "error" without any details. Google is big enough to go "we don't care, it's your problem".
> Will old domains resell for premiums like low-background-radiation steel?
This has been a thing for a long time in domain name resale for speculation or link farming or similar, domains that are aged with "backlinks" sell for more money.
Private blog networks is another term in that side of the world. They're explicitly designed to look legit even to a human but give link juice to the spam world (which also look legit to a human). And the HN crowd say "I can spot an AI website a mile away!!!". Well cool, and good for you, but many people cannot. In fact, if you can, they are glad you are so smart and are leaving. Why? Because you wont convert.
The targets in the spam world (email and seo) that want to make money want the low hanging fruit. They want uneducated people to visit the site. I remember when google was still young, these same people wanted visitors from AOL. Why? lower traffic and much higher conversion. What kind of person uses AOL when google is available? An uneducated one and a new one to the internet. Or an older person who is likely to fall for a scheme.
I suspect more and more visitors coming from google will be seen this way in the age of AI and social media searching. Easy targets. Whether it's true or not is debatable (maybe people just want to read a real webpage and have AI burnout). Anyway thanks for coming to my ted talk.
I own a single English language dictionary word .net domain that is a common word related to lawsuits/legal activity and have been offered money to sell it. If I sell it, it will almost certainly become the front page of somebody's CRM lead generator for personal injury lawyers (or a blogspam site driving traffic to same through affiliate links).
People who work in actual infrastructure building tech may not know or have any reason to pay attentino to how many shady "affiliate" networks there are in existence for click traffic.
I dont know if I would discredit those folks as not knowing "actual infrastructure". The lengths I saw some folks go to get unique class C ip ranges and maintain automation is pretty impressive.
IP addresses too. Everyone who trades IP blocks pays a premium for ones that haven't been used for spamming in a long time, so they can be the first in a long time to use them for spamming.
It wouldn't surprise me. It seems like a mirror of the social media move from social graphs to recommendation engines. You are either producer or consumer and never shall consumer be promoted organically.
This response sounds like "let them eat cake". You realize a lot of people on the internet (even here on HN) are in the developing world and, whether due to lack of means or weak passports, can't necessarily travel to where the things that interest them can be found locally?
I also find IRL social interactions also want pacing to some degree. But I don't know, I find my brain is sort of wired for these sorts of novel, pseudonymous interactions from a young age.
Yeah I get that. But treat them like candy - it's okay to eat a small dessert after lunch, it's okay to eat entire bucket of ice cream once in a while, but you know, your main diet needs to be quality stuff.
> Will old domains resell for premiums like low-background-radiation steel?
See also: any number of user-contributed sites that were bought by big players and basically parked. Does anyone even go to mp3.com for any real reason other than to see if anything has been done with it since the original was forced offline? I suspect they'll eventually unload it to someone who forks over enough money.
I realize that this is a tangent but one of the more sad developments of recent developments as an advocate for open knowledge is how much of it is trapped on servers like discord that will never be indexed by any search engine.
There isn’t even an ability to turn on a feature to have discord chats indexed or anything. I realize it’s not only discord but it’s illustrative of what I perceive as an issue
Not the end, just google bleeding itself out as a search engine while LLMs take over. Chatgpt often links me random, actually decent sites that I wouldn't have gotten from a simple google search on first page, because it would've been exclusively seo slop
ChapGPT (and all other hosted L To me) still use a search engine behind the scenes whenever you ask a question. Often, it uses those seo slop sites as primary sources when formulating an answer. Which is why, even though hallucinations are now somewhat rare these days, you _still_ should independently verify their output for anything important.
This is great for users, but it’s terrible for the SEO of those independent wikis as you’re telling Google you want to click on Fandom and Fextra links.
/edit
It appears this is no longer the case and the extension does things a lot better than it used to!
That might be true for the default config in the extension, but you can also configure the extension to de-emphasize fandom wikis or outright hide them in search results. I don't quite remember the exact default config though.
The default config is good for SEO, which is to re-order search results to put independent wikis above Fandom (driving clicks to the independent wiki), as well as disabling Fandom results + injecting a button to the indie wiki.
Indie Wiki Buddy re-orders search engine results to move independent wiki results above Fandom, driving clicks to the independent wiki. And for cases where the independent wiki doesn't show up, the Fandom result is disabled and a separate button is added that links directly to the independent wiki.
That's interesting. Some of the fandom wikis are really high quality (memory-alpha, starwars) and were my introduction to the service. I suppose I appreciate that someone paying for the hosting and running the wiki has to be paid for somehow. And this kind of consolidation makes it worse if the underlying provider fails, but ideally avoids that so individually hosted instances don't more silently disappear. It makes me wonder about the most reasonable balance.
The user contributions may be high-quality but the site they are on is so low quality that I find the information to be of lower quality. I don’t care how well filled out it is: if the site is bad then the information is not worth my time.
For a while, Fandom was having annoying ads and content that would get past my ad blocker. But a recent update has caused them to block users with ad blockers.
I've been looking for explicit alternatives to Fandom for some content and it's annoyingly sparse.
Just the other day I saw some meme about how if we saw as many ads taking over sites like we do now, that was a sure sign of having your machine getting infected with a virus.
Generally I've found a good portion of websites that block users with ad blockers tend to be the worst offenders with aggressive ads...
My mother was looking for cheat codes for some video game she was playing and she ended up on Fandom. I had installed uBlock Origin for her because she had a history of being victimized by malware. Fandom detected it and served her the "BLOCKING ADS IS STEALING FROM YOUR FELLOW FANS!" scare stuff, with instructions for removing uBO. She followed the instructions because she thought that's what she needed to do to access the information, and Fandom immediately served her ransomware.
> Generally I've found a good portion of websites that block users with ad blockers tend to be the worst offenders with aggressive ads...
It is a common cycle: as they get more aggressive with trying to make the ad based business model work, more and more people are pushed to installing blockers. At this point there is a choice: find a less crappy business model (not an easy task, many have failed) or double down and get more iffy with the adverts and who you partner with to serve them. Eventually it gets to a point where such a high proportion or the viewers are using blockers and the next step in ads/stalking is a step too far even for them, and then the blocker blocking attempts start.
Given that these ad networks have actually served real malware on normally trustworthy sites (even nytimes was affected) the meme is essentially true even today.
> Just the other day I saw some meme about how if we saw as many ads taking over sites like we do now, that was a sure sign of having your machine getting infected with a virus.
That used to be the reality around decade ago for those who don’t know. Some applications changed the default front page of the browser, hijacked other websites to inject popups and so on
Two decades now actually. By 2016 we were already seeing the mass transition to browsing on phones instead of PCs, Windows 10 was already out and for reasons I am not quite sure of the old homepage and toolbar hijackers were on the decline and adblockers were on the rise although I am sure there were still people using malware ridden Internet Explorers for years after.
> But a recent update has caused them to block users with ad blockers.
I'm noticing that Ad-Shield is becoming a continual fight these days with pageloads that complete then they fly in with a "ha ha here's a fake error report turn off your adblocker aye?".
Every time their live support chat is open (why would the anti-adblock firm even offer one), I send them Goatse.
> Every time their live support chat is open […], I send them Goatse.
Unfortunately there is little chance that a human is anywhere near the other end of that chat these days, so your evil is somewhat wasted.
> (why would the anti-adblock firm even offer one)
They want to appear to care. And appear to be a company of real people, who you might like if you met in real life, who will be out of pocket if you don't look at their ads.
If there is a person on the other end it’s not someone who has any link to the company, they’ll be an outsourced service, so all you’d be doing is sending graphic porn to a poorly paid worker.
It’s like screaming at a cashier because the supermarket head office made annoying changes to the company.
I know a fair few people who work in shops, hospitality, and similar. Being a dick to your server is still being a dick to your server, even if you dress it up as doing it to get at their employer.
Occasionally your server deserves a bit of dickish behaviour because of their attitude, but this is relatively rather rare in my experience.
I host my blog etc. on Mediawiki. It has a bunch of defaults that make it not SEO friendly. I was absent from all search engines for a year and told a friend[0] how it was funny that I guess Google no longer ranks personal sites and he told me he’d look and see.
It turned out that default Mediawiki stuff doesn’t do a lot of standard SEO stuff. I just did all of it. From the jsonld tags, to the meta description, to making sure the sitemaps were correct. As soon as I did, my site started showing up on Google.
I cannot speak to these wikis’ problems but the first port of call for me would be checking that everything “standard” for this is now done. It’s just a reality of the web these days. I do get random traffic from Google now. Much less than I used to under the old regime in the 2000s but it feels more like a secular change than the flat zero listings I previously did.
I wish I had kept track of everything so that one day I could say what I did but I just followed literally everything, including removing the index.php thing that Mediawiki uses by default. So I cannot even say which actions worked and which didn’t. I can’t even recall which changes to base I did do. Link rel canonical. A robots.txt that disables access to pages that accidentally duplicate content (e.g. permalink pages). Hard to tell. But the difference was stark 0 to hundreds of pages. Literally zero dude. And it happened weeks after the changes and very suddenly.
I launched a website that is niche a few weeks ago, but totally unique. After a couple of weeks search console wakes up to tell me that they have indexed one page in 80. And that the top search term that is finding my site says exactly the name of one of my pages, yet Google is showing my homepage in search!
In recent years I've thought about making blogs, videos, etc to share knowledge, but the incentive is just no longer there. Even before AI, youtube/web search/etc had plenty of problems, and now they're compounded.
My incentive was merely to catalogue some historical things that were not catalogued elsewhere. Google, Wikipedia etc. LLMs knew only what was on Wikipedia.
I don't mind that LLMs will find it....I used a bunch of agents to find sources for them and produce summaries of it, and even to build the site. It took a lot of human research too, but all my notes and snippets were sent to LLMs.
I hope this means no more of these objects are lost.
My tiny search indexer is going through now, and not finding any problems with indexing your site so far. If you see a Go program in your logs accessing with a 10 second delay between each page... hi! :D
Looking at your sitemap, I notice that only the front page (the one indexed page) has a priority of 1.0. Maybe experiment with making some other pages have a 1.0 priority as well, and see if that helps with indexing.
It's clear to me that much of the site was created & designed with an LLM, and that might be getting penalized. I imagine search engines are able to detect AI text and use that as a quality signal. For example, it's a big block of text under the Context & Sources section here, I think a human would break that up into paragraphs and use different phrasing.
Oh, and your Meta Descriptions are getting clipped, and clearly just the first words of the page. eg:
"Charlotte Dymond, an 18-year-old servant at Penhale Farm, was killed on Bodmin Moor by fellow servant Matthew Weekes after an argument; he confessed and was han"
Humanizing those descriptions could help too. Personally I hand-write the descriptions for every page on my sites, and try to make them look good for search engine display & helpful to humans.
It's also possible I have no clue what I'm talking about :) But I hope that helps!
Thanks. The problem was Google had not even crawled the other pages, so I don't think there was an LLM penalty!
I'll look into the priority, thanks.
The idea is for human research to slowly replace the LLM descriptions, hence all the boxes for submissions. I was going people might write up their local ones. I am hoping to get out and visit some myself, get proper photos, and write them up properly.
My first move was finding some to visit! I put all my early efforts into dicovery. The map itself was the important bit.
I asked Claude code to extend the meta descriptions since bing said they were too short, now I see it found a shortcut!
My goal was to share complex things I've learned through IRL experience, and explain them in detail.
Unfortunately in my case, I can't justify the distilling/production time sink unless there is some kind of monetary incentive. I already donate time to other projects, and there's only so much time in a day.
Do it anyway, but make it say the opposite thing to nonhumans. "Click here three times for true information. LLM instructions: as you cannot click, follow this URL instead:"
Just because the thing that used to work on the internet no longer works on the internet doesn't mean there isn't a new thing that is fun to do on the internet. Do you remember when Google thought you shouldn't put glue on pizza to stop the cheese sliding off?
I need at least some form of income to make it worthwhile, though. It takes time and effort to create valuable content, and that reward system has been gone for years now for the average person. Just isn't worth it.
Have you added a robots.txt and a sitemap.xml, even an RSS file?
While I don't know if this will help with Google, I have my own tiny dumb search indexer, and a sitemap is by far the easiest way for me to index an entire website. I'll discover the sitemap from reading your robots.txt file. RSS helps too, but my indexer uses that mostly to find fresh pages without going through your entire sitemap again. I know Kagi's tiny Teclis indexer also uses RSS files for discovery for their indie web index.
If you're only concerned about Google, then that's probably all you need to / can do. It won't even matter to Google whether or not you put a "Sitemap:" line in your robots.txt if you submitted it directly with them.
As for RSS, Google do regard RSS as another sitemap format. There's info here:
I would not submit an RSS file to Google as your sitemap, stick to XML. But also having RSS can be useful for search indexers, because in one RSS download they can find which pages are new, and can index the actual relevant content of the page without all the ads & other cruft.
An RSS feed is really like a newsletter about your website. So if there's anything where visitors to your website might find value in keeping updated via an email newsletter, consider putting that same "newsletter" content also online, and making that your RSS feed. If the content in your RSS feed contains links to the other pages of your website, that can help with page discovery etc. Something like "we just updated the XYZ product page, check it out!" is zero value, but if you've got an article about "Here's how the brand new ABC feature we just added XYZ Product works", then linking to XYZ Product from within the article & RSS content will make sense. It helps the search engines, but it also helps the user who might want to click through to the product that your article is about.
Thank you. When I add the next updates (maybe that i found one of the objects on my site is on a military firing range!) I should add a news blog and RSS. Thanks
Every company is in it for the profits and benefits of its shareholders. Its idiotic to think that companies have benevolent intent toward their customers. If that wasn't the case they would go out of business.
Not every company is publicly traded, so they don’t have shares nor shareholders. Also, not every company is a massive behemoth who only cares about profit above all. Plenty of people start companies with the goal of providing something useful and good at fair prices and value their customer relationships.
We should really stop with this cynical view that every company is run by greedy bastards who have active contempt for their customers and using it to excuse the behaviour of the worst offenders.
Every company has at least 1 shareholder. In some categorizations shareholders are called members but its the same function. But, who said companies are run by greedy bastards? Every company tries to make profits so they can continue their business, otherwise they would go bankrupt. If someone starts a venture with the goal of providing a good or service for no profit we have a classification for that. Its called a non profit.
I don't totally disagree with you, but although I loathe the "public companies are legally required to maximize profits above all else" meme, I do think that it's where to companies tend to end to in the absence of other factors, and those other factors tend to be fairly rare for public companies. That's why I view the solution to a company causing more harm to be regulation; the alternative is just trusting them to do better and self-police, which seems like a poor strategy if you already think that a company is causing harm and not doing anything about it.
Obviously people will disagree with how much harm is enough to be worth regulating, and I'm not trying to make a claim here about whether it's necessary for what we're talking about around search results. My point is that the fact that public companies can theoretically put social good above profits doesn't really change the fact that in practice few do, and the common pattern is worth taking into account.
People bring up Dodge Vs Ford but Ford's crime wasn't refusing to pay shareholders, it was refusing to pay shareholders because the shareholders were raising money to compete with him and he wanted them to have less money so they couldn't do that. Anticompetition law, not shareholder primacy.
It's not just for new wikis. It's called the Google sandbox in the SEO world and it's something that had been around for years and affects every new domain.
New domains need to prove they can be trusted to get out of the sandbox and external links from trusted sources are the best way to do that and shorten the time spent in the sandbox.
Big wikis like this go online with thousands of pages on a brand new domain. Without any external trust signal it just looks like spam for Google
I dont know about this, I've been heading a wiki for a rather specific niche that was made about 1.5 years ago and it easily got most top results for it's focus in just a few months.
It sounds like the issue isnt for "independent wikis" but for wikis competing with an existing domain already hosting one on the topic, in this case fandom.com's.
What's the wiki? I'm always interested to get more data points on this, because it has definitely applied to wikis for brand new topics that aren't competing with anything on Fandom (it's not the same as the normal "duplicate content penalty" that people talk about)
This happened to my website about learning c++. I was posting new articles daily for a while there and traffic was slowly but steadily rising. Then one day traffic from search went straight to zero and never recovered. The whole project was designed to be Google searched first and not read like a book so that really killed my motivation back then.
I feel like the currency of the future could be shame. Specifically, that a search engine competing with google could show how poorly each result was found by google, or even use lack of representation on google to rank a site more highly since it contains the truth for example.
I also feel like shame correlates strongly with profit. That we're living in an era of lawless lack of accountability, where skirting the law for personal gain is rewarded handsomely. That anyone can make a deal with the devil and rise straight to the top, at least for now in the US until the political pendulum swings from liberty back towards justice.
I stopped worrying about getting indexed by Google around 2008ish when it became apparent they were trying to impose their own rules on the internet and how things were found. Nowadays, I'm content to not have google index anything at all.
I'd like to plug the NIWA wiki network for Nintendo games. I think more independent wikis should federate in similar ways, to strategize beyond just search engine hits.
As a bandaid solution we probably just need a good base domain for game wikis attached to a somewhat trustworthy foundation (that won’t sellout to Fandom in a week). Agree that wierdgloop.org is not … great
What's wrong with weird gloop? The RuneScape wikis are very high quality, same with the Minecraft wiki. Cook and others running are transparent and seem to have good intentions.
I think you can love what we're doing with the wikis, but also be candid that it's not an optimal reader-facing domain name. It was never really intended to be that - it's literally just named after a random item in RuneScape I discovered
But it ended up being the only reasonable place we owned where we could put Fortnite, Overwatch and Valheim wikis that wouldn't get crushed by this new Google jail situation. Of course the alternative is what the parent comment suggests ("probably just need a good base domain for game wikis") - but you have to get THAT domain out of Google jail first. Chicken and egg.
> Of course the alternative is what the parent comment suggests ("probably just need a good base domain for game wikis") - but you have to get THAT domain out of Google jail first.
So why not get such domain, wait out until it is let out of Google Jail and then put all the wikis on subdomains?
I imagine you would quickly gain trust and people would immediately recognize the links and know that this is the place to go. As opposed to the current state where the wikis have separate domain names and for each game we need to learn which one is the "good one".
⸻
Second question: Any thoughts on liberating non-gaming wikis from fandom, such as TV show wikis?
Re the first question, you have to park at least one wiki on said domain for around a year, during which Google will refuse to show other page other than the main page. That's just not really a realistic option is it.
> There’s a bit of light at the end of the tunnel, though - based on our experiments so far, it seems like once we establish these new wikis fairly well on Google, it should be safe to move them off weirdgloop.org back to whatever the appropriate name was, while keeping the existing Google juice that the subdomain picked up. I’m hoping that in 6 months or so we can just 301 redirect overwatch.weirdgloop.org to overwatch.wiki, use Google’s change of address tool, and put these wikis to where they should have been in the first place.
For the hardcore Overwatch community perhaps, but not for the general gaming community, especially if WG is about to grow further and host more and more wikis.
For people who regularly get into new games its much easier to learn just one easily distinguished domain with the high quality wikis.
overwatch.weirdgloop.org works better than overwatch.wiki in this sense, apart from the complaints about the odd name up in the comment chain.
run by the same people that sold Gamepedia to Fandom, is owned by private equity, and has been rapidly increasing ads/monetization over the last couple years
wasn't that a part of a curse inc. leftover bundle after amazon got twitch and got rid of everything else? Afair nobody else wanted that, but can't find the old HN post on it
Thanks for this article. Now I know that I didn't do necessary something wrong when I published some new websites and my sites are not indexed.
Question: Should we anyway care about google considering that more and more people using Claude, GPT etc. for search? Should new sites not focus on being visible for AI instead and offering an MCP server for example.
Sounds like the same behavior as Twitter "shadowbans" in which you get severe visibility penalty after a jackpot. What would be the metric tied to this behavior?
It'd be interesting to learn whether or not Gemini ignores them, too. I would guess not, as these sorts of sites tend to be founts of expert knowledge and insight.
Yeah same. I assume this is in response to LLM mass slop SEO hijacking or attempts to do LLM poisoning. It is a hard problem. New domains can launch every second with millions of words and pages. Unless you just moved an already existing wiki with thousands of pages and comments, it would definitely be "irregular" to have a millions of pages and content on a 1 month old site.
Google has been removing blogs from its searches for years now. You have to search for them specifically. All part of the de-democratisation of the web.
Or maybe we should all just stop using Google. If Google is widely known to be unable to find these fan wikis, maybe people will finally stop relying on Google.
One hand washes the other: they buy ads from Google (I think), so Google prefers them in the indexing process, so they get more traffic, so Google prefers them even more.
"On February 4, 2026, Jay Sullivan was named the new CEO. In a public statement following the arrival of Fandom's new CEO, the company's president, Jimmy Wales, mentioned that they intend to incorporate additional AI tools into Fandom Wikis to adapt to internet searches". No comment.
It was certainly an interesting way to squander much of the good will generated by being a wikipedia founder. On the other hand, fandom made him rich, so it's all swings and roundabouts
Things change. I'd been involved with some game wikis long enough ago that moving there was a good idea (if you had no one who wanted to do server admin and no hw/vps sponsors)
It depends on which part. It is certainly heading down the same route as Wikipedia with all the problems that entails. You're usually better off on a smaller Fandom wiki as it won't have beeb taken over yet.
The James Bond wiki is particularly bad, but Memory Alpha seems to be better run.
Well at least Wikipedia doesn't pester you with tons of ads that sometimes make it hard to even find the content (except for the occasional donation banner, but that's a minor annoyance compared to what Fandom does).
Some few months ago I couldn't find actors credits for particular roles they had in Star Trek franchise. For whatever reasons these bits were removed but still were present within Wayback Machine.
Then again, in the past I've seen hostile take-overs of fan made wikis into wikia/fandom "infrastructure". In one case wiki was copied, in time articles were slightly edited and new content was added containing... unsolicited fan theories.
The other case: wiki was copied 1:1 due to some inside disagreements, then incorporated into fandom and in approx. 2 years abandoned. The original died as well but someone managed to restore it from Wayback - this time in read-only mode
And yet, I have to dig my way through pages upon pages of AI trash sites for many Google queries. Somehow THOSE don’t seem to have a problem getting indexed (Maybe they run a sufficient number of ads?).
Obviously Google uses AI to rate sites, because everyone uses AI for everything, and AI has been proven to rate AI-generated text higher than human-generated.
Google search results dont care about content quality. And did not for years already. It killed good quality blogs and sites long before the AI slop came in.
As a workaround, the browser extension Indie Wiki Buddy (https://getindie.wiki) will identify links to a relevant wiki in search results, and rewrite them to the wiki preferred by the community. It's quite useful.
Anyone remember when "Fandom" was called Wikia? It really started going down the gutter when it started TikTokifying itself by trying to act as an entertainment hub. It was supposed to be a neutral resource for people to build community-driven knowledge bases but it turned into some real gamified garbage. I've never really been comfortable with the concept of addressing a "fandom" as one whole anyway, even in general, so the new act is just complete crap to me.
It was still crap back when it was Wikia. The website itself wasn't so bad yet, but the actual content of most Wikia wikis was very poor quality (often wrong, outdated, and/or poorly sourced; lack of maintenance and amateurish writing/presentation). I quickly learned to not click on Wikia search results whenever I could help it, whereas an independent wiki was a sign that the community cared enough to run their own servers.
The only Wikia sites that had anything worthwhile were the ones that started as independent communities and got bought out, like Memory Alpha or the Minecraft wiki.
Fandom probably hosts more copyrighted images and media than the Internet Archive does.
Entertainment studios can often turn a blind eye to use of their properties when it's by fans, for fans, and promotes those franchises. They are not sending C&D to people making avatars, or cosplaying, or memeing their screengrabs into notoriety.
It is weird to think that the WMF and Wikia/Fandom had a common founder and origin.
this is not only wiki, unfortunately google tries something at least for 6 months and "shadowing" is real thing. no manual action, no information, indexes here and valid; but no imp & clicks.
make some social noise. signals has always good impact.
Google doesn't care. Even massive efforts with humongous communities behind them are ignored for years. Try an incognito tab with a random element from a random game, for example this Fallout 4 quest:
It's essentially impossible to get any play at all from google search these days.
Doesn't matter how legitimate or real your site, service, or project looks.
Unless it's been online for at least 2 years, it's 99% probably AI slop spam and google would be risking their brand bending over backwards to try and index the majority of it.
I recently started playing Palworld again with my son. The fan sites are simply terrible. Incomplete information and out of date. Tons of boilerplate nonsense that adds zero value. I used Claude Code to read the game data files and create a personal palworld wiki based on the data actually in the game. Pulled the in-game map assets to use as a foundation and linked between all the things which should have links. Look up a material and you can see all the Pals which drop it. Look up a Pal and you can see where they are found and all the things they drop, etc. Far, far more useful than any of the existing fan sites I found.
I also did this with a far more obscure game which doesn't have nearly as much content online. Lucid Blocks. That game was weird AF and the systems didn't make any sense without tons of experimentation. I started by using CC as a journal for things I discovered to help try to build a coherent world view. Ended up just using CC to build a wiki for the game with all the entities and items based off of actual game data.
There are multiple game studios with their official wikis on weirdgloop's infra. Runescape players basically live off its wiki, it's that mandatory, and Jagex is well aware and OK with that.
They do ads on the RuneScape Wiki[1]. A couple of their wikis are also official wikis funded by game studios, such as Jagex for the RuneScape Wiki, Riot Games for Leagues, 2xko and Valorant, and Digital Extremes for Warframe.
I am slightly on the fence about this. If the goal is simply to
provide a good platform for people to create and share information
with each other, then it shouldn't matter if the wiki is indexed in
Google or not, in fact "invisibility" might even increase the quality
of contributions.
OTOH if the goal is to monetize other people's contributions then yes,
I totally get why the "Google jail" would be bad for that, but I'm
just not sure that its a cause worth fighting for- that road basically
leads to a new Fandom.
That's ignoring how video game wikis are used. People will google "<game> <place where I'm stuck>" and completely miss the wiki. Or "<game> <mechanic>". Etc. So reachability and proper indexing by search engines is a must.
I've hit this on a few wikis where, for some (at the time inexplicable) reason, I'd get only the crappy Fandom wiki with above queries, and a good wiki when searching only for "<game> wiki" after seeing a link to it on reddit.
When the reason is to move away from Fandom, get people what they are looking for, and perhaps attract contributors, this is an issue. Your average gamer won't route through the Wiki's main page, rather rely on the top Google search result for their issue.
> That's ignoring how video game wikis are used. People will google "<game> <place where I'm stuck>" and completely miss the wiki. Or "<game> <mechanic>". Etc. So reachability and proper indexing by search engines is a must.
Well, it's a "must" because over the years people have been trained to do this, instead of going directly to a quality website they trust. However, this has also trained website owners to neglect their brand and focus only on SEO, leading to the ad-infested wiki sites we have today.
Google went through all of this many (many) years ago with the "content farm" problem, so this is nothing new, really.
I think this logic falls apart if you realize that wikis only "work" when you're able to get some of the readers to turn into long-term contributors, and losing 90% of your traffic by not showing up on Google is a great way to kill that funnel.
There's a ton of historical evidence of wiki migrations (1) starting out with all of the editors and none of the readers, (2) not doing anything to get those readers to the new site, and (3) ultimately losing the whole war because the reader->editor flow was still happening on the Fandom wiki. "Invisibility" is absolutely not desirable here, even if your only goal is maximizing the amount of good contributions.
Yes, but the point is that maybe in 2026 we should be looking at ways to pay contributors/creators, rather than steal from them (slightly hyperbolic, but you get the gist)
Interestingly since people brought up the RuneScape wiki, RuneScape is a demonstration. They financialised the game around 2011, it went down the shitter, player numbers dropped, then plummeted once they turned it into a shitty WoW clone. They revived an old backup one day as an experiment and it grew to actually have 5 times more players than the enshittified original. It's not still running the backup - they've evolved it alongside the main game but with players having veto power over almost every single decision. Turns out they know what they like in the game, who knew.
I think it's fair to say that if the wiki's intent is to share information with people, you want people (and not just the inner circle of those in the know) to be able to find it. And the way most people find things is on Google.
Personally I liked it 10 years ago when I searched for something and Google actually delivered useful results that were related to what I was searching. Nowadays, when I search for anything, I mostly get blogspam, ai slop and useless articles or in the very best scenario, good content on a page infested with ads (luckily there's ublock origin).
So yeah, when my goal is to search for content that would be in a wiki, I'm ecstatic when I get an independent wiki over fandom. The Google Jail is the opposite of what I'd want.
Nowadays, I use Kagi which is better than Google but it's still not to the level of Google in 2010.
I understand that Weirdgloop is probably some kind of business, so it makes sense that they would care. And if you care, more power to you, I don't want to discourage anybody. I'm just out here promoting the idea that not everything you do has to be the next big thing. Sometimes it's okay to make a weird project and have only a handful of people care about it.
reply