Reddit now blocking search engines and AI bots from showing its results, except for Google
Jul 26, 2024 at 9:30 AM

Reddit now blocking search engines and AI bots from showing its results, except for Google

Google has become the exclusive search engine capable of surfacing results from Reddit, following an update to Reddit's Robots Exclusion Protocol (robots.txt) last month. This update aims to prevent unauthorized data scraping by AI bots. As a result, searches for Reddit content on rival engines like Bing now yield no results, a fact confirmed by 404 Media. This exclusion extends to other search engines such as DuckDuckGo, Mojeek, Qwant, and more.

The exclusivity likely stems from a $60 million deal between Google and Reddit, allowing Google to train its AI models on Reddit's content. Reddit's decision to restrict access to its data is seen as a strategic move to safeguard its information and create an additional revenue stream. This action aligns with Reddit's increasing efforts over the past year to protect its data and satisfy new investors.

Jul 26, 2024 by Paul

HeelNagisaSTBayAreaPatstoyangenov
Heel found this interesting

Comments

Tubby 9417
CommentJul 30, 2024

Well, as a duckduckgo user I am not to happy to know that interesting results from reddit may no longer showup. Time to switch to another search engine which doesn't bother with robots.txt I guess ?

0
zerush
CommentJul 28, 2024

Not a problem with Andisearch to access reddit content, despite it use AI and don't use Google. Seems that the Reddit filter isn't so good as they want.

Andi icon Andi

2
Sam Lander
CommentJul 27, 2024

The arrival of Reddit dismantled a large number of niche forums, because it's easier to sign up for one service. And the quality was never the same anymore. Those actual professionals just didn't bother to go to Reddit. Fortunately good ones still exist, like Creative Cow.

And Reddit is a trash echo chamber. It's practically the left-wing echo chamber like Gab is a right-wing echo chamber.

It's good if it doesn't show up in my searches. I don't need the opinion of some random 20-year-old who just learned how to wipe.

2
guck_foogle
CommentJul 26, 2024

I love how shitbag companies like Reddit and Google talk about "their data" and "their content" when it actually comes from it's users.

3
k 4u
CommentJul 26, 2024

Not cool... I've gotta use my damn bookmarks now that are stuck who knows where. Not a chance to get back to google though xD I have the strange feeling that wouldn't be the last shit update they do and drive it further in the ground...

1 reply
zerush

As said, Andisearch can still access without problems

Reply written Jul 28, 2024

If this reply contains spam or other abuse, notify admins about it.
2
catalin560
CommentJul 26, 2024

never liked reddit... it has garbage UI&UX and recently with the "modern" UI its even slower than before... also the content is so-so, you can barely find anything useful on it, the most useful thing that I can think of is the piracy list that seems to be updated regularly...

2 replies
Azazel

Last UI update is pretty good on my end, visually and performance-wise. And can you name another similar platform where content isn't "so-so"?

Reply written Jul 27, 2024

If this reply contains spam or other abuse, notify admins about it.
catalin560

I meant that "so-so" from my perspective, I usually hang out on platforms that focus more on a single thing rather than an amalgamation of stuff... and I also prefer forums since they have way better search functionality and topics are neatly organized under different categories...

Reply written Jul 28, 2024

If this reply contains spam or other abuse, notify admins about it.
3
TBayAreaPat
CommentJul 26, 2024

That's not all bad.. Reddit seems to junk up search results anyways when using other AI search engines

2
Gu