Only some search engines can access and index Reddit content

30 July 2024

Google, and Brave Search, are apparently the only search engines permitted to crawl Reddit, and index content published there. Other search engines, including Bing and Duck Duck Go, are presently being prevented from accessing the “front page of the internet” forum and discussion website.

Reddit appears to be concerned that search engines other than Google and Brave may use content on their website for training AI chatbots. The move has nothing to do — or so we are told — with a sixty-million dollar deal between Google and Reddit, made earlier this year.

This deal precisely allows Google to use content published on Reddit to train its AI apps though. It seems like it’s ok to take Reddit content for these purposes, so long as you’ve stumped up the cash. I wonder if Reddit members, who wrote the content in the first place, see any of this money.