AI scraper bots like your website content, you should feel grateful

28 March 2025

Herman Martinus, creator of the Bear Blogging platform:

Bear is hit daily by bot networks requesting tens of thousands of pages in short time periods, and while I now have systems in place to prevent it actually taking down the server, when it started happening a few months ago it certainly had an impact on performance.

I check my website stats every morning, and high hopes that something I wrote might have gone viral, wanes almost immediately when I realise AI scraper bots have been at it again.

I considered trying to block the data scrapers, but read that such methods are often ignored. I suppose I should feel faltered that developers of AI bots think the content published here is worthy of training one of their LLMs. There seems little else I can really do.

RELATED CONTENT

artificial intelligence, technology, trends