Does ahrefsbot just straight up ignore your robots.txt
@sir those types of crawlers usually have an option to ignore robots.txt when being set up.
@sir Maybe the server should just straight up ignore ahrefsbot. :)
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!