DrUrls-Bot
If you found this page in your server logs, a Dr Urls user asked us to audit a site — usually their own. The crawler identifies itself as DrUrls-Bot/1.0 (+https://drurls.com/bot) on every request.
What it does
- Fetches HTML pages only, following your sitemap and internal links, up to the page cap the requesting plan allows. It does not run JavaScript, submit forms or sign in.
- Reads and obeys
robots.txt. A disallow rule stops it; a 5xx or a timeout onrobots.txtstops it too — it fails closed. - Keeps a small number of connections per host and never retries a page that refused it.
- Never opens private addresses: every hop is checked before the socket opens.
How to block it
User-agent: DrUrls-Bot Disallow: /
Takes effect on the next audit. If a scan of your site was run by someone else and you want it removed from our public directory, or you see traffic that does not match the above, write to support@drurls.com with your domain.
Want to see what it found? Run the free check on any page.