Semhack fetches web pages with this user agent:
SemhackBot/1.0 (+https://semhack.com/bot)
When it visits
- Free tools: when a visitor checks a URL with one of our free tools, we fetch that single URL (and its redirects or robots.txt) once. These requests are made on demand by a person, so like other user-triggered fetchers they don't consult robots.txt.
- Site audits: when a customer starts a site audit, SemhackBot crawls that website from its homepage, following internal links and the sitemap, up to the page limit of the customer’s plan. It obeys robots.txt rules for
SemhackBot(and skips anything disallowed for Googlebot), honorsCrawl-delayup to 10 seconds, and makes at most two requests at a time with a pause after each.
How to block it
Add this to your robots.txt to stop crawls by SemhackBot:
User-agent: SemhackBot Disallow: /
Problems?
If SemhackBot causes trouble on your site, email hello@semhack.com with the affected domain and approximate times, and we'll look into it right away.