SemrushBot-OCOB user agent, IP ranges and robots.txt

Semrush operates SemrushBot-OCOB, an SEO-tool crawler that Semrush says crawls sites for its Content Toolkit reports. It obeys robots.txt under the SemrushBot-OCOB User-agent token, which is the only place Semrush documents it. Semrush publishes no IP ranges and no reverse DNS suffix, stating it does not use consecutive IP blocks, so no verification method is documented.

Semrush does not publish a full user agent string for this crawler, so the only documented identifier is the user agent token SemrushBot-OCOB. The operator's own wording is brief and ties the bot to one product, the Content Toolkit and its reports. Semrush gives no further description of what the visits collect or how often they occur, so anything beyond the token and the product name is undocumented.

Because this is an SEO-tool crawler rather than a search engine, allowing or blocking it has no documented effect on where your site appears in search results. Blocking it removes your site's data from the Content Toolkit reports; allowing it lets those reports include your pages. Semrush documents the bot on its /bot/ page only as a robots.txt User-agent line paired with Disallow: /, which is the exact rule to add if you want it to stop crawling.

Verification is the weak point. Semrush states that it does not use any consecutive IP blocks, and it publishes neither an IP list nor a reverse DNS suffix, so there is no operator-backed way to confirm that a request claiming to be SemrushBot-OCOB came from Semrush. Site owners have the user agent token alone. JavaScript execution is also undocumented; Semrush does not say whether the crawler renders pages.

Operator note. Semrush documents this token only as a robots.txt User-agent that can be blocked with "Disallow: /" on its bot page. JavaScript execution and IP addresses are not documented.