Our crawler
Clickwalker/1.0
If you saw Clickwalker/1.0 (+https://clickwalker.com/bot) in your logs, this page explains what it is.
What it does
Clickwalker runs site checks and flow tests that its users set up for their own websites. A check loads one page, its images and scripts, and sends a quick request to the links on that page to see whether they work. A flow test clicks through the steps the site owner wrote.
It doesn't crawl whole sites, index content, or train AI models on what it sees.
How it behaves
- It identifies itself with
Clickwalker/1.0 (+https://clickwalker.com/bot)at the end of a normal Chrome user agent. - When checking links, it sends at most three requests at a time to any one server, usually a HEAD request.
- Runs are started by a person, or by a schedule they chose. Scheduled runs happen at most every six hours.
Allowing it through a firewall
If you use Cloudflare or another bot protection service and want to test your own site, allow requests whose user agent contains Clickwalker.
Stopping it
If Clickwalker is visiting a site you run and you didn't set it up, email hello@clickwalker.com with the domain and we'll stop the checks and remove the site from the account that added it.