ReoncoreBot
About our crawler
What ReoncoreBot, our crawler, fetches from regulators' websites, how politely, and how webmasters can reach us.
Last updated:
ReoncoreBot is the fetcher behind RuleScan AI (operated by Reoncore Labs, Dhaka). It reads public regulatory notices so that businesses can be told when an official document changes. This page says exactly what it does and how to reach us.
How to recognise it
- User-Agent: ReoncoreBot/0.1 (+https://regulationradar.vercel.app/en/bot; hello@reoncorelabs.com).
- HTTPS only, with full certificate verification. We never downgrade TLS or ignore certificate errors.
- The name deliberately avoids the word "scan", which some government firewalls treat as a vulnerability scanner.
What it fetches
- Only the public listing pages of regulators named on our coverage page, and the official PDF files those pages link to.
- Nothing behind a login, CAPTCHA or bot challenge, and no private or undocumented APIs. If a site blocks automated access, we list it as a coverage gap instead of working around it.
How often
- Each listing page is read at most once an hour, most every 12 hours, as one request for the newest entries.
- At least 1.5 seconds between two requests to the same host, a 30-second timeout and conditional requests (ETag / Last-Modified) when re-checking files.
- Each PDF is downloaded once and then only re-checked occasionally to detect a silent replacement.
What we do with it
- We keep a copy of each official file as evidence and always link users to the original URL on your site.
- We never present our copy as the official version.
Contact or opt out
Write to hello@reoncorelabs.com with your domain. We will slow down or stop fetching your site within one working day. We check robots.txt before adding a source; none of the sites we read restricts automated access today.