DekipiBot

What is DekipiBot

DekipiBot is a web crawler that powers the Dekipi Search index of web pages.

Our bots index accurate, up-to-date information about websites and how they link to each other. This data enables you to find high-quality content and all the information you need on Dekipi Search.

DekipiBot obeys robots.txt

DekipiBot strictly adheres to robots.txt, including both disallow and allow rules

Customizing DekipiBot access

Since DekipiBot strictly follows the robots.txt file on your website, you have full control over what it does.

To change how often DekipiBot visits your site, simply specify the minimum delay you are willing to accept between consecutive requests in your robots.txt file:

User-agent: DekipiBot
Crawl-delay: [value]

Crawl-Delay value is time in seconds.

If you want to stop DekipiBot from visiting your site, add these two lines to the robots.txt file on your server:

User-agent: DekipiBot
Disallow: /

Please note that it may take DekipiBot some time to pick up changes to your robots.txt file.

Also, if your robots.txt file contains errors, DekipiBot may only recognise and follow the commands it can parse.

If you think that DekipiBot is misbehaving on your website or if you have any questions, please reach out to [email protected] and we'll do our best to help.