We run internet bot to gather some of our data. Our bot is following the internet standards.
What is SMTBot?
SMTBot is a focused web crawler, it does not try to index your entire website but instead will crawl small amount of pages (not more than 200) of your site.
Does SMTBot respects robots.txt?
Yes, it follows robots.txt rules for "SMTBot" user-agent.
How do we use the data?
We use the data to identify technologies on websites more accurately.
Need more information? Contact us