FeedFetcher-Mojeek

FeedFetcher-Mojeek is an RSS/Atom feed crawler for Mojeek news search engine. Although every attempt has been made to be considerate of webmaster, site owner, and hosts, unfortunately mistakes and errors are inevitable. If you have noticed our bot misbehaving in any way, crawling a page or directory it shouldn't, or you just have general enquiries please contact us, we would appreciate your feedback.

Crawl Frequency

FeedFetcher-Mojeek should not request, whether successful or not, more than one feed from your site within the same 2 second time period.

Verify FeedFetcher-Mojeek

To verify it's a real FeedFetcher-Mojeek visiting your site perform two steps, first, a reverse dns lookup on the visiting ip address:-

> host 5.102.173.71
71.173.102.5.in-addr.arpa domain name pointer crawl-5-102-173-71.mojeek.com.

This should resolve to a name within the mojeek.com domain. Now check this is not a false reverse dns by performing a forward dns lookup on the above response:-

> host crawl-5-102-173-71.mojeek.com
crawl-5-102-173-71.mojeek.com has address 5.102.173.71

This should now return the original visiting ip address, if not, it's not a genuine FeedFetcher-Mojeek user agent.

Contact Us

If you have any further questions or comments regarding our bot, please do not hesitate to contact us.