Tuesday, December 09, 2008

Block Scrapers Who Translate

Scrapers use translation services in order to access and scrape your site. Check your logs to see where that translation traffic is coming, you'll probably see a lot of countries normally associated with hacking, trojans, and viruses. If I wanted my site viewed in another language I'd translate it myself.

One reason not to block translation services is that it might cause a dropoff of links from sites where members converse in other languages. However, if you're receiving frequent translations from rogue countries, you might have more pressing reasons to ban translation services.

Banning Google's translation service is easy. You just have to drop a meta tag into your head section:

<meta name="google" value="notranslate">

Banning BabelFish requires adding the following code to your htaccess:

SetEnvIfNoCase Referer "^http://babelfish.yahoo.com/" BadReferrer
order deny,allow
deny from env=BadReferrer

Good luck with it!