How to block evil majestic bot

[jrmarino]

I was looking in my logs, here is a sample:

151.48.23.53 - - [22/Dec/2007:10:00:05 -0600] "GET /index.php?id=35&player=1598&&league=4701&day=1025013 HTTP/1.1" 200 50647 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
151.48.23.53 - - [22/Dec/2007:10:00:06 -0600] "GET /index.php?id=19&day=1075011&&league=4701 HTTP/1.1" 200 47855 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
88.156.197.5 - - [22/Dec/2007:10:00:04 -0600] "GET /index.php?id=3&mavid=12&&league=1601&day=1065013 HTTP/1.1" 200 105094 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
88.156.197.5 - - [22/Dec/2007:10:00:04 -0600] "GET /index.php?id=66&game=5&&league=1601&day=1065002 HTTP/1.1" 200 74373 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"

The logs are filled with this majestic bot, but the robots.txt instructs to disallow all robots to the entire document root. According to this page:

http://www.majestic12.co.uk/projects/dsearch/mj12bot.php

There is an evil botnot spoofing an older version of a legitimate bot. They suggest filtering out the request based on this text "MJ12bot/v1.0.8"

Exactly how do I do that? I am guessing it is the "Filter Request" tab on the server panel?

Can you outline the settings necessary to block this nuisance?

 

[jrmarino]

I found the apache way to block this:

SetEnvIfNoCase User-Agent "^MJ12bot(.*)1\.0\.8" stay_out 

<Location />
Order Allow,Deny
Deny from env=stay_out
Allow from all
</Location>

It is detailed in this blog:
http://toti.hexia.net/faces/blog/entry.do?id=7a26c046ab99f812615b4110d8656170&entry=79125

I'm still not quite sure what is supposed to go in the new "Request Filtering Rule Set" definition.

 

[mistwang]

Use rewrite rule or request fileter

Rewrite Rule

RewriteCond %{HTTP_USER_AGENT} ^MJ12bot/v1.0.8 [NC,OR]
RewriteRule .*  - [F]

filter rule

SecFilterSelective HEADER_USER_AGENT ^MJ12bot/v1.0.8

 

[jrmarino]

Thanks a lot!

 

[killed]

My apology to this thread (long time): I implementing this to block Firefox/3
But not effect

113.162.128.27 - - [16/Feb/2012:15:01:41 +0700] "POST /forum/login.php?do=login HTTP/1.1" 200 4464 "http://svtoantin.com/forum/forumdisplay.php?f=122" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100722 AskTbFXTV5/3.8.0.12304 Firefox/3.6.8"

Here:
_Rule: SecFilterSelective HEADER_USER_AGENT "Firefox/3"

 

[killed]

Up this thread

 

[webizen]

Does this rule in .htaccess work for you?
%

RewriteCond %{HTTP_USER_AGENT} Firefox/3 [NC,OR]
RewriteRule .* - [F]

 

[killed]

Does this rule in .htaccess work for you?
%

Yes, in .htaccess work fine. But I need "hard" prevent for all host (with 100 host, use .htaccess impossible)

Thank webizen replied.

 

[webizen]

you can also try mod_security rule as follows:

SecFilterSelective HTTP_USER_AGENT "Firefox/3"

 

[killed]

I tried both:
SecFilterSelective HTTP_USER_AGENT "MSIE\ 6"
SecFilterSelective HEADER_USER_AGENT "MSIE\ 6"

But nothing to effect

113.162.74.188 - - [02/Mar/2012:12:52:58 +0700] "GET /auto.html HTTP/1.1" 200 935 "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
113.162.74.188 - - [02/Mar/2012:12:52:58 +0700] "GET /hinh/ads.jpg HTTP/1.1" 200 15668 "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"

 

[masood_y]

How can block "majestic bot" on litespeed server?

 

[NiteWave]

in .htaccess,

RewriteCond %{HTTP_USER_AGENT} majestic[NC,OR]
RewriteRule .* - [F]

can block it.