- Joined
- Sep 5, 2011
- Messages
- 12
119.63.196.49 is a Baidu spider (China search engine) not complying to robots.txt so I want to ban it - actually I have banned it some of the IPs via iptables server side - problem is that now I get loads of these buggers coming from 119.63.196.xx. Can I safely ban that whole range 119.63.196.xx without affecting other people?
The reason of concern is that my server runs websites based in Japan (where this Baidu search is), so I don't want to block normal folk by accident. Biggest issue is now that I have Russian spiders and other Chinese search people crawling my site - its only a matter of time before I get spammed like mad - any ideas?
Also if I want to ban an ip range - how do I write that in SSH or webmin? is it 119.63.196.0/xxx (xxx= the max IP I can find?)
Cheers,
C
The reason of concern is that my server runs websites based in Japan (where this Baidu search is), so I don't want to block normal folk by accident. Biggest issue is now that I have Russian spiders and other Chinese search people crawling my site - its only a matter of time before I get spammed like mad - any ideas?
Also if I want to ban an ip range - how do I write that in SSH or webmin? is it 119.63.196.0/xxx (xxx= the max IP I can find?)
Cheers,
C