China bots - oh how they love me...

Jon123456

Enthusiast
Joined
Nov 2, 2009
Messages
132
In the last couple of weeks, I do seem to have a rather large number of guests. Like on a Sunday, I might have typically about 200 guests, but this time it is 2,000! I am getting a lot of vBulletin error messages in my email, saying session table is full. But the IP addresses are all from China Telecom. And the urls that they are on tend to be things like calendar, top posters, tags etc., rather than normal threads.

How do you deal with these? What is the best solution?

Someone pointed me to this: https://www.google.com/recaptcha/intro/v3.html

Anyone used that?
 

Cyburbia

Fan
Joined
Jan 14, 2004
Messages
741
Just tweak your .htaccess


My experience: https://www.cyburbia.org/forums/threads/aggressive-bots-from-chinanet-and-china-unicom-ips.56183/
 

Jon123456

Enthusiast
Joined
Nov 2, 2009
Messages
132
Thanks for the link. I got onto my host and they have implemented modsecurity. They said it might have an impact on site functionality, but they are not sure, so I am monitoring the situation.
 

zappaDPJ

Administrator
Joined
Aug 26, 2010
Messages
7,507
Thanks for the link. I got onto my host and they have implemented modsecurity. They said it might have an impact on site functionality, but they are not sure, so I am monitoring the situation.

If your host has configured it properly you shouldn't encounter too many issues. If they haven't it can cause all manner of problems including '403' or 'access denied' errors, '404' errors, login issues and so on.
 

Jon123456

Enthusiast
Joined
Nov 2, 2009
Messages
132
Oh wow, so some significant stuff then. How do I find out if it is causing problems?
 

zappaDPJ

Administrator
Joined
Aug 26, 2010
Messages
7,507
It's really just something to keep very much in mind when someone reports an error because once installed it's so easy to forget about it.

If someone reports an error it's best to first establish if they are using an ad-blocker because they can cause similar errors. If there's no ad-blocker involved and nothing much has changed in terms of your installation then it's fairly safe to assume it's an issue with ModSecurity. To be sure you can turn it off in your .htaccess.txt to check that fixes the issue.
 

Jon123456

Enthusiast
Joined
Nov 2, 2009
Messages
132
zappaDPJ, thanks for that. As a side note and perhaps a bit off topic from the thread, was TAZ using vBulletin 3.8 before and if so, did you notice any drop off in traffic after switching over? Did user engagement change at all? Just doing my due diligence!
 

Kyrie

Habitué
Joined
Sep 2, 2009
Messages
1,014
zappaDPJ, thanks for that. As a side note and perhaps a bit off topic from the thread, was TAZ using vBulletin 3.8 before and if so, did you notice any drop off in traffic after switching over? Did user engagement change at all? Just doing my due diligence!


The important thing to remember is your niche. Is your community programmers/gamers from the year 99? They may not like XF...
 

Jon123456

Enthusiast
Joined
Nov 2, 2009
Messages
132
Does blocking China with ipset have a negative impact on performance?
 

mysiteguy

Migration Expert
Joined
Feb 20, 2007
Messages
3,172
Does blocking China with ipset have a negative impact on performance?

No. Blocking them in .htaccess will. Apache must scan every single deny rule for every page, image, javascript, css and other items fetched.

IPset works at the firewall level before it even get's to Apache. And it uses a binary tree to match up rules, so it's extremely quick - you can have a hundred thousand rules in it and it will add virtually nothing to overhead.
 

TLChris

Administrator
Joined
Jan 2, 2020
Messages
277
Couldn't you use a service like CloudFlare to block countries as well?
 

mysiteguy

Migration Expert
Joined
Feb 20, 2007
Messages
3,172
Couldn't you use a service like CloudFlare to block countries as well?

Cloudflare hurts TTFB of page content when it comes to dynamic content so I don't use them. For CDN purposes (js, images, css) I find MaxCDN to be a better solution, and I use them for those types of files, since they are static and will be cached as a result, decreasing their TTFB.
 

Kyrie

Habitué
Joined
Sep 2, 2009
Messages
1,014
Cloudflare hurts TTFB of page content when it comes to dynamic content so I don't use them. For CDN purposes (js, images, css) I find MaxCDN to be a better solution, and I use them for those types of files, since they are static and will be cached as a result, decreasing their TTFB.

After reading a review this part startled me via Digitial.com

Fewer security options than competitors like CloudFlare, which focus more on security than speed.

Do you consider this to be the case?
 

mysiteguy

Migration Expert
Joined
Feb 20, 2007
Messages
3,172
Digital.com doesn't show any testing on their "review". They are merely repeating/quoting cloudflare's claims. A CDN won't speed up dynamic content - it can't. It can only speed up static content.

Digital.com's "review" is merely a front to earn affiliate commissions through Cloudflare's partner program, and they are going through extraordinary steps to hide this by opening in a new window, through a go-between URL and javascript trying to obfuscate the referring URL on the cloudflare destination page.
 

Kyrie

Habitué
Joined
Sep 2, 2009
Messages
1,014
Digital.com doesn't show any testing on their "review". They are merely repeating/quoting cloudflare's claims. A CDN won't speed up dynamic content - it can't. It can only speed up static content.

Digital.com's "review" is merely a front to earn affiliate commissions through Cloudflare's partner program, and they are going through extraordinary steps to hide this by opening in a new window, through a go-between URL and javascript trying to obfuscate the referring URL on the cloudflare destination page.


Couldn't it, hypothetically, slow-down the content though? Also, I didn't realize what I was reading so thanks for that... Doesn't that seem illegal to you?
 
Top