Forum becoming inaccessible

[sethypeety]

The Forums unfortunately have been becoming harder and harder for me to use, almost to the point where I was ready to send this directly to the feedback@mxsimulator.com Email address because I thought I wasn't going to be able to post it here.

Copy/Paste of the error I continue getting:

General Error
SQL ERROR [ mysqli ]

MySQL server has gone away [2006]

An sql error occurred while fetching this page. Please contact an administrator if this problem persists.

What's going on here? Is this a problem on my end? Is the forum being flooded by bots? I have a track I'm almost ready to upload but I have to refresh 5-10 times every time I enter a new page on the Forum!

[MarshDylan_]

Same here, I'm sure everyone else is dealing with the same as well.

[317]

Same problem here for the past couple days.. I noticed a couple days ago the "users online" was at 4,000 something.
I also saw another forum post where JLV was talking about the 'spambots'?

There usually isn't 4,000 people online at once... so the forum must be receiving some sort of spam.

Definitely has been an annoyance though with hopping on here. I am sure JLV with fix it as soon as he can put his attention to it.

[jlv]

Damn I hope they didn't brute force 4000 passwords. I've been IP blocking them. I'll start changing the forum password more often. That seems to trip them up since playing the game online to get the password is apparently an obstacle for them.

[jlv]

It was really bad this afternoon so I had a look at the IP addresses hammering the site. It was Amazon, Apple and Microsoft. After I blocked the IP addresses, I tried adding a robots.txt file that asks them to only do 1 request every 10 seconds. Somehow I doubt it'll be honored.

[jlv]

Thought I'd check if anything actually downloads robots.txt. It looks like they do. Here are the top 10 ranked by how many times they've downloaded robots.txt. Definitely some AI scraping going on. Hopefully if they're bothering to download robots.txt they'll honor it.

Code: Select all

     91  "Mozilla/5.0 (compatible; Barkrowler/0.9; +https://babbar.tech/crawler)"
     94  "Mozilla/5.0 (compatible; MojeekBot/0.11; +https://www.mojeek.com/bot.html)"
    117  "AliyunSecBot/Aliyun (AliyunSecBot@service.alibaba.com)"
    169  "Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)"
    225  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
    288  "Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)"
    302  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot"
    422  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"
    477  "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
    786  "Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)"
    

[jlv]

I blocked a lot of IPs tonight. If any got any real people by accident send me an email.

It's a little crazy. There are 900 different IP addresses scraping the site in the last 2 minutes even after the blocking.

[317]

I blocked a lot of IPs tonight. If any got any real people by accident send me an email.

It's a little crazy. There are 900 different IP addresses scraping the site in the last 2 minutes even after the blocking.

I just looked at the bottom of board index, and it says "Most users ever online was 5,787 on Sat Jul 19, 2025 1:55 am"

So a little over 5 hours ago there were 5,787 connections to the forum. Crazy. And it is currently 7:03am EST for me right now.

[TeamHavocRacing]

Way better now. Thanks.

[jlv]

It was really crazy last night. I was playing with the maxclients setting to make the server run better and the better the server ran the more bots would show up. Half of them are doing searches for some reason so I disabled searching for guest accounts. Almost like they're trying to do the worst thing possible since searching uses a lot of cpu time.

[ShortyMX]

Hey jlv, have you heard of anubis? It might help fend off the ai scrapers better than blocking ips.
I've read about it on HN a while ago, so I'm not sure if it's still working effectively.

If I recall correctly, cloudflare also offers solutions to this problem, if you wanna go with them.

[DBRider251]

I've been doing a bit of scraping for my stats, but it was on the server results page. I usually run it in small batches though

[jlv]

Hey jlv, have you heard of anubis? It might help fend off the ai scrapers better than blocking ips.
I've read about it on HN a while ago, so I'm not sure if it's still working effectively.

If I recall correctly, cloudflare also offers solutions to this problem, if you wanna go with them.

I didn't but the article was interesting! Not sure if I like the idea of requiring Javascript and having a proof of work challenge if I can avoid it though. The collective scraper IP blacklist sounds useful though!

Disabling search for guests seems like it did enough for now. Hopefully they don't get worse.

[jlv]

I've been doing a bit of scraping for my stats, but it was on the server results page. I usually run it in small batches though

It's tough because there are legit scrapers. I keep blocking RF's server by accident which is the last thing I want to do.

[DBRider251]

I've been doing a bit of scraping for my stats, but it was on the server results page. I usually run it in small batches though

It's tough because there are legit scrapers. I keep blocking RF's server by accident which is the last thing I want to do.

I wonder if it's some kids attacking or if it's legit AI bot scraping