Forum becoming inaccessible

[jlv]

I wonder if it's some kids attacking or if it's legit AI bot scraping

Some of the IP addresses are residential so it does kind of look like a botnet. Not sure what the point would be though. They keep hitting the search and the phpbb faq for some reason. I just changed the search to return an empty page and they suddenly stopped.

[DBRider251]

Interesting.

I'm not sure if this is related, but we've seen an uptick in races not getting printed to the server page here recently from the na.mxbattlegrounds.com server (probably more, that's just the race server we're using for RF right now). I know both 450 Am motos got missed on Sunday and 450 Moto 2 from EU yesterday was missed. I was thinking of trying to curl the results to a backup server too in case the results page doesn't get generated. Especially since it's gone forever after a restart

[jlv]

The server page is on the same server as the forum so that'd be expected.

Definitely a good idea to keep a backup of the server results.

[Jakob Hubbard]

The server page is on the same server as the forum so that'd be expected.

Definitely a good idea to keep a backup of the server results.

I assume you implemented some sort of request throttling for the servers for individual addresses. My guess is since the na.mxbattlegrounds.com servers 19801-19810 are also being used as qualifying servers, every time those servers restart it also sends request from the na.mxbattlegrounds.com ip. So it probably fills up the request limit from the qualifying servers, then when the race server wants to send a result, it gets blocked from the rate limit.

That's my theory anyways, could it be possible to remove the restrictions from that ip address? I'll probably have backups made regardless. Do lobby servers have this issue currently? I know there's hundreds of those servers so I wonder if those server owners are having problems or if they're already whitelisted.

[Jakob Hubbard]

That's my theory anyways, could it be possible to remove the restrictions from that ip address? I'll probably have backups made regardless. Do lobby servers have this issue currently? I know there's hundreds of those servers so I wonder if those server owners are having problems or if they're already whitelisted.

Actually after thinking about it I'm not sure why I immediately assumed you had a throttler. It could just be too many active connections to the server to where it throws a general error. I should probably have the post request write output to a file and check if it failed, and if it did then make a backup. Right now it just sends the output to /dev/null lol

[jlv]

Maybe just move the results to a directory that acts like a queue and then have a separate script retry uploading until it succeeds. Something like this:

Code: Select all

#!/bin/bash
while true
do
  for f in *
  do
    if test "$f" != '*' && uploadresults "$f"
    then
      mv "$f" ../backups
    fi
    break; # only upload one file
  done
  sleep 60
done

Create a directories called "queue" and "backups" and then for the results command use "mv results.txt queue/results-$(date '+%s').txt" and run the script in the queue directory.

I'm not doing any IP throttling. I just put some restrictions on the phpbb search.

[DBRider251]

That's basically the script I was testing, great minds think alike JLV