« previous next »
Pages: [1]

Author Topic: Bots on the server  (Read 4151 times)

0 Members and 1 Guest are viewing this topic.

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 3059
  • Last Login:Today at 06:49:20 am
Bots on the server
« on: September 29, 2025, 06:53:37 pm »
Bots that look to be in China are hitting the server hard seemingly often the forum. It much more than I recall seeing before and enough to affect performance of the server.  I am considering taking down the forum temporarily and possibly suspending TM overall while I figure out how to limit them...
Logged

Offline vdeane

  • Sr. Member
  • ****
  • Posts: 487
  • Gender: Female
  • Last Login:Yesterday at 10:19:05 pm
    • New York State Roads
Re: Bots on the server
« Reply #1 on: September 29, 2025, 09:12:18 pm »
Could what's used on the AARoads Wiki be used here?
Logged
Please note: All comments here represent my own personal opinion and do not reflect the official position of NYSDOT or its affiliates.

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 3059
  • Last Login:Today at 06:49:20 am
Re: Bots on the server
« Reply #2 on: September 29, 2025, 09:35:21 pm »
I've asked for advice from the Siena network admins, and now the load is back to normal or less.  Hopefully it's something they've done.  I'll leave the forum active for now.
Logged

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 3059
  • Last Login:Today at 06:49:20 am
Re: Bots on the server
« Reply #3 on: September 30, 2025, 12:54:40 pm »
Bots hitting the forum remain very low.  I still plan to block web requests from all of (at least) China on the server itself or have that done at the network level before the requests even get there.
Logged

Offline rschen7754

  • Jr. Member
  • **
  • Posts: 52
  • Gender: Male
  • Last Login:October 05, 2025, 10:53:10 pm
    • Rschen7754
Re: Bots on the server
« Reply #4 on: September 30, 2025, 09:26:18 pm »
We had to both use Anubis (https://anubis.techaro.lol/) and block Alibaba IP addresses at the firewall level in order to get things to a manageable state.
Logged

Offline Jim

  • TM Collaborator
  • Hero Member
  • *****
  • Posts: 3059
  • Last Login:Today at 06:49:20 am
Re: Bots on the server
« Reply #5 on: October 08, 2025, 03:42:52 pm »
Chinese bots were bogging things down again today, so I took the time to implement the firewall updates.  As far as I can tell, it's working.  All traffic to the server from all China IP blocks is being denied.  It looks like we're still being crawled by at least one bot in Vietnam but it's not hammering things like the China ones were.

Some other web crawlers look to be loading every single showroute link and loading everyone's user page and following every link there and loading every archived log file...
Logged
Pages: [1]
« previous next »