Sysadmin Log: Difference between revisions
(Created page with "=1/25/26= The access log for the nginx_proxy is not available, so this is the check recommended for bot activity for the prior 15 minutes: docker logs --since 15m nginx_proxy \ | grep 'nginx.1' \ | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' \ | sort | uniq -c | sort -nr | head -20 \ | while read count ip; do org=$(whois $ip 2>/dev/null | awk -F: '/OrgName|Organization|netname/ {print $2; exit}' | xargs) echo -e "$count\t$ip\t$org" done It can take up to 30 seconds to com...") |
|||
| Line 1: | Line 1: | ||
=1/25/26= | =1/25/26= | ||
AI bot throttling - see also [[Wiki Bot Throttling]] | |||
The access log for the nginx_proxy is not available, so this is the check recommended for bot activity for the prior 15 minutes: | The access log for the nginx_proxy is not available, so this is the check recommended for bot activity for the prior 15 minutes: | ||
Latest revision as of 03:16, 26 January 2026
1/25/26
AI bot throttling - see also Wiki Bot Throttling
The access log for the nginx_proxy is not available, so this is the check recommended for bot activity for the prior 15 minutes:
docker logs --since 15m nginx_proxy \ | grep 'nginx.1' \ | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' \ | sort | uniq -c | sort -nr | head -20 \ | while read count ip; do org=$(whois $ip 2>/dev/null | awk -F: '/OrgName|Organization|netname/ {print $2; exit}' | xargs) echo -e "$count\t$ip\t$org" done
It can take up to 30 seconds to complete.
I wrote a wrapper script to make it easier to run as the "dadm" user:
$ showbots 1701 172.18.0.1 Internet Assigned Numbers Authority (IANA) 1112 120.0.0.0 UNICOM-HE 1039 35.227.147.165 Google LLC (GOOGL-2) 902 35.208.19.233 Google LLC (GOOGL-2) 672 216.73.216.189 Amazon.com, Inc. (AMAZO-4) 223 119.0.0.0 CHINANET-GZ 215 544.0.0.42 136 187.156.194.102 123 544.0.0.32 88 216.244.66.203 Wowrack.com (WOWTEC-1) 84 141.179.46.10 RIPE Network Coordination Centre (RIPE) 83 144.0.0.0 Asia Pacific Network Information Centre (APNIC) 80 172.18.0.4 Internet Assigned Numbers Authority (IANA) 77 133.0.0.0 76 66.249.72.195 Google LLC (GOGL) 69 139.0.0.0 Asia Pacific Network Information Centre (APNIC) 62 136.0.0.0 Ace Data Centers II, L.L.C. (ADCIL) 58 31.145.16.12 YONCU 58 185.50.71.199 OS 55 135.0.0.0 CIK Telecom INC (CIKTE)
Turns out that "IANA" is resulting from reverse DNS lookups from the other bot activity.
I just checked it again and the server is getting hammered by CMNET (China Mobile Network):
13637 112.0.0.0 CMNET 2055 172.18.0.1 Internet Assigned Numbers Authority (IANA) 935 216.73.216.189 Amazon.com, Inc. (AMAZO-4) 160 172.18.0.4 Internet Assigned Numbers Authority (IANA) 133 143.0.0.0 Latin American and Caribbean IP address Regional Registry (LACNIC) 98 66.249.72.195 Google LLC (GOGL) 98 134.0.0.0 RIPE Network Coordination Centre (RIPE) 96 189.94.6.146 91 144.0.0.0 Asia Pacific Network Information Centre (APNIC) 90 544.0.0.42 88 413.0.0.41 86 47.189.220.137 Frontier Communications Corporation (FCC-211) 86 174.234.212.39 Verizon Business (MCICS) 81 88.182.195.57 FR-PROXAD-ADSL 73 105.68.191.0 INWI-ADSL002 63 138.0.0.0 Latin American and Caribbean IP address Regional Registry (LACNIC) 61 31.145.16.12 YONCU 61 140.0.0.0 Asia Pacific Network Information Centre (APNIC) 61 139.0.0.0 Asia Pacific Network Information Centre (APNIC) 60 185.50.71.199 OS
Not sure what to do about CMNET, as it's a BIG network, like DOD and RIPE.
Anyway, the list of IPs to block can be updated by becoming root and editing this file:
/etc/nginx/conf.d/blocked_ips.conf
Then become "dadm" and run:
docker exec nginx_proxy nginx -s reload