Image

Knowledge base → Protection from flooding (frequent regular crawling of pages) of nginx server

[Virtual servers]
Date of publication: 10.09.2025

Any server, sooner or later, is subjected to flooding, which leads to increased load on services and elevated resource usage. Numerous bots on the internet scan websites for vulnerabilities and data collection.

Such bots exhibit malicious behavior, and to avoid wasting server resources on them, they should be blocked.

In this article, we are discussing one specific method of combating flooding. Other services and methods are used to combat DDoS attacks.

Example of such requests:

100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /aa.php HTTP/1.1" 404 27 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /file.php HTTP/1.1" 301 162 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /file.php HTTP/1.1" 404 27 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /wp-file.php HTTP/1.1" 301 162 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /wp-file.php HTTP/1.1" 404 27 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /k.php HTTP/1.1" 301 162 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /k.php HTTP/1.1" 404 27 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /al.php HTTP/1.1" 301 162 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /al.php HTTP/1.1" 404 27 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:57:59 +0000] "GET /wp- HTTP/1.1" 301 162 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:58:00 +0000] "GET /wp- HTTP/1.1" 404 48273 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:58:00 +0000] "GET /admin.php HTTP/1.1" 301 162 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:58:00 +0000] "GET /admin.php HTTP/1.1" 404 27 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:58:00 +0000] "GET /file2.php HTTP/1.1" 301 162 "-" "-"
100.xx.188.xx - - [10/Sep/2025:00:58:00 +0000] "GET /file2.php HTTP/1.1" 404 27 "-" "-"

As seen from the log, the bot makes about 9 requests per second to the server, which is quite a lot, especially if you consider that there could be several such bots with different IP addresses. This can create a significant load on a virtual server.

Our task is to block IP addresses that make too many regular requests, while excluding static files – we will only count dynamic requests, as these are the ones that create the load.

1. Installing Fail2ban

apt install fail2ban

2. Configuration

nano /etc/fail2ban/jail.local
[DEFAULT]
# Main settings
ignoreip = 127.0.0.1/8 ::1
bantime = 3600
findtime = 600
maxretry = 200

# Email settings
destemail = corp@domain.tld
sender = fail2ban@domain.tld
sendername = Fail2Ban
mta = sendmail

# Default action (with email notification)
action = %(action_mwl)s

[nginx-flood]
enabled = true
port = http,https
filter = nginx-flood
logpath = /var/log/nginx/*access*.log
maxretry = 200
findtime = 60
bantime = 3600
# action is inherited from DEFAULT, so no need to duplicate

2.1 The Filter Itself

nano /etc/fail2ban/filter.d/nginx-flood.conf
[Definition]
# Exclude static files from counting
failregex = ^<HOST> -.*"(GET|POST|HEAD) (?!.*\.(jpg|jpeg|png|gif|ico|css|js|woff|woff2|ttf|eot|svg|pdf|txt|xml)(\?.*)?$).* HTTP/\d\.\d".*
ignoreregex = 
datepattern = ^[^\[]*\[({DATE})\s+({TIME})\s+[^\]]*\]

2.2 Edits

Don't forget to change the email to your own:

destemail = corp@domain.tld
sender = fail2ban@domain.tld

2.3 Trigger Threshold

If necessary, adjust the parameters for your needs to avoid false positives:

maxretry = 200
findtime = 60
bantime = 3600

3. Starting Services

systemctl restart fail2ban
systemctl enable fail2ban

4. Viewing and Management

4.1 Viewing Blocked IPs

For iptables

iptables -L -n

For fail2ban

fail2ban-client status nginx-flood

4.2 Testing

Temporarily add a test IP to the ban list

fail2ban-client set nginx-flood banip 1.2.3.4




No Comments Yet