Anyone who runs a server with open ports knows that systems with questionable intent will come knocking. Sometimes thousands of them. fail2ban is software that that checks your server logs and detects multiple failures, for example 5 failed SSH logins in a row, and bans the source IP address a period of time, e.g. for an hour. This helps prevent password-guessing and brute force attacks. It might be useful to share information about those questionable IP addresses with others so that we can block them proactively.
One such list of IP addresses that I found is blocklist.de. Since I am primarily concerned with systems that are trying to SSH into my system, I looked specifically at their SSH blocklist:
All IP addresses which have been reported within the last 48 hours as having run attacks on the service SSH.
I found this forum post on the blocklist site that describes how to add the blocklist to
Import Blocklist to fail2ban (PHP Cronjob)
My ability to read German is minimal, but I do speak code. The code provides 3 things:
fail2banjail, jailing offending IP addresses for an hour (3600 seconds)
- A PHP script to retrieve the blocklist of IP addresses and ban each one via the jail
- A cron task to run the script hourly
In general, the PHP script works, but there’s no need for it to be PHP and it could be improved. I wrote a Bash shell script instead that additionally:
- Verifies the MD5 checksum of the blocklist file
- Processes only lines in the blocklist file that match a basic IP address regex
I also left out overwriting an
empty.log file with a hyphen (“-“):
exec("echo "-" > /var/log/fail2ban.blocklist.log"); //force reload ip-locks
As far as I can tell that does nothing. I’ll come back to the log file later, though.
The SSH blocklist contained 3614 IP addresses when I checked, which means that on the hour, the script will retrieve the list and run the
fail2ban-client 3614 times. How long does that process take? I timed the original PHP version, which took over 8 minutes:
$ sudo sh -c "time /usr/bin/php /etc/fail2ban/blocklist.php" real 8m19.708s user 5m0.565s sys 1m31.705s
fail2ban-client seems like a strange way to go about this.
fail2ban is designed to process log files. I had the idea use the IP blocklist to generate a log file, and then have
fail2ban read from that log:
while read IP_ADDRESS do echo $(date +'%b %d %T') $HOSTNAME sshd: $IP_ADDRESS >> /var/log/fail2ban.blocklist.log done < ssh.txt
I specified the new log file in the
logpath = /var/log/fail2ban.blocklist.log
$ sudo systemctl restart fail2ban
fail2ban did not block IPs added to that log. My custom log format did not match any of the
failregex lines in
I created a custom filter (relying heavily on
fail2ban‘s Developing Filters documentation) named
ssh-blocklist.conf that contains a single failregex that matches what I’m writing to the custom log:
[INCLUDES] before = common.conf [Definition] _daemon = sshd failregex = ^%(__prefix_line)s<HOST>$
fail2ban started processing new items added to the
/var/log/fail2ban.blocklist.log. I confirmed that the addresses were added by reviewing the
iptables REJECT entries:
$ sudo iptables -L -w -n | grep REJECT
As it turns out, if you add 3614 log entries to the blocklist log,
fail2ban doesn’t process all those instantly either. I haven’t timed it, but anecdotally it appears to take about 30 minutes this way, even longer than using
This brings up a good question: is the effort worth it? That is, what is gained by spending human time (setup/config) and processor time (hourly cron) to block these 3614 IPs? How many of these IPs will actually attempt an SSH connection with the newly-configured server? It would be interesting to log connection attempts from the blocklist IP addresses. This would require updating associated fail2ban action to add an
iptables LOG rule, rather than a
DROP rule — a project for another day.
fail2ban is already running with a sshd jail, offending IPs would be blocked after n failed attempts anyway. Even if a substantial portion of the blocklist IPs attempt connections, they would be blocked quickly. The extra effort seems rather pointless for a single SSH server, but if the same blocklist were applied at a network firewall in front of hundreds or even thousands of hosts, then makes more sense. Adding 3614 block rules to prevent a handful of connection attempts is one thing, adding 3614 block rules to prevent thousands of connection attempts is another.
The files I created are available at fail2ban-blocklist on GitHub.