Blocking a DNS DDOS using the fail2ban package

Are you tired of getting multi-thousand line emails from the logcheck package that contain multiple reports of denied queries from named? If so this article will show how you can reject these DDOS attempts via the fail2ban package.

These events look something like this:

System Events
=-=-=-=-=-=-=
Jan 21 06:02:13 www named[32410]: client 66.230.128.15#15333: query (cache)
+'./NS/IN' denied

You can get the whole story about this DOS attack at http://isc.sans.org/diary.html?storyid=5713

However, in summary, the source ip address is falsified by a large bot army. Each soldier of the bot army sends one DNS packet per second to your DNS server. Your dns server replies with a fail message to the falsified source address, causing a DOS attack on that source address.

Tired of your DNS server being used as someone's DOS amplifier weapon? Try Debian's fail2ban package. The homepage for fail2ban is www.fail2ban.org

First install the Debian fail2ban package. By default it only watches and bans ssh. That is probably a good idea, further discussion of which is somewhat beyond the scope of this article.

apt-get install fail2ban

Then inspect the contents of /etc/fail2ban/jail.conf
As per the notes at the end of that file, you'll need to modify your bind logging so fail2ban can understand it.

First make the directory for the bind log file.

mkdir /var/log/named
chmod a+w /var/log/named

I'm sure a reader will complain about making a log file a+w, but it is the simplest way to make this demo work. In your spare time, once everything works, find a better way.

Next, edit /etc/bind/named.conf.local and add the following lines

logging {
    channel security_file {
        file "/var/log/named/security.log" versions 3 size 30m;
        severity dynamic;
        print-time yes;
    };
    category security {
        security_file;
    };
};

Restart Bind using /etc/init.d/bind9 restart
Test bind to make sure it's still working and also verify the log file /var/log/named/security.log is filling up with lines like this:

21-Jan-2009 07:19:54.835 client 66.230.160.1#28310: query (cache) './NS/IN' denied

OK, now to set up fail2ban. Edit the /etc/fail2ban/jail.conf file and change from:

[named-refused-udp]

enabled  = false

to:

[named-refused-udp]

enabled  = true

and from:

[named-refused-tcp]

enabled  = false

to:

[named-refused-tcp]

enabled  = true

Then restart fail2ban in the usual manner,

/etc/init.d/fail2ban restart

Now verify that fail2ban is doing something by checking out the log file located at /var/log/fail2ban.log it should contain something like

2009-01-21 07:34:32,800 fail2ban.actions: WARNING [named-refused-udp] Ban 76.9.16.171
2009-01-21 07:34:32,902 fail2ban.actions: WARNING [named-refused-tcp] Ban 76.9.16.171

Verify that fail2ban is modifying the iptables rules

iptables -L

Now verify that fail2ban's iptables rules are actually stopping access

tail -f /var/log/named/security.log

DNS error messages should be several minutes apart rather than multiple per second.

Now for some fine tuning.

First we have to modify logcheck to look at the new location of named error messages. Edit /etc/logcheck/logcheck.logfiles and add this to the end of the file:

/var/log/named/security.log

Next modify logcheck to report what fail2ban is doing. edit the same file, /etc/logcheck/logcheck.logfiles and add this line to the end of the file:

/var/log/fail2ban.log

Now verify you are getting both named and fail2ban messages in your hourly logcheck emails.

It would be a good idea to research, change, and test the [DEFAULT] ignoreip stanza in the /etc/fail2ban/jail.conf file. Maybe the package default should change to ignore all RFC1918 addresses. Probably you should ignore your LAN, just in case. At least ignore the source ip address of the machine that you usually use to SSH into your DNS server.

It would also be a good idea to think about the bantime = 600 setting in the /etc/fail2ban/jail.conf file. Maybe more than ten minutes would be appropriate. Since the DDOS attacks last "days", perhaps "hours" would be most appropriate.

As per above, think of a better way to assign permissions to the logfile directory /var/log/named. Maybe modify /etc/bind/named.conf.local and /etc/logcheck/logcheck.logfiles and /etc/fail2ban/jail.conf to not use that directory... maybe just use a file named /var/log/named.log

Last, but certainly not least, please compliment the maintainer of the debian package Yaroslav Halchenko and also the author of fail2ban, Cyril Jaquier

Thanks, and have a pleasant day.

Vince Mulhollon vince@mulhollon.com aka vlm@debian.org