sukria.net

Link Spammers are getting more and more agressive these days and one shouldn’t ignore them. As I said before, I set up a small but effective system to blacklist IPs from accessing my pages and that works pretty well.

After more than ten days of active blacklisting, I have an interesting file to share, the Apache RewriteMap I use for listing unwanted IP adresses.
That file gets updated every day, with the new IP adresses that attacked my website.

This makes me wondering if we could set up a volatile package which would be dedicated to set up a webserver shield. It could just provide an httpd.conf configuration example (showing how to use the RewriteMap) and some maps (IP addresses, fake referers and user agents).

I might work on a prototype package and post again on that topic, stay tuned …

If you run your own webserver, you are certainly a blog spammer’s target. Yes, you are.

They use compromised boxes or open proxies to launch their bots on your website, posting comments or sending trackbacks on your blog, or simulating referers hits with their domain names. All this to increase their visibility.

This new way of polluting the World Wide Web must become as obstructing as mail spams. Here is how I proceed to block those kind of attacks, using basic and well known tools: mod_rewrite for denying access, bash for writing a simple IP addresses grabber script and Cron for scheduling.

The strategy here is to block requests that match one or more of those conditions:

1. The user agent is known to be a spambot.
2. The IP address is blacklisted.
3. The referer is known to be a fake one.

1. Grabbing IP addresses

Let’s start by greping your accesslog for finding the IP addresses related to the attacks.
That tiny shell script will help you to do this job. It takes as its only argument a pattern used for performing the grep in the accesslog. Your only job is to use a good pattern.

Once you have lauched this script, the file /usr/share/apache/maps/ip-baned.txt (or whatever you chose) will contain all the IP addresses you don’t want to serve.

2. Apache configuration

We can now update the Apache configuration in order to setup mod_rewrite:

First filter the User Agents:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (AGENT_1) [OR]
…
RewriteCond %{HTTP_USER_AGENT} (AGENT__N)
RewriteRule ^(.*) - [F]

Then use the blacklisted IP addresses:

RewriteMap ipbaned txt:/usr/share/apache/maps/ip-baned.txt
RewriteCond ${ipbaned:%{REMOTE_ADDR}|NOTFOUND} !=NOTFOUND
RewriteRule ^(.*) -                                     [F]

And filter fake referers:

RewriteCond %{HTTP_REFERER} (DOMAIN_1) [NC,OR]
…
RewriteCond %{HTTP_REFERER} (DOMAIN_N) [NC]
RewriteRule ^(.*) - [F]

Restart Apache and enjoy all the 403 errors you’ll send to the spammers.

3. Using Cron for updating the blacklist

The last thing to do is to setup a cron script to periodically update your IP blacklist using the little script I provide.

You’ll then receive a mail from Crond whenever a change appears in the blacklist file, seeing which IP addresses are added.

Using this simple solution works great for me, my log analyzer shows more than 1400 hits refused with a 403 error in less than 3 days of use…

You might know that I made an unofficial package of Apache Lingerd some months ago, and by now, a couple of users pinged me back to let me know that they use it on their production servers.

The package itself seems to work properly (I personnaly use it for my website) and as the package was made with following Fabionne’s advices, I guess that everything would be ok for asking for an upload.

Well, not really, there was an important point to clarify before going on: security issues.
Does Lingerd has a security history that could block its inclusion in Debian? That’s the question to ask, and that’s why I mailed the Debian Security Team.

Martin Schulze’s answer sounds like uploading apache-lingerd to the archive is something possible. Great to read such a mail.

To be continued…

Do you know the Apache module named mod_spambot?
It’s a simple but powerful blacklist engine that prevents unwanted site downloads.

Let’s quote the upstream description instead of rewriting it:

Mod_Spambot is an Apache plugin which monitors the data being downloaded from a server. When the number of requests for a client exceeds a preset level no more downloads are allowed for a preset time. When this happens the client received a tailored message informing them of what has happend. Many of the features can be tailored to the needs of the webmaster to help to prevent false positives and to customise the definition of a client to be blacklisted.

Upstream author, Nigel Horne, made a RFP a couple of days ago and I’d like to close that new wnpp meta bug.

So here we are, a first version of a Debian package of the Spambot module is available on my repository, it’s the Apache 1.3 version of the module - I am working on the Apache 2 version - and is working great (congratulations Nigel) !

I’m currently looking for a sponsor for uploading my package to the archive, please let me know if you are interested in sponsoring me.