Dave over at Dave’s Chalkboard has a whole category on referrer spamming and how to put an end to it using .htaccess. Dave uses ExpressionEngine, much as we do here, and EE has a built-in referrer spam blacklist so I was curious why he’d go through the trouble of using .htaccess instead. Turns out there’s a good reason: Bandwidth.
EE’s blacklist works pretty well at blocking sites from showing up in the referrer list that EE maintains itself, but it doesn’t stop it from showing up in any other tracking services you might use nor does it stop the bandwidth from being used when you’re hit constantly by these assholes. Dave estimates that the referrer spam attempts were eating up about around 3.6GB to 7GB of bandwidth a month, which isn’t a big deal when you have 50GB of bandwidth to play with, but for a site like SEB which has a mere 19GB a month that bandwidth lost to referrer spamming can quickly add up. So I’m thinking of following Dave’s lead and see if I can’t figure out how to massage my .htaccess file a bit and cut back on some of the wasted bandwidth. It’ll probably be a little trickier for me considering that I have two domains to protect, but I’ll let you know how it goes.
Update: After studying Dave’s .htaccess file and reading up on regular expressions I’ve gone ahead and implemented my own attempt at referrer spam blocking. Dave had a pretty good sized file full of URLs, but I wanted to make it as simple as possible. So rather than using full URLs I’ve put together one that makes use of pattern matching to filter out most of the crap that comes along. There’s a few sites that managed to keep their domain names free of the popular keywords so I had to add in lines just for them, but overall I think this is going to catch a lot of it with a minimal amount of work.
Testing things out with the helpful wannaBrowser appears to confirm that I’m in good shape. Still, there’s a chance my patterns are overly broad so if you find you’re getting a Forbidden error when trying to follow a link from someplace to SEB, please be sure to let me know about it.