Category Archives: Security

Guest Post: Be careful who you work for

Posted on by
Reply

Hey everybody, my name is Alexander and I have spent most of my career working on the computer, primarily on the internet. I began working with computers when I was about 11 years old, you’d be amazed at how I progressed over the years, and how quickly as well. The main idea behind this post, is to let all you (Randy’s blog readers) know how dangerous it could be, getting involved with working from home on the computer, when you choose to work with the wrong people. I am going to use an example, that happened not long ago (Just over a year or so).

So my main line of work is Information Technology, this position just so happened to be for managing the Internet Web Servers for a community strong within the Virtual Pet Gaming industry, defined by websites such as Marapets.com and Neopets.com. I spent a couple of years working with the owner of this community, free of charge as the website was not intended to generate revenue. So, my duties were to monitor and take care of the structural integrity of this computer, that hosted the website. This is a pretty easy job, but seemingly complicated for those whom do not have the acquired skills to take care of Internet Web Servers.

One day, the website went down, the entire server at that, it was out of my control. At first I believed that the owner had forgotten to pay his bill, but soon after we realized he had been hacked. Not only was the server hacked, but his personal accounts as well (Gmail, AdSense, Etc). Might I remind all internet users, to never use the same password for everything, it’s not safe! So, what happened is the owner of the website targeted me, as the only person who had access to the system, to be the person who compromised all of his accounts. I gave my input, and showed my innocence, but he needed somebody to blame because there was no evidence of who had done the attacks. He made numerous claims that he had direct proof of my wrongdoing, but in truth, all evidence that was said to have existed, was destroyed when the system(s) were compromised.. Meaning, a name will never be put to the face of the Hacker who destroyed this once thriving community.

So all-in-all, to this day I have been cleaning up the mess, contacting blog owners and press release websites just to prove my innocence, to have my name and personal information removed from their websites, so that my name could be cleared within online communities. Nothing is worst than googling your name, and returning results all over the place with your personal details claiming you made attacks against your employer. The internet is a big place, and word travels fast. It’s very complicated to remove something from the internet, once it’s been posted.

So be careful who you work for, be careful what you do and most of all be careful what you put online, you never know who might find it! Also, once again, never use the same password for anything. Thank you for your time Randy, I appreciate you allowing me to make this post, and I hope that we can work together in the future!

Fighting Spam: Giving Back

Posted on by
Reply

Most of my posts about fighting spammers, email harvesters and content scrapers focus on blocking them from your website. Blocking spammers and scrapers is all fine and well, but it’s a little selfish isnt it? I mean, if you (or your systems/modules/mods/firewall) identifies a spammer or content-scraper’s IP address why keep it to yourself? Especially if you are using shared or open systems like Akismet or BadBehavior which makes use of ProjectHoneypot.org‘s http:BL you might even feel guilty for not “giving back” to the anti-spam community! Ok, well maybe you never thought about it, but after you read how easy it is to give back you should feel guilty if you don’t!

How To Help The Fight Against Spammers, Harvesters and Scrapers:

You can easily help identify and catch spammers and harvesters by contributing resources to Projecthoneypot.org. Projecthoneypot is (i think) the largest free, open collection of honeypots on the internet. Projecthoneypot makes all the data it collects freely available via their http:BL (via BadBehavior, MOD_HTTPBL, http:BL for WordPress, and more).

There are 2 very easy ways you can contribute to the fight, and 1 slightly-less easy way:

First, you need to go over to ProjectHoneypot.org and create a free account. You will need to create an account either if you want to use their spam-blocking services above, or if you want to contribute resources. After you create your free account you will be issued an http:BL API key which is needed for any of the http:BL spam-blocking systems and you will then also be able to contribute spammer-blocking data in one of three ways:

1) “Borrow” someone’s shared honey-pot via a Projecthoneypot QuickLink: This is super easy! After you create your Projecthoneypot account, just click ‘Manage Quicklinks‘. After answering a question or two about your site, you will be given your own Honeypot link that you can paste on your site. You put this link on your website so that only bots/scrapers/spammers can ‘see’ it (it’s simple – and there are full instructions). Your visitors will never see the links, but spammers and scrapers will (there is a hidden honey-pot link in this post.. do you see it?) – they will follow the link to the shared-honeypot, where their IP information will be caught and published on the public block-lists. This is literally as simple as copying and pasting a link onto your site!

2) Install your own honeypot to catch spammers! I avoided installing my own honeypot for a long time because frankly I just assumed it was too complicated. But after finally taking the time (2 or 3 minutes) to read the instructions, I was surprised at how simple it was. To create and install your own honeypot, log-in to ProjectHoneypot.org and click ‘install a honeypot‘. Answer a few questions like what the URL is for your website, if you want to share it with others, and what language you want the honeypot in (probably PHP), and a custom honeypot script will be generated for you to download – full, step-by-step instructions are provided, but, basically you upload ONE SINGLE .PHP file to your website, open that file through your browser, click a link, and that’s it! You then put invisible links to your new honeypot on your website. The whole process, start to finish should not take more than 2 minutes.. 3 if you read slow!

3) Donate an MX record: Donating an MX record will allow project honeypot to generate unique (fake) email address to catch spammers. These fake email addresses are posted in honeypots for spammers to find and the more unique domains available, the better. Donating an MX record does not use your email system or any of your resources because all of the (fake spam) email goes directly to Projecthoneypot.org and unspam.com’s email servers. To donate an MX record you need to be able to edit your DNS Zones/DNS MX records – it’s not as simple as using a honeypot, but full instructions are given, and it’s not difficult. If you are comfortable with editing DNS entries it’s a snap, if you aren’t comfortable monkeying with your DNS settings, you may want to stick with method’s #1 and #2 above. I was uncomfortable donating and configuring an MX record at first, but after doing the first one, i’ve since donated several more.

Now that you know how easy it is to help in the fight against spammers/email harvesters/scrapers you have no excuse not to help – NOW you will feel guilty if you didn’t feel guilty before.

Note about links: You may notice that the links to Projecthoneypot.org above are ‘referral’ links. Projecthoneypot referrals are not paid referrals. Instead Projecthoneypot keeps track of how many referrals each member has for “karma points”.. We don’t really get anything else out of it other than feeling better about ourselves for helping spread the word about how easy it is to fight spam :-)

How To Troubleshoot Mod_Security Rules

Posted on by
4

So you’ve read all my posts about how wonderful and necessary Mod_Security is and decided to configure it on your server. You activate a bunch of new rules from GotRoot or OWASP and now suddenly nobody can access your website, you can’t create new posts, or things are otherwise FUBAR – NOW what do you do?

First – don’t panic. Premade Modsecurity rulesets have hundreds of rules that must work across a variety of websites and CMS’s. They are usually 99% perfect, but there is always that 1% that may cause problems on your particular site or configuration. If you are coming to this post via Google, and you are already panicking, just go into your HTTP.CONF or MODSEC2.CONF file (you will need to figure out where your ModSecurity directives are being stored) and add the following line:
SecFilterEngine Off
Then restart Apache. This should stop Mod_Security from blocking anything, but still let you see what it’s doing in the logs. Now you can relax and start troubleshooting.

Troubleshooting Mod_Security 2.5:
The first step is to identify which rule is blocking your users or breaking your site. To see what is happening, check your Apache Error log or your Mod_Security log. In your logs, you will see something like this:

Pattern match "(?:cd |\;|php |echo |perl |killall |python |rpm |yum |apt-get |emerge |lynx |links |mkdir |elinks |wget |lwp-(?:download|request|mirror|rget) |id|uname |cvs |svn |(?:s|r)(?:cp|sh) |net(?:stat|cat) |rexec |smbclient |t?ftp |ncftp |curl |telnet |g?cc |cp ..." at REQUEST_URI. [file "/etc/modsecurity/10_asl_rules.conf"]
[line "587"] [id "340037"] [rev "3"] [msg "Atomicorp.com - FREE UNSUPPORTED DELAYED FEED - WAF Rules: Generic command injection"] [severity "CRITICAL"]
yoursite.com 81.99.13.12 340037 [07/Jan/2011:22:43:29 --0800]
Pattern match "(?:cd |\;|php |echo |perl |killall |python |rpm |yum |apt-get |emerge |lynx |links |mkdir |elinks |wget |lwp-(?:download|request|mirror|rget) |id|uname |cvs |svn |(?:s|r)(?:cp|sh) |net(?:stat|cat) |rexec |smbclient |t?ftp |ncftp |curl |telnet |g?cc |cp ..." at REQUEST_URI. [file "/etc/modsecurity/10_asl_rules.conf"] [line "587"] [id "340037"] [rev "3"] [msg "Atomicorp.com - FREE UNSUPPORTED DELAYED FEED - WAF Rules: Generic command injection"] [severity "CRITICAL"]
[07/Jan/2011:22:43:29 --0800] THcUgdXS8AADiSIm4AX 81.99.13.12 42556 42.22.93.39 80
--c349556d-B--
GET /modules/forum/hide-comments.png HTTP/1.1
Host: yoursite.com
User-Agent: Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5
Referer: http://yoursite.com/the-referring-page.html
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate

Look through the log for the “id” and file – in the example above, it’s [file/etc/modsecurity/10_asl_rules.conf] and the ID is rule #340037. Now you know what mod_security file to focus on (10_asl_rules.conf) and where in that file to start looking (rule/id 340037).

Open up file 10_asl_rules.conf and search until you find 340037 -

Rule 340037 looks like this:

# Rule 340037: generic attack signature
SecRule REQUEST_URI "(?:cd |\;|php |echo |perl |killall |python |rpm |yum |apt-get |emerge |lynx |links |mkdir |elinks |wget |lwp-(?:download|request|mirror|rget) |id|uname |cvs |svn |(?:s|r)(?:cp|sh) |net(?:stat|cat) |rexec |smbclient |t?ftp |ncftp |curl |telnet |g?cc |cpp |g\+\+ |/bin/(xterm|id|bash|sh|echo|kill|chmod|ch?sh|python|perl|nasm|ping|mail|ssh))" \
"id:340037,rev:3,severity:2,msg:'Atomicorp.com - FREE UNSUPPORTED DELAYED FEED - WAF Rules: Generic command injection'"

Now you need to figure out what it is about this rule that is going wrong, so back to the error log:
The error (above) says “Pattern match” (then all the match-rules) at REQUEST_URI. “REQUEST_URI” is the Mod_Security directive to inspect the URL being requested. (For a list of all the directives, check the MODSECURITY online-manual). To see what the URL was, find the GET command a little further down – in the example above, it’s GET /modules/forum/hide-comments.png

So we now know that rule #340037 located in file 10_asl_rules.conf does not like “modules/forum/hide-comments.png” – So let’s figure out why.

Rule 340037 (above) is made up of just a few lines (like most rules). The first line, starting with the ‘#’ is a comment describing the rule: # Rule 340037: generic attack signature

The next line:
"SecRule REQUEST_URI "(?:cd |\;|php |echo |perl |killall |python |rpm |yum |apt-get |emerge |lynx |links |mkdir |elinks |wget |lwp-(?:download|request|mirror|rget) |id|uname |cvs |svn |(?:s|r)(?:cp|sh) |net(?:stat|cat) |rexec |smbclient |t?ftp |ncftp |curl |telnet |g?cc |cpp |g\+\+ |/bin/(xterm|id|bash|sh|echo|kill|chmod|ch?sh|python|perl|nasm|ping|mail|ssh))"

Basically says if the URL matches this very complex regular expression, then do what’s in the next line, which reads:
"id:340037,rev:3,severity:2,msg:'Atomicorp.com - FREE UNSUPPORTED DELAYED FEED - WAF Rules: Generic command injection'"

This ID’s the rule as ’340037′ with revision#3, sets the severity level for logging, and assigns the message to show in the logs – because elsewhere in the mod_security configuration the default action as been set to “deny” – this page/url will been blocked – but WHY?

To figure out what it is about about the path “modules/forum/hide-comments.png” that the expression above does not like you can try to read-through it and figure it out (good luck!) or use a RegEx (Regular Expression) tool such as www.gskinner.com/RegExr to see what the match is. In our example, it is the “id” in the path ‘hide-comments.png’ that matches (?:download|request|mirror|rget) |id|uname |cvs |svn |(?:s|r) in the rule.

Now we know where and why Mod_Security has blocked this particular page.. Now what?

Now you have to decide what to do about it – here are some choices:
1) Disable the entire rule
2) Disable the rule only for this particular URL (“ByLocation”)
3) Create a slightly modified custom rule that overrides this rule and works with this particular page/URL

Which route you take depends on many things and I can’t tell you which to choose – that part you will have to figure out on your own.

Option 1 – Disable the entire rule:
You could simply delete all the lines associated with rule 340037, or comment them out by putting a ‘#’ on each line, then restart Apache. The problem with this method is that if you ever update your rules, your changes will be overwritten with the new file, and the rule would be active again. A better way to disable the rule is to add the following line in your HTTP.CONF or MODSEC2.CONF file:

SecRuleRemoveById 340037

Restart Apache and now rule 340037 will be disabled

Option 2 – Disable the rule just for this particular URL:
You can tell Mod_security to ignore rule 340037 only for the URL /modules/forum/hide-comments.png by adding the following line to your HTTP.CONF or MODSEC2.CONF file:

<LocationMatch modules/forum/hide-comments.png>

SecRuleRemoveById 340037

</LocationMatch>

Restart Apache and now rule 340037 will be ignored/disabled only for the URL /modules/forum/hide-comments.png

Option 3 – Replace rule 340037 with your own, slightly customized rule:
You can create your own custom rule that does not include the “id” match, then disable the ‘real’ rule 340037.

First, create a new file named something like “99_my_rules.conf” and place in the same directory as your other Mod_Security rules (probably at /etc/modsecurity). Copy and paste rule 340037 from 10_asl_rules.conf and put into your new custom-rules file and make the following changes:

Change “340037″ to 990037 – This will keep you from confusing it with the real rule “340037″

Change the message from : ‘Atomicorp.com – FREE UNSUPPORTED DELAYED FEED – WAF Rules: Generic command injection’
to something like: ‘My custom rule that replaces 340037′

Find the part of the expression that we want to get rid of – in this case it is the “id” at this part:

|request|mirror|rget) |id|uname |cvs |svn

Remove the “id|”, which should leave just:

|request|mirror|rget) |uname |cvs |svn

and save the file.

In your HTTP.CONF or MODSEC2.CONF file, disable the original rule 340037 by following Option #1 above

Add the following line in your HTTP.CONF or MODSEC2.CONF file to tell Mod_security to load your new custom rules:
Include /etc/modsecurity/99_my_rules.conf — place this line right after the other rules listed. Be sure to use the correct path for your system.

Restart Apache and now the original rule 340037 will be disabled and your new rule 990037 will be active. If you ever need to create any other custom rules, just add them into your new 99_my_rules.conf file (and restart Apache).

Some things to keep in mind when changing Mod_Security rules:
Before you change any files, make a copy of it first. It’s very easy to make a syntax error, or screw something up.

Each time you restart Apache, make sure it actually starts and check your site! If Apache will not start, just put back the backup file that you made, and start Apache again – then figure out what went wrong.

If you disabled blocking in Mod_Security with the SecFilterEngine Off directive, be sure to change it back to SecFilterEngine ON

Most important – watch your logs very closely and verify that ModSecurity is only blocking things you really want blocked.

Questions? Did I miss anything? Post a comment!

MOD_SECURITY vs Bad Behavior

Posted on by
1

Bad Behavior and MOD_Security are both great tools to help block spammers, bots, scrapers, proxies, and application-level attacks – but which one should you use?

Bad Behavior:
+ Easy to install and configure, especially on WordPress. The Drupal Bad Behavior module is no longer supported so it will not work “out of the box” with the newest version of Bad Behavior, but you can make it work by following these instructions. (I have the fully-patched and working module that you can drag’n Drop into your modules directory – contact me if you would like a copy)
+ Many built-in rules that block a wide variety of spammers, bots, scrapers, proxies and other bad stuff.
- No control over the rules and no visibility into what is blocked or why; Bad Behavior has an on/off “strict mode” setting – but it’s a mystery as to what it changes. Bad Behavior also allows you to whitelist IP’s, but gives you no other control of what is or is not blocked
- Must be installed and separately administered on every site/domain/sub-domain.
- May be slower due to the additional PHP overhead

MOD_SECURITY:
+ Full, fine-grained control over blocking rules
+ Create your own rules, or get (free) fully-tested rules from GotRoot (and customize to your needs)
+ Fully integrates with CSF Firewall
+ Protects the entire server with the same ruleset
+ Very lightweight/fast – does not use PHP resources
- Must have root-access to install
- Can only run on a dedicated or VPS server
- More difficult to install for non-Linux types

I started with BadBehavior, then used both Mod_Security and BadBehavior together for a number of years with a very light-set of ModSecurity rules, allowing Bad Behavior to block the rest. Recently I grew tired of maintaining Bad Behavior across all my sites and frustrated with the lack of control in Bad Behavior. I translated all of the Bad Behavior rules into Mod Security and now use it exclusively.

So which one is right for you? Only you can decide. Now you know the differences.

Mailbag: How To Block A Bad Bot

Posted on by
Reply

Isabel has contacted me with the following issue about bots hitting her site:

I have come upon your website in a recent google search on how to ban proxy bots. I am experiencing a problem on my forum which you seem to be all too familiar with.

As of yesterday, a malicious user has setup a BOT to attempt to gain access to administrative and member accounts by inputting passwords. At first, we IP banned everything used. But with over 20 IP bans the BOT is still at it even as of today. It hasn’t stopped since it began. It uses IP addresses from around the world.

I read your article on stopping proxies but it all sounded greek to me. I don’t know how to use or setup any of those programs. Basically, I’d like to ban this user from my site and his BOT but I don’t know the first step into figuring out how.

I have contacted my host, my forum service, and website designers for help.

Is there a way that I can simply backtrack this BOT to the real owner? Is there a way to ban this bot regardless of IP addresses used if I could find the name of it? How could I find out what BOT is causing the issue?

Sincerely,
Isabel

Presumably both your forum service, website designers or maybe even your web-host can help you with this, and of course exactly what you can do will depend on your specific server/site configuration, but here is a general outline of what you can do:
The first step in getting this bot blocked from your site is identifying some sort of a ‘signature’ to reliably identify it. Once you know a uniquely-identifying signature for the bot you can then use various methods to block it from accessing your website or server. To ID the bot and it’s signature you will have to look in your forum logs, Apache access-logs and Apache error logs.

Here are some of the things I look for:

  • IP address or IP range
  • Geographical location (based on the IP)
  • Type of access (proxy, bot-net, etc)
  • User-Agent or other header information
  • “What” it’s doing – pages/path it’s hitting, repeated page-not-found, etc

Once you have a unique signature for the bot, or way to see this bot and pick it out from all of your other visitors you can then take steps to block it. How you block the bot will depend on the level of server access you have and the tools at your disposal – here are a few tools you can use:

As an example, this is what I recently did to block a bot that was repeatedly trying create fake user-accounts at one of my sites:

The bot was coming from random IP’s from all over the world – probably from infected PC’s. None of the IP’s (or very few) were on any block-lists, none were from proxies and the user-agent and headers were indistinguishable from ‘real’ visitors. However, by looking through my Apache error logs I was able to see that every time this bot tried to create an account, it first tried to access a “sign up” page that does not exist at my site (probably does exist for some other type of forum) – So by using MOD_SECURITY and CSF I was able to put together some rules that went something like this:

IF the path="/thatspecificpage.html" THEN BLOCK with error 403 page
AND IF the same IP accesses "/thatspecificpage.html" < 3 times THEN ADD IP to firewall (permanently block)

Another example in the past was when a person was attempting to ‘brute force’ attack random accounts by trying different passwords (much like your issue) – at that time I simply installed a module for my forum-software that would ‘lock out’ accounts for a specified amount of time after a specified number of failed login attempts. By looking in my logs I was eventually able to get the IP of the person, close their account and block the IP address they were using.

Some hypothetical ways to block your bad-bot:

  • Just install BadBehavior! BadBehavior will automatically block a very large percentage of bots and may take care of the problem!
  • If the IP’s are coming from proxies or a BOTNET, use an RBL or DNSBL to block them – to determine if the IPs are in any DNSBL or RBLs check them at a site like MXToolbox.
  • If the IP’s are all in one ‘CIDR’ or a few CIDRs (ranges of IP addresses), or are all coming from the same geographical area that you don’t care about (Turkey, China, etc), block them using your firewall or .HTACCESS
  • If the bot has a particular user-agent, block it with MOD_SECURITY or your .HTACCESS file

As far as identifying the real owner or master of the bot – forget it. Unless the bot-owner is an idiot you would probably never be able to track him or her down. And, even if you could identify him, then what?  Are you  going to go over to his house and kick his ass? (yah, sounds satisfying, but he might be big!). Sure, you could report the bot-owner to the authorities, but in all reality, they probably have bigger fish to fry.  Just figure out how to get it blocked – and, just so you know, there will be more!  Welcome to the interwebs!

Again – what you do, and specifically how you do it, will depend on many things, but hopefully this outline can give you some guidance.

DRUPAL: How To Use dnsBL and RBLs

Posted on by
Reply

Most of my recent yammering about using DNSBLs to block spammers, trolls and other riff-raff has focused on using the DNSBL with Mod_security rules and IPTables firewall which requires either a dedicated server or at least a VPS – leaving all you shared-hosting users out in the cold to fend for yourselves..

But Drupal users in shared-hosting environments rejoice and fear not! You can easily incorporate some of the fun and function of these DNSBLs too! You won’t have the same ability to totally drop their packets but you can block the posts that they make or redirect them to an error page.. Here’s how:

The Drupal TROLL module will not only allow you to block IP’s of users or other IP’s that you add to the block-list, but it will also make-use of DNS Blacklists (DNSBL) to prevent the spammers/trolls, etc from making any posts. They can still read your pages, they just can’t make posts. To use the DNSBL option in TROLL, install as usual, then on the DNS BLACKLIST tab in the configuration page click the “Enable Operation” checkbox and specify how many blacklists the IP must appear on before being “blacklisted” (1 should be plenty).

By default the Troll module uses dnsbl.sorbs.net, bl.spamcop.net, dnsbl.njabl.org, cbl.abuseat.org, & sbl-xbl.spamhaus.org. I would be careful with some of those – particularly spamcop.net and the sbl-xlb.spamhaus.org list. I would change sbl-xbl.spamhaus.org to XBL.spamhaus.org and maybe replace the others with some from my favorite black lists. Remember – be careful. I dont think the Troll module will log when IP’s are blocked so you wont have a good gauge of who is being blocked.

The other way which you can use instead-of Troll or in addition-to Troll is one of my other favorite Drupal Modules, Bad Behavior. The newer version of BadBehavior includes httpBL support from ProjectHoneyPot which will catch/stop a lot of spammers and trolls. The Drupal BadBehavior module is no longer officially supported by the Author, but the community has been doing a good job at patching it and keeping it current. To get the newer versions of BadBehavior to work with Druapal and also enable httpBL support read-through this page at Drupal: Updated For BadBehavior 2.1. If all the posts in that thread are too much to follow, I have the fully-patched (with updated whitelists) version and I’d be happy to ZIP/gZIP it up and send to anyone that needs it.