Web Server Security Checklist

Here is a quick checklist of items I have found to be important in securing and monitoring the security of outward-facing web servers.

Architecture

Typical components surrounding a web server include an external firewall to protect the web server itself from attacks and an internal firewall to protect the internal corporate systems in case the web server is breached.

Hardening

Operating systems and web server software packages often come with additional components that may not be necessary. Rather than leaving unused but potentially vulnerable software available on a web server, it is wise to disable and/or remove any unused software.

Ensure directories and files have appropriate access permissions - does the web server process really need read access to the entire filesystem?

Remove default system accounts. Ensure accounts with access to the server have appropriate passwords.

If you have a host-based firewall on the web server, limit access to administrative functions (SSH or remote terminal services). Limit outbound network connections from the server to only necessary sites and/or protocols.

Patching & Updating

It is amazing how many web servers I have found that are running operating systems, web server software, or web applications that are long outdated and likely to have substantial vulnerabilities. It's important to stay abreast of known vulnerabilities and vendor patches, and have a working plan to evaluate, apply, and test patches for all the software on the web server as well as the other servers and network devices associated with the web server.

I subscribe to the SANS @Risk Consensus Security Alert mail list to stay informed of vulnerabilities and patches in major operating systems and applications.

Web application firewall

Even the best-run and maintained systems can have latent vulnerabilities hiding in the software and/or configuration. Web application firewalls can help protect against attacks such as SQL injection which are otherwise all too commonly successful.

I have made use of the mod_security Apache plug-in module and rules to protect web servers. Many commercial web application firewall devices are available, and even cloud-based web application firewall services are available.

Penetration testing

It's not a bad idea to have a third-party check your web site using penetration testing techniques to check for potential network, operating system, web server, and web application vulnerabilities and mis-configurations.

I have not used it, but I understand the BackTrack bootable Linux CD provides a nice collection of tools to perform penetration testing. Otherwise, there seems to be quite a few consultants willing to perform penetration testing.

Log monitoring

Of course, a busy web site can generate a large amount of log data every day. Tools like awstats can be useful to build an understanding of typical usage loads, top pages, and user demographics.

I have also found looking at failed requests (4xx responses) to be interesting because one can see what approaches attackers are using against web sites, and can help make sure that defenses are working properly.

Auditing

If a system seems to be running OK, why bother looking for trouble?

• What if you have outdated administrative accounts, some of which probably have poor passwords?
• What if a piece of software was installed at some point that unexpected opened access permissions in the filesystem?
• What if, during a hasty period of diagnosing and resolving a significant issue, permissions were changed in the filesystem or in the web server configuration and never were restored?
• Or, what if an attacker has gained access to the server and is siphoning data into a hidden directory for later download?

Make time to audit your web server regularly and look for unexpected changes in files, permissions, or access, check logs, and verify installed software and patches.

Data loss monitoring and prevention

Data loss monitoring and prevention systems should have a place in high-stakes web services. These systems can monitor the type and quantity of data that is coming out of a web server or the database, and raise alerts or block results that violate rules. These systems can be put in place either in front of the web server or the database server to monitor requests and responses.

Data Loss Prevention: Technology or Strategy?

As often happens in the computer industry, nomenclature is unwieldy and flexible as technologists, sales & marketing, and the rest of the world clash.

My case in point is the phrase "data loss prevention" or DLP. In other articles, I have talked about DLP as a technology -- in that it is used to analyze the content of a document or message, determine whether the content references a concept confidential or protected in nature, and uses rules or reporting to handle the content. As the concept of DLP was developed in the last decade, the industry struggled to find an appropriate phrase that defined it: phrases including content monitoring & filtering, content analysis, deep packet inspection, and others were used, but the industry and analysts settled on data loss prevention.

Many companies are marketing "data loss prevention" in relation to their technologies, but not in the context of analysis of document content. Instead, their approaches include building a wall around all corporate data (such as on a mobile device, or in a cloud-based document-sharing service), or providing some regular expression matching for message content. This is well and good, but I would suggest these technologies fall under the larger strategy of information protection rather than being specifically about "data loss prevention".

This goes to the heart of the matter: when we build true data loss prevention systems, the intent is to protect confidential information rather than just bits and bytes of raw data. Under the fundamentals of information theory, data is just bits and bytes, but information is found where there is entropy, or value, in the data. This is what distinguishes data loss prevention technology from other data protection technologies, and perhaps the better phrase for the technology would be "information loss protection."

Practically, though, we are probably stuck with the labels that have been adopted. So, I suppose we can accept a variety of technologies under the strategy of data loss prevention, including the technology of data loss prevention itself. Unfortunately, this will continue to be confusing to those inside and outside of the industry and troublesome for sales and marketing.

Five Stages of Cloud Acceptance

Denial: We'll never put anything in the cloud because of security/reliability/performance/etc.

Anger: You already put WHAT in the cloud? How are we going to do backups/switch providers/manage identity/etc.???

Bargaining: OK, we'll move X into the cloud if/when the cloud becomes secure/reliable/etc.

Depression: The CFO/CEO/etc. wants us to start using cloud to save money/reduce costs/expand functionality. We can't use the cloud. What about my job running the data center? What about our bandwidth? What about PCI DSS/HIPAA/GLBA?

Acceptance: It works, I can do more to enable my company's business, and reduce capital expenditures. Let's put everything into the cloud!

Seriously, I have had some of these reactions myself. I hear some of these reactions from people when we talk about using cloud services and realize there truly is a road to acceptance for many people.

Changing Face of "Spam" Email

As a network engineer involved in bringing up some of the first Internet connections in the upper midwest in the late 1980s and early 1990s, I also managed email systems in the 1990s as spam email started becoming a nuisance. In the past decade, spam has been more than a nuisance - email systems must have effective spam filters to keep email usable for end users.

There is an interesting trend I see now - I am getting a fair bit of relevant business-related marketing email in my inbox. The amount of "online pharmacy" spam is way down, but I still get a fair amount of complete junk, including a lot of Cyrillic and Mandarin spam that is completely unintelligible to me. Fortunately, my company's spam filter, including up-to-date SpamAssassin rule lists and a good blacklist, are doing a good job discarding and classifying the useless spam, while allowing through the reasonable marketing queries (I think).

A few years back, the sales team at my employer emailed potential customers asking if they could setup meetings to introduce the company's software - not an unusual email message, especially nowadays. One particular recipient hit the roof and replied with a rant worthy of a response to the first massive Usenet spam from the green card lawyers back in the day.

Are people's attitudes changing about spam? Is there an increasing acceptance of reasonable marketing-type contact via email?

Security Technology Musings

Each security technology that comes along has its set of "use cases" -- that is, it improves confidentiality, integrity, or availability for certain uses.  Trying to apply that security technology outside of its useful situations results in either a false sense of security or complete failure.

For example, full disk encryption is a useful security technology intended to keep the entire contents of a disk drive relatively safe from an attacker who might steal the physical disk drive (or the system in which it is installed, such as a laptop).  However, when the computer is in operation, full disk encryption has nothing to do with whether files can be accessed -- that is the function of the access control technology built into the operating system.

When we began building Data Loss Prevention (DLP) some years ago, my idea was that content analysis (looking at the textual content of a document) was a powerful way to determine whether a document should be shared outside of an organization.  However, the documents that would be visible to the DLP system for analysis would depend on a number of factors: logical placement of the DLP functionality in an organization's computing system, whether the DLP system would be able to see documents as plaintext, and how an adversary might try to circumvent the system.

As we have further developed DLP technology and the industry has settled on standard implementations (data-in-motion, data-at-rest, data-at-use), customers have become comfortable with the functionality and capability of DLP systems. We're finding that DLP is a very useful tool -- helping significantly reduce exposure of confidential information, and improving standing in risk & compliance audits -- for our customers. It's become one part of the security management arsenal.

Are Anti-Virus and a Firewall Enough?

I thought after all the commotion from the many significant data breaches of the past several months that data security would be top-of-mind at nearly every company. Perhaps people outside the information security industry have become tired of the breach news, or perhaps the lesson didn't sink in. Maybe more likely is the idea that "we haven't been hit yet, so we don't need more security yet."

Computer viruses were such a big problem in the late 80's and 90's (and still today) that companies became accustomed to buying anti-virus software.

The Internet was such a wild and wooly place that companies didn't dare connect their LANs to the 'net without a firewall of some sort to keep the outside world from instantly pwning everything.

People in the information security industry know these two main tools, anti-virus and firewalls, have significant limitations.  Anti-virus tools have limited effectiveness in the era of morphing malware. Firewalls often are configured to allow HTTP/HTTPS (web traffic) and SMTP (email traffic) without any limits, and everyone always has browsers and email clients running. The result is that attackers have a fairly easy time exploiting problems with browsers, email programs, and the users themselves.

Today, organizations need deeper defenses to handle the problems. Intrusion Detection Systems (IDS/IPS), Data Loss Prevention (DLP), patch management, web filter, and Security Information & Event Management (SIEM) are the important systems to have in place in addition to firewalls and anti-virus.

Web servers need to have a Web Application Firewall (WAF) in front of them to protect against attacks on the applications running on the web servers. If you have a good hosting provider for your web server, you may already have a WAF protecting your web server.

If you don't have these systems in place, you can prioritize based on an analysis of your organization's risks.

Web Servers as an Attack Vector

For a long time in computer security, we have been focused on protecting workstations, and rightly so.  Viruses, worms, remote access Trojans, and other malware has targeted the end-user workstation, and unfortunately, the attacks continue to be quite successful.  A number of recent high-profile data leaks have occurred using workstations as the initial point of attack.

However, a point of attack in several other high-profile data leaks have involved attacks on web servers.  Citigroup, Barracuda, and now Pacific Northwest National Laboratory (PNNL) were attacked through web servers.  This makes me a bit nervous -- I do like to make sure a public-facing web server is hardened and running software that is fully-patched, but there are several techniques attackers can use to find and take advantage of any holes in the server.

One of the problems that I saw disclosed today, CVE 2011-2688, involves a SQL injection attack against the mod_authnz_external module, an Apache authentication module.  It is worrisome that a well-known attack is successful on this security-critical component that may be in use on many web servers.  Many other attacks, including parameter tampering,

Web servers and the web applications running under them are proving to be all too vulnerable.  With high-value data accessible in a web server, such as customer accounts at an online banking website, any exploitable vulnerability in the web server or web application can result in significant loss. As the events at PNNL illustrated, even a web server that may not be high-value can still be an entry point for an attacker into more valuable networks and systems.

It seems that web servers need backstops.  We need to be able to filter and/or monitor requests coming into a web server, and to filter and/or monitor data returned by a web server.  And, we need to be able to do this in the cloud with web servers that automatically scale.  Something to think about.