Apache web application security

This document summarises various ways of setting up web applications to run under the Apache httpd server in a secure and isolated fashion.

Secure? Isolated?

The observant system administrator will have noticed that their operating system provides a way to isolate resources into different security contexts. Usually this means that each resource is associated with a particular user id, and that resources may only interact with each other if they are associated with the same user id.

In a typical operating system such as Debian GNU/Linux, resources refer to files and processes. Files are ‘owned by’ a user, and processes are said to ‘run as’ a user.

I guess this terminology came from the days when Time sharing was a new and exciting concept. A (human) user would be given an account on a computer system, and so the term ‘user’ also came to refer to the persona which the human operated as while he was using the computer.

Only some time later was it realised that it would be a good idea to assign the resources associated with unrelated services to different security contexts; and by then the term ‘user’ had become common. If we were going to give the term ‘user’ a more logical name, we might have chosen 'role' or 'security principal'.

Let's give an example for those who are scratching their head. A computer system runs two services: a domain name server runs as the 'bind' user, while a web server runs as the 'www-data' user. This means that in the event of the DNS server bring compromised and made to execute arbitrary code, the attacker will only be able to interfere with the files and processes owned by/running as the bind user―files and processes relating to the web server will be off-limits.

httpd is the new inetd... only everything is run as the same user

Unfortunately, Apache's default setup does not make isolating web applications from one another easy: Apache normally runs as a single user, putting all your web applications into the same security context. Back when web servers only served static files, this did not matter, but these days, a typical web server will provide access to both simple PHP scripts and complex web applications.

If your blog is compromised and the web server is made to execute code of an attacker's design, you don't want him to be able to capture the passwords that you input into phpMyAdmin to access your database server. A customer on a shared web server does not want other customers to be able to read the configuration file where his web applications' database passwords are installed.

Doing something about it

There are a variety of ways to configure Apache to isolate web applications from one another.

Simple solutions

What these solutions lack in novelty or glamour, they make up for in reliability. They are very well tested and work well, within their limitations.

Multiple web servers

The simplest way to isolate web applications is to run multiple web servers, each running as a different user. Managing all the separate processes is not difficult if the admin uses a decent service supervisor like runit or upstart, or if he is able to write his own init scripts.

The problem with this approach is that each web server will want to open a TCP socket on port 80 to receive requests from clients. Unless you have one IP address for each web server to listen on, you will have to set up an http proxy server to analyse requests and route them to the appropriate web server.

Apache itself can do this with mod_proxy. In this example configuration, http://phpmyadmin.example/, http://personal.example/blog and http://personal.example/wiki all point to separate web servers, which are presumably running as different users.

<VirtualHost phpmyadmin.example>
 ProxyPass / http://localhost:10090/
</VirtualHost>

<VirtualHost personal.example>
 <Location /blog>
  ProxyPass / http://localhost:10091/
 </Location>

 <Location /wiki>
  ProxyPass / http://localhost:10092/
 </Location>
</VirtualHost>

This method is presumably rather inefficient, although I have not benchmarked it. It may also be inconvenient in mass hosting situations.

suEXEC

The suEXEC documentation says it best:

• The suEXEC feature provides Apache users the ability to run CGI and SSI programs under user IDs different from the user ID of the calling web server.

If your web applications use CGI then mod_cgi/mod_cgid combined with suEXEC is ideal.

suEXEC always runs the CGI executables as the user specified with the SuexecUserGroup directive in Apache's configuration file.

Some of the security checks performed by the suexec helper executable can be annoying if you want to use suEXEC for mass hosting, although these can be worked around (or, if you rebuild the suexec helper executable, disabled—but only if you know exactly what you are doing!).

CGI applications perform relatively poorly, although as always, avoid premature optimisation—if CGI is fast enough for your purposes, use it!

I have seen references to a technology known as "PHPSuExec" referred to on several forums. I believe that this is a mythical beast, and that the posters are actually referring to the combination of PHP's CGI SAPI and suEXEC.

suEXEC is a part of Apache proper, so it is presumably the most well-tested solution listed here.

suPHP

suPHP is a slightly more flexible alternative to suEXEC. It has security checks similar to suEXECs, but they can be fine-tuned or disabled in its config file. It can call chroot(2) before handing control over to the target executable.

Despite its name, suPHP will work with any program that uses CGI; its configuration file allows for different executables to be used depending on the handler that Apache associates with a request.

Whereas suEXEC always runs executables as the user specified in Apache's configuration file, suPHP will by default run an executable as the user who owns it. This behaviour can be changed at compile-time with the setid-mode option.

suPHP will presumably perform poorly when used with a multi-threaded MPM, for the same reasons that mod_cgi will. There is no suPHP equivalent of mod_cgid to overcome these issues.

suPHP packages are present in Debian.

More invasive solutions

The solutions presented so far have their performance limited by the nature of how CGI works. It is far more modern and cool to modify Apache itself to assign a different user to each virtual host. Unfortunately, it is also very hard to get this right.

perchild MPM

The perchild MPM allows a user to be assigned to each virtual host in your Apache configuration. The Apache process for each virtual host runs as the assigned user; thus all aspects of all requests processed by the virtual host run as that user.

This means that the script interpreters that live inside the Apache process when using modules such as mod_php or mod_python can be used without having to sacrifice security. In practice however, the perchild MPM uses a multi-threaded process model and so is not compatible with mod_php and other Apache modules that are not thread-safe.

Unfortunately, the perchild MPM was never particularly popular, and never received the testing it required to become suitable for a production system. Development was abandoned and it was removed before the release of Apache 2.2.

metux MPM

The metux MPM is "a successor for perchild". There is not much information available about it, and its web site has been down for some time.

Like the perchild MPM, metux uses a multi-threaded process model and so is incompatible with mod_php and other non-thread-safe Apache modules. I also seem to remember the web site saying that it was incompatible with mod_ssl.

The metux MPM is not packaged in Debian but instructions were available on the web site before it went down. According to chatter on the debian-apache list, the metux developers do not seem particularly interested in getting the metux MPM cleaned up and accepted into Apache itself.

peruser MPM

The peruser MPM is based on the metux MPM, itself based on the perchild MPM. Again, since it uses multiple threads, there are issues with non-thead-safe Apache modules. It has several other issues described on its home page.

It is not currently present in Debian.

ITK MPM

The ITK MPM is another MPM for Apache. Similar to perchild and metux, it allows a user to be assigned to each virtual host. However it uses a traditional, Unix-style prefork process model and so does not have the same threading issues shared by the other MPMs.

Best to quote from the ITK home page (emphasis mine):

• Warning: since mpm-itk has to be able to setuid(), it runs as root until the request is parsed and the vhost determined. This means that any security hole before the request is parsed will be a root security hole. (The most likely place is probably in mod_ssl...) Without implementing socket passing (which is the primary reason why perchild/metux doesn't really work well; it's complex enough in the difficult cases that nobody has bothered finished their socket passing implementation) or using some sort of special SELinux functionality, this is not going to change in the near future.

As always, consider the various trade-offs before implementing this solution.

The ITK MPM is packaged in Debian.

What's old is new again

The only nonstandard MPM I would consider using in production is the ITK MPM. While I haven't had any problems with it, I did come to wonder whether adding a module for each language you want to run web applications in is the right solution. The memory eaten up, and bugs introduced, by each of mod_php, mod_python, mod_perl, and so on, will all add up eventually.

I have now come to believe that the original CGI approach is superior except for the overhead for each request caused by waiting for the CGI script to start. It just so happens that someone devised a replacement for CGI that does away with this problem.

This replacement is called FastCGI, and although it has been around for over a decade, it has only entered widespread use in the last few years.

If your web application can act as a FastCGI application (even PHP's CGI SAPI can do so, so there is no excuse!) then you can benefit from the security of the traditional CGI + suEXEC arrangement, without suffering from the performance penalty.

This approach allows us to remove all the language-specific modules from Apache. It makes Apache into a lightweight proxy that passes requests to external processes that run as separate users.

mod_fastcgi

mod_fastcgi is the original implementation of a FastCGI module for Apache. It takes care of two jobs: first, it spawns FastCGI applications as requests come in for them; second, it communicates with the applications via the FastCGI protocol, passing them requests and returning their responses.

mod_fastcgi can run FastCGI applications in one of three modes:

static

Apache launches an application server that will host the web application. If suEXEC is configured, the application server is run as the user specific in Apache's configuration. If the server dies, Apache starts another one.

dynamic

Apache spawns application servers dynamically as requests come in. As with the static mode, if suEXEC is configured then the servers will run as the user specified in Apache's configuration. This mode is what you would use in a mass hosting environment, since users do not have to modify Apache's configuration in order to set up new dynamic FastCGI applications.

external

Apache does not spawn the application at all; it merely hands off requests to an external FastCGI application via a well known socket. This is basically the same as the original 'multiple web servers' configuration, only faster.

mod_fastcgi does not appear to have been developed for a while, and there are statements on the Web that it is buggy when run with Apache 2, however I have not seen any concrete reports of problems and it is still widely used.

mod_fastcgi is packaged in Debian's non-free section.

mod_fcgid

mod_fcgid is another implementation of FastCGI for Apache that claims to be less buggy than mod_fastcgi. Although documentation is sparse, it does seem to be actively developed.

mod_fcgid acts like the ‘dynamic’ mode of mod_fastcgi. That is, it spawns FastCGI applications itself as they are requested by clients, making it suitable for mass hosting. As with mod_fastcgi, processes are spawned as the user specified in the suEXEC configuration.

mod_fcgid is present in Debian.

mod_proxy_fcgi

Another alternative FastCGI implementation, this acts like the ‘external’ mode of mod_fastcgi. It is present in Apache's trunk, but has not yet been backported to the 2.2.x branch.

There is also a mod_proxy_fcgi SourceForge project. It seems to be an older version of the same code. Presumably it was submitted upstream and so the SourceForge project is now obsolete.

mod_scgi

The SCGI protocol is similar to FastCGI but designed to be easier to implement. Its Apache module mod_scgi behaves similar to mod_fastcgi in external mode, or mod_fcgi_proxy. Perhaps it will one day be incorporated into Apache itself in the form of a mod_scgi_proxy module.

mod_scgi acts like the ‘static’ mode of mod_fastcgi above; that is, it passes requests through to an already-running server process, rather than spawning such processes as needed.

mod_scgi is present in Debian.

mod_jk

mod_jk is another lightweight proxy designed to let Apache talk to web application servers using the Apache JServ Protocol. Its purpose is to allow Apache to be the front-end for JSP containers.

Best solution

Obviously, the combination of the above features that is best for your situation depends very much on the set of web applications that you wish to serve. When configuring my web server, my goals were:

security

Each virtual host to run dynamic content under separate user accounts. This means that they cannot interfere with one another.

performance

Performance does not have to be amazing, but it would be nice if requests for dynamic resources were handled quickly, without the overhead of a fork and exec for each request as with CGI.

configurability

It should not be necessary to modify the Apache configuration when a new web application is to be installed. It should be a matter of dropping the right kind of files into the correct directories.

reliability

Reasonably well-tested solutions are preferred, it's no good if the web server has to be constantly restarted.

With these criteria in mind, I settled on mod_fcgid and suEXEC. My configuration is detailed at ApacheConfiguration.

CategoryTechnote