New Apache instance with Reverse Proxy
This article covers the process of setting up a new Apache instance using port 81 and then hiding it behind a reverse proxy. Some details are specific to Apache 2 running under Debian GNU/Linux, but the process should be similar for other flavours.
Why run multiple instances?
There are many reasons for running multiple instances of Apache.
- If you host different websites with different requirements in terms of Apache modules you can save memory by running multiple stripped-down versions for each website or group of websites.
- Running separate instances also helps secure the webserver for multiple users as you can assign each instance a different user, group, configuration and log directories.
- You may want to run different versions of PHP or other modules on different instances to cater for legacy code or to allow for testing.
- Apache can also run as a proxy or reverse proxy to deliver content from a different port, or even a different machine or network, as if it was being served locally. For this you would normally use port 8080 and include a caching function such as Squid.
In this example we're setting up a second apache instance to listen on port 81 and hiding it behind a reverse proxy on port 80 on the same server, though it could just as easily be between two separate servers or networks.
Apache 2 to the rescue
Running multiple instances is now such a common requirement that Apache 2 has provided both documentation and a script for getting the process started.
Many of the apache2 command-line functions have also been updated to be capable of distinguishing between different instances of apache depending on how they are called.
The provided documentation can be read online here, and should be read carefully to avoid common pitfalls:
And as you can see a shell script is also provided to initialise a new instance:
The original apache2 remains unchanged. Each added instance is given a suffix, such as apache2-xxx, apache2-proxy or apache2-forperl. Use whatever makes sense to your situation.
The setup script simply creates a duplicate of the configuration directory and included files, a new logs directory and logrotate script:
/etc/apache2-xxx
/etc/apache2-xxx/apache2.conf
/etc/apache2-xxx/conf.d
/etc/apache2-xxx/envvars
/etc/apache2-xxx/httpd.conf
/etc/apache2-xxx/logs
/etc/apache2-xxx/magic
/etc/apache2-xxx/mods-available
/etc/apache2-xxx/mods-enabled
/etc/apache2-xxx/ports.conf
/etc/apache2-xxx/sites-available
/etc/apache2-xxx/sites-enabled
/etc/logrotate.d/apache2-xxx
/var/log/apache2-xxx
The assumption is that you're setting up a completely separate environment with no shared configuration files. Any upgrades or changes to the original apache settings will not automatically affect the instances, requiring an intelligent file management system.
If you do want to share directories such as conf.d or mods-available then delete the copies and replace them with symlinks. You can also have different apache instances use the same logs directory and logrotate settings if you want, though separate error logs are probably a good idea.
Some files, particulary ports.conf, need to be unique to prevent binding conflicts. And if you want to run different modules then mods-enabled also can't be a symlink.
The provided script also creates a wrapper for the init script and symlinks for command-line tools:
/etc/init.d/apache2-xxx
/usr/local/sbin/apache2ctl-xxx
/usr/local/sbin/a2dissite-xxx
/usr/local/sbin/a2ensite-xxx
/usr/local/sbin/a2dismod-xxx
/usr/local/sbin/a2enmod-xxx
The new commands are just links to the original scripts which have been modified to detect the suffix and target the relevant instance.
Changing how you Listen
A webserver with a single instance will have been listening on port 80 (:80), and maybe :443, for requests to one or more ip addresses. This is defined in ports.conf, for example:
Listen 80
Listen 443
This tells this instance of apache to handle all http and https requests for all active ip addresses. A second or subsequent webserver cannot have any overlap or it will fail to bind:
Address already in use: make_sock: could not bind to address 0.0.0.0:80
These and similar errors can also appear if you only reload apache after editing ports.conf instead of stopping and starting the server so it releases the relevent ip address/port combination.
A useful command for diagnosing these problems is netstat -nlp which shows which programs are currently bound to which Local Address.
To separate your apache instances you can distinguish them by ip address, assuming you have multiple ones available:
/etc/apache2/ports.conf Listen XXX.XX.XX.121:80
/etc/apache2-xxx/ports.conf Listen XXX.XX.XX.122:80
by port number:
/etc/apache2/ports.conf Listen 80
/etc/apache2-xxx/ports.conf Listen 81
or by a combination of address and port, as long as there is no overlap:
/etc/apache2/ports.conf Listen XXX.XX.XX.121:80
Listen XXX.XX.XX.122:80
/etc/apache2-xxx/ports.conf Listen XXX.XX.XX.121:81
/etc/apache2-yyy/ports.conf Listen XXX.XX.XX.122:81
Once you have configured ports.conf, and checked the rest of the configuration, you should be able to start using the new instance just as you would a normal apache:
service apache2-xxx start
a2ensite-xxx example.com
a2ensite-xxx example.net
service apache2-xxx reload
Hiding the port number
After setting up a new instance to listen on an alternative port, such as 81, 82, 83, 84 or 8080, websites will only be visible if the request includes the port number:
http://www.example.net:81/
http://www.example.com:81/about
This is because all browser requests arrive on port 80 by default and :80 traffic is still handled by the original apache which doesn't know about the xxx instance or associated websites. This might be suitable for an intranet or web application, but not for a public website.
The solution is to set up a reverse proxy allowing the original apache to accept requests on :80, funnel them internally to :81 and display the relevant content.
To do this we need to add the modules mod_proxy and mod_proxy_http to the original apache:
a2enmod proxy
a2enmod proxy_http
The following configuration in /etc/apache2/mods-enabled/proxy.conf should be suitable for a server with name-based virtual hosting:
<IfModule mod_proxy.c>
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPreserveHost On
ProxyRequests Off
ProxyVia Off
</IfModule>
It will also help to install mod_rpaf on the new instance to allow for accurate logging:
apt-get install libapache2-mod-rpaf
a2enmod-xxx rpaf
service apache2-xxx reload
With configuration:
<IfModule mod_rpaf.c>
RPAFenable On
RPAFsethostname On
RPAFproxy_ips 127.0.0.1 XXX.XX.XX.121 XXX.XX.XX.122
</IfModule>
Configuring the websites
Using a reverse proxy means adding a VirtualHost to the original apache including instructions for the proxy. Something like the following:
<VirtualHost XXX.XX.XX.121:80>
ServerName www.example.net
ServerAlias example.net
ProxyPass / http://XXX.XX.XX.122:81/
ProxyPassReverse / http://XXX.XX.XX.122:81/
</VirtualHost>
This takes any requests for www.example.net on :80 which arrive at XXX.XX.XX.121 and funnels them internally to the new ip address and port: XXX.XX.XX.122:81. If you like you can also add logging here as well as on the proxied server.
The mod_proxy configuration means that the hostname is passed along with the request so we don't need to do anything special on the new apache instance to display the correct content
<VirtualHost XXX.XX.XX.122:81>
ServerName www.example.net
ServerAlias example.net
DocumentRoot /var/www/example.net
CustomLog logs/example-net-combined.log combined
</VirtualHost>
The mod_rpaf configuration means that instead of XXX.XX.XX.121 in the request header and log files every time we instead see the original remote ip address.
With this setup :81 can remain firewalled from outside as all external requests will arrive through the default http port (:80). At the proxy stage any number of functions can be applied, from blocking and filtering, to compressing, optimising, caching and even load balancing.
The reverse proxy is only needed if you are serving content from a non-standard port or a different server. If you have multiple apache instances listening to :80 on different IP addresses then you only need to divide websites between them.
References
- Running Multiple Apache Instances
- Multiple Instances of Apache
- Apache Module mod_proxy
- Using Apache with mod_proxy
- Preserving Remote IP/Host while proxying