Using Gunicorn on the ActivePython AMI

Last month we posted a screencast from Raymond Hettinger on Getting Started with the ActivePython AMI on EC2. For this follow-up post, Mike Ivanov walked me through how to swap the default Apache web server for Gunicorn (aka Green Unicorn, a Python WSGI web server) which is a popular replacement for Django environments.

This tutorial continues on from the Building a Python-centric WebServer in the Cloud tutorial, but to make it more broadly useful we're going to edit the configuration files in place on the server, rather than using Fabric for deployment as mentioned in the tutorial. Using fab is ultimately more flexible and powerful, but editing files on the server works too.

1) Install Gunicorn

This part is easy using PyPM. Log in to your EC2 instance using SSH and run the following command:

/opt/ActivePython-2.6/bin/pypm install gunicorn

2) Change the config files

We'll start by creating a Gunicorn configuration file which will serve the same purpose as the demosite.wsgi configuration file in the Apache setup. Substitute vim with your favorite shell editor.

sudo vim /var/www/demosite/etc/gunicorn.conf.py

Paste in this:

import os

os.environ['PRODUCTION'] = 'TRUE'
os.environ['DEMOSITE_DB_HOST'] = open('/var/www/demosite/etc/dbhost').read().strip()

def num_cpus():
    if not hasattr(os, "sysconf"):
        raise RuntimeError("No sysconf detected.")
    return os.sysconf("SC_NPROCESSORS_CONF")
    
bind = "127.0.0.1:8080"
workers = num_cups() * 2
max_requests = 1000
preload = True
pid = "/var/run/gunicorn-demosite.pid"

Nothing special here. This config would be almost exactly the same for another project. Note that there's no mention of our project ("demosite") here. That's because we're not going to run Gunicorn by itself, it's just a Python script running a WSGI server. To make a system daemon out of it, we need to use an external tool. We'll use supervisor, which is a convenient process control system for UNIX-like systems. It's installed with ActivePython on the AMI.

Create a supervisor config file:

sudo vim /var/www/demosite/etc/supervisor.conf

Paste in this configuration to tie the pieces together:

[program:demosite]
command=/var/www/demosite/env/bin/gunicorn_django -c /var/www/demosite/etc/gunicorn.conf.py
directory=/var/www/demosite/
user=nobody
group=www-data
autostart=True
autorestart=True
redirect_stderr=True

The command= setting tells supervisor to run the gunicorn_django launcher script with our new gunicorn.conf.py configuration file. The directory= line tells supervisor where the site is. If you've been using fab it would be something like /var/www/demosite/releases/current instead.

Note the user and group. These are the same as Apache's default user/group combination for WSGI processes. This could be specified in the gunicorn config, but is easier to do here.

3) Symlink to /etc/supervisor/conf.d/

To get supervisor to bring up demosite at boot, we need a symlink in supervisors conf.d directory:

sudo ln -s /var/www/demosite/etc/supervisor.conf /etc/supervisor/conf.d/demosite.conf

Again, if you're using fab the path will have 'releases/current' in it.

4) Try it out!

Since Apache is already running by default, we should stop that service before starting up Gunicorn using supervisorctl:

sudo service apache2 stop
sudo supervisorctl demosite start

We can do a quick check using curl to make sure we can connect directly to Gunicorn:

curl localhost:8080

On the ActivePython AMI, Nginx is sitting in front of Gunicorn for better performance and to protect against denial of service attacks. It forwards our demosite on to port 80 where it can be seen by the outside world.

So now we have a running Django stack that uses a server with native support for WSGI and Django with a much more Pythonic approach to configuration.