Setup uWSGI to run python scripts via HTTP request

This post was auto converted and may contain formatting errors.

What I want to achieve here is to run a python script on the server whenever a certain URL is requested, and possibly with some arguments passed to the python script. For example, when I enter the URL yuluyan.com/app?x=1&?y=2, the python script will be called with two arguments x=1 and y=2.

uWSGI, named after WSGI (Web Server Gateway Interface), is a complete solution for hosting services. But it also works with other servers like Nginx, which is perfect for my case. This post will cover the very basics of setup uWSGI with python on an Nginx server.

Prerequisites

Upgrade the system:

sudo apt update
sudo apt upgrade

Install C compiler and Python

apt install build-essential python-dev python-pip

Install uWSGI through pip

pip install uwsgi

Create python application

This is the example from the official documentation:

# helloworld.py
def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    return [b"Hello World"]

Save it as helloworld.py.

Configuration

Change the config file inside /etc/nginx/. The following is a minimal example:

location /app/ {
    include uwsgi_params;
    uwsgi_pass 127.0.0.1:3905;
}

It means: whenever the URL yuluyan.com/app/ is requested, it is passed to uWSGI. Now restart Nginx

sudo service nginx restart

Start uWSGI (the port should match)

uwsgi --socket 127.0.0.1:3905 --wsgi-file ~/helloworld.py --master --processes 1 --threads 1

or run it using screen:

screen -dmS uwsgi uwsgi --socket 127.0.0.1:3905 --wsgi-file ~/helloworld.py --master --processes 1 --threads 1

A better way would be using .json or .ini config files:

# config.ini
[uwsgi]
# socket = addr:port
socket = 127.0.0.1:3905

# The python application file
wsgi-file = helloworld.py

# master = [master process (true of false)]
master = true

# processes = [number of processes]
processes = 1

# threads = [number of threads]
threads = 1

and run with it

screen -dmS uwsgi uwsgi --ini config.ini

Now if you enter URL yuluyan.com/app, a page with Hello World will show up.

WSGI basics

The following code shows the environment variable passed to the python script.

def application (env, start_response):
    response = '<h2>Content in environment variable</h2>' + '\n'.join([
        '<p><span style="font-weight: bold;">%s</span>: %s</p>' % (k, v) for k, v in sorted(env.items())
    ])
    start_response('200 OK', [('Content-Type','text/html')])
    return [response]

When the URL yuluyan.com/app/func?x=1&y=2 is requested, the content of the environment variable is set correspondingly. The following shows the keys that are important to us.

[nc]Content in environment variable
CONTENT_LENGTH:
CONTENT_TYPE:
... ...
PATH_INFO: /app/func
QUERY_STRING: x=1&y=2
... ...
REQUEST_METHOD: GET
REQUEST_SCHEME: https
REQUEST_URI: /app/func?x=1&y=2
SERVER_NAME: yuluyan.com
... ...

For our case, we only need to take care of the two keys PATH_INFO and QUERY_STRING. PATH_INFO can be processed by simply split while QUERY_STRING can be processed by parse_qs function from library urlparse. The following script is an example of how to use them:

from urlparse import parse_qs

def application (env, start_response):
    path_info = env.get('PATH_INFO', '').strip('/').split('/')
    query_string = parse_qs(env.get('QUERY_STRING', ''))

    response = '<h4>PATH_INFO</h4>' + '[' + ', '.join(path_info) + ']'
    response += '<h4>QUERY_STRING</h4>' + '\n'.join([
      "%s: %s" % (k, '[' + ', '.join(v) + ']') for k, v in query_string.items()
    ])

    start_response('200 OK', [('Content-Type','text/html')])
    return [response]

If we enter the URL

[nc]yuluyan.com/app/path1/path2/path3/func?x=1&y=2&z=3&x=4

we will get the following result:

[nc]PATH_INFO
[app, path1, path2, path3, func]
QUERY_STRING
y: [2] x: [1, 4] z: [3]

Dispatch function calls

Here is a toy implementation of a function dispatcher. In the config dictionary, the key is the url path and the value is the function to be called.

from urlparse import parse_qs

# if use python version >3.3, use
# from inspect import signature
from funcsigs import signature

def func1(t):
    return "You've input " + t

def func2(x, y):
    return int(x) + int(y)

def application(env, start_response):
    config = {
        'app/func1': func1,
        'app/func2': func2
    }
    try:
        result = Dispatcher(config)(env)
        response = 'The result is ' + str(result)
    except ValueError as e:
        response = '<span style="color:red;">Error: ' + str(e) + '</span>'

    start_response('200 OK', [('Content-Type', 'text/html')])
    return [response]


class Dispatcher():
    def __init__(self, config):
        self.config = {}
        for path, func in config.items():
            self.config[path.strip('/')] = {
                'function': func,
                'arguments': [ param.name for param in signature(func).parameters.values()]
            }

    def __call__(self, env):
        path = env.get('PATH_INFO', '').strip('/')
        query_string = parse_qs(env.get('QUERY_STRING', ''))

        if path in self.config:
            args = []
            for param in self.config[path]['arguments']:
                if param in query_string:
                    args.append(query_string[param][0])
                else:
                    raise ValueError('No argument named %s provided!' % param)
            return self.config[path]['function'](*args)
        else:
            raise ValueError('No matched function at path %s!' % path)

Let’s test it!

[nc]yuluyan.com/app/func1?t=some+text
# The result is You've input some text

yuluyan.com/app/func1?x=some+text
# Error: No argument named t provided!

yuluyan.com/app/func1?t=some+text&x=nonsense
# The result is You've input some text

yuluyan.com/app/func2?x=1&y=2
# The result is 3

yuluyan.com/app/func2?x=1&z=2
# Error: No argument named y provided!

yuluyan.com/app/func2?x=1&z=2&y=3
# The result is 4

yuluyan.com/app/func2?x=1&y=b
# Error: invalid literal for int() with base 10: 'b'

yuluyan.com/app/func3
# Error: No matched function at path app/func3!

yuluyan.com/app/foobar/func3?x=3
# Error: No matched function at path app/foobar/func3!

It works well for this simple case. But be sure not to allow dangerous things in the functions! For more complex use case, there are plenty of frameworks available, e.g., Django, Flask, etc..