Setting up a django project with uWSGI

In this article, we will see how to setup a django project with a uWSGI server for production usage. In the past I have only been using the internal development server provided directly in django for my test projects, but that development server should definitely not be used in production: instead, setting up a wsgi connection behing a proper web server such as Apache or nginx seems to be the recommanded way to proceed for production setup.

Also, in the process, I'm also now using a MariaDB backend for the database instead of a simple sqlite3 database: again, this seems to be the recommended path in production, so we will see how to set this up too.

• In this session, we used the article Setting up Django and your web server with uWSGI and nginx as main reference, so you might also want to check this.

• First think I noticed, was that my pip installation for python3 was out dated [of course…], so to upgrade it:

  sudo pip3 install --upgrade pip

• Then we need to setup a virtual python env for our django project, which will be called nvserver in my case (and thus, I use that name too for the virtual env here):

  cd $venvs_root_dir
  virtualenv -p /usr/bin/python3 nvserver

• We then need to “enter” or “activate” that virtual env:

  source nvserver/bin/activate

• Once the virtualenv is activated we have to install some pip packages in that env:

      pip install Django
      pip install mysqlclient
      pip install uwsgi

• Start django project:

  cd $projects_root_dir
  django-admin.py startproject nvserver

• Test running the project:

  cd nvserver
  python manage.py runserver

• To run the test server (or development server if you will) on a different port and all available interfaces we can use:

  python manage.py runserver 0.0.0.0:8090

⇒ After changing the port I could finally get the project to run when opening the correponding web page (ie. http://192.168.2.20:8090 in my case)

• For the setup of the database I use this article as base reference.

• First we need to create the database/user with the following code:

  mysql -u root -p -h 192.168.1.20 -P3307
  CREATE DATABASE nvserver CHARACTER SET UTF8;
  CREATE USER nvserveruser@localhost IDENTIFIED BY 'password';
  GRANT ALL PRIVILEGES ON nvserver.* TO 'nvserveruser'@'localhost';
  GRANT ALL PRIVILEGES ON nvserver.* TO 'nvserveruser'@'192.168.1.20' IDENTIFIED BY 'password';
  FLUSH PRIVILEGES;
  EXIT

• Then to set up the django project to use the mariadb database, we have to edit our nvserver/settings.py file:

  DATABASES = {
      # 'default': {
      #     'ENGINE': 'django.db.backends.sqlite3',
      #     'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
      # }
      'default': {
          'ENGINE': 'django.db.backends.mysql',
          'NAME': 'nvserver',
          'USER': 'nvserveruser',
          'PASSWORD': 'password',
          'HOST': '192.168.1.20',
          'PORT': '3307',
      }
  }

⇒ Once the database access is configured we can apply the migrations with:

python manage.py makemigrations
python manage.py migrate

All the initial migrations went just fine for me. Then we can create the superuser:

python manage.py createsuperuser

When the superuser is created we can start the server again and then nagivate to http://192.168.1.20:8090/admin to check that we can indeed login with the superuser credentials: OK

To be honest the first test I actually did on my side was to only install uwsgi globally (ie. system wide) with the command:

sudo -H pip3 install uwsgi

Then I tried to serve my django project with it directly :

uwsgi --http :8090 --wsgi-file nvserver/wsgi.py

But this didn't work as expected… [and in fact this makes complete sense]: when trying to load the file, the uwsgi server will produce an error because the django module is not found in the global python environment. So this means we should really run the uwsgi server from the virtual python env (and that's why we installed the package accordingly as reported above)

⇒ When executing the same command from the python virtual env, the uwsgi server will start properly, and we can navigate to http://192.168.1.20:8090 to see the django project running as desired.

Next we need to configure the webserver:

server {
  listen 80;
  server_name   api.nervtech.org;
  
  rewrite       ^ https://$server_name$request_uri? permanent;
}

# cf. https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html
# the upstream component nginx needs to connect to
upstream django {
    # server unix:///path/to/your/mysite/mysite.sock; # for a file socket
    server 192.168.1.20:8181; # for a web port socket (we'll use this first)
}

server {
    listen 443 ssl;
    server_name api.nervtech.org;
    charset     utf-8;

    ssl_certificate           /etc/letsencrypt/live/nervtech.org/fullchain.pem;
    ssl_certificate_key       /etc/letsencrypt/live/nervtech.org/privkey.pem;

    ssl_prefer_server_ciphers on;
    ssl_protocols         TLSv1.2;

    ssl_ciphers 'ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4';

    client_max_body_size 20m;

    access_log            /var/log/nginx/api.access.log;
    error_log            /var/log/nginx/api.error.log;

    add_header Strict-Transport-Security max-age=15768000;

    # Django media
    location /media  {
        alias /mnt/web/nvserver/media;  # your Django project's media files - amend as required
    }

    location /static {
        alias /mnt/web/nvserver/static; # your Django project's static files - amend as required
    }

    # Finally, send all non-media requests to the Django server.
    location / {
        uwsgi_pass  django;
        include     /mnt/web/nvserver/uwsgi_params; # the uwsgi_params file you installed
    }
}

Minor detail: from this step I started to use the port 8181 instead of 8090 Just don't get confused on this point.

My project setup is a bit non-standard I guess, but: I have the django project inside a git repository that I checkout on my host computer while my nginx server is running in a container that doesn't have access to the location where I checked out the project. So the idea I have in mind is to serve the django dynamic stuff with uWSGI while copying all the static files into a location that can be accessed by the nginx container afterwards. And it turns out this is very easy to do: you just need to specify an “out of source” STATIC_ROOT folder in the settings.py file:

# STATIC_ROOT = os.path.join(BASE_DIR, "static/")
STATIC_ROOT = "/mnt/array1/web/nvserver/static/"

Then we can collect all the static files with:

python manage.py collectstatic --clear --noinput

This will fill the folder /mnt/array1/web/nvserver/static/ with content, and that content is then available from the nginx container as /mnt/web/nvserver/static/: all good for us !

At first I couldn't connect nginx with the uWSGI server, and I eventually realized this was due to the fact I was still using an “http” setup instead of a proper “socket” setup, so the command to start the server should be updated here to:

uwsgi --socket 192.168.1.20:8181 --wsgi-file nvserver/wsgi.py

To use a unix socket file instead of a socket port, we update our uwsgi start command again to something like:

uwsgi --socket /mnt/array1/web/nvserver/uwsgi.sock --wsgi-file nvserver/wsgi.py --enable-threads

Then we update the nginx site config file to use that socket file:

upstream django {
    server unix:///mnt/web/nvserver/uwsgi.sock; # for a file socket
    # server 192.168.1.20:8181; # for a web port socket (we'll use this first)
}

Important note: We must also ensure that nginx has the permissions to access the socket file, so for my part I also had to add the command line option --chmod-socket=666 to the uwsgi start command [This is not really recommended, but I don't think this could be a problem somehow anyway].

We now create a config file to start our django project uwsgi server:

# nvserver_uwsgi.ini file
[uwsgi]

# Django-related settings
# the base directory (full path)
chdir           = /mnt/array1/dev/projects/NervSeed/python/django/nvserver
# Django's wsgi file
module          = nvserver.wsgi
# the virtualenv (full path)
home            = /mnt/array1/dev/projects/NervSeed/python/envs/nvserver

# process-related settings
# master
master          = true
# maximum number of worker processes
processes       = 10
# the socket (use the full path to be safe
socket          = /mnt/array1/web/nvserver/uwsgi.sock
# clear environment on exit
vacuum          = true

chmod-socket    = 666
uid             = www-data
gid             = www-data
enable-threads  = true

Then we can ensure that the project is still working properly using the command line:

uwsgi --ini nvserver_uwsgi.ini

And finally we can run uwsgi in emperor mode in the global python environment with:

uwsgi --emperor /mnt/array1/app_data/uwsgi --uid www-data --gid www-data

To ensure that uWSGI is started when the system boots I added the command to my custom function “nv_start_all_services” with the additional content:

  logDEBUG "Starting uWSGI emperor"
  /usr/local/bin/uwsgi --emperor /mnt/array1/app_data/uwsgi --uid www-data --gid www-data --daemonize /mnt/array1/app_data/nginx_server/logs/uwsgi_emperor.log

And finally we should call that script function in /etc/rc.local:

su kenshin -c "bash /home/kenshin/scripts/start_all_services.sh"

With the script content:

lfile="/mnt/array1/admin/logs/nv_start_all_services.log"

# source /home/kenshin/scripts/profile.sh 2>&1 >> $lfile
source /home/kenshin/scripts/profile.sh
nv_start_all_services 2>&1 >> $lfile

⇒ With this setup in place the uWSGI server is correctly started when the system boots, and will monitor any change on the config files inside /mnt/array1/app_data/uwsgi.

And this is it for the setup of the uWSGI server for django, in the end, this was less complex that I thought it would be ! Now I just hope I could help clarifying some points on how to do it here.

Next thing I need now is to provide support to register/login users with my django app, but this is another story, so let's just stop here for this work session.