abidibo.net

search

Deploy django applications with nginx, uwsgi, virtualenv, south, git and fabric, part 5

deploy django fabric git nginx programming south uwsgi virtualenv

This is the fifth and last part of the django deploy environment, you may find the fourth part here.

In this part we'll see how to automate the deployment using fabric.

What the hell have fabric to do?

Well, fabric have to do all the work for me in the sense that once I've done some changes to my project and pushed them to the bare repository I want to update the production environment with only one command. I'd like also to setup initially the whole environment with only one command.

How does it work?

First of all install it, so with the virtualenv activated:

$ pip install fabric

Done.

Now reading from the official site:

Fabric is a Python (2.5 or higher) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks. It provides a basic suite of operations for executing local or remote shell commands (normally or via sudo) and uploading/downloading files, as well as auxiliary functionality such as prompting the running user for input, or aborting execution.

Fabric provides a command line tool (fab) which executes the tasks indicated in a fabfile.py. Such file contains all the operation to do in remote (and also in locale if desired) in order to update the environment.

Setup the environment with fabric

When doing the initial setup we have to do the following operations (assuming that the remote virtualhost is ready as the remote user and the database, see previous posts):

  • create all the directories we need
  • clone the bare erpository in remote
  • checkout the latest version
  • install all the requirement (do you remember pip freeze?)

Deploy to remote with fabric

In this case we have to perform the following operations:

  • checkout the latest revision
  • install the requierments (if new ones are necessary)
  • doing some symlinks (we keep trace of a previous release if we want to do a rollback)
  • migrate the database (South)
  • restart the webserver and uwsgi

The fabfile.py

All these things are achieved with the following fabfile (customize paths users etc...):

  1. from fabric.api import *
  2. # Default release is 'current'
  3. env.release = 'current'

  5. def production():
  6.   """Production server settings"""
  7.   env.settings = 'production'
  8.   env.user = 'myproject'
  9.   env.path = '/home/%(user)s/sites/myproject' % env
  10.   env.hosts = ['mydomain.com']

  12. def setup():
  13.   """
  14.   Setup a fresh virtualenv and install everything we need so it's ready to deploy to
  15.   """
  16.   run('mkdir -p %(path)s; cd %(path)s; virtualenv --no-site-packages .; mkdir releases; mkdir shared;' % env)
  17.   clone_repo()
  18.   checkout_latest()
  19.   install_requirements()

  21. def deploy():
  22.   """Deploy the latest version of the site to the server and restart nginx"""
  23.   checkout_latest()
  24.   install_requirements()
  25.   symlink_current_release()
  26.   migrate()
  27.   restart_server()

  29. def clone_repo():
  30.   """Do initial clone of the git repo"""
  31.   run('cd %(path)s; git clone /home/%(user)s/git/repositories/myproject.git repository' % env)

  33. def checkout_latest():
  34.   """Pull the latest code into the git repo and copy to a timestamped release directory"""
  35.   import time
  36.   env.release = time.strftime('%Y%m%d%H%M%S')
  37.   run("cd %(path)s/repository; git pull origin master" % env)
  38.   run('cp -R %(path)s/repository %(path)s/releases/%(release)s; rm -rf %(path)s/releases/%(release)s/.git*' % env)

  40. def install_requirements():
  41.   """Install the required packages using pip"""
  42.   run('cd %(path)s; %(path)s/bin/pip install -r ./releases/%(release)s/requirements.txt' % env)

  44. def symlink_current_release():
  45.   """Symlink our current release, uploads and settings file"""
  46.   with settings(warn_only=True):
  47.     run('cd %(path)s; rm releases/previous; mv releases/current releases/previous;' % env)
  48.   run('cd %(path)s; ln -s %(release)s releases/current' % env)
  49.   """ production settings"""
  50.   run('cd %(path)s/releases/current/; cp settings_%(settings)s.py myproject/settings.py' % env)
  51.   with settings(warn_only=True):
  52.     run('rm %(path)s/shared/static' % env)
  53.     run('cd %(path)s/releases/current/static/; ln -s %(path)s/releases/%(release)s/static %(path)s/shared/static ' %env)

  55. def migrate():
  56.   """Run our migrations"""
  57.   run('cd %(path)s/releases/current; ../../bin/python manage.py syncdb --noinput --migrate' % env)

  59. def rollback():
  60.   """
  61.   Limited rollback capability. Simple loads the previously current
  62.   version of the code. Rolling back again will swap between the two.
  63.   """
  64.   run('cd %(path)s; mv releases/current releases/_previous;' % env)
  65.   run('cd %(path)s; mv releases/previous releases/current;' % env)
  66.   run('cd %(path)s; mv releases/_previous releases/previous;' %env)
  67.   restart_server()

  69. def restart_server():
  70.   """Restart the web server"""
  71.   with settings(warn_only=True):
  72.     sudo('kill -9 `cat /tmp/project-master_helpmamme.pid`')
  73.     sudo('rm /tmp/project-master_helpmamme.pid /tmp/uwsgi_helpmamme.sock')
  74.   run('cd %(path)s/releases/current; %(path)s/bin/uwsgi --ini %(path)s/releases/current/uwsgi.ini' % env)
  75.   sudo('/etc/init.d/nginx restart')

Now the good part.

Setup the production environment

Just cd into the locale project container folder, and

$ fab production setup

Just insert the required password when needed and the environment is set up.

What does it happens?

  • the function production is called and some variables are defined, in particular the host, the path to the project container folder and the user with ssh access.
  • the function setup is called, so:
    • the previous path is created
    • a new virtualenv is created inside path
    • two directories: releases and shared are created inside path
    • the bare repository is cloned inside path/repository
    • the new repository is updated (pull) looking at the bare one, and then copied into a subfolder of path/releases which name contains the actual datetime (the .git directory is removed).
    • all the project requirements are installed reading the requirements.txt.

So now we have the bare repository connected to the remote project repository and the locale repository that allow to keep the project syncronized with the local version. We have a releases folder which contains the last release of the project. We have our new virtualen with all the project dependencies installed.

Submit changes to the production environment

Just type

fab production deploy

Done!

What does it happens?

  • the function production, same as above.
  • the function deploy is called, so:
    • the new repository is updated (pull) looking at the bare one, and then copied into a subfolder of path/releases which name contains the actual datetime (the .git directory is removed).
    • all the project requirements are installed reading the requirements.txt (if something has changed since the setup).
    • a symbolic link is created in order to serve the last project version at the path releases/current. At the same time the current release (if any) is moved to releases/previous, and the previous (if any) is deleted.
    •  the settings.py is overwritten by the production_settings.py
    • a symbolic link is created in order to serve the current release static folder at the path shared (the old shared symlink is deleted first)
    • the database is migrated with no input (only the database structure is touched)
    • the web server is restarted and so the uwsgi application

So if all goes well the new release is up and running.

What if something went wrong?

Don't worry, just perform a rollback:

$ fab production rollback

and the previous version is restored

What's next

Luckily this series of posts ends here. Surely I'll write a summary post which will contain all the main arguments written in these five episodes with links to the specific parts. For questions problems etc... feel free to comment here.

What if we want to set up the project on another machine?
We may work from home and office for example, if you're interested read here.

Bibliography

Such series of posts came from a wild google search, there are some good articles about these stuffs and all gets out in the first google results, I've learned much from any of them, above all for the generation of the fabfile and the creation of the entire architecture. Then I've tried to collect all these things together and write something more recent than the used sources.

See also

Subscribe to abidibo.net!

If you want to stay up to date with new contents published on this blog, then just enter your email address, and you will receive blog updates! You can set you preferences and decide to receive emails only when articles are posted regarding a precise topic.

I promise, you'll never receive spam or advertising of any kind from this subscription, just content updates.

Subscribe to this blog

Comments are welcome!

blog comments powered by Disqus

Your Smartwatch Loves Tasker!

Your Smartwatch Loves Tasker!

Now available for purchase!

Featured

Django admin and bootstrap 5

Bootstrap 5 has come, currently in beta release, and seems already very stable.

So the question is: are you looking for ...

About code optimization, learn from exercises

Let's see an example of exercise you can face during a job interview, and let's use it to understand some ...

Notes on the Pearson correlation coefficient

The Pearson correlation coefficient is a measure of the linear correlation between two variables X and Y. It has a ...