How to speed up tests with Django and PostgreSQL

5comments

20th July 2015 in Coding Tags: django, python, test

Over time I realized that waiting more than 15 minutes for my entire test suite to complete was very, very wrong. I've neglected spending time on fine tuning end eningeering tests and now each time I had to run some tests I found myself annoyed, bored, distracted. Quite a productivity killer.

Here's how I achieved a drastic speed up of an entire test suite (roughly 300 tests) of an open source django project designed for crowdsourcing geographic data called nodeshot, which has many database tables running on PostgreSQL (9.1) and Postgis 2.

Hopefully you'll be able to replicate this setup and enjoy the same benefits I'm enjoying now!

Measure execution time

First of all I measured my tests by importing this module from the main manage.py, which adds two flags to the test command: "--time" and "--detailed", which output the execution time of each test class and each single test function.

I noticed that there were tests which were taking many seconds to complete, but most tests were not very slow. So I understood that I needed to act on both fronts: in general, by speeding up the database creation procedure and in detail by working on specifc slow test functions.

Tune your PostgreSQL configuration

I tried out quite a few settings, I tried UNLOGGED tables, I even tried to run the database in memory only, but i'll save you time and get straight to the point, the three settings that really made a difference for me were the following ones:

# /etc/postgresql/9.1/main/postgresql.conf
fsync = off
synchronous_commit = off
full_page_writes = off

With these settings alone the total time taken to execute the tests was less than half.

Keep in mind these settings are for development only, I'm assuming you won't care if you lose data in your development environment.

Find out up-to-date information about PostgreSQL non durable options.

Update your project to django 1.8

Django 1.8 has a new built in feature in the django-admin command:

--keepdb

This flag will avoid destroying the database at the end of each run, which will drastically speed up test execution time. Another good consequence of using this flag is that tests will start really quickly, so running specific test classes for specific modules will take only seconds, which is obviously a great news if you're doing TDD or refactoring your code often.

Find out more about keepdb flag in django 1.8

Tune your django test settings

I fine tuned the default django settings for tests to keep only a few essential middlewares, use the md5 password hasher, and set the celery broker to "memory".

This is more a micro optimization compared to the previous boosts.

Use mocking whenever possible

I had a few test classes extending LiveServerTestCase in order to perform http requests to the development server, these test classes took a much longer time to complete than the others.

I refactored these tests to mock http responses, the main tool that I used is the great responses library.

Results

Now the entire test suite takes about 50 seconds to complete, while specific tests for specific modules take seconds, the result is obvious: enhanced productivity, faster refactoring.

While working on this issue I also discussed about the topic in the django-developers mailing list, which in turn resulted in this small addition to the django documentation.

I hope I saved some of your time, I hope as well that you'll do better than me and share your results with the open source community.

Retweet

Comments

  1. 1.

    Ryan said:

    ( on 20th of September 2015 at 19:27 )

    IMHO, making migrations mandatory has seriously damaged the ability to do TDD in Django. Using --keepdb can reduce the time *between* tests but in my experience it doesn't decrease the time to *start* your test suite which has now become much longer due to the need to run do migrations first. Or perhaps I'm just doing it wrong, I don't know. All I know is it now takes much longer to run my tests.

  2. 2.

    Federico said:

    ( on 23rd of September 2015 at 16:39 )

    Even though I agree that mandatory migrations have complicated things quite a bit, I believe you must be doing something wrong, because my tests now run very fast.

  3. 3.

    Facundo said:

    ( on 11th of December 2015 at 00:00 )

    Thank you! With those settings for Postgres my tests now run in 10 seconds instead of 40.

  4. 4.

    Federico said:

    ( on 11th of December 2015 at 17:04 )

    @Facundo: glad it helped you achieving a good result! :-)

  5. 5.

    Alois said:

    ( on 24th of February 2016 at 15:18 )

    Thanks !

Leave your comment

Categories

Let's be social

Popular posts

Latest Comments

  1. Great news Aymará! Very happy to know this post has inspired you to experiment :-)

    By Federico Capoano in First DjangoGirls Rome wrap-up & afterthoughts

  2. Hi!! I'm a Django Girls coach too. Here, in Argentina, made just what you suggested, splited the workshop in two days. The experiment went just great! Most of the girls achieved to publish the blog from ground 0. It feels great to be helpfull ...

    By Aymará in First DjangoGirls Rome wrap-up & afterthoughts

  3. Send any question to the interop-dev mailing list or open an issue on github.

    By Federico in Network Topology Visualizer: django-netjsongraph

  4. I have a question about Network Topology Collector, can you brief me pls?

    By Nasrin Akter in Network Topology Visualizer: django-netjsongraph

  5. Thanks !

    By Alois in How to speed up tests with Django and PostgreSQL

Popular Tags

battlemesh censorship creativity criptography django event google-summer-of-code ibiza inspiration javascript jquery linux nemesisdesign netjson ninux nodeshot open-source openwisp openwrt performance photo programming python security staticgenerator talk upload wireless-community