This is first art from series about Celery 4 configurations and quirks. I will show you how to properly use Celery with Django projects.
At that moment I assume, that you know at least python basics.
Followed by Wikipedia:
Celery is an open source asynchronous task queue or job queue which is based on distributed message passing. While it supports scheduling, its focus is on operations in real time.
This article is about Celery 4.0.2
Celery 4 gives you ability to place all configuration in
settings.py with rest Django settings, but from my experience .. you don’t want to do that. It is much cleaner to keep Celery settings in separate module.
In main project app, next to
settings.py, create new module
Structure should looks similar to this:
proj |-- proj | |-- __init__.py | |-- celery.py | |-- settings.py | |-- tasks.py | |-- urls.py |-- manage.py |-- requirements.txt
I prefer to use Redis as message broker because it is more familiar to me then RabbitMQ, but Celery is ready to work with both of them with the same configuration.
from celery import Celery redis = 'redis://localhost:6379/' app = Celery('proj', broker=redis, backend=redis) app.autodiscover_tasks()
Celery instance is assign to
app variable by convention. Keep your project simple and consistent. Celery instance should be named same as project.
For example if project’s name is “gift-catalogue” then
app = Celery('gift-catalogue', broker=redis, backend=redis)
At least autodiscover method is called to load all tasks.
autodiscover_tasks() searches a list of packages for a
tasks.py module or use related_name argument, more about autodiscover.
Last thing to do. To
proj/proj/__init__.py add this lines to load celery on Django startup:
from proj.celery import app as celery_app __all__ = ['celery_app']
From now on we can call tasks with
from proj import celery_app @celery_app.task def add(a, b): return a + b add.delay(4, 5) add.apply_async(args=(4, 5))
Enable virtualenv with installed celery then be sure you are in root project directory. You have to be one directory above main app directory, exactly where
Run worker with
-A proj argument where
proj is your main django app directory name, where
(venv) $ celery -A proj worker -l info