| Version: | 2.3.0a1 |
|---|---|
| Web: | http://celeryproject.org/ |
| Download: | http://pypi.python.org/pypi/celery/ |
| Source: | http://github.com/ask/celery/ |
| Keywords: | task queue, job queue, asynchronous, rabbitmq, amqp, redis, python, webhooks, queue, distributed |
--
Celery is an open source asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
The execution units, called tasks, are executed concurrently on one or more worker nodes using multiprocessing, Eventlet or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
Celery is used in production systems to process millions of tasks a day.
Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks.
The recommended message broker is RabbitMQ, but limited support for Redis, Beanstalk, MongoDB, CouchDB and databases (using SQLAlchemy or the Django ORM) is also available.
Celery is easy to integrate with Django, Pylons and Flask, using the django-celery, celery-pylons and Flask-Celery add-on packages.
This is a high level overview of the architecture.
The broker delivers tasks to the worker nodes. A worker node is a networked machine running celeryd. This can be one or more machines depending on the workload.
The result of the task can be stored for later retrieval (called its "tombstone").
You probably want to see some code by now, so here's an example task adding two numbers:
from celery.task import task
@task
def add(x, y):
return x + y
You can execute the task in the background, or wait for it to finish:
>>> result = add.delay(4, 4) >>> result.wait() # wait for and return the result 8
Simple!
The latest documentation with user guides, tutorials and API reference is hosted at Github.
You can install Celery either via the Python Package Index (PyPI) or from source.
To install using pip,:
$ pip install Celery
To install using easy_install,:
$ easy_install Celery
Download the latest version of Celery from http://pypi.python.org/pypi/celery/
You can install it by doing the following,:
$ tar xvfz celery-0.0.0.tar.gz $ cd celery-0.0.0 $ python setup.py build # python setup.py install # as root
You can clone the repository by doing the following:
$ git clone git://github.com/ask/celery.git
For discussions about the usage, development, and future of celery, please join the celery-users mailing list.
Come chat with us on IRC. The #celery channel is located at the Freenode network.
If you have any suggestions, bug reports or annoyances please report them to our issue tracker at http://github.com/ask/celery/issues/
http://wiki.github.com/ask/celery/
Development of celery happens at Github: http://github.com/ask/celery
You are highly encouraged to participate in the development of celery. If you don't like Github (for some reason) you're welcome to send regular patches.
Be sure to also read the Contributing to Celery section in the documentation.
This software is licensed under the New BSD License. See the LICENSE
file in the top distribution directory for the full license text.