Difference between revisions of "Celery"
Jump to navigation
Jump to search
Rafahsolis (talk | contribs) (Created page with "== Install == pip install Celery") |
Rafahsolis (talk | contribs) m Tag: visualeditor |
||
| (8 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | == | + | =Install= |
| − | + | pip install Celery | |
| + | ==Create Virtual Host and User== | ||
| + | sudo rabbitmqctl add_user <username> <password> | ||
| + | sudo rabbitmqctl add_vhost <vhostname> | ||
| + | sudo rabbitmqctl set_permissions -p <vhostname> <username> ".*" ".*" ".*" | ||
| + | |||
| + | ==With RabbitMQ== | ||
| + | sudo apt-get install rabbitmq-server | ||
| + | # Create rabbitmq user | ||
| + | sudo rabbitmqctl add_user myuser mypassword | ||
| + | # Create rabbitmq vhost | ||
| + | sudo rabbitmqctl add_vhost myvhost | ||
| + | sudo rabbitmqctl set_user_tags myuser mytag | ||
| + | sudo rabbitmqctl set_permissions -p myvhost myuser ".*" ".*" ".*" | ||
| + | |||
| + | Starting/Stopping the RabbitMQ server | ||
| + | To start the server: | ||
| + | $ sudo rabbitmq-server | ||
| + | you can also run it in the background by adding the -detached option (note: only one dash): | ||
| + | $ sudo rabbitmq-server -detached | ||
| + | Never use kill to stop the RabbitMQ server, but rather use the rabbitmqctl command: | ||
| + | $ sudo rabbitmqctl stop | ||
| + | |||
| + | =Workers= | ||
| + | ==Launch== | ||
| + | celery -A {ProjectName} worker -l info --queues=queuename1,queuename2 | ||
| + | |||
| + | == SystemD == | ||
| + | |||
| + | === Service File === | ||
| + | <syntaxhighlight lang="bash"> | ||
| + | [Unit] | ||
| + | Description=CIA Celery Scheduler | ||
| + | After=network.target | ||
| + | |||
| + | [Service] | ||
| + | User=tenkaichi | ||
| + | Group=tenkaichi | ||
| + | EnvironmentFile=/etc/default/celerybeat | ||
| + | WorkingDirectory=/home/tenkaichi/CIA | ||
| + | ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} beat --loglevel=${CELERYBEAT_LOGLEVEL} --scheduler=${CELERYBEAT_SCHEDULER} --logfile=${CELERYBEAT_LOG_FILE}' | ||
| + | Restart=always | ||
| + | RestartSec=3 | ||
| + | |||
| + | [Install] | ||
| + | WantedBy=multi-user.target | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | === /etc/default/celerybeat === | ||
| + | <syntaxhighlight lang="bash"> | ||
| + | CELERY_BIN="/home/tenkaichi/.virtualenvs/cia/bin/celery" | ||
| + | CELERY_APP="CIA" | ||
| + | CELERYBEAT_CHDIR="/home/tenkaichi/CIA" | ||
| + | CELERYBEAT_SCHEDULER="django_celery_beat.schedulers:DatabaseScheduler" | ||
| + | CELERYBEAT_OPTS="" | ||
| + | CELERYBEAT_LOGLEVEL=debug | ||
| + | CELERYBEAT_LOG_FILE="/home/tenkaichi/CIA/logs/celery_beat.log" | ||
| + | DJANGO_SETTINGS_MODULE="CIA.settings" | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | ==Run task at specific time== | ||
| + | <source lang="Python">import datetime | ||
| + | |||
| + | import celery | ||
| + | |||
| + | @celery.shared_task(bind=True) | ||
| + | def add_tag(task, user_id, tag): | ||
| + | User.objects.filter(id=user_id, tags__ne=tag).update(push__tags=tag) | ||
| + | return True | ||
| + | |||
| + | user_id = '582ee32a5b9c861c87dc297e' | ||
| + | tag = 'new_tag' | ||
| + | started_at = datetime.datetime(2018, 3, 12, tzinfo=datetime.timezone.utc) | ||
| + | add_tag.apply_async((user_id, tag), eta=started_at)</source> | ||
| + | https://vinta.ws/code/run-a-celery-task-at-a-specific-time.html | ||
Latest revision as of 09:28, 13 January 2020
Install[edit]
pip install Celery
Create Virtual Host and User[edit]
sudo rabbitmqctl add_user <username> <password> sudo rabbitmqctl add_vhost <vhostname> sudo rabbitmqctl set_permissions -p <vhostname> <username> ".*" ".*" ".*"
With RabbitMQ[edit]
sudo apt-get install rabbitmq-server # Create rabbitmq user sudo rabbitmqctl add_user myuser mypassword # Create rabbitmq vhost sudo rabbitmqctl add_vhost myvhost sudo rabbitmqctl set_user_tags myuser mytag sudo rabbitmqctl set_permissions -p myvhost myuser ".*" ".*" ".*"
Starting/Stopping the RabbitMQ server To start the server: $ sudo rabbitmq-server you can also run it in the background by adding the -detached option (note: only one dash): $ sudo rabbitmq-server -detached Never use kill to stop the RabbitMQ server, but rather use the rabbitmqctl command: $ sudo rabbitmqctl stop
Workers[edit]
Launch[edit]
celery -A {ProjectName} worker -l info --queues=queuename1,queuename2
SystemD[edit]
Service File[edit]
[Unit]
Description=CIA Celery Scheduler
After=network.target
[Service]
User=tenkaichi
Group=tenkaichi
EnvironmentFile=/etc/default/celerybeat
WorkingDirectory=/home/tenkaichi/CIA
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} beat --loglevel=${CELERYBEAT_LOGLEVEL} --scheduler=${CELERYBEAT_SCHEDULER} --logfile=${CELERYBEAT_LOG_FILE}'
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
/etc/default/celerybeat[edit]
CELERY_BIN="/home/tenkaichi/.virtualenvs/cia/bin/celery"
CELERY_APP="CIA"
CELERYBEAT_CHDIR="/home/tenkaichi/CIA"
CELERYBEAT_SCHEDULER="django_celery_beat.schedulers:DatabaseScheduler"
CELERYBEAT_OPTS=""
CELERYBEAT_LOGLEVEL=debug
CELERYBEAT_LOG_FILE="/home/tenkaichi/CIA/logs/celery_beat.log"
DJANGO_SETTINGS_MODULE="CIA.settings"
Run task at specific time[edit]
import datetime
import celery
@celery.shared_task(bind=True)
def add_tag(task, user_id, tag):
User.objects.filter(id=user_id, tags__ne=tag).update(push__tags=tag)
return True
user_id = '582ee32a5b9c861c87dc297e'
tag = 'new_tag'
started_at = datetime.datetime(2018, 3, 12, tzinfo=datetime.timezone.utc)
add_tag.apply_async((user_id, tag), eta=started_at)
https://vinta.ws/code/run-a-celery-task-at-a-specific-time.html