Difference between revisions of "Celery"

From RHS Wiki
Jump to navigation Jump to search
m
Tag: visualeditor
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Install =
+
=Install=
 
  pip install Celery
 
  pip install Celery
== Create Virtual Host and User ==
+
==Create Virtual Host and User==
 
  sudo rabbitmqctl add_user <username> <password>
 
  sudo rabbitmqctl add_user <username> <password>
 
  sudo rabbitmqctl add_vhost <vhostname>
 
  sudo rabbitmqctl add_vhost <vhostname>
 
  sudo rabbitmqctl set_permissions -p <vhostname> <username> ".*" ".*" ".*"
 
  sudo rabbitmqctl set_permissions -p <vhostname> <username> ".*" ".*" ".*"
  
== With RabbitMQ ==
+
==With RabbitMQ==
 
  sudo apt-get install rabbitmq-server
 
  sudo apt-get install rabbitmq-server
 
  # Create rabbitmq user
 
  # Create rabbitmq user
Line 23: Line 23:
 
  $ sudo rabbitmqctl stop
 
  $ sudo rabbitmqctl stop
  
= Workers =
+
=Workers=
== Launch ==
+
==Launch==
 
  celery -A {ProjectName} worker -l info --queues=queuename1,queuename2
 
  celery -A {ProjectName} worker -l info --queues=queuename1,queuename2
 +
 +
== SystemD ==
 +
 +
=== Service File ===
 +
<syntaxhighlight lang="bash">
 +
[Unit]
 +
Description=CIA Celery Scheduler
 +
After=network.target
 +
 +
[Service]
 +
User=tenkaichi
 +
Group=tenkaichi
 +
EnvironmentFile=/etc/default/celerybeat
 +
WorkingDirectory=/home/tenkaichi/CIA
 +
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} beat --loglevel=${CELERYBEAT_LOGLEVEL} --scheduler=${CELERYBEAT_SCHEDULER} --logfile=${CELERYBEAT_LOG_FILE}'
 +
Restart=always
 +
RestartSec=3
 +
 +
[Install]
 +
WantedBy=multi-user.target
 +
</syntaxhighlight>
 +
 +
=== /etc/default/celerybeat ===
 +
<syntaxhighlight lang="bash">
 +
CELERY_BIN="/home/tenkaichi/.virtualenvs/cia/bin/celery"
 +
CELERY_APP="CIA"
 +
CELERYBEAT_CHDIR="/home/tenkaichi/CIA"
 +
CELERYBEAT_SCHEDULER="django_celery_beat.schedulers:DatabaseScheduler"
 +
CELERYBEAT_OPTS=""
 +
CELERYBEAT_LOGLEVEL=debug
 +
CELERYBEAT_LOG_FILE="/home/tenkaichi/CIA/logs/celery_beat.log"
 +
DJANGO_SETTINGS_MODULE="CIA.settings"
 +
</syntaxhighlight>
 +
 +
==Run task at specific time==
 +
<source lang="Python">import datetime
 +
 +
import celery
 +
 +
@celery.shared_task(bind=True)
 +
def add_tag(task, user_id, tag):
 +
    User.objects.filter(id=user_id, tags__ne=tag).update(push__tags=tag)
 +
    return True
 +
 +
user_id = '582ee32a5b9c861c87dc297e'
 +
tag = 'new_tag'
 +
started_at = datetime.datetime(2018, 3, 12, tzinfo=datetime.timezone.utc)
 +
add_tag.apply_async((user_id, tag), eta=started_at)</source>
 +
https://vinta.ws/code/run-a-celery-task-at-a-specific-time.html

Latest revision as of 09:28, 13 January 2020

Install[edit]

pip install Celery

Create Virtual Host and User[edit]

sudo rabbitmqctl add_user <username> <password>
sudo rabbitmqctl add_vhost <vhostname>
sudo rabbitmqctl set_permissions -p <vhostname> <username> ".*" ".*" ".*"

With RabbitMQ[edit]

sudo apt-get install rabbitmq-server
# Create rabbitmq user
sudo rabbitmqctl add_user myuser mypassword
# Create rabbitmq vhost
sudo rabbitmqctl add_vhost myvhost
sudo rabbitmqctl set_user_tags myuser mytag
sudo rabbitmqctl set_permissions -p myvhost myuser ".*" ".*" ".*"
Starting/Stopping the RabbitMQ server
To start the server:
$ sudo rabbitmq-server
you can also run it in the background by adding the -detached option (note: only one dash):
$ sudo rabbitmq-server -detached
Never use kill to stop the RabbitMQ server, but rather use the rabbitmqctl command:
$ sudo rabbitmqctl stop

Workers[edit]

Launch[edit]

celery -A {ProjectName} worker -l info --queues=queuename1,queuename2

SystemD[edit]

Service File[edit]

[Unit]
Description=CIA Celery Scheduler
After=network.target

[Service]
User=tenkaichi
Group=tenkaichi
EnvironmentFile=/etc/default/celerybeat
WorkingDirectory=/home/tenkaichi/CIA
ExecStart=/bin/sh -c '${CELERY_BIN} -A ${CELERY_APP} beat --loglevel=${CELERYBEAT_LOGLEVEL} --scheduler=${CELERYBEAT_SCHEDULER} --logfile=${CELERYBEAT_LOG_FILE}'
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

/etc/default/celerybeat[edit]

CELERY_BIN="/home/tenkaichi/.virtualenvs/cia/bin/celery"
CELERY_APP="CIA"
CELERYBEAT_CHDIR="/home/tenkaichi/CIA"
CELERYBEAT_SCHEDULER="django_celery_beat.schedulers:DatabaseScheduler"
CELERYBEAT_OPTS=""
CELERYBEAT_LOGLEVEL=debug
CELERYBEAT_LOG_FILE="/home/tenkaichi/CIA/logs/celery_beat.log"
DJANGO_SETTINGS_MODULE="CIA.settings"

Run task at specific time[edit]

import datetime

import celery

@celery.shared_task(bind=True)
def add_tag(task, user_id, tag):
    User.objects.filter(id=user_id, tags__ne=tag).update(push__tags=tag)
    return True

user_id = '582ee32a5b9c861c87dc297e'
tag = 'new_tag'
started_at = datetime.datetime(2018, 3, 12, tzinfo=datetime.timezone.utc)
add_tag.apply_async((user_id, tag), eta=started_at)

https://vinta.ws/code/run-a-celery-task-at-a-specific-time.html