7. Integration with Celery

If your project runs in a Celery enabled environment, you can use its worker to send out queued emails. This setup has a big advantage that emails are sent immediately after they are added to the queue. The delivery is performed asynchronously in a separate task to prevent blocking request/response-cycle.

Warning

Current version of sendmail uses Django ORM select_for_update(skip_locked=True) method in celery task for locking sent emails. Not all database backends support it.

“Using select_for_update() on backends which do not support SELECT … FOR UPDATE (such as SQLite) will have no effect. SELECT … FOR UPDATE will not be added to the query, and an error isn’t raised if select_for_update() is used in autocommit mode.” (read more in Django QuerySet documentation)

You should configure celery so that you celery.py setup invokes autodiscover_tasks

Celery must also be enabled in sendmail configurations in settings.py:

SENDMAIL = {
# other settings
'CELERY_ENABLED': True,
}

Now you can start celery worker:

python -m celery -A your_project worker -l info --concurrency=5

Adjust number of concurrent processes to meet your needs.

You should see something like this:

     -------------- celery@mykhailo-Latitude-5540 v5.4.0 (opalescent)
--- ***** -----
-- ******* ---- Linux-6.8.0-45-generic-x86_64-with-glibc2.39 2024-10-08 13:36:49
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         celery_project:0x7e00fa50c710
- ** ---------- .> transport:   redis://localhost:6379//
- ** ---------- .> results:     redis://localhost:6379/
- *** --- * --- .> concurrency: 5 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . sendmail.tasks.cleanup_mail
  . sendmail.tasks.send_queued_mail

send-queued-mail tasks is triggered when email is created. We might want to process queue automatically to sent scheduled emails or retry failed ones. This can be scheduled with a simple Celery beat configuration, for instance through

app.conf.beat_schedule = {
'send-queued-mail': {
    'task': 'sendmail.tasks.send_queued_mail',
    'schedule': 600.0,
    },
}

Now Beat can be started as:

python -m celery -A your_project beat -l info

The email queue now will be processed every 10 minutes. If you are using Django Celery Beat, then use the Django-Admin backend and add a periodic tasks for sendmail.tasks.send_queued_mail.

Depending on your policy, you may also want to remove expired emails from the queue. This can be done by adding another periodic tasks for sendmail.tasks.cleanup_mail, which may run once a week or month.