Andal.LND/edx-platform
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Only instantiate one celery app per process.

Browse Source 
Ticket: BOM-2086

Currently there are parts of the LMS that import content from the CMS
APP and vice-versa.  When this happens, we end up with 2 instances of
the celery app and some tasks get registered to the wrong one. The
tasks that were getting registered to the wrong one are never able to
run and result in lots of production errors on celery workers.

The timing of the CMS celery app instantiation is non deterministic
so different tasks get lost depending on when it's imported by some
code in the LMS.

As long as SERVICE_VARIANT is set, this code should prevent the
instantiation of both celery apps.
This commit is contained in:
Feanil Patel
2020-12-09 14:13:40 -05:00
parent 643fbe82a0
commit ab6bf348d4
2 changed files with 6 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
cms/__init__.py
View File

@@ -5,7 +5,7 @@ registration and discovery can work correctly.
Import sorting is intentionally disabled in this module.
isort:skip_file
"""
import os
# We monkey patch Kombu's entrypoints listing because scanning through this
# accounts for the majority of LMS/Studio startup time for tests, and we don't
@@ -20,4 +20,5 @@ kombu.utils.entrypoints = lambda namespace: iter([])
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import APP as CELERY_APP
if os.environ.get('SERVICE_VARIANT', 'cms').startswith('cms'):
from .celery import APP as CELERY_APP

4
lms/__init__.py
View File

@@ -2,6 +2,7 @@
Celery needs to be loaded when the cms modules are so that task
registration and discovery can work correctly.
"""
import os
# We monkey patch Kombu's entrypoints listing because scanning through this
# accounts for the majority of LMS/Studio startup time for tests, and we don't
@@ -16,4 +17,5 @@ kombu.utils.entrypoints = lambda namespace: iter([])
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import APP as CELERY_APP
if os.environ.get('SERVICE_VARIANT', 'lms').startswith('lms'):
from .celery import APP as CELERY_APP