13 Commits

Author SHA1 Message Date
usamasadiq
8556d670a2 Applied pylint-amnesty 2021-02-02 15:29:41 +05:00
Tim McCormack
0c57a02119 Enforce a Celery singleton across cms and lms by using shared module (#25840)
This should prevent the issues we've seen recently where cms modules are
imported by the running lms process, resulting in two celery instances
being created and tasks intermittently being registered to the wrong
instance (and therefore effectively lost.)

In commit ab6bf348d4/PR #25822 we tried to ensure that only one or the
other of the instances was created by adding a startup check.
Unfortunately, there's an external shared library that refers directly
to the lms celery, causing a startup failure in cms, so we had to revert
it. Rather than waiting to fix that library, this commit collapses
the two instances together so that there is only ever one.
2020-12-10 17:54:31 +00:00
Tim McCormack
a9bcf22169 Revert "Only instantiate one celery app per process." 2020-12-10 14:14:42 +00:00
Feanil Patel
ab6bf348d4 Only instantiate one celery app per process.
Ticket: BOM-2086

Currently there are parts of the LMS that import content from the CMS
APP and vice-versa.  When this happens, we end up with 2 instances of
the celery app and some tasks get registered to the wrong one. The
tasks that were getting registered to the wrong one are never able to
run and result in lots of production errors on celery workers.

The timing of the CMS celery app instantiation is non deterministic
so different tasks get lost depending on when it's imported by some
code in the LMS.

As long as SERVICE_VARIANT is set, this code should prevent the
instantiation of both celery apps.
2020-12-09 14:13:40 -05:00
Kyle McCormick
d21a330d0a Tell isort not to sort certain imports in cms/ 2020-10-19 09:34:31 -04:00
Feanil Patel
9cf2f9f298 Run 2to3 -f future . -w
This will remove imports from __future__ that are no longer needed.

https://docs.python.org/3.5/library/2to3.html#2to3fixer-future
2019-12-30 10:35:30 -05:00
David Ormsbee
294bbe7fe0 Test speedup: Monkey patch Kombu entrypoints fn.
We monkey patch Kombu's entrypoints listing because scanning through
this accounts for the majority of LMS/Studio startup time for tests,
and we don't use custom Kombu serializers (which is what this is for).
Still, this is pretty evil, and should be taken out when we update
Celery to the next version where it looks like this method of custom
serialization has been removed.

FWIW, this is identical behavior to what happens in Kombu if
pkg_resources isn't available.
2017-03-14 13:45:40 -04:00
Feanil Patel
6289b190d9 Fix pep8 and pylint. 2015-05-08 09:01:50 -04:00
Feanil Patel
27223ae80f Load celery for CMS as well. 2015-05-08 09:01:48 -04:00
Calen Pennington
cfae1cdf62 Pep8 autofixes 2013-02-06 11:13:50 -05:00
Calen Pennington
7d8095cd97 Initial port of templates into filesystem based storage 2012-09-25 10:10:44 -04:00
Calen Pennington
2554413291 Hook up link to create modules from templates 2012-09-25 10:10:44 -04:00
Calen Pennington
8134bcbc3d Get the cms up to the point of rendering a template 2012-06-08 14:01:41 -04:00