Doing modulestore lookups is expensive, so commit 695b036 created a
course_publish listener that would materialize the discussion ID to
XBlock usage key mapping into the CourseDiscussionSettings model.
However, the signal wasn't hooked up to the Studio process, so that
async task was never called. When hooking it up, I also discovered that
bok choy tests related to partitioning were failing because of a race
condition where multiple processes are overwriting the discussion
settings. To make sure this wasn't an issue, I moved the mapping to
its own table.
This is part of ARCH-111, and the overall Course Structures API
deprecation.
PSA was monolothic, now split, with new features, like
a DB-backed partial pipeline. FB OAuth2 version also upped.
Partial pipelines don't get cleared except when necessary.
They persist for special cases like change of browser while
still mid-pipeline (i.e. email validation step).
Refactor, cleanup, and update of a lot of small things as well.
PLEASE NOTE the new `social_auth_partial` table.
There are a number of Django Signals that are on the modulestore's
SignalHandler class, such as SignalHandler.course_published. These
signals can trigger very expensive processes to occur, such as course
overview or block structures generation. Most of the time, the test
author doesn't care about these side-effects.
This commit does a few things:
* Converts the signals on SignalHandler to be instances of a new
SwitchedSignal class, that allows signal sending to be disabled.
* Creates a SignalIsolationMixin helper similar in spirit to the
CacheIsolationMixin, and adds it to the ModuleStoreIsolationMixin
(and thus to ModuleStoreTestCase and SharedModuleStoreTestCase).
* Converts our various tests to use this new mechanism. In some cases,
this means adjusting query counts downwards because they no longer
have to account for publishing listener actions.
Modulestore generated signals are now muted by default during test runs.
Calls to send() them will result in no-ops. You can choose to enable
specific signals for a given subclass of ModuleStoreTestCase or
SharedModuleStoreTestCase by specifying an ENABLED_SIGNALS class
attribute, like the following example:
from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase
class MyPublishTestCase(ModuleStoreTestCase):
ENABLED_SIGNALS = ['course_published', 'pre_publish']
You should take great care when disabling signals outside of a
ModuleStoreTestCase or SharedModuleStoreTestCase, since they can leak
out into other tests. Be sure to always clean up, and never disable
signals outside of testing. Because signals are essentially process
globals, it can have a lot of unpleasant side-effects if we start
mucking around with them during live requests.
Overall, this change has cut the total test execution time for
edx-platform by a bit over a third, though we still spend a lot in
pre-test setup during our test builds.
[PERF-413]
This specifically enables/disables the underlying comment service client
used to make calls to the service. When disabled, this client will now
throw an exception which can be propagated upwards so that callers can
make the right decision about how to notify users of the error, or
handle retry, etc etc.
Firstly, we're now explicitly instructing the comments service to not
return thread responses/comments if the request isn't AJAX. So, if you
load the URL for a single discussion thread in your browser, this would
be a non-AJAX call and we'll avoid loading the responses for the entire
thread behind-the-scenes. Big win here for large threads.
Next, we removed a redundant "get threads" call which was also happening
behind-the-scenes. This call was redundant as the front-end JS also
grabs the thread list when a topic is chosen, so we were making an
extranenous call for no benefit. Poof, gone!
Finally, we added some caching of database queries that are required to
drive a lot of the permissions/cohorts machinery around discussion.
This will have a minimal effect but introduced a cleaner way to apply
general memoization at the per-request level which will let us further
cache things as we identify them as issues.
We're often grabbing the metadata of a specific thread to then be able
to perform other operations, but we never need the actual responses or
comments of a thread unless we're displaying it in the normal forum
view.
This change sets a default of with_responses=False, which instructs the
comment service to not send back the responses/comments for the given
thread. We only ask for responses in the case of rendering a single
thread or inline discussion.
2. Update COMPREHNSIVE_THEME_DIR to COMPREHENSIVE_THEME_DIRS
3. Update paver commands to support multi theme dirs
4. Updating template loaders
5. Add ENABLE_COMPREHENSIVE_THEMING flag to enable or disable theming via settings
6. Update tests
7. Add backward compatibility for COMPREHEHNSIVE_THEME_DIR
By default, disable all caching in tests, to preserve test independence.
In order to enable caching, inherit from CacheSetupMixin, and specify
which cache configuration is needed.
[EV-32]
