Test Speedup: Isolate Modulestore Signals

There are a number of Django Signals that are on the modulestore's
SignalHandler class, such as SignalHandler.course_published. These
signals can trigger very expensive processes to occur, such as course
overview or block structures generation. Most of the time, the test
author doesn't care about these side-effects.

This commit does a few things:

* Converts the signals on SignalHandler to be instances of a new
  SwitchedSignal class, that allows signal sending to be disabled.

* Creates a SignalIsolationMixin helper similar in spirit to the
  CacheIsolationMixin, and adds it to the ModuleStoreIsolationMixin
  (and thus to ModuleStoreTestCase and SharedModuleStoreTestCase).

* Converts our various tests to use this new mechanism. In some cases,
  this means adjusting query counts downwards because they no longer
  have to account for publishing listener actions.

Modulestore generated signals are now muted by default during test runs.
Calls to send() them will result in no-ops. You can choose to enable
specific signals for a given subclass of ModuleStoreTestCase or
SharedModuleStoreTestCase by specifying an ENABLED_SIGNALS class
attribute, like the following example:

    from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase

    class MyPublishTestCase(ModuleStoreTestCase):
        ENABLED_SIGNALS = ['course_published', 'pre_publish']

You should take great care when disabling signals outside of a
ModuleStoreTestCase or SharedModuleStoreTestCase, since they can leak
out into other tests. Be sure to always clean up, and never disable
signals outside of testing. Because signals are essentially process
globals, it can have a lot of unpleasant side-effects if we start
mucking around with them during live requests.

Overall, this change has cut the total test execution time for
edx-platform by a bit over a third, though we still spend a lot in
pre-test setup during our test builds.

[PERF-413]

cms/djangoapps/contentstore/tests/test_gating.py

@@ -13,6 +13,8 @@ class TestHandleItemDeleted(ModuleStoreTestCase, MilestonesTestCaseMixin):
     """
     Test case for handle_score_changed django signal handler
     """
+    ENABLED_SIGNALS = ['course_published']
+
     def setUp(self):
         """
         Initial data setup

cms/djangoapps/contentstore/tests/test_import.py

@@ -181,13 +181,13 @@ class ContentStoreImportTest(SignalDisconnectTestMixin, ModuleStoreTestCase):
         # we try to refresh the inheritance tree for each update_item in the import
         with check_exact_number_of_calls(store, 'refresh_cached_metadata_inheritance_tree', 28):
 
-            # _get_cached_metadata_inheritance_tree should be called twice (once for import, once on publish)
-            with check_exact_number_of_calls(store, '_get_cached_metadata_inheritance_tree', 2):
+            # _get_cached_metadata_inheritance_tree should be called once
+            with check_exact_number_of_calls(store, '_get_cached_metadata_inheritance_tree', 1):
 
                 # with bulk-edit in progress, the inheritance tree should be recomputed only at the end of the import
-                # NOTE: On Jenkins, with memcache enabled, the number of calls here is only 1.
-                #       Locally, without memcache, the number of calls is actually 2 (once more during the publish step)
-                with check_number_of_calls(store, '_compute_metadata_inheritance_tree', 2):
+                # NOTE: On Jenkins, with memcache enabled, the number of calls here is 1.
+                #       Locally, without memcache, the number of calls is 1 (publish no longer counted)
+                with check_number_of_calls(store, '_compute_metadata_inheritance_tree', 1):
                     self.load_test_import_course(create_if_not_present=False, module_store=store)
 
     @ddt.data(ModuleStoreEnum.Type.mongo, ModuleStoreEnum.Type.split)

cms/djangoapps/contentstore/tests/test_orphan.py

@@ -100,8 +100,8 @@ class TestOrphan(TestOrphanBase):
         self.assertIn(unicode(location), orphans)
 
     @ddt.data(
-        (ModuleStoreEnum.Type.split, 9, 6),
-        (ModuleStoreEnum.Type.mongo, 34, 13),
+        (ModuleStoreEnum.Type.split, 9, 5),
+        (ModuleStoreEnum.Type.mongo, 34, 12),
     )
     @ddt.unpack
     def test_delete_orphans(self, default_store, max_mongo_calls, min_mongo_calls):

cms/djangoapps/contentstore/views/tests/test_course_index.py

@@ -331,6 +331,8 @@ class TestCourseOutline(CourseTestCase):
     """
     Unit tests for the course outline.
     """
+    ENABLED_SIGNALS = ['course_published']
+
     def setUp(self):
         """
         Set up the for the course outline tests.
@@ -579,6 +581,8 @@ class TestCourseReIndex(CourseTestCase):
     """
     SUCCESSFUL_RESPONSE = _("Course has been successfully reindexed.")
 
+    ENABLED_SIGNALS = ['course_published']
+
     def setUp(self):
         """
         Set up the for the course outline tests.

cms/djangoapps/contentstore/views/tests/test_gating.py

@@ -17,6 +17,7 @@ class TestSubsectionGating(CourseTestCase):
     Tests for the subsection gating feature
     """
     MODULESTORE = TEST_DATA_SPLIT_MODULESTORE
+    ENABLED_SIGNALS = ['item_deleted']
 
     def setUp(self):
         """