Commit Graph

109 Commits

Author SHA1 Message Date
Nimisha Asthagiri
e7ea1b47e5 Merge pull request #14572 from edx/beryl/cbs_config
Storage-backed versioned Block Structures: Config
2017-02-23 17:37:41 -05:00
David Ormsbee
8423ecb1c8 Student Dashboard CourseOverviews with one query.
Pre-load the course overviews attached to CourseEnrollments on the
Student Dashboard, if possible. This will only grab the CourseOverviews
that already exist, and will not generate new ones. Any missing
CourseOverviews fall back to the lazily-created one-at-a-time behavior
they've always had. That's mostly because I wanted to optimize for the
common case in the least invasive way possible, and I don't want to get
caught up in locking issues.
2017-02-23 15:50:44 -05:00
David Ormsbee
2051c90924 Test Speedup: Isolate Modulestore Signals
There are a number of Django Signals that are on the modulestore's
SignalHandler class, such as SignalHandler.course_published. These
signals can trigger very expensive processes to occur, such as course
overview or block structures generation. Most of the time, the test
author doesn't care about these side-effects.

This commit does a few things:

* Converts the signals on SignalHandler to be instances of a new
  SwitchedSignal class, that allows signal sending to be disabled.

* Creates a SignalIsolationMixin helper similar in spirit to the
  CacheIsolationMixin, and adds it to the ModuleStoreIsolationMixin
  (and thus to ModuleStoreTestCase and SharedModuleStoreTestCase).

* Converts our various tests to use this new mechanism. In some cases,
  this means adjusting query counts downwards because they no longer
  have to account for publishing listener actions.

Modulestore generated signals are now muted by default during test runs.
Calls to send() them will result in no-ops. You can choose to enable
specific signals for a given subclass of ModuleStoreTestCase or
SharedModuleStoreTestCase by specifying an ENABLED_SIGNALS class
attribute, like the following example:

    from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase

    class MyPublishTestCase(ModuleStoreTestCase):
        ENABLED_SIGNALS = ['course_published', 'pre_publish']

You should take great care when disabling signals outside of a
ModuleStoreTestCase or SharedModuleStoreTestCase, since they can leak
out into other tests. Be sure to always clean up, and never disable
signals outside of testing. Because signals are essentially process
globals, it can have a lot of unpleasant side-effects if we start
mucking around with them during live requests.

Overall, this change has cut the total test execution time for
edx-platform by a bit over a third, though we still spend a lot in
pre-test setup during our test builds.

[PERF-413]
2017-02-23 10:31:16 -05:00
Nimisha Asthagiri
30b5822b0e Storage-backed versioned Block Structures: Config 2017-02-23 09:04:53 -05:00
Douglas Hall
58f0154ee2 Add support for multi-org sites
WL-926
2017-02-13 19:33:47 -05:00
Douglas Hall
450d35a799 Revert "Added instructor info in course overview."
This reverts commit 180cf299a3.
2017-02-03 11:30:17 -05:00
Hasnain
180cf299a3 Added instructor info in course overview. 2017-02-01 15:19:52 +05:00
Eric Fischer
0dd617897b Log course on no-retry update_course_in_cache errors 2017-01-25 12:41:39 -05:00
Nimisha Asthagiri
784ef04221 Merge pull request #14359 from edx/beryl/command_generate_cb_tasks
Update generate_course_blocks management command to enqueue tasks.
2017-01-24 17:24:30 -05:00
Nimisha Asthagiri
c04f4401dc Update generate_course_blocks management command to enqueue tasks 2017-01-24 09:11:57 -05:00
Nimisha Asthagiri
408ed01b1d Block Structure: Don't invalidate immediately upon course publish
Introduces the use of Waffle to put the change behind a Switch.
TNL-6323
2017-01-23 11:27:30 -05:00
Cliff Dyer
6d0af9d423 Merge pull request #14157 from edx/efischer/tnl-5799
update_course_in_cache error handling cleanup.
2017-01-04 12:40:27 -08:00
Gregory Martin
1b2fbbd61d delete datetext functions, implement dateutils 2017-01-03 10:18:26 -05:00
Calen Pennington
47e21ca5b0 Revert "Merge pull request #14078 from edx/yro_remove-datetimetext-functions"
This reverts commit 8c0098812d, reversing
changes made to 5b6e2dd5ee.
2016-12-22 09:29:33 -05:00
J. Cliff Dyer
9259156565 review response. 2016-12-20 14:00:47 -05:00
Gregory Martin
b0bf19a741 delete datetext functions, implement dateutils 2016-12-20 11:10:06 -05:00
Eric Fischer
9d545eca4c update_course_in_cache exception handler cleanup 2016-12-15 14:59:24 -05:00
Gregory Martin
f0cd29f02a Revert "Merge pull request #13915 from edx/revert-13794-yro_implement-dateutil"
This reverts commit d59ab18b27, reversing
changes made to 0ebab35e89.
2016-11-07 17:04:13 -05:00
Jesse Zoldak
ec880138d2 Revert "Implement DateUtils in LMS" 2016-11-04 07:50:26 -04:00
Gregory Martin
3fd16eb6be Implement DateUtils in LMS 2016-11-03 10:06:10 -04:00
Eric Fischer
8f0134b7e4 update_course_in_cache should retry on exceptions
Previously, retry() was never called, so the task was never retried despite
defining several retry-related variables.
2016-10-19 14:09:28 -04:00
John Eskew
3614eb2800 Move xmodule_django app to openedx.core.djangoapps 2016-10-06 14:50:40 -04:00
J. Cliff Dyer
1371739f45 Register block_structure djangoapp.
TNL-5274
2016-09-06 11:42:38 -04:00
Nimisha Asthagiri
b37d2d739a Make BlockStructure tasks more resilient against failures.
https://openedx.atlassian.net/browse/TNL-5041
2016-09-02 08:02:49 -04:00
Calen Pennington
bc20a37c2b Simplify how shards are selected in CI 2016-08-03 16:23:02 -04:00
Kevin Kim
0bf8fc4b44 Converts the dates on the dashboard, sidebar navigation, and important course dates to user specified time zone. 2016-07-05 17:45:13 +00:00
E. Kolpakov
33b3dfcc92 Converts Discussion XModule to Discussion XBlock
* Renames discussion_module to discussion_xblock
* Moves common/lib/xmodule/xmodule_discussion to openedx/core/lib/xblock_builtin/xblock_discussion
2016-06-29 06:47:04 +12:00
Nimisha Asthagiri
8513e823bc Add block_metadata_utils 2016-06-23 20:22:33 -04:00
Nimisha Asthagiri
57dde6516a Update ENABLED_CACHES values 2016-06-23 20:22:33 -04:00
Nimisha Asthagiri
f016647fef Enable Block Transformer Registry in tests 2016-06-23 20:22:31 -04:00
Eric Fischer
fdc6d91588 update_in_cache on lms worker (#12689)
This commit "undoes"a previous hotfix, and allows a cms course_publish
signal to trigger a block_structure update_course_in_cache task, which
is run on an lms worker queue.

Changes:
    -exposes ALTERNATE_QUEUE_ENVS
    -adds routing layer in celery.py
    -moves prior dev_with_worker settings file to devstack_with_worker
    -moves course_block api functionality into openedx/core/djangoapps/content/block_structure
2016-06-16 18:01:06 -04:00
Nimisha Asthagiri
7da42ec769 Hotfix: Revert block structure caching changes 2016-05-25 13:26:16 -04:00
Nimisha Asthagiri
d37499c29d Hotfix for BlockStructure data not being invalidated upon Course Publish. 2016-05-20 16:21:06 -04:00
Calen Pennington
853bfe7a36 Add a TestCase mixin for enabling caches in tests
By default, disable all caching in tests, to preserve test independence.
In order to enable caching, inherit from CacheSetupMixin, and specify
which cache configuration is needed.

[EV-32]
2016-05-04 14:51:30 -04:00
Ben Patterson
b9558a90f1 Create a 3rd explicit shard for unit tests.
More accurately, this creates a 4th shard because the
last shard is always the default.
2016-04-28 10:09:32 -04:00
Ned Batchelder
2e1238b83e Remove unused imports 2016-04-08 11:23:05 -04:00
Clinton Blackburn
1185fff1ec Added pacing field Course API
ECOM-3994
2016-03-30 14:36:43 -04:00
Toby Lawrence
85304b8b9d Make CDNifying of course over image URLs only happen for relative URLs.
We don't want to blindly assemble the base CDN URL with whatever an
image URL happens to be, since it might be an absolute URL and now the
result is a broken URL.  We take a more selective approach now.
2016-03-22 13:56:40 -04:00
Ben Patterson
c504029b39 Create a third shard for lms unit tests.
This will leverage the nose attrib plugin to allow us
to break tests apart on our build system to run
in parallel on separate nodes.
2016-03-17 12:01:42 -04:00
Kevin Falcone
631bfc08a3 Migrate correctly from scratch also
Unfortunately, instrospection.get_table_description runs
select * from course_overview_courseoverview, which of course
does not exist while django is calculating initial migrations, causing
this to fail.  Additionally, sqlite does not support information_schema,
but does not do a select * from the table.

Lift the main part of mysql's get_table_description up to the migration itself
and just inspect it directly.  Continue to call the API for sqlite.
2016-03-04 11:46:25 -05:00
Kevin Falcone
7a129b0e9f Noop the 0008 and conditional the 0009 migration
We removed a column in the same release that we removed it
from the model.  This creates a gap where the code still looks for
a column which has been dropped until the new code has been deployed.

The initial fix was to put the column back, but that creates a window
during the alterations where views will error.

This noops the 0008 migration and effectively noops 0009 unless you've
run the old migration.
2016-02-26 17:14:12 -05:00
Nimisha Asthagiri
33e47d317b Course Overview migrations fix: Readd the Facebook URL field. 2016-02-26 10:20:36 -05:00
Nimisha Asthagiri
fa8ca11d8c Remove unused Mobile Social Facebook endpoint 2016-02-18 12:12:04 -05:00
Ned Batchelder
d8b9dec51e Update this command for Django 1.8 2016-02-11 15:39:35 -05:00
Julien Paillé
ea2633d485 Fix generate_course_structure command. 2016-02-10 10:23:20 +01:00
David Ormsbee
1a5b9172d8 Allow course images to be served from the CDN.
Previously, course images on the student dashboard were always
served from the localhost, even if the CDN was enabled.
2016-02-09 16:45:18 -05:00
David Ormsbee
be25bb8aba Clear caches between tests for ModuleStoreTestCase/SharedModuleStoreTestCase
Cached values were leaking across tests, causing difficult to debug errors,
particularly when using Config Models. As part of this work, certain tests
that had query counts that relied on those values being cached needed to
be adjusted up.
2016-02-06 00:01:15 -05:00
Nimisha Asthagiri
05767b433d Collect Course Blocks on Course Publish and Management Command
MA-1368
2016-02-03 16:56:22 -05:00
David Ormsbee
c4b0c7b483 Move test_get_all_courses_by_mobile_available to CourseOverviewTestCase
It somehow ended up in CourseOverviewImageSetTestCase (merge artifact?).
2016-01-11 04:30:44 -05:00
David Ormsbee
0e70676ba7 Prevent course image thumbnail race condition.
Without this change, we'd throw a TransactionManagementError in the case
where we have a race condition and generate multiple CourseOverviewImageSets
for the same CourseOverview concurrently.
2016-01-11 04:30:35 -05:00