Commit Graph

98 Commits

Author SHA1 Message Date
Sanford Student
5388d5d1fc Revert "Bulk-reads and Request caching in Course Grade Report"
This reverts commit 16e9636513.
2017-05-09 16:25:46 -04:00
Nimisha Asthagiri
16e9636513 Bulk-reads and Request caching in Course Grade Report 2017-05-09 08:58:49 -04:00
J. Cliff Dyer
77f5e2dd00 Fix batch size error and kwargification of compute_v2. 2017-04-26 16:30:27 -04:00
Nimisha Asthagiri
6cdfeb98cb Merge pull request #14946 from edx/neem/refactor_grades_misc
Grades API misc cleanup
2017-04-25 10:14:25 -04:00
Nimisha Asthagiri
16114f585b Miscellaneous grades code cleanup - pending from previous PRs 2017-04-24 17:58:18 -04:00
Eric Fischer
5c26acc115 Re-kill ICRV block
This reverts commit 1224e341de. I've also added
NotImplementedPartitionScheme, which allows deprecated partition types to have
a valid entry point despite being unusable.

TNL-6675
2017-04-19 15:55:13 -04:00
Nimisha Asthagiri
1503e5f768 Grade only engaged learners
Add a new waffle switch that allows us to assume zero grades for
learners who have no entry previously recorded, and another to
disable persisting grades for unengaged learners.

TNL-6691
2017-04-11 10:41:57 -04:00
cahrens
e5035746bf Introduce EnrollmentTrackUserPartition.
TNL-6674
2017-03-30 12:24:48 -04:00
J. Cliff Dyer
a3154b923f Create task to process a batch of grade computations.
TNL-6690
2017-03-28 12:46:51 -04:00
Nimisha Asthagiri
634ac8a825 Still compute grades asynchronously when PersistentGrades is enabled 2017-03-16 18:26:17 -04:00
Nimisha Asthagiri
1224e341de Revert "Merge pull request #14588 from edx/efischer/kill_stuff"
This reverts commit 83f2be4929, reversing
changes made to 01ed17426f.
2017-03-08 13:10:48 -05:00
Eric Fischer
83f2be4929 Merge pull request #14588 from edx/efischer/kill_stuff
Kill edx-reverification-block
2017-03-06 09:29:36 -05:00
Nimisha Asthagiri
00d466224e Consolidate Block Structure folders
TNL-6518
2017-03-01 21:19:28 -05:00
Eric Fischer
7b85c0ea00 Remove in-course reverification
This is unused, and will no longer be supported on edx.org. Also adds a
DeprecatedModel utility class.

TNL-6126
2017-03-01 10:46:29 -05:00
Nimisha Asthagiri
02c24cb259 Block Structure query count and API changes 2017-02-28 23:54:57 -05:00
David Ormsbee
2051c90924 Test Speedup: Isolate Modulestore Signals
There are a number of Django Signals that are on the modulestore's
SignalHandler class, such as SignalHandler.course_published. These
signals can trigger very expensive processes to occur, such as course
overview or block structures generation. Most of the time, the test
author doesn't care about these side-effects.

This commit does a few things:

* Converts the signals on SignalHandler to be instances of a new
  SwitchedSignal class, that allows signal sending to be disabled.

* Creates a SignalIsolationMixin helper similar in spirit to the
  CacheIsolationMixin, and adds it to the ModuleStoreIsolationMixin
  (and thus to ModuleStoreTestCase and SharedModuleStoreTestCase).

* Converts our various tests to use this new mechanism. In some cases,
  this means adjusting query counts downwards because they no longer
  have to account for publishing listener actions.

Modulestore generated signals are now muted by default during test runs.
Calls to send() them will result in no-ops. You can choose to enable
specific signals for a given subclass of ModuleStoreTestCase or
SharedModuleStoreTestCase by specifying an ENABLED_SIGNALS class
attribute, like the following example:

    from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase

    class MyPublishTestCase(ModuleStoreTestCase):
        ENABLED_SIGNALS = ['course_published', 'pre_publish']

You should take great care when disabling signals outside of a
ModuleStoreTestCase or SharedModuleStoreTestCase, since they can leak
out into other tests. Be sure to always clean up, and never disable
signals outside of testing. Because signals are essentially process
globals, it can have a lot of unpleasant side-effects if we start
mucking around with them during live requests.

Overall, this change has cut the total test execution time for
edx-platform by a bit over a third, though we still spend a lot in
pre-test setup during our test builds.

[PERF-413]
2017-02-23 10:31:16 -05:00
Eric Fischer
fd1ed2ce6d Remove unneeded validation on grades model
These checks are causing SQL query numbers to scale linearly with the
number of subsections being created/updated, and the errors they
check for have not been seen in prod.

TNL-6225
2017-02-17 09:48:56 -05:00
Nimisha Asthagiri
71cce9bb73 Block Transformers: distinguish between READ and WRITE versions
TNL-6522
2017-02-13 12:20:22 -05:00
Eric Fischer
6f8383199f Allow null edited_timestamp
Some old mongo courses do not have this field, and the team has opted
to allow null values rather than inserting a default. This change
affects both course and subsection grades.

TNL-6408
2017-01-30 13:19:36 -05:00
Sanford Student
dd93496f7a add task id to logging and log database exceptions
for TNL-6332, TNL-6373
2017-01-26 10:06:34 -05:00
Nimisha Asthagiri
5a9179bf00 Update has_database_updated_with_new_score to handle all block types
TNL-6331
2017-01-23 15:50:16 -05:00
Eric Fischer
3fd86614b0 sub_api get_score return value is a dict
This access pattern was wrong, and was causing errors in production. I've
included an additional happy-path test.
2017-01-09 11:50:18 -05:00
Eric Fischer
a7c2a3d926 Retry on all exceptions on failed subsection grade update 2017-01-09 11:50:18 -05:00
Eric Fischer
69eae7b8fb Test for missing submission score
Follow-up to this morning's change for TNL-6236
2017-01-03 14:47:23 -05:00
Nimisha Asthagiri
99e6d53939 Preload definitions when collecting block structures 2016-12-29 08:51:26 -05:00
Eric Fischer
5a98a90eba Use timestamps in _has_database_updated_with_new_score check
Also fixes issue where ORA grades were never updated, by including
them in this check.

TNL-5994
TNL-5995
2016-12-09 13:46:34 -05:00
Sanford Student
5f2ffbe75b implementation of grading events
for TNL-5045
2016-12-07 14:47:08 -05:00
Sanford Student
4eb5311ece Separating problem raw score changed and problem weighted score changed signals
For TNL-5993
2016-11-30 10:15:07 -05:00
J. Cliff Dyer
5350e31667 fixup: extract some common code. 2016-11-22 15:53:30 -05:00
J. Cliff Dyer
0e2534f3ec Wrap update_or_create in transaction to allow validation after save 2016-11-22 12:12:15 -05:00
J. Cliff Dyer
86c12e67cc fixup: Error instead of correcting on invalid inputs. 2016-11-22 11:48:33 -05:00
Nimisha Asthagiri
a6b93f1ab7 Fix missing import due to merge conflict between parallel PRs 2016-11-18 15:05:09 -05:00
Nimisha Asthagiri
47de5e6c68 Merge pull request #14021 from edx/beryl/null-subsection-error
Handle DAGs with inaccessible Subsections on Problem Score update
2016-11-18 13:57:25 -05:00
Nimisha Asthagiri
bc551b9b25 Merge pull request #14001 from edx/beryl/get-max-score-opt
Optimize max_score by moving from XModule to Descriptor
2016-11-18 11:41:55 -05:00
Nimisha Asthagiri
89d916918f Handle DAGs with inaccessible Subsections on Problem Score update
TNL-5886
2016-11-18 10:34:01 -05:00
Nimisha Asthagiri
fda2ceac8d Fix Delete-Student-State CSM entry deletion 2016-11-18 10:27:12 -05:00
Eric Fischer
b9adb42f23 Remove Extraneous Subsection Grade DatabaseError protection
These lines are now being run in async tasks anyways, so we can remove this
decorator and rely on the async task to retry itself. Also cleans up a few
pylint violoations noticed in subsection_grade.py.

Tests have been updated to match new behavior.
2016-11-15 10:49:32 -05:00
Nimisha Asthagiri
266bfb1ab3 Optimize async grades computation
TNL-5909
2016-11-08 19:22:30 -05:00
Sanford Student
6e52fd791f fixes from robust grades bug bash
Update course grade query parameters and check scores when updating subsection grade

For TNL-5861
2016-11-08 14:16:03 -05:00
Nimisha Asthagiri
8ae20e3812 Add grades feature check to subsection task handler. 2016-11-01 21:21:04 -04:00
Eric Fischer
a54c4ea4f9 Revert "Unrevert "Fix Gating to use grades API, instead of its own calculations""
This reverts commit 7aacae9a4f.
2016-11-01 15:02:04 -04:00
Eric Fischer
caeac6921a Fast-fail grades tasks if feature flag disabled 2016-11-01 13:34:10 -04:00
Sanford Student
a1d5ea905d Persistent Course Grades Glue Layer
Initial commit

TNL-5484
2016-10-27 14:56:45 -04:00
Eric Fischer
6a47311479 Update Persisted Course Grade on Subsection Grade update
Makes use of the new SUBSECTION_SCORE_CHANGED signal to trigger a task that
updates persisted course grade values. We've also renamed SCORE_CHANGED to
PROBLEM_SCORE_CHANGED to head off any issues with unclear signal names.

TNL-5740
2016-10-25 14:47:31 -04:00
Nimisha Asthagiri
7aacae9a4f Unrevert "Fix Gating to use grades API, instead of its own calculations" 2016-10-24 12:33:59 -04:00
Nimisha Asthagiri
aa000c1a3d Support for rescoring a problem only if the new score is higher
TNL-5046
2016-10-20 15:08:54 -04:00
Eric Fischer
529889f3f2 Recalculate Subsection Grade task retry logic
The task will now be retried if an IntegrityError is encountered, up to
the default of 3 times. Test included.

TNL-5739
2016-10-19 09:45:34 -04:00
Eric Fischer
70e029b1f5 Update persistent grades asynchronously
For better user-facing performance, the SCORE_CHANGED signal is now handled by
enqueueing an async task to update the relevant stored grade, rather than
making the request wait until that operation finishes.

TNL-5738
2016-10-13 14:06:06 -04:00