Andal.LND/edx-platform
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
60,428 Commits 1 Branch 0 Tags
72ec00cbb71d0a1d7201dfdcd92462156dbd6152
Commit Graph

105 Commits

Author SHA1 Message Date
connorhaugh
b01e773d2b feat: cache course index queries (#29107) 
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
 2021-10-27 13:53:10 -04:00
Dillon Dumesnil
78fbd28f41 chore: AA-1058: Update edx-when version 
Updates query counts for the additional query to look up the Schedule
in edx-when
 2021-10-22 09:36:04 -04:00
David Ormsbee
ae124bd554 Revert "feat: store split modulestore's course indexes in Django/MySQL" 
This reverts commit 96e5ff8dce.
 2021-10-07 15:07:42 -04:00
Braden MacDonald
96e5ff8dce feat: store split modulestore's course indexes in Django/MySQL 
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
 2021-10-07 10:59:47 -04:00
Christie Rice
912f2fa431 feat!: Enable the allowlist for all courses (#27468) 
MICROBA-1013
 2021-05-03 11:15:04 -04:00
Usama Sadiq
228c54d222 refactor: pyupgrade second iteration 2021-05-03 14:55:47 +05:00
Dillon Dumesnil
c9197d3cfc refactor: AA-677: Switch relative dates to CourseWaffleFlag 
We have been bucketing all users into the relative dates experiment
since May 18, 2020. We no longer need to keep this as an
ExperimentWaffleFlag and can convert to a CourseWaffleFlag (so it
continues to support exemptions).
 2021-04-08 10:12:54 -04:00
Jawayria
b31c18d6db BOM-2352: Removed unused imports from lms/djangoapps/{grades, instructor} 2021-03-09 17:25:54 +05:00
M. Zulqarnain
e505d99237 pyupgrade on lms gating and grades apps (#26532) 2021-02-22 12:58:41 +05:00
Aarif
906b6f7fed replaced unittest assertions pytest assertions (19) (#26545) 2021-02-18 18:00:30 +05:00
Christie Rice
8845685fd8 MICROBA-918 Check for the allowlist on invalidation and failing grades (#26356) 2021-02-04 13:42:33 -05:00
M. Zulqarnain
0213fde968 BOM-2281 : Pylint amnesty for grades,gating and instructor app (#26343) 
* pyint amnesty for grades,gating and instructor app
 2021-02-03 18:14:35 +05:00
Kyle McCormick
151bd13666 Use full names for common.djangoapps imports; warn when using old style (#25477) 
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
 2020-11-10 07:02:01 -05:00
Régis Behmo
2307dff4c9 Deprecate WaffleFlag.override method 
This allows us to get rid of waffle_utils' custom WaffleFlag method.
 2020-11-03 19:25:37 +01:00
Robert Raposa
77e490f057 ARCHBOM-1305: remove deprecated flag_undefined_default (#24426) 
This is the final step in removing the deprecated
flag_undefined_default as explained by the following ADR:
https://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/waffle_utils/docs/decisions/0001-refactor-waffle-flag-default.rst

Notes:

* All uses of flag_undefined_default=False were always
  supposed to have been no-ops.
* All uses of flag_undefined_default=True that are removed
  in this PR have been replaced by migrations in past PRs.
* The temporary metric temp_flag_default_used id no longer
  reporting any data.

ARCHBOM-1305
 2020-07-09 09:31:31 -04:00
Michael Terry
e724a4bcb7 Add ExperimentWaffleFlag 
This is a helpful class when running an experiment, to help
bucket users and keep track of which enrollments to consider
as part of the experiment.

AA-53
 2020-03-03 14:39:02 -05:00
Calen Pennington
98328ea426 On publish, add evenly spaced dates to self-paced courses 2020-02-21 10:50:22 -05:00
Feanil Patel
046feb0cf0 Merge pull request #22649 from edx/feanil/fix_pep8 
Fix all E303 pep8 errors.
 2019-12-30 13:32:26 -05:00
Feanil Patel
6e3fe00fff Fix all E303 pep8 errors. 2019-12-30 12:25:38 -05:00
Feanil Patel
2df8b8226b Merge pull request #22643 from edx/feanil/2to3_asserts 
Run `2to3 -f asserts . -w` on edx-platform.
 2019-12-30 12:13:42 -05:00
Feanil Patel
9cf2f9f298 Run 2to3 -f future . -w 
This will remove imports from __future__ that are no longer needed.

https://docs.python.org/3.5/library/2to3.html#2to3fixer-future
 2019-12-30 10:35:30 -05:00
Feanil Patel
70294b0ad2 Run 2to3 -f asserts . -w on edx-platform. 
https://docs.python.org/3.5/library/2to3.html#2to3fixer-asserts
 2019-12-30 10:22:19 -05:00
Matthew Piatetsky
bd76a71019 delete old holdback code 2019-11-07 09:51:39 -05:00
Jeremy Bowman
553d35e53e Upgrade mock BOM-596 (#21717) 2019-09-24 10:14:17 -04:00
Ayub khan
a075e44e72 BOM-273 
tests fixed
 2019-09-19 14:26:07 +05:00
Dave St.Germain
cb5ec5a65d Grade api support for bulk grading 2019-06-03 13:53:30 -04:00
Stu Young
22856fdc1a INCR-315 (#20610): python-modernize lms/djangoapps/grades/tests 
* run python modernize
* run isort
* fix pylint issues
 2019-05-28 11:21:34 -04:00
Dave St.Germain
b4ccd03740 This adds a new django app -- edx-when -- that will copy start and due dates to mysql and allow per-learner overrides in the instructor dashboard, using the existing IDDE interface. 
It adds a data migration for existing IDDE data.
 2019-04-24 12:26:19 -04:00
Michael Youngstrom
3221c2b91f Remove lms pytest shards 2019-02-12 11:03:46 -05:00
jansenk
726fb0f6f0 fail a certificate if grades are modified to failing 2019-01-29 15:57:15 -05:00
Alex Dusenbery
577315f2a7 Refactor how grade override values are persisted. 2019-01-08 15:21:04 -05:00
Bill Filler
684d6e3669 check if user is in holdback before showing new track selection 2018-12-17 16:05:13 -05:00
Matthew Piatetsky
14a899a65d REVE-104 & REVE-108: Ensure Course Roles' Access 2018-12-07 16:39:09 -05:00
Calen Pennington
0cb48dfe1c Add StackedConfigurationModels for managing content_type_gating and course_duration_limits 2018-11-21 12:18:10 -05:00
Alex Dusenbery
c11a2c0a6e EDUCATOR-3471 | Add a grade override bulk update view. 2018-10-25 14:52:12 -04:00
Simon Chen
929ca61e6c Default the waffle flag to be on 2018-10-05 11:54:04 -04:00
Nimisha Asthagiri
ddafbb0d29 POC: Stop-gap implementation to freeze grades. 2018-10-04 08:13:35 -04:00
Michael Terry
fd33130d73 Add initial grade signaling 
Tells the Credential service about user grades when a cert is awarded
or a grade for a course with an awarded cert changes.

LEARNER-5051
 2018-06-25 11:42:08 -04:00
Michael Youngstrom
26b4e30833 Remove django 1.8 shim 2018-06-06 10:34:56 -04:00
Alex Dusenbery
ef477ee2a4 LEARNER-5123 | Cache VisibleBlocks per course AND user, only fetch the ones the user needs for subsection scoring. 2018-05-14 13:44:56 -04:00
Alex Dusenbery
2d4dd3750b LEARNER-5123 | Hotfix - don't regrade everything for a user in a course with too many visible blocks. 2018-05-10 14:49:15 -04:00
Stuart Young
1fd2167144 rebalance python unittests onto new shards 2018-05-03 11:16:05 -04:00
John Eskew
3915c47d71 Bump query counts for grading in Django 1.11. 2018-01-25 11:23:31 -05:00
Alex Dusenbery
1ea419c2c4 EDUCATOR-1428 | Add time_limit to course/subsection grading tasks. 2017-10-17 15:28:01 -04:00
Nimisha Asthagiri
b5435e6f5a Course Grade reports: eliminate CSM calls, user-transformed structures 
EDUCATOR-558
 2017-10-05 01:12:49 -04:00
Nimisha Asthagiri
f8e3a3c461 Grades Cleanup 
EDUCATOR-1405
EDUCATOR-1451
 2017-10-02 19:03:26 -04:00
Nimisha Asthagiri
1febdbfac9 Grades cleanup: remove read_only param and create method 
EDUCATOR-171
 2017-09-28 07:48:25 -04:00
Nimisha Asthagiri
083ccdb95a Remove unneeded WRITE_ONLY_IF_ENGAGED waffle switch 2017-09-25 16:13:07 -04:00
Nimisha Asthagiri
22edaf7d24 Grades: move out of temporary new folder 
EDUCATOR-1400
 2017-09-22 11:49:46 -04:00
Tyler Hallada
a9c1e1cf43 Squashed commit of the following: 
commit 12880fa21eac0c2e69b2d832d42829f00484561d
Author: Tyler Hallada <thallada@edx.org>
Date:   Fri Aug 11 11:17:22 2017 -0400

    Tweak exam override messaging in progress.html

commit 6f90352dc27b3a199eea7efaf84b72df50f570a9
Merge: e0e9f83789 7c535f5fb9
Author: Tyler Hallada <thallada@edx.org>
Date:   Thu Aug 10 17:38:13 2017 -0400

    Merge remote-tracking branch 'origin/master' into EDUCATOR-926

commit e0e9f83789efa837577cedbcd5807c6fdba249e4
Author: Tyler Hallada <thallada@edx.org>
Date:   Thu Aug 10 17:35:26 2017 -0400

    Revert "Comment out actual override and log instead"

    This reverts commit 4953cf30d9.

commit 27d6b537c8915ecd1af8af51c8f9509db6ea0a26
Author: Tyler Hallada <thallada@edx.org>
Date:   Thu Aug 10 17:34:56 2017 -0400

    Revert "Fix tests"

    This reverts commit 1b2fec2129.

commit cabddd504ddcb4f92aeed8b5fec4bff0d64a8de3
Author: Tyler Hallada <thallada@edx.org>
Date:   Thu Aug 10 17:34:36 2017 -0400

    Revert "Remove override behavior and log instead"

    This reverts commit 1ab1880337.

commit dc9407f4606804e00d5aee4e80d309c7d7f9c437
Author: Tyler Hallada <thallada@edx.org>
Date:   Thu Aug 10 17:34:15 2017 -0400

    Revert "Fix python tests and use proctoring 1.0.0"

    This reverts commit 842ce8365f.
 2017-08-11 13:53:34 -04:00