Commit Graph

70 Commits

Author SHA1 Message Date
Rodrigo Mendez
12a46e6463 feat: AuthZ for course authoring compatibility layer (#38013) 2026-03-06 09:35:17 -07:00
Feanil Patel
57cdbfa013 test: Remove test variants that test without a cache.
Since the cache is now always on, remove test cases that test the case
when it's disabled.
2024-11-05 10:44:16 -05:00
Maria Grimaldi
809ffc3743 feat: connect teams with content groups using dynamic partition generator (#33788)
Implements the connection from the teams feature to the content groups feature. This implementation uses the dynamic partition generator extension point to associate content groups with the users that belong to a Team.

This implementation was heavily inspired by the enrollment tracks dynamic partitions.
2024-04-25 13:02:49 -04:00
Feanil Patel
40eedc6889 test: Update query counts now that we've removed a waffle flag.
Various tests with query counts need to be updated now that we have
removed a waffle check that was in their codepath.
2023-01-22 18:39:51 -05:00
0x29a
dd9f6936bc refactor: xmodule/video_module -> xmodule/video_block 2022-12-19 17:48:49 +01:00
Sagirov Eugeniy
8f88422c4a test: prepare tests for removing support for children in Old Mongo 2022-10-28 11:43:20 -04:00
Sagirov Eugeniy
3f3d0d25d8 feat!: Remove inheritance-related code from Old Mongo 2022-10-24 11:58:44 -04:00
Chris Chávez
ebd96cba8a fix: Bug showing hidden sections for learners of a public course fixed (#27925)
The bug is explained in https://openedx.atlassian.net/browse/CRI-233. Only is missing add the `VisibilityTransformer` in `get_blocks()` when the user is not enrolled to the course.
On the test, `html_block` is visible only for staff and `vertical_block` is a normal block. The new behaviour hides the `html_block` and show the `vertical_block` to anonymous users
2022-08-02 12:40:23 -04:00
Eugene Dyudyunov
adb156ac03 test: update tests query counts 2022-05-03 13:16:43 +03:00
Julia Eskew
0eb0f99d00 feat: Add org-level waffle overrides to allow a waffle flag to be overridden for all
courses in a particular organization, without needing to make a separate course-level
waffle override for each individual course for an organization.
2022-01-28 08:00:18 -05:00
Jawayria
59002b5702 chore: Applied lint-amnesty on lms/djangoapps/{course_api, course_blocks} 2021-12-17 16:05:40 +05:00
connorhaugh
b01e773d2b feat: cache course index queries (#29107)
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
2021-10-27 13:53:10 -04:00
Braden MacDonald
6c85668099 feat: write split modulestore's course indexes to Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-26 10:06:52 -07:00
Dillon Dumesnil
78fbd28f41 chore: AA-1058: Update edx-when version
Updates query counts for the additional query to look up the Schedule
in edx-when
2021-10-22 09:36:04 -04:00
David Ormsbee
ae124bd554 Revert "feat: store split modulestore's course indexes in Django/MySQL"
This reverts commit 96e5ff8dce.
2021-10-07 15:07:42 -04:00
Braden MacDonald
96e5ff8dce feat: store split modulestore's course indexes in Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-07 10:59:47 -04:00
jawad khan
15f33dc1b2 revert: show empty container blocks from the course blocks api when called by mobile (#27707)
REVE-52 is merged and mobile
traffic from older app versions is < 5% of the mobile traffic.
Therefore we dont need changes from this PR 325fbe4eb2 (diff-c061bf405edf96531766b26cc2bcc2bb3639d851e5d9f844612c2ea57c35e56e)

LEARNER-8377
2021-05-25 20:30:26 +05:00
Dillon Dumesnil
c9197d3cfc refactor: AA-677: Switch relative dates to CourseWaffleFlag
We have been bucketing all users into the relative dates experiment
since May 18, 2020. We no longer need to keep this as an
ExperimentWaffleFlag and can convert to a CourseWaffleFlag (so it
continues to support exemptions).
2021-04-08 10:12:54 -04:00
Usama Sadiq
afa102e125 refactor: ran pyupgrade on lms/djangoapps/course_api (#26735) 2021-03-09 12:54:42 +05:00
Aarif
a332aac1f6 replaced unittest assertions pytest assertions (#26541) 2021-02-19 12:27:11 +05:00
Jawayria
e9b167e1fd Applied pylint-amnesty to course_api 2021-02-04 13:22:45 +05:00
Régis Behmo
3a29cff016 Get rid of calls to set_request_cache_with_short_name
This method from the toggle legacy classes should not actually be
exposed to all. So we get rid of it by manually setting the cached
value. While we are at it, we convert the STORAGE_BACKING_FOR_CACHE
legacy waffle switch to its modern version. As the flag is not being
used elsewhere, this should not break anything.

We take the opportunity to modernize waffle switches from
block_structure.config: to do so we convert the INVALIDATE_CACHE_ON_PUBLISH and
RAISE_ERROR_WHEN_NOT_FOUND waffle switches from legacy classes to their modern
equivalents. These switches are not used outside of edx-platform, so this
change should not trigger any error.
2021-01-12 16:52:08 +01:00
Kyle McCormick
151bd13666 Use full names for common.djangoapps imports; warn when using old style (#25477)
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
2020-11-10 07:02:01 -05:00
Régis Behmo
4586002956 Import waffle classes from edx_toggles instead of waffle_utils
Those classes were ported to edx_toggles. The imports remain in
waffle_utils.__init__ for backward compatibility.
2020-11-03 19:25:37 +01:00
Régis Behmo
3b127f8c92 Deprecate WaffleSwitch.override* methods
This allows us to get rid of the custom WaffleSwitch and
WaffleSwitchNamespace classes from waffle_utils in favour of
edx_toggles.toggles classes.
2020-11-03 19:25:37 +01:00
Régis Behmo
2307dff4c9 Deprecate WaffleFlag.override method
This allows us to get rid of waffle_utils' custom WaffleFlag method.
2020-11-03 19:25:37 +01:00
Robert Raposa
624012e11c remove ENABLE_VIDEO_URL_REWRITE
The temporary waffle flag ENABLE_VIDEO_URL_REWRITE is no
longer needed, becaues the rollout is complete.
See https://openedx.atlassian.net/browse/PROD-62

ARCHBOM-1304
2020-06-23 11:19:32 -04:00
Michael Terry
e724a4bcb7 Add ExperimentWaffleFlag
This is a helpful class when running an experiment, to help
bucket users and keep track of which enrollments to consider
as part of the experiment.

AA-53
2020-03-03 14:39:02 -05:00
Calen Pennington
98328ea426 On publish, add evenly spaced dates to self-paced courses 2020-02-21 10:50:22 -05:00
Feanil Patel
046feb0cf0 Merge pull request #22649 from edx/feanil/fix_pep8
Fix all E303 pep8 errors.
2019-12-30 13:32:26 -05:00
Feanil Patel
6e3fe00fff Fix all E303 pep8 errors. 2019-12-30 12:25:38 -05:00
Feanil Patel
2df8b8226b Merge pull request #22643 from edx/feanil/2to3_asserts
Run `2to3 -f asserts . -w` on edx-platform.
2019-12-30 12:13:42 -05:00
Feanil Patel
9cf2f9f298 Run 2to3 -f future . -w
This will remove imports from __future__ that are no longer needed.

https://docs.python.org/3.5/library/2to3.html#2to3fixer-future
2019-12-30 10:35:30 -05:00
Feanil Patel
70294b0ad2 Run 2to3 -f asserts . -w on edx-platform.
https://docs.python.org/3.5/library/2to3.html#2to3fixer-asserts
2019-12-30 10:22:19 -05:00
Matthew Piatetsky
bd76a71019 delete old holdback code 2019-11-07 09:51:39 -05:00
DawoudSheraz
942c79e606 add video url transformer 2019-09-27 14:26:52 +05:00
Stu Young
b0e6870271 INCR-286 Run python-modernize on lms/djangoapps/course_api/blocks/tests (#20593)
* run python modernize

* run isort

* Fix quality
2019-05-21 11:24:39 -04:00
Dave St.Germain
b4ccd03740 This adds a new django app -- edx-when -- that will copy start and due dates to mysql and allow per-learner overrides in the instructor dashboard, using the existing IDDE interface.
It adds a data migration for existing IDDE data.
2019-04-24 12:26:19 -04:00
Matthew Piatetsky
51b991c128 fix bug and add temporary waffle flag 2019-04-09 15:38:55 -04:00
Michael Youngstrom
3221c2b91f Remove lms pytest shards 2019-02-12 11:03:46 -05:00
Calen Pennington
77922ae073 Merge pull request #19478 from edx/hide-empty-content-course-blocks
REVE-181: Hide empty container blocks from the course blocks api when called by the mobile app
2018-12-21 10:22:59 -05:00
Calen Pennington
325fbe4eb2 Hide empty container blocks from the course blocks api when called by the mobile app
All code in this PR should be removed after REVE-52 is merged and mobile
traffic from older app versions falls to < 5% of the mobile traffic
to the course_blocks api
2018-12-20 15:17:24 -05:00
Bill Filler
684d6e3669 check if user is in holdback before showing new track selection 2018-12-17 16:05:13 -05:00
Matthew Piatetsky
14a899a65d REVE-104 & REVE-108: Ensure Course Roles' Access 2018-12-07 16:39:09 -05:00
Calen Pennington
0cb48dfe1c Add StackedConfigurationModels for managing content_type_gating and course_duration_limits 2018-11-21 12:18:10 -05:00
Michael Youngstrom
26b4e30833 Remove django 1.8 shim 2018-06-06 10:34:56 -04:00
Stuart Young
1fd2167144 rebalance python unittests onto new shards 2018-05-03 11:16:05 -04:00
Amir Qayyum Khan
aa6f88382d Fixed edX block structure api to load override data 2018-02-16 11:10:17 +05:00
Troy Sankey
e649d2d782 Shim query count assertions due to added SAVEPOINTs 2018-01-25 11:38:46 -05:00
Andy Armstrong
79acb5c5be Reorder LMS imports using isort 2017-06-11 21:48:06 -04:00