Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
course outline, which don't need block dates below the subsection level of a course.
Pass the course's published version to all the appropriate places where edx-when's API
is called - to allow edx-when to more efficiently cache queried/processed results.
TNL-8061
Reverting because of errors seen in proctoring. The error from splunk:
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/edx_proctoring/views.py", line 590, in get
data['onboarding_release_date'] = effective_start.isoformat()
AttributeError: 'NoneType' object has no attribute 'isoformat'
The effective date is coming from learning_sequences, which ultimately
comes from edx-when.
The new filter, called `permissions`, allows callers to filter courses
per access granted to the specified username. Callers can now filter
courses per roles, actions, etc.
course outline, which don't need block dates below the subsection level of a course.
Pass the course's published version to all the appropriate places where edx-when's API
is called - to allow edx-when to more efficiently cache queried/processed results.
TNL-8061
The get_course_members API returns a dict of users associated with a course.
This is a potentially expensive operation on a large course, so there is a
control in place to limit its cost. If a course has more than
settings.COURSE_MEMBER_API_ENROLLMENT_LIMIT enrollments, then the function
raises an OverEnrollmentLimitException.
This API was added to help implement the LTI 1.3 Names and Roles Provisioning
service.
Jira references: [BD-24] [BB-2726] [TNL-7330]
Pull request: #25843
Co-authored-by: Giovanni Cimolin da Silva <giovannicimolin@gmail.com>
We discovered a subsection that contained a unit without any content
inside, but because of our logic requiring children, it would never be
marked complete (meaning the subsection, section, and course could thus
never be marked complete). This fixes that by removing the children
check from setting completion, but first gating that code path on the
xblock being an aggregator (to prevent leaves from marking as true
simply because there are no children).
Test fixes include adding a test for the empty aggregator case as
well as some changes to not have an entire course marked complete
because they are all empty aggregators.
We have been bucketing all users into the relative dates experiment
since May 18, 2020. We no longer need to keep this as an
ExperimentWaffleFlag and can convert to a CourseWaffleFlag (so it
continues to support exemptions).
The /jump_to/ LMS endpoint is used in a number of places
to direct users to courseware. It currently only redirects to
Legacy courseware URLs, which then conditionally may
redirect to the Learning MFE.
Two issues with this:
1. Performance Impact: In most cases, going to Legacy first
is just an extra redirect.
2. Confusion for Privileged Users: Neither course nor global
staff are auto-redirected from the Legacy experience to the
MFE. Thus, these priviliged users confusingly never see the
MFE by default; they must always manually click into it.
This commit makes it so that /jump_to/ directs
users to whatever the default courseware experience is
for them. For staff of courses active in the new experience,
this will impact (at a minimum) the "View Live"
links in Studio, all links on the old and new LMS
course outline, and the "Resume" links on the course
dashboard. Learners should see no difference other than
a performance improvement when following courseware links
from the LMS.
This also adds an optional 'experience=[legacy|new]'
query param to /jump_to/, allowing us to specifically
generate Legacy courseware URLs for the
"View in Legacy Experience" tool.
TNL-7796
A variety of updates were made to improve the toggle documentation:
* Added comments to help ensure that the waffle(), waffle_switches(),
waffle_flags() anti-pattern won't be contagious (copied).
* Some minor toggle_description updates.
* Removed empty toggle_target_removal_date annotations for
non-temporary toggles.
* Removed empty optional toggle_warnings annotations.
* Removed empty optional toggle_tickets annotations.
* Removed deprecated toggle_category, toggle_status,
and toggle_expiration_date annotations.
* Fixed some indents, use cases, and implementations.
ARCHBOM-1721
For now only the discussion blocks were supported. If we had a custom XBlock that specified `completion_mode = XBlockCompletionMode.EXCLUDED`, then it could never be marked as completed on the course outline page, despite being marked as such inside the learning sequence.
This adds support for displaying completion on the course outline page, to remove the discrepancies between this view and the learning sequence. It also simplifies course outline page by reusing existing APIs for determining completion state and finding the "Resume block"'s target.
django-not-configured is an error raised by pylint (with
the pylint-django plugin) when it's not correctly configured.
We should not be applying lint amnesty for such a violation.
* LEARNER-8158
Fixed completion param issue
- There is a case where we are sending requested_fields in params as comma separated list
e.g. requested_fields=children,show_gated_sections,graded,special_exam_info,completion.
- We didn't test for this case in first place and test cases were sending requested_fields as list.
- Now we can also handle this comma separated completion field which was getting ignored before.
This method from the toggle legacy classes should not actually be
exposed to all. So we get rid of it by manually setting the cached
value. While we are at it, we convert the STORAGE_BACKING_FOR_CACHE
legacy waffle switch to its modern version. As the flag is not being
used elsewhere, this should not break anything.
We take the opportunity to modernize waffle switches from
block_structure.config: to do so we convert the INVALIDATE_CACHE_ON_PUBLISH and
RAISE_ERROR_WHEN_NOT_FOUND waffle switches from legacy classes to their modern
equivalents. These switches are not used outside of edx-platform, so this
change should not trigger any error.
Commit 7f59688 attempted to solve this using
`CourseOverview.course_exists`, but that check is case-
insensitive. This commit instead does a small refactor
to `lms.djangoapps.courseware.get_course` so that we can
handle a failed course lookup without broadly catching
a `ValueError`.
By explicitly importing the legacy namespace classes, we make it clear
that we are using soon-to-be-deprecated classes. We will then be able to
start removing the legacy classes, one module at a time.
