This adds support for course staff masquerading as any other user when
viewing their own course, via user={username} querystring parameter.
Rules:
* Anonymous users are repesented by a blank "user" value.
* If there is no "user" parameter at all, render for the user making the
request.
* Global staff can view any course as any user.
* Course staff and instructors can view their own course as any user
(including other staff, the anonymous user, or users not enrolled in
their course).
This commit supports TNL-8330 (switching the frontend-app-learning MFE
to use the Learning Sequences API instead if Course Blocks).
and CourseSectionSequence to ensure that cascading deletes will occur to delete
the relation upon deletion of a foreign-keyed object.
This commit is phase 3 of 4 in order to ensure a smooth deploy. The phases:
1) Add separate through models for user partition groups, add fields to point to the
separate models, and start writing to those fields as well. (COMPLETE)
2) After all models have been re-generated, switch code over to use separate through
model fields backed by the separate through models *and* stop writing to the original
ManyToManyField field. (COMPLETE)
3) After phase 2 is deployed smoothly, remove the original ManyToManyField fields. (This PR!)
4) After phase 3 is deployed smoothly, rename the new fields to have the same name
as the old fields, so new_user_partition_groups -> user_partition_groups.
This PR is designed to be able to easily be rolled-back if anything goes wrong.
[MICROBA-1289]
* Remove use of the CourseOverview model directly in the Certificates app
* Introduce a few new Python API functions in the `course_overview` and `catalog` django apps to replace functionality in the Certificates app.
and CourseSectionSequence to ensure that cascading deletes will occur to delete
the relation upon deletion of a foreign-keyed object.
This commit is phase 2 of 3 in order to ensure a smooth deploy. The phases:
1) Add separate through models for user partition groups, add fields to point to the
separate models, and start writing to those fields as well. (COMPLETE)
2) After all models have been re-generated, switch code over to use separate through
model fields backed by the separate through models *and* stop writing to the original
ManyToManyField field. (This PR!)
3) After phase 2 is deployed smoothly, remove the original ManyToManyField fields
and rename the new fields to have the same name as the old fields.
This PR is designed to be able to easily be rolled-back if anything goes wrong.
The course settings `certificate_available_date` (CAD) and
`certificates_display_behavior` (CDB) were previously acting indedependantly
of one another. They now work in tandem. This change:
- limits CDB to a dropdown
- removes "early_with_info" and adds "end_with_date"
- only takes CAD into account if "end_with_date" is selected
- Moves CDB to the main course schedule settings page
- updates CourseOverview to validate these fields and choose sane
defaults if they aren't expected values
Certificates will now show under the following circumstances:
"Immediately upon passing"
certificate_availability_date = null
certificates_display_behavior = "early_no_info"
"End date of course"
certificate_availability_date = null
certificates_display_behavior = "end"
"A date after the course end date"
certificate_availability_date = <date>
certificates_display_behavior = "end_with_date"
and CourseSectionSequence to ensure that cascading deletes will occur to delete
the relation upon deletion of a foreign-keyed object.
This commit is phase 1 of 3 in order to ensure a smooth deploy. The phases:
1) Add separate through models for user partition groups, add fields to point to the
separate models, and start writing to those fields as well.
2) After all models have been re-generated, switch code over to use separate through
model fields backed by the separate through models.
3) After phase 2 is deployed smoothly, remove the original ManyToManyField fields
and rename the new fields to have the same name as the old fields.
* feat: Course Apps API
This adds a new concept called course apps. These are exposed via a new
"openedx.course_app" entrypoint, which helps the LMS and studio discover such
apps and list them in a new rest api for the same.
These course apps will drive the pages and resources view in the course authoring
MFE. This system will track which apps are enabled and which are disabled. It
also allows third-party apps to be listed here by using the plugin entrypoint.
* Apply feedback from review
A common usage pattern is to make an exam that is restricted to a
particular enrollment track. Since there is no UI in Studio to do this
at the sequence/subsection level, course authors instead restrict every
unit in the sequence to that track (e.g. "verified"). The legacy
courseware experience could handle this, but the Course Blocks API does
not. It is likely that we'll want to permit toggling this at the
sequence level, but regardless of whether we do that going forward, we
still have to deal with this kind of content data in existing courses.
An entirely empty sequence is useless, and currently breaks the
Courseware MFE browsing experience. This commit introduces the first
part of that fix, invoking the new Learning Sequences API to remove
sequences that the user shouldn't be allowed to know about. The plan is
to switch over the entire course outline from the Course Blocks API to
the Learning Sequences API, and this is the first small step in that
direction.
This also creates CourseWaffleFlag learning_sequences.use_for_outlines
to control the gradual rollout of this feature. This will start as a
very limited rollout to address courses that show this specific bug
(TNL-8377).
New learning_sequences public API call: public_api_available
Code in ./common/lib/xmodule/xmodule should
be imported as `from xmodule`, since `xmodule`
is a locally-installed package.
This is weird, but as long as it is the case,
we should be consistent.
(In BOM-2584, I propose moving the files to
./xmodule, which would quell this confusion.)
In order to upgrade to Django > 2.2.20, we can't continue to use
absolute paths in the block_structure's data FileField. This used to
work for S3, but it will not work going forward due to a security fix
in Django 2.2.21.
This data migration will remove the starting '/' from any paths in the
block_structure table. The resulting locations in S3 should be
unaffected.
The edx-platform codebase already includes quite a few type annotations, but
they were not regularly checked. This may cause problems, when the annotations
themselves include errors (as we found out in some of the learning_sequences
annotations). So here, we add mypy as a dev requirement and introduce a make
command to run mypy regularly. Mypy runs on a very small portion of the total
edx-platform, as configured in mypy.ini. Our hope is that developers will add
more and more modules to this configuration file, until we can eventually run
mypy on the full code base.
See discussion: https://discuss.openedx.org/t/dev-notes-running-mypy-on-edx-platform/4860
In DE-1822, we believed we needed to switch to start_date and end_date.
It was determined this was not the case, so this updates the comment
to ensure future users use the correct fields (start and end) and
updates any pieces of code that may have used start_date or end_date.
We already had function traces to see how long each processor took when
generating a UserCourseOutlineData. This adds a similar tracking around
making a UserCourseOutlineDetailsData, to help track down performance
issues observed in what I think is the special exams processor.
[MICROBA-1178]
- remove modulestore usage in `generation_handler.py`
- add duplicate functions that utilize a CourseKey or CourseOverview to remove dependence on modulestore (this will be cleaned up (if possible) at a later part of this refactor)
- add python API function to `content`/`course_overview` app that will retrieve a single CourseOverview (rather than a serialized list of dicts of CourseOverview data)
NOTE: This will require a forced backfill of course outlines to update
the course content data in learning_sequences:
python manage.py cms backfill_course_outlines --force
Without this backfill, the learning_sequences API will continue to serve
stale content data that has no user partition group data. It won't cause
errors, but it won't do the exclusions properly.
Commit summary:
* Created EnrollmentTrackPartitionGroupsOutlineProcessor to process the
enrollment_track User Partition Group, allowing Sequences and Sections
to be removed based on their group_access settings.
* Added user_partition_groups attribute to CourseLearningSequenceData
and CourseSectionData in learning_sequences/data.py, along with
backing model data.
* get_outline_from_modulestore now extracts group_access settings from
Sections and Sequences. It also bubbles up group_access settings from
Units, meaning that if a Sequence with no group_access setting has
Units that are all set to show only to the Verified enrollment track,
then the Sequence will only show to the Verified enrollment track.
This commit adds model-level support for all user partition groups by
capturing all the content group associations (group_access), but it only
implements the code checks for the enrollment track partition. It's not
clear that we want to generalize, since there's only one other partition
type (A/B testing) that is applicable at the outline level.
It's important to note that there is no way to set the group_access for
a Section or Sequence in Studio today. It's only possible by direct
editing of the OLX for import. That being said, the block structures
framework supports applying course groups at this level, and this commit
moves learning_sequences closer to feature parity.
The bubbling up from Units to the parent Sequence was done to mitigate
confusion when a Sequence is entirely composed of Units that are not
visible to the user because of content group restrictions. It's not
clear whether this is something we want to do in the long term, since it
would simplify the code to always specify group_access at the Sequence
level. This first pass is done partially to collect better data about
places in our courses where this kind of usage is already happening.
Most of the EnrollmentTrackPartitionGroupsOutlineProcessor code and its
tests were written by @schenedx.
* MST-734 Fix production issue on Learner Onboarding Status Panel
Fix the prod issue where learning sequence service object missing the needed get_user_course_outline service API
The user web API call currently returns 500
I had the wrong attribute before this commit, and Django Admin let
me get away with it because it doesn't explode when you try to grab
relations that aren't there–it just quietly returns None in some of
those cases.
* Introduces the idea of content errors into the learning_sequences
public API, accessible using get_content_errors().
* Makes course outline generation much more resilient to unusual
structures (e.g. Section -> Unit with no Sequence in between),
with the understanding that anything that doesn't conform to the
standard structure will simply be skipped.
* Improves the Django Admin for learning_sequences to display
content errors and improve sequence data browsing within a course.
* Switches the main table viewed in the Django admin from
LearningContext to CourseContext, which is appropriate since only
course runs generate outlines.
This was done as part of TNL-8057, with the end goal of making
course outline generation resilient enough to switch over apps
to using the learning_sequences outline API. The types of course
structure errors that this PR addresses cause display issues even
in the current Outline Page experience, but would break the outline
generation for learning_sequences altogether.
The approach for error messages here is very generic, to keep
modulestore concepts from seeping into learning_sequences (which is
not aware of the modulestore/contentstore). We may need to address
this later, with a more normalized content error data model.
While the Django admin page is backwards compatible with the old
versions of the models, we should run the backfill_course_outlines
management command after deploying this change, to get the full
benefits.
Because the available date update to the CourseOverview happens inside a
view's signal and we have atomic requests on, the read that was
happening inside the task happened *before* the write was commited to
the database. To avoid the unknown bugs that would come from disabling
atomic transactions for that view (since it's large), this passes the
date we want down through the signals and tasks so we can skip the DB
read at the end.
This will make it possible to make a New Relic dashboard for the
learning_sequences API calls that tracks call performance across
different transactions (by querying Spans). Our goal would be to
offer SLAs around 99th percentile performance.
Course IDs and User ID metrics are also added, so that we can see
outliers in Span reporting, for later investigation.
* Adds the backfill_course_outlines management command to contentstore
* Adds a read-only Django admin interface to learning_sequences for the
support team and debugging.
* Adds two new functions to the learning_sequences public API:
key_supports_outlines and get_course_keys_with_outlines
The learning_sequences app isn't supposed to know about contentstore or
modulestore, as it's intended to be extracted out of edx-platform in the
long term. Therefore, the backfill_course_outlines command is in
contentstore, and not learning_sequences.
This work was tracked in TNL-7983, but it also fixes a bug where we were
trying to generate course outlines for libraries (TNL-7981).
All Open edX instances upgrading to Lilac should run the
backfill_course_outlines command as part of their upgrade process.
COURSE_CERT_DATE_CHANGE was being called before saving the new data in
the course overview. The listeners were expecting to pull the data out
of the course overview, and thus were only right about half the time.
This moves the signal to trigger after the course publish signals are
handled.
This commit removes several waffle toggles that have been enabled
on edx.org for years. It's time to remove the rollout gating for
these features and enable them by default.
This doesn't directly change any behavior. But it does create new
database objects by default now and allows for enabling other
schedule based features more easily.
Specifically, the following toggles were affected.
schedules.create_schedules_for_course
- Waffle flag removed as always-enabled
- We now always create a schedule when an enrollment is created
schedules.send_updates_for_course
- Waffle flag removed as always-enabled
- Course update emails are sent as long as the ScheduleConfig
allows it.
- This is not a change in default behavior, because ScheduleConfig
is off by default.
dynamic_pacing.studio_course_update
- Waffle switch removed as always-enabled
- Course teams can now always edit course updates directly in Studio
ScheduleConfig.create_schedules
ScheduleConfig.hold_back_ratio
- Model fields for rolling out the schedules feature
- Schedules are now always created
- This commit only removes references to these fields, they still
exist in the database. A future commit will remove them entirely
This commit also adds a new has_highlights field to CourseOverview.
This is used to cache whether a course has highlights, used to
decide which course update email behavior they get. Previously every
enrollment had to dig into the modulestore to determine that.
