refactor: move editor_saved to VideoConfigService (#37829)
* This moves edx-platform-specific logic out of the VideoBlock,
in preparation for the VideoBlock extraction
Adds API to fetch all legacy library content blocks that are ready to be updated to use library v2 and convert to item banks.
Also adds API to update all the references via a user celery task and to fetch its status.
Prior to this, if ENABLE_ORGANIZATION_STAFF_ACCESS_FOR_CONTENT_LIBRARIES
was enabled, we would not return the orgs that someone had course
creator rights on, even if ENABLE_CREATOR_GROUP was enabled. (For the
moment, we are conflating "can create courses" with "can create
libraries" for a given org, even though we should probably eventually
split those apart.)
For legacy library_content references in courses, this PR:
- **Removes the spurious sync after updating a reference to a migrated
library**, so that users don't need to "update" their content _after_
updating their reference, _unless_ there were real content edits that
happened since they last synced. We do this by correctly associating a
DraftChangeLogRecord with the ModulestoreBlockSource migration artifact,
and then comparing that version information before offering a sync.
(related issue:
https://github.com/openedx/frontend-app-authoring/issues/2626).
- **Prompts users to update a reference to a migrated library with higher
priority than prompting them to sync legacy content updates for that
reference**, so that users don't end up needing to accept legacy content
updates in order to get a to a point where they can update to V2 content.
- **Ensures the library references in courses always follow the correct
migration,** as defined by the data `forwarded` fields in the data model,
which are populated based on the REST API spec and the stated product UI
requirements.
* For the migration itself, this PR:
- **Allows non-admins to migrate libraries**, fixing:
https://github.com/openedx/edx-platform/issues/37774
- **When triggered via the UI, ensures the migration uses nice title-based
target slugs instead of ugly source-hash-based slugs.** We've had this as an
option for a long time, but preserve_url_slugs defaulted to True instead of
False in the REST API serializer, so we weren't taking advantage of it.
- **Unifies logic between single-source and bulk migration**. These were
implement as two separate code paths, with drift in their implementations. In
particular, the collection update-vs-create-new logic was completely
different for single-souce vs. bulk.
- **When using the Skip or Update strategies for repeats, it consistently
follows mappings established by the latest successful migration** rather than
following mappings across arbitrary previous migrations.
- **We log unexpected exceptions more often**, although there is so much more
room for improvement here.
- **Adds more validation to the REST API** so that client mistakes more often
become 400s with validation messages rather than 500s.
For developers, this PR:
- Adds unit tests to the REST API
- Ensures that all migration business logic now goes through a general-purpose
Python API.
- Ensures that the data model (specifically `forwarded`, and
`change_log_record`) is now populated and respected.
- Adds more type annotations.
Unhandled exception while migration legacy xblocks into new library stops the migration process abruptly causing following issues:
* Components not being collected into Collections for successful migrations
* Data being corrupted for already migrated blocks most likely due to incomplete transaction.
We previously fixed this when the CourseLimitedStaffRole was applied to
a course but did not handle the case where the role is applied to a user
for a whole org. The underlying issue is that the CourseLimitedStaffRole
is a subclass of the CourseStaffRole and much of the system assumes that
subclesses are for giving more access not less access.
To prevent that from happening for the case of the CourseLimitedStaffRole,
when we do CourseStaffRole access checks, we use the strict_role_checking
context manager to ensure that we're not accidentally granting the
limited_staff role too much access.
There is no way to resume either the backup or restore library actions,
i.e. if you navigate away from it, you have to do it again. This is a
limitation of the current UI because we wanted to get something quick
and simple in for Ulmo, but it also reflects the fact that library
backup/restore should be much faster than course import/export has
historically been.
In any case, sending an email for a 5-10 second task is unnecessary and
distracting, so this commit suppresses the email.
Note: I'm using local imports to get around the fact that the
content_libraries public API is used by content_libraries/tasks.py
which defines the tasks. I can't import from content_libraries/tasks.py
directly, because that would violate import linter rules forbidding
other apps from importing things outside of api.py. This isn't ideal,
but it keeps the fix small and it keeps the logic in the
content_libraries app.
* Updates `ModulestoreBlockMigration` table to allow storing `null` values in `target` field for blocks that failed to migrate/import.
* Adds `unsupported_reason` field to store reason for failure.
* Add number of children blocks in failed block `unsupported_reason` field.
* Fixes issue with blocks like `openassessment` where `url_name` field is not included in its olx during serialization.
BREAKING CHANGE: All references to the hardcoded 'proctortrack' string have
been removed from the codebase, as well as the `studio.show_review_rules`
waffle flag. These were used to determine whether an escalation email is
required and whether review rules should be shown. These decisions are now made
based on the value of 'requires_escalation_email' (default False) and 'show_review_rules'
(default True) config items in the PROCTORING_BACKENDS entry.
Additionally:
* The proctoring info api will now return the list of providers which require an escalation
email so that frontend-app-learning does not need to use a hardcoded check agaist
the provider name 'proctortrack'.
* Removed translation commands, mock variables and user facing strings that contained
'proctortrack'.
* Updated all test cases that were using proctortrack to use fake providers names.
Part of: https://github.com/openedx/edx-platform/issues/36329
This change drops the legacy studio custom pages UI aka. the tab edit
page.
This work is part of https://github.com/openedx/edx-platform/issues/36108
BREAKING CHANGE: The 'legacy_studio.custom_pages' waffle flag has been
removed and the code will work as if this flag is permanently set to
False.
Co-authored-by: Kyle McCormick <kyle@axim.org>
Remove the legacy studio updates page and its related artifacts. They
have been replaced by API and a new UI in the authoring MFE.
This cleanup is a part of https://github.com/openedx/edx-platform/issues/36108
BREAKING CHANGE: The 'legacy_studio.updates' waffle flag will no longer
be respected. The system will behave as if the flag is set to false
permanently.
The legacy textbooks page has been replaced with an authoring MFE
equivalent. We don't need to keep the old one around.
This work is part of https://github.com/openedx/edx-platform/issues/36108
BREAKING CHANGE: With this change the `legacy_studio.textbooks` waffle
flag will no longer be respected and the system will behave as if the
flag is always set to False.
As part of the ongoing effort to deprecate and eventually remove xmodule,
we’ve started gradually migrating the necessary code files from xmodule
to more appropriate locations within the codebase.
Ticket: https://github.com/openedx/public-engineering/issues/445
Also: this tweaks importlinter ignores & add follow-up issue links
Co-authored-by: Kyle McCormick <kyle@axim.org>
This PR adds the `/api/modulestore_migrator/v1/library/:libraryId/migrations/courses/` endpoint, which returns all course migrations for a target library.
The OpenAPI docs (at /api-docs) for the modulestore_migrator app were missing
some info, had duplicated info, and had some incorrect info. This brings them
up to a state where they're accurate and should have all the info needed for
someone to integrate with the API. For example:
* We move the viewset class docstrings, which have the info needed for someone
to POST correctly, into the actual POST handler method docstring. That way,
they show up under that POST API endpoint rather than the GET API endpoint.
* We fix the target keys to be v2 keys instead of v1 keys. We add detail on
all the different migration options and why a user would want to specify
them.
* We fix the docs for `parameters` field to explain that they are always
a _list_ of parameter objects, even for non-bulk migration tasks.
This also removes confusing & unnecessary endpoints:
* POST /api/modulestore_migrator/v1/bulk_migration/cancel
* DELETE /api/modulestore_migrator/v1/migrations/<uuid>
Finally, it sorts this endpoint to be newest-first:
* POST /api/modulestore_migrator/v1/migrations
Fixes: https://github.com/openedx/edx-platform/issues/37566
These changes fix a bug in how LTI-based exam due dates are computed and written to the exams service. Prior to this change, an LTI exam due date was computed irrespective of the course pacing type. In certain cases, this caused incorrect due dates to be written to the exams service for LTI-based exams.
For example, if a course team initially develops a course as an instructor-paced course and sets a due date on an exam subsection, that subsection due date is written to the modulestore. If the course team subsequently changes that course pacing type to self-paced, then that due date remains in the modulestore to allow course teams to switch pacing types without erasing due dates. The impact of this is that, when the course is published, the exam subsection due date is written to the exams service as the due date, even though there are no static due dates in a self-paced course. Frequently, these due dates are in the past (e.g. for course reruns), so learners automatically cannot access exams. Even if the due date is manually corrected in the exams service, every course publish reverts the due date to the incorrect due date.
This change computes the due date of LTI-based exams as...
* the exam subsection due date if the course is instructor-paced, if the subsection has a due date; else None
* the course end date if the course is self-paced, if the course has an end date; else None
In order to correct any incorrect due dates, course teams should republish their courses.
* Consolidates and renames the runtime used as a base for all the others:
* Before: `xmodule.x_module:DescriptorSystem` and
`xmodule.mako_block:MakoDescriptorSystem`.
* After: `xmodule.x_module:ModuleStoreRuntime`.
* Co-locates and renames the runtimes for importing course OLX:
* Before: `xmodule.x_module:XMLParsingSystem` and
`xmodule.modulestore.xml:ImportSystem`.
* After: `xmodule.modulestore.xml:XMLParsingModuleStoreRuntime` and
`xmodule.modulestore.xml:XMLImportingModuleStoreRuntime`.
* Note: I would have liked to consolidate these, but it would have
involved nontrivial test refactoring.
* Renames the stub Old Mongo runtime:
* Before: `xmodule.modulestore.mongo.base:CachingDescriptorSystem`.
* After: `xmodule.modulestore.mongo.base:OldModuleStoreRuntime`.
* Renames the Split Mongo runtime, the which is what runs courses in LMS and CMS:
* Before: `xmodule.modulestore.split_mongo.caching_descriptor_system:CachingDescriptorSystem`.
* After: `xmodule.modulestore.split_mongo.runtime:SplitModuleStoreRuntime`.
* Renames some of the dummy runtimes used only in unit tests.
Meilisearch index documents were not synced properly when any block with children blocks like units, subsections, sections etc. were being deleted as the `XBLOCK_DELETED` is only triggered for the deleted block.
This PR fixes it by deleting all index documents that contain the deleted block in its `breadcrumbs` field as only blocks that are children of this block will have it its breadcrumbs field.
Similarly, the entity links that store links between course and library blocks was not synced properly due to children `ContainerLinks` not being deleted.
