This commit introduces several improvements to database migration
scripts to enhance compatibility between MySQL and PostgreSQL, ensure
case-sensitive behavior where needed, and improve migration safety and
correctness. The changes include dynamic SQL generation based on the
database engine, improved transaction handling, and adjustments to
field types and adapters for better cross-database support.
Database compatibility and case sensitivity improvements:
- Migration scripts in split_modulestore_django and learning_sequences
now dynamically generate SQL statements for altering column case
sensitivity and uniqueness based on whether the database is MySQL or
PostgreSQL, ensuring correct behavior across both backends.
- common/djangoapps/split_modulestore_django/migrations/0001_initial.py
- openedx/core/djangoapps/content/learning_sequences/migrations/0001_initial.py
- The courseware.fields module now checks for "postgresql" in the
database engine string instead of a specific backend name, improving
compatibility with different PostgreSQL drivers.
- lms/djangoapps/courseware/fields.py
- The 0011_csm_id_bigint migration in courseware now supports both MySQL
and PostgreSQL for altering column types, with specific SQL for each
backend.
- lms/djangoapps/courseware/migrations/0011_csm_id_bigint.py
- The 0009_readd_facebook_url migration in course_overviews now
introspects the table structure using backend-specific SQL for MySQL
and PostgreSQL, ensuring correct detection of existing fields.
- openedx/core/djangoapps/content/course_overviews/migrations/0009_readd_facebook_url.py
Migration safety and correctness:
- Service user creation and deletion in the commerce app is now wrapped
in atomic transactions to ensure database consistency.
- lms/djangoapps/commerce/migrations/0001_data__add_ecommerce_service_user.py
- The move_overrides_to_edx_when migration in courseware now specifies
a no-op reverse migration, preventing accidental data loss on migration
rollback.
- lms/djangoapps/courseware/migrations/0008_move_idde_to_edx_when.py
Adapter registration and code cleanup:
- The common_initialization app now registers custom adapters for
CourseLocator and related classes in psycopg2 when using PostgreSQL,
ensuring proper serialization of these types.
- openedx/core/djangoapps/common_initialization/apps.py
- Minor code cleanup and formatting improvements in migration files,
including import order and field formatting for readability.
- lms/djangoapps/grades/migrations/0015_historicalpersistentsubsectiongradeoverride.py
- Returns top parent key instead of boolean in upstream info api
- Adds edited_on raw time in course outline api
- Adds has_changes to course details api
The openedx-learning repo was recently refactored to consolidate its
authoring apps into a single openedx_content app:
https://github.com/openedx/openedx-platform/pull/37924
This commit makes the following changes to accommodate this:
- Bumps the openedx-learning version to 0.31.0 to get the changes.
- Creates new migrations in content_libraries, contentstore, and
modulestore_migrator to foreign key references to authoring apps
to point to the new openedx_content app. This is done without
actually making database changes, since the openedx_content app
models are taking over the existing tables that the authoring apps
once pointed to.
- Creates new squashed migrations in these apps that create these
foreign keys to reference openedx_content app models from the start.
The full rationale for how and why this was done is in the following
openedx-learning ADR:
https://github.com/openedx/openedx-learning/blob/main/docs/decisions/0020-merge-authoring-apps-into-openedx-content.rst
These migrations should run fine from either a from-scratch scenario
(i.e. a new install or CI), or when upgrading from an Ulmo-or-later
database state. If you have a database state that comes from the middle
of the Ulmo development cycle (e.g. October 2025), you may encounter
migration errors in content_libraries, contentstore, or
modulestore_migrator, with an error message complaining about missing
tables. If you receive this message, run the following command:
python manage.py lms migrate openedx_content 0001
Then try to run the migrations for the app that failed. Repeat if
necessary for multiple apps.
- Add GET /api/instructor/v2/courses/{course_id}/group_configurations
- Add GET /api/instructor/v2/courses/{course_id}/group_configurations/{id}
- Create shared constants module for course groups
* refactor: moved remaining feature dicts settings into top-level settings.
* refactor: moved remaining feature dicts settings into top-level settings.
* fix: fixed the test files
* fix: fixed tehe pylint errors
* fix: fixation of the cms ci failure
* fix: fixed remaining feature settings for cms
* fix: added fix for requirements
* fix: added fix for lms tests
* fix: resolved the test views issue
* fix: configured views file and test_views
* fix: fixed lint errors and assertion issues
* fix: added fix for base url issue in test view
* fix: added fix for base_url and assertion issue
* fix: added configurations for base utl fix
* fix: handled none issue for mfe config
* fix: corrected override settings in test views
* fix: added getattr defensive technique for view settings
* fix: reverted views and test_views file
* fix: added settings in views file
* fix: added with patch within functions in test view
* fix: rearranged the features in default_legacy_config
* fix: fixing the tests with clearing cache
* fix: reverted test views to verify the CI check
* fix: added cache clear in mfe config test
* fix: fixed the patch toggles to override settings
* fix: fixed the lint errors
* fix: changed patch toggle to override settings
This commit updates the logic in the build_block_structure function to
ensure that block locations are consistently normalized by removing
branch and version information. This change addresses issues when
creating a BlockStructure from modulestore using the published_only
branch. Without this change, we end up comparing versioned keys to
unversioned ones later on, which always yields False.
Sphinx was complaining about these code-block annotations so don't claim
a language since the code doesn't fully match that language and the
parsing fails.
* refactor(certificates): replace direct model imports with data classes and APIs
* fix: use Certificates API to create certificates
* docs: update docstring for get_certificate_for_user
* fix: remove trailing whitespace
---------
Co-authored-by: coder1918 <ram.chandra@wgu.edu>
Co-authored-by: Deborah Kaplan <deborahgu@users.noreply.github.com>
refactor: move editor_saved to VideoConfigService (#37829)
* This moves edx-platform-specific logic out of the VideoBlock,
in preparation for the VideoBlock extraction
Adds API to fetch all legacy library content blocks that are ready to be updated to use library v2 and convert to item banks.
Also adds API to update all the references via a user celery task and to fetch its status.
The "overview" and "about_sidebar_html" fields in the
CoursewareInformation view (/api/courseware/course/{courseId}) were
returning unsanitized HTML and relying on the client to sanitize it.
This commit shifts that work to the server side (clean_dangerous_html)
to remove potentially dangerous tags when generating the response. The
source of this data is modified in the "Settings and Details" section
of a course in Studio.
There is no way to resume either the backup or restore library actions,
i.e. if you navigate away from it, you have to do it again. This is a
limitation of the current UI because we wanted to get something quick
and simple in for Ulmo, but it also reflects the fact that library
backup/restore should be much faster than course import/export has
historically been.
In any case, sending an email for a 5-10 second task is unnecessary and
distracting, so this commit suppresses the email.
Note: I'm using local imports to get around the fact that the
content_libraries public API is used by content_libraries/tasks.py
which defines the tasks. I can't import from content_libraries/tasks.py
directly, because that would violate import linter rules forbidding
other apps from importing things outside of api.py. This isn't ideal,
but it keeps the fix small and it keeps the logic in the
content_libraries app.
