This commit introduces several improvements to database migration
scripts to enhance compatibility between MySQL and PostgreSQL, ensure
case-sensitive behavior where needed, and improve migration safety and
correctness. The changes include dynamic SQL generation based on the
database engine, improved transaction handling, and adjustments to
field types and adapters for better cross-database support.
Database compatibility and case sensitivity improvements:
- Migration scripts in split_modulestore_django and learning_sequences
now dynamically generate SQL statements for altering column case
sensitivity and uniqueness based on whether the database is MySQL or
PostgreSQL, ensuring correct behavior across both backends.
- common/djangoapps/split_modulestore_django/migrations/0001_initial.py
- openedx/core/djangoapps/content/learning_sequences/migrations/0001_initial.py
- The courseware.fields module now checks for "postgresql" in the
database engine string instead of a specific backend name, improving
compatibility with different PostgreSQL drivers.
- lms/djangoapps/courseware/fields.py
- The 0011_csm_id_bigint migration in courseware now supports both MySQL
and PostgreSQL for altering column types, with specific SQL for each
backend.
- lms/djangoapps/courseware/migrations/0011_csm_id_bigint.py
- The 0009_readd_facebook_url migration in course_overviews now
introspects the table structure using backend-specific SQL for MySQL
and PostgreSQL, ensuring correct detection of existing fields.
- openedx/core/djangoapps/content/course_overviews/migrations/0009_readd_facebook_url.py
Migration safety and correctness:
- Service user creation and deletion in the commerce app is now wrapped
in atomic transactions to ensure database consistency.
- lms/djangoapps/commerce/migrations/0001_data__add_ecommerce_service_user.py
- The move_overrides_to_edx_when migration in courseware now specifies
a no-op reverse migration, preventing accidental data loss on migration
rollback.
- lms/djangoapps/courseware/migrations/0008_move_idde_to_edx_when.py
Adapter registration and code cleanup:
- The common_initialization app now registers custom adapters for
CourseLocator and related classes in psycopg2 when using PostgreSQL,
ensuring proper serialization of these types.
- openedx/core/djangoapps/common_initialization/apps.py
- Minor code cleanup and formatting improvements in migration files,
including import order and field formatting for readability.
- lms/djangoapps/grades/migrations/0015_historicalpersistentsubsectiongradeoverride.py
The openedx-learning repo was recently refactored to consolidate its
authoring apps into a single openedx_content app:
https://github.com/openedx/openedx-platform/pull/37924
This commit makes the following changes to accommodate this:
- Bumps the openedx-learning version to 0.31.0 to get the changes.
- Creates new migrations in content_libraries, contentstore, and
modulestore_migrator to foreign key references to authoring apps
to point to the new openedx_content app. This is done without
actually making database changes, since the openedx_content app
models are taking over the existing tables that the authoring apps
once pointed to.
- Creates new squashed migrations in these apps that create these
foreign keys to reference openedx_content app models from the start.
The full rationale for how and why this was done is in the following
openedx-learning ADR:
https://github.com/openedx/openedx-learning/blob/main/docs/decisions/0020-merge-authoring-apps-into-openedx-content.rst
These migrations should run fine from either a from-scratch scenario
(i.e. a new install or CI), or when upgrading from an Ulmo-or-later
database state. If you have a database state that comes from the middle
of the Ulmo development cycle (e.g. October 2025), you may encounter
migration errors in content_libraries, contentstore, or
modulestore_migrator, with an error message complaining about missing
tables. If you receive this message, run the following command:
python manage.py lms migrate openedx_content 0001
Then try to run the migrations for the app that failed. Repeat if
necessary for multiple apps.
Required upgrade of Django.
Commit generated by workflow `openedx/openedx-platform/.github/workflows/upgrade-one-python-dependency.yml@refs/heads/master`
Add a new endpoint to get the unit extensions for a course. Update edx-when from 3.0.0 to 3.1.0 in order to get data necessary for response.
---------
Co-authored-by: Daniel Wong <danieleduardo.wongfa@wgu.edu>
There was a bug in celery 5.6.1 that caused having eta/countdown
parameters being stuck in received and never executing until we
downgraded to 5.6.0. This issue appears to be been fixed in 5.6.2.
For some reason sphinx is being listed as a thing that requires
`roman-numerals` during the upgrade call but not during
`compile-requirements`. This just commits the expectations from
compile-requirements for now so we can land the package update.
This commit implements a comprehensive solution for test score integration in the
enhancement system along with improvements to the score rendering mechanism. Key
changes include:
- Add event handler for rendering blocks with edx-submissions scores
- Implement event-based mechanism to render XBlocks with scoring data
- Create signal handlers in handlers.py to process external grader scores
- Develop specialized XBlock loader for rendering without HTTP requests
- Add queue_key propagation across the submission pipeline
- Register submission URLs in LMS routing configuration
- Add complete docstrings to score render module for better code maintainability
- Add ADR for XBlock rendering with external grader integration
- Add openedx-events fork branch as a dependency in testing.in
- Upgrade edx submission dependency
These changes support the migration from traditional XQueue callback HTTP requests
to a more robust event-based architecture, improving performance and reliability
when processing submission scores. The included ADR documents the architectural
decision and implementation approach for this significant improvement to the
external grading workflow.
* chore: Upgrade Python requirements
* build: Downgrade edx-submissions for now.
The latest version changes a method signature and so code here will need
to be updated before that can land. There is already a PR to pick up
that change https://github.com/openedx/edx-platform/pull/37528 so it
doesn't seem worth it to add the constraint so just downgrade the
package in this PR so we can land the rest of the updates.
---------
Co-authored-by: Feanil Patel <feanil@axim.org>
BREAKING CHANGE: All references to the hardcoded 'proctortrack' string have
been removed from the codebase, as well as the `studio.show_review_rules`
waffle flag. These were used to determine whether an escalation email is
required and whether review rules should be shown. These decisions are now made
based on the value of 'requires_escalation_email' (default False) and 'show_review_rules'
(default True) config items in the PROCTORING_BACKENDS entry.
Additionally:
* The proctoring info api will now return the list of providers which require an escalation
email so that frontend-app-learning does not need to use a hardcoded check agaist
the provider name 'proctortrack'.
* Removed translation commands, mock variables and user facing strings that contained
'proctortrack'.
* Updated all test cases that were using proctortrack to use fake providers names.
Part of: https://github.com/openedx/edx-platform/issues/36329
Previously we would upgrade pip before we upgrade pip-tools. This
breaks when the latest version of pip is not compatible with the current
version of pip-tools as happened with https://github.com/jazzband/pip-tools/issues/2252
If we re-order the steps so that we upgrade pip-tools first, we know
that this upgrade call will work since it will run with the versions of
pip and pip-compile that ran the last full upgrade.
However in this case the pip.txt file is redundant as the pip-tools.txt
file already has the latest version of pip that is compatible with the
current version of pip-tools being installed.
This changeset also updates the compile-requirements command to ignore
the common_constraints entry for pip so that we can verify the upstream
fix of pip-compile before we remove the entry from
common_constraints.txt upstream.
* feat: add the authz check to the library api function
feat: add the authz publish check in rest_api blocks and containers
feat: add the authz checks in libraries and refactor
feat: add collections checks
feat: update enforcement in serializer file
refactor: refactor the permission check functions
fix: fix value error
fix: calling the queries twice
* test: add structure for test and apply feedback
refactor: refactor the tests and apply feedback
fix: apply feedback
Revert "refactor: refactor the tests and apply feedback"
This reverts commit aa0bd527dd7bc7dec4a7ad7adb41a3c932f4a587.
refactor: use constants and avoid mapping
test: fix the test to have them in order
docs: about we rely on bridgekeeper and the old check for two cases
docs: update openedx/core/djangoapps/content_libraries/api/libraries.py
Co-authored-by: Maria Grimaldi (Majo) <maria.grimaldi@edunext.co>
refactor: use global scope wildcard instead of *
refactor: allow receiving PermissionData objects
refactor: do not inherit from BaseRolesTestCase to favor CL setup methods
If both BaseRolesTestCase and ContentLibrariesRestApiTest define a method
with the same name (e.g., setUp()), Python will use the one found first
in the MRO, which is the one in BaseRolesTestCase because it is
listed first in the class definition leading to unexpected behavior.
refactor: remove unnecessary imports and indent
* chore: bump openedx-authz version
This pulls in publishing dependency changes from:
https://github.com/openedx/openedx-learning/pull/369
This fixes a bug where publishing a Content Library v2 container would
publish only its direct children instead of publishing all ancestors.
Co-authored-by: Kyle McCormick <kyle@axim.org>
This dependency is specific to edx.org and should not be in the default
version of the edx-platform. There is related code in edx-platform but
circuit breakers should keep that from running for now until we can
clean it up.
BREAKING CHANGE: If you are relying on the edx-name-affirmation app
working, you should install it yourself before running the platform.
