Description
This is a follow up to #29058 and #29413. This is the next step in moving part of the modulestore data (the course indexes / "active versions" table) from MongoDB to MySQL.
There are four steps planned in moving course index data to MySQL:
Step 1: create the tables in MySQL, start writing to MySQL + MongoDB ✅ done
Step 2: migrate all remaining courses to MySQL ✅ done
Step 3: switch reads from MongoDB to MySQL (this PR)
Step 4 (much later, once we know this is working well): stop writing to MongoDB altogether.
Supporting information
OpenCraft Jira ticket: MNG-2557
Status
✅ Tested with a large Open edX instance is in progress.
Testing instructions
Try making changes in Studio and verify that they work fine.
Deadline
None
It's long past time that the default test modulestore was Split,
instead of Old Mongo. This commit switches the default store and
fixes some tests that now fail:
- Tests that didn't expect MFE to be enabled (because we don't
enable MFE for Old Mongo) - opt out of MFE for those
- Tests that hardcoded old key string formats
- Lots of other random little differences
In many places, I didn't spend much time trying to figure out how to
properly fix the test, and instead just set the modulestore to Old
Mongo.
For those tests that I didn't spend time investigating, I've set
the modulestore to TEST_DATA_MONGO_AMNESTY_MODULESTORE - search for
that string to find further work.
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
Use the lazy module to create cached attributes
Ensure tests for localized date formatting are insulated from changes in default formatting
Other minor fixes
repairs https://openedx.atlassian.net/browse/TNL-2007 and https://openedx.atlassian.net/browse/TNL-2009
Original Commit Messages:
Update ccx dashboard display to match new course display
Fix display of CCX dates on dashboard to use dates from the CCX instead of the related course.
The fix is implemented by providing date- and course-related APIs on the CCX object itself. These APIs are mimicked from the same APIs available on a course descriptor.
bring the methods for printing start and end datetimes into line with course by adding UTC under the proper circumstances
add test coverage for new APIs defined on the CCX model
fix quality violation
Final official name is Custom Courses for EdX (CCX), rename all code to remove previous name.
Rename the FEATURE constant used to identify this feature
Rename the middleware for the CCX change
rename the constant used for storing the current poc id on the session.
rename the _PocContext threading local
rename the override provider in all places where it is referenced
`PersonalOnlineCoursesOverrideProvider` -> `CustomCoursesForEdxOverrideProvider`
generally rename symbols from overrides.py to replace `poc` with `ccx` where possible without changing model names or attributes
rename more symbols from poc to ccx
rename util functions from app utils module
general symbol renaming poc -> ccx in views.py and related url changes
Rename the coach role wherever it is used.
reword poc_coach to ccx_coach
UI rename
replace POC with CCX globally
template context variable renamed
rename poc_ to ccx_ in urls and all related locations (views, js, scss etc)
remove pocs migrations
Final massive renaming, including models. Re-built migration.
cleaning up a few tailing references
Fix reference typo in schedule template JS
undo modifications made on the fly in test setup to ensure that our tests are properly isolated from the rest of the system tests.
Fixesjazkarta/edx-platform#38
Clean up some leftover strings and comments
fixing more strings and comments in python files
fix a naming error in the schedule tab that was causing problems in deleting courseware items.
Fixesjazkarta/edx-platform#36
updating tests and utility code to match changes in infrastructure from latest rebase
