Description
This is a follow up to #29058 and #29413. This is the next step in moving part of the modulestore data (the course indexes / "active versions" table) from MongoDB to MySQL.
There are four steps planned in moving course index data to MySQL:
Step 1: create the tables in MySQL, start writing to MySQL + MongoDB ✅ done
Step 2: migrate all remaining courses to MySQL ✅ done
Step 3: switch reads from MongoDB to MySQL (this PR)
Step 4 (much later, once we know this is working well): stop writing to MongoDB altogether.
Supporting information
OpenCraft Jira ticket: MNG-2557
Status
✅ Tested with a large Open edX instance is in progress.
Testing instructions
Try making changes in Studio and verify that they work fine.
Deadline
None
When doing import, we ran into an issue where there was a learner with an external program key that was the
same as an existing, completely unrelated edX account username. Rather than try to guess which learner we want
or do the lookups backwards, it seemed that splitting these columns to avoid any ambiguity would be the most
straightforward and simple approach
It's long past time that the default test modulestore was Split,
instead of Old Mongo. This commit switches the default store and
fixes some tests that now fail:
- Tests that didn't expect MFE to be enabled (because we don't
enable MFE for Old Mongo) - opt out of MFE for those
- Tests that hardcoded old key string formats
- Lots of other random little differences
In many places, I didn't spend much time trying to figure out how to
properly fix the test, and instead just set the modulestore to Old
Mongo.
For those tests that I didn't spend time investigating, I've set
the modulestore to TEST_DATA_MONGO_AMNESTY_MODULESTORE - search for
that string to find further work.
- Adds Enhanced Staff Grader (ESG) backend-for-frontend (BFF) in `lms/djangoapps/ora_staff_grader`
- Adds routing to ESG BFF at `{lms_url}/api/ora_staff_grader/*`
- Adds mock implementation routing at `{lms_url}/api/ora_staff_grader/mock/*`
- Adds `ORA_GRADING_MICROFRONTEND_URL` setting for routing to ESG microfrontend (MFE)
- Updates to the teams app:
- Add`get_teams_in_teamset` to the teams API.
- Add `get_team_names` to teams service.
- Adds `openassessment.staffgrader` app for appropriate ORA migrations.
- Modifies management commands for creation of users.
- Updates test factory to return display org with course overview.
Co-authored-by: jansenk <jkantor@edx.org>
Co-authored-by: Leangseu Kim <lkim@edx.org>
Co-authored-by: Ben Warzeski <bwarzeski@edx.org>
* feat!: Change the way tabs are ordered
The change imposes a new ordering for tabs based on their new priority. When reordering tabs, this ordering will be maintained.
* fix: Apply suggestions from code review
Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>
* fix: review feedback
Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>
* feat: Adds a new discussion topic configuration mechanism
The new discussion configuration system links discussion topics directly to the course structure. This change adds a new task that sends a discussion update signal if there are any changes to the course. This signal includes all the context needed to update the configuration of the course.
The handler for this new event will create a new entry for each unit that needs a topic in the database. In the future this will be used to see the topics in the course.
* fix: add support for marking a provider as supporting LTI
* fix: review feedback
* feat: Add a new way to enable/disable teams
Adds a new mechanism for enabling/disabling the team feature in a course using an 'enabled' field to the teams config.
If this field is set to true, teams is enabled (team sets/groups) still need to be defined. If this is set to false then teams is disabled whether or not team sets are defined.
* fix: review feedback
Instead of hard-coding the "Learn More" and potentially other links for course
apps in the course authoring MFEs this change loads those URLs from the
django settings as part of each individual course app.
* feat: Course Apps API
This adds a new concept called course apps. These are exposed via a new
"openedx.course_app" entrypoint, which helps the LMS and studio discover such
apps and list them in a new rest api for the same.
These course apps will drive the pages and resources view in the course authoring
MFE. This system will track which apps are enabled and which are disabled. It
also allows third-party apps to be listed here by using the plugin entrypoint.
* Apply feedback from review
django-not-configured is an error raised by pylint (with
the pylint-django plugin) when it's not correctly configured.
We should not be applying lint amnesty for such a violation.
This deprecates `save=False` for several functions and removes all known
usages of the parameter but does not actually remove the parameter.
Instead, it will emit a deprecation warning if the parameter is used.
We can remove the parameter as soon as we feel sure nothing is using it.
Now that we have refactored `anonymous_id_for_user` to always prefer
retrieving an existing ID from the database -- and observed that only a
small fraction of calls pass save=False -- we can stop respecting
save=False. This opens the door for future improvements, such as generating
random IDs or switching to the external user ID system.
Metrics: I observe that 1 in 16 requests for new, non-request-cached
anon user IDs are made with save=False. But 71% of all calls are served
from the request cache, and 99.7% of the misses are served from the DB.
save=False only appear to come from intermittent spikes as reports are
generated and are low in absolute number.
Also document usage/risk/rotation of secret in anonymous user ID
generation as indicated by `docs/decisions/0008-secret-key-usage.rst`
ADR on `SECRET_KEY` usage.
ref: ARCHBOM-1683
By explicitly importing the legacy namespace classes, we make it clear
that we are using soon-to-be-deprecated classes. We will then be able to
start removing the legacy classes, one module at a time.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
OeX_ES-44: Update edx-search elasticsearch.
* removes redundant doc_types from the lms/cms, replaces courseware
index with doc_types to two indices
* code polishing
* Use a new config variable for ES7 deployment. (#2226)
Co-authored-by: Golub-Sergey <1.golub.sergey.1@gmail.com>
Co-authored-by: Diana Huang <diana.k.huang@gmail.com>
Instead of going up the stacktrace to find the module names of waffle
flags and switches, we manually pass the module __name__ whenever the
flag is created. This is similar to `logging.getLogger(__name__)`
standard behaviour.
As the waffle classes are used outside of edx-platform, we make the new
module_name argument an optional keyword argument. This will change once
we pull waffle_utils outside of edx-platform.
Note that the module name is normally only required to view the list of
existing waffle flags and switches. The module name should not be
necessary to verify if a flag is enabled. Thus, maybe it would make
sense to create a `add` class methor similar to:
class WaffleFlag:
@classmethod
def add(cls, namespace, flag, module):
instance = cls(namespace, flag)
cls._class_instances.add((instance, module))
