Don't assume that there will be an extra `context` kwarg when using the
bookmark serializer. We use it this way in the current code but that's
specific to us and not comon to all serializers. There are a lot of API
documentation tools that automate introspecting serializers but they
won't know that they have to send in a `context` to the serializer.
To make this serializer behave more like other serializers without
changing the behavior, we just need to check that the `context` value is
defined before we dig into it. In the case that there is no `context`
we just treat it the same as if there is no `request` in the `context`.
* test: warn about dependencies from cms->openedx->lms and vice versa
* test: warn about importing from package's internal implementation code
* chore: Update some imports to use public APIs only
* chore: Update 'bookmarks' app to have stricter public API
* fix: we are sharing 'adapters' from olx_rest_api to content_staging
Description
This is a follow up to #29058 and #29413. This is the next step in moving part of the modulestore data (the course indexes / "active versions" table) from MongoDB to MySQL.
There are four steps planned in moving course index data to MySQL:
Step 1: create the tables in MySQL, start writing to MySQL + MongoDB ✅ done
Step 2: migrate all remaining courses to MySQL ✅ done
Step 3: switch reads from MongoDB to MySQL (this PR)
Step 4 (much later, once we know this is working well): stop writing to MongoDB altogether.
Supporting information
OpenCraft Jira ticket: MNG-2557
Status
✅ Tested with a large Open edX instance is in progress.
Testing instructions
Try making changes in Studio and verify that they work fine.
Deadline
None
Convert more tests from MONGO_AMNESTY to SPLIT modulestores.
This is in preparation for just wholesale denying access to Old
Mongo, so I either converted tests to split or just deleted some
test variants that were Old Mongo specific. (e.g. ddt lines)
It's long past time that the default test modulestore was Split,
instead of Old Mongo. This commit switches the default store and
fixes some tests that now fail:
- Tests that didn't expect MFE to be enabled (because we don't
enable MFE for Old Mongo) - opt out of MFE for those
- Tests that hardcoded old key string formats
- Lots of other random little differences
In many places, I didn't spend much time trying to figure out how to
properly fix the test, and instead just set the modulestore to Old
Mongo.
For those tests that I didn't spend time investigating, I've set
the modulestore to TEST_DATA_MONGO_AMNESTY_MODULESTORE - search for
that string to find further work.
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
django-not-configured is an error raised by pylint (with
the pylint-django plugin) when it's not correctly configured.
We should not be applying lint amnesty for such a violation.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
* Moving plugins infrastructure to edx-django-utils
This PR extracts the code that enables plugins in edx-platform and puts it in edx-django-utils. This is done to allow other IDAS to add plugin functionality.
* Renamed OAuth2Authentication to BearerAuthentication
* Added back OAuth2Authentication name
-there are libraries such as edx-enterprise that still import OAuth2Authentication. The OAuth2Authentication class should be fully removed when everything is importing BearerAuthentication correctly
- Removing deprecated auth classes
- Replacing OAuth2AuthenticationAllowInactiveUserDeprecated with OAuth2AuthenticationAllowInactiveUser
- the old class was a child class of a deprecated class that we are removing and replacing.
* Added new authentication class(meets drf standards)
The new class replaces the deprecated oauth2authetnication class from rest_framework_auth library(repo django-rest-framework-oauth).
Majority of the code is combination of copy-pasta from old oauth2authentication class and Oauth2AuthenticationAllowInactiveUser class
* Added ability to switch to new authentication class in bookmarks app
* Changed error type reported by Outh class. It now outputs a json rather than a string.
Currently, we are working on removing the rest_framework_auth library from edx-platform. For this push, we need to remove the oauth2Authentication class. This PR creates a new class oauth2AuthenticationDeprecated that adds additional new relic metrics. The metrics would allow us to see how often this class is used and its success rate. The hope is that this information will help us with transitioning to a different authentication class.
made changes to test with previous version of django-waffle
updated the query count to test
testing with version 0.13
testing with version 0.14
testing with version 0.15
added version 0.14
updated the django-waffle version to use 0.18
updated the django-waffle version to use 0.16
updated the query counts to pass tests
ran make upgrade
updated the django-waffle to support django 2.2
made changes to test with previous version of django-waffle
updated the query count to test
testing with version 0.13
testing with version 0.14
testing with version 0.15
added version 0.14
updated the django-waffle version to use 0.18
updated the django-waffle version to use 0.16
updated the query counts to pass tests
removed the pdb statements
ran make upgrade
The API documentation decorators do not have to leak which solution we
use to generate the docs. Here, and as discussed in PR #21820, we rename
the `openapi` module to `apidocs`, and we make sure that this module
includes all the right functions to document API Views without referring
to Open API.
