Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
django-not-configured is an error raised by pylint (with
the pylint-django plugin) when it's not correctly configured.
We should not be applying lint amnesty for such a violation.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
* Moving plugins infrastructure to edx-django-utils
This PR extracts the code that enables plugins in edx-platform and puts it in edx-django-utils. This is done to allow other IDAS to add plugin functionality.
* Renamed OAuth2Authentication to BearerAuthentication
* Added back OAuth2Authentication name
-there are libraries such as edx-enterprise that still import OAuth2Authentication. The OAuth2Authentication class should be fully removed when everything is importing BearerAuthentication correctly
- Removing deprecated auth classes
- Replacing OAuth2AuthenticationAllowInactiveUserDeprecated with OAuth2AuthenticationAllowInactiveUser
- the old class was a child class of a deprecated class that we are removing and replacing.
* Added new authentication class(meets drf standards)
The new class replaces the deprecated oauth2authetnication class from rest_framework_auth library(repo django-rest-framework-oauth).
Majority of the code is combination of copy-pasta from old oauth2authentication class and Oauth2AuthenticationAllowInactiveUser class
* Added ability to switch to new authentication class in bookmarks app
* Changed error type reported by Outh class. It now outputs a json rather than a string.
Currently, we are working on removing the rest_framework_auth library from edx-platform. For this push, we need to remove the oauth2Authentication class. This PR creates a new class oauth2AuthenticationDeprecated that adds additional new relic metrics. The metrics would allow us to see how often this class is used and its success rate. The hope is that this information will help us with transitioning to a different authentication class.
made changes to test with previous version of django-waffle
updated the query count to test
testing with version 0.13
testing with version 0.14
testing with version 0.15
added version 0.14
updated the django-waffle version to use 0.18
updated the django-waffle version to use 0.16
updated the query counts to pass tests
ran make upgrade
updated the django-waffle to support django 2.2
made changes to test with previous version of django-waffle
updated the query count to test
testing with version 0.13
testing with version 0.14
testing with version 0.15
added version 0.14
updated the django-waffle version to use 0.18
updated the django-waffle version to use 0.16
updated the query counts to pass tests
removed the pdb statements
ran make upgrade
The API documentation decorators do not have to leak which solution we
use to generate the docs. Here, and as discussed in PR #21820, we rename
the `openapi` module to `apidocs`, and we make sure that this module
includes all the right functions to document API Views without referring
to Open API.
The `swagger_auto_schema` view function decorator allows us to
auto-document API endpoints. All swagger arguments are passed as kwargs
to the decorator, but this feels a bit unnatural: we would like to use
the function documentation as the endpoint summary and description. We
introduce this feature here by reading the `__doc_`_attribute of the
view function.
Some notes:
* The completion API was pulled out into a new repo, but left behind a
url() entry. That entry used a different namespace than the
pulled-out repo, so I had to fix the one place we used the namespace.
* Two urls couldn't be anchored because they broke tests:
url(r'users/(?P<user_id>\w+)/followed$', views.followed_threads, name='followed_threads'),
url(r'users/(?P<user_id>\w+)$', views.user_profile, name='user_profile'),
This adds middleware that will create custom parameter metrics in
New Relic to track the size of all the cookies being received for
our domain. The custom fields are "cookies_total_size" and a
separate named parameter for every cookie size, e.g.
"cookies.csrftoken.size".
This is intended to help us track cookie growth and better diagnose
issues where users lose their sessions. It is toggled by the
'request_utils.capture_cookie_sizes' Waffle Flag.
