Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.
By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
Sanitizes Markdown that goes back and forth between the server and
client side, to strip out data: links, so that they cannot be abused.
There is no present vulnerability to this issue–modern browsers disallow
data links in the first place, and we already filter this out in both
client-side code as well as the HTML generated in the REST API (it's run
through bleach). But we're adding this anyway, to further reduce the
odds that some client-side mistake could cause a vulnerability. This is
part of TNL-8589.
This helper is used by the LMS, CMS, _and_ `openedx.core`,
so let's move it to `openedx.core` to reduce import complexity.
The following files no longer import from LMS:
- cms/djangoapps/contentstore/management/commands/edit_course_tabs.py
- lms/djangoapps/ccx/migrations/0006_set_display_name_as_override.py
- openedx/core/djangoapps/ccxcon/api.py
- openedx/core/djangoapps/verified_track_content/models.py
- openedx/features/course_experience/plugins.py
Note: The LTI XBlock has a dependency on this import path (!?);
a fix can be found here [1].
- [1] https://github.com/edx/xblock-lti-consumer/pull/154
* Python code cleanup by the cleanup-python-code Jenkins job.
This pull request was generated by the cleanup-python-code Jenkins job, which ran
```
cd lms/djangoapps/dashboard; find . -type f -name '*.py' | while read fname; do sed -i 's/ # lint-amnesty, pylint: disable=super-with-arguments//; s/ # lint-amnesty, pylint: disable=import-error, wrong-import-order//; s/ # lint-amnesty, pylint: disable=wrong-import-order//' "$fname"; done; find . -type f -name '*.py' | while read fname; do pyupgrade --exit-zero-even-if-changed --py3-plus --py36-plus --py38-plus "$fname"; done; isort --recursive .
```
The following packages were installed:
`pyupgrade,isort`
* feedback done
Co-authored-by: Zulqarnain <muhammad.zulqarnain@arbisoft.com>
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
- Looks at masquerading config for dates, outline, metadata, and
celebration APIs in course_home_api / courseware_api.
- Consolidates and cleans up places we check whether masquerading
gives us full access to a course.
This is a follow up from MST-16, which was commited
in 3858036a4e.
Changes:
* Enrich course teams_configuration from a plain Dict
to a custom XBlock field that uses the new TeamsConfig
wrapper class.
* Remove teams_conf property from course, as the previous
change made it redundant.
* Update teams_enabled implementation.
* Remove teams_max_size field from course, which is
no longer semantically correct, as max team size
is now defined on a teamset level.
* Remove teams_topics in order to discourage use of raw
teams config dict.
* Add convenience properties teamsets and teamsets_by_id
to course.
* Allow periods and spaces in teamset IDs to avoid breaking
existing course teams.
Some parts of the code still use the old raw config data
(identifiable by searching "cleaned_data_old_format"),
which we expect to be slowly factored away as we build
new teams features. MST-40 has been created to remove any
remaining references if necessary.
MST-18
* fix: bokchoy test
* fix: remove pdb break
Request object was mocked but mocked value was never assigned.
In py2 moked object call is always successful. But in py3 if
value is not properly assigned it returns None. Causing the test
to fail. Removing the mocked object also fails the test so
removing it as broken test.
