Commit Graph

23 Commits

Author SHA1 Message Date
Sagirov Evgeniy
42246c858d feat!: remove code related to Old -> Split migration (#30955)
This commit removes code that was used to copy Old Mongo courses into
new Split Mongo courses. This includes both the migrate_to_split
management command, as well as the backend code that would be invoked
to re-run Old Mongo courses as Split courses using Studio (the UI for
this was already removed in b429e55c).

This is a part of the Old Mongo removal effort tracked in:
  https://github.com/openedx/public-engineering/issues/62
2022-09-19 10:51:40 -04:00
Jawayria
c42e356f95 chore: Applied lint-amnesty on cms/djangoapps/contentstore/tests 2021-12-10 19:16:43 +05:00
Braden MacDonald
6c85668099 feat: write split modulestore's course indexes to Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-26 10:06:52 -07:00
David Ormsbee
ae124bd554 Revert "feat: store split modulestore's course indexes in Django/MySQL"
This reverts commit 96e5ff8dce.
2021-10-07 15:07:42 -04:00
Braden MacDonald
96e5ff8dce feat: store split modulestore's course indexes in Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-07 10:59:47 -04:00
M. Zulqarnain
fa271dafe9 BOM-2369 (C): pyupgrade on contentstore/tests (#26759)
* pyupgrade on contentstore/tests
2021-03-04 14:37:39 +05:00
usamasadiq
7bbde8f0f5 Applied pylint-amnesty 2021-02-08 13:00:22 +05:00
Kyle McCormick
151bd13666 Use full names for common.djangoapps imports; warn when using old style (#25477)
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
2020-11-10 07:02:01 -05:00
Kyle McCormick
538dbf696b Import CMS code using fully qualified module names
or, using proper the proper Python 3 relative imports
(e.g., 'from .views import x') which are standard and
unambiguous.
2020-10-19 09:34:31 -04:00
Feanil Patel
9cf2f9f298 Run 2to3 -f future . -w
This will remove imports from __future__ that are no longer needed.

https://docs.python.org/3.5/library/2to3.html#2to3fixer-future
2019-12-30 10:35:30 -05:00
Ayub khan
171b66114c INCR-329 python3 compatibility 2019-07-09 13:17:51 +05:00
cahrens
6f71706b17 Remove SlashSeparatedCourseKey from tests. 2017-08-07 14:49:30 -04:00
Andy Armstrong
ebb04ca396 Reorder Studio imports using isort 2017-05-30 17:00:50 -04:00
Awais Jibran
5e91c37b4e Configuration error on course re-run
TNL 2098
2015-05-11 16:19:21 +05:00
Brandon DeRosier
046e295322 Change has_course_access definition and doc string
- has_course_access renamed to has_course_author_access for clarity
- Changed doc string to clearly state that it determines whether or not
  a user has write access to a course
2014-12-02 22:08:58 -05:00
Brandon DeRosier
bdc64a7c70 Moved has_course_access from CMS to student.auth (common) 2014-12-02 19:36:38 -05:00
Ben McMorran
62643a5941 Fix broken rerun test 2014-08-21 15:43:22 -04:00
Don Mitchell
85f5c49bb7 Unit tests for rerun
along w/ fixes to make them work
LMS-11011
2014-08-21 12:11:29 -04:00
Don Mitchell
3c3a316f08 Make it branch agnostic
LMS-11210

Conflicts:
	common/lib/xmodule/xmodule/modulestore/split_mongo/split_draft.py
2014-08-18 16:19:46 -04:00
Don Mitchell
639658913d Refactor split migrator
LMS-2936

Also, a bunch of ancillary cleanups.

Conflicts:
	common/lib/xmodule/xmodule/modulestore/tests/test_publish.py

Conflicts:
	cms/djangoapps/contentstore/management/commands/migrate_to_split.py
	cms/djangoapps/contentstore/management/commands/tests/test_rollback_split_course.py
	common/lib/xmodule/xmodule/modulestore/__init__.py
	common/lib/xmodule/xmodule/modulestore/mixed.py
	common/lib/xmodule/xmodule/modulestore/mongo/draft.py
	common/lib/xmodule/xmodule/modulestore/split_migrator.py
	common/lib/xmodule/xmodule/modulestore/split_mongo/split.py
	common/lib/xmodule/xmodule/modulestore/tests/test_split_migrator.py
	common/lib/xmodule/xmodule/modulestore/tests/test_split_w_old_mongo.py
2014-07-14 11:20:29 -04:00
Mat Peterson
98649072bd skipping test_clone_course 2014-07-11 15:40:09 -04:00
Nimisha Asthagiri
ce9aea80b1 Move context_managers to base class, overridden by mixed. 2014-07-09 15:55:53 -04:00
Don Mitchell
cc6dfbbc75 Asset support in split
LMS-2876
2014-07-07 17:11:43 -04:00