Commit Graph

322 Commits

Author SHA1 Message Date
Kyle McCormick
739e8a1e71 Stringify keys in backfill_orgs_and_org_courses (#25802)
The command was failing when it encountered both Old Mongo
and Split Mongo course keys, as it tried to `sort` the keys,
but Opaque Keys are only comparable if they are of the same
type. The solution is to convert them to strings before sorting.
The edx-organizations code that instruments the backfill knows
to parse them back into CourseKeys.
2020-12-08 09:36:46 -05:00
Kyle McCormick
4dda73d797 [BD-14] Migrate all environments to use database-backed organizations (#25153)
* Install `organizations` app into LMS and Studio non-optionally.
* Add toggle `ORGANIZATIONS_AUTOCREATE` to Studio.
* Remove the `FEATURES["ORGANIZATIONS_APP"]` toggle.
* Use the new `organizations.api.ensure_organization` function to
  either validate or get-or-create organizations, depending
  on the value of `ORGANIZATIONS_AUTOCREATE`,
  when creating course runs and V2 content libraries.
  We'll soon use it for V1 content libraries as well.
* Remove the `util.organizations_helpers` wrapper layer
  that had to exist because `organizations` was an optional app.
* Add `.get_library_keys()` method to the Split modulestore.
* Add Studio management command for backfilling organizations tables
  (`backfill_orgs_and_org_courses`).

For full details, see
https://github.com/edx/edx-organizations/blob/master/docs/decisions/0001-phase-in-db-backed-organizations-to-all.rst

TNL-7646
2020-12-02 13:58:40 -05:00
Kyle McCormick
9f239ffe8f Add reset_course_content Studio management command (#25653)
Given a course key and a split-mongo version GUID,
it resets the course run's draft branch to a specified
version and publishes.

The purpose of this is to allow us to restore overwritten
course content without having to write to Mongo via a
dbshell.

Adds `reset_course_to_version` method to modulestore API.

TNL-7705
2020-11-23 15:14:10 -05:00
Diana Huang
65d0c11591 Catch individual errors instead of failing the entire job for course
indexing.
2020-11-17 13:23:13 -05:00
Diana Huang
afeea75d9e Add changes to support ES7 and edx-search 2.0 back into the tree. 2020-11-17 13:23:13 -05:00
Diana Huang
8d652e6a4b Revert "Merge pull request #25515 from edx/diana/edx-search"
This reverts commit 9714d7412f, reversing
changes made to 5bc7c3996b.
2020-11-13 13:16:16 -05:00
Diana Huang
33d0a18d56 Update edx-platform to use Elasticsearch 7.
This reverts commit b73f755487.
2020-11-13 11:13:01 -05:00
Kyle McCormick
151bd13666 Use full names for common.djangoapps imports; warn when using old style (#25477)
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
2020-11-10 07:02:01 -05:00
Feanil Patel
b73f755487 Revert "Feanil/edx search update (#25406)" (#25412)
This reverts commit 581b20056d.
2020-10-22 15:53:57 -04:00
Feanil Patel
581b20056d Feanil/edx search update (#25406)
OeX_ES-44:  Update edx-search elasticsearch.


* removes redundant doc_types from the lms/cms, replaces courseware
index with doc_types to two indices
* code polishing

* Use a new config variable for ES7 deployment. (#2226)

Co-authored-by: Golub-Sergey <1.golub.sergey.1@gmail.com>
Co-authored-by: Diana Huang <diana.k.huang@gmail.com>
2020-10-22 14:18:19 -04:00
Kyle McCormick
a3deb6e317 Sort imports in cms/
Command: `isort --recursive cms/`
2020-10-19 09:34:31 -04:00
Kyle McCormick
538dbf696b Import CMS code using fully qualified module names
or, using proper the proper Python 3 relative imports
(e.g., 'from .views import x') which are standard and
unambiguous.
2020-10-19 09:34:31 -04:00
Kyle McCormick
7538691484 Patch CMS app code in tests using fully qualified names 2020-10-19 09:34:31 -04:00
Troy Sankey
26276c796d export_olx management command: write bytes to stdout correctly
This must have been broken ever since we upgraded from Python 2 to 3.

DENG-379
2020-09-10 21:17:17 -04:00
mariajgrimaldi
003ccf4f1f changed insert to insert_one (#23908)
I wanted to make a byte-sized contribution but there were no Jira tickets so we decided, thanks to a conversation with @jmbowman through the Open Edx Community #incr (Slack) channel, to collaborate in the elimination of warnings listed in the Warnings Report at https://build.testeng.edx.org/job/edx-platform-python-pipeline-master/warning_5freport_5fall_2ehtml/

This PR contributes to the elimination of deprecation warnings, specifically the one mentioned above and reported in the Warnings Report.

Changed insert to insert_one in the following file:

* cms/djangoapps/contentstore/management/commands/tests/test_cleanup_assets.py

This warning occurs due to deprecation: https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert
2020-05-08 12:04:29 -04:00
Aarif
6ee2089077 fixed warnings for wrong-import-order 2020-04-08 23:43:06 +05:00
Awais Qureshi
195d225044 BOM-1417 (#23476)
Fixing management command.

Django PR removed force_str method: dc8834cad4 (diff-55346957c9b0473492d19a390a7adb47L36)

None is not working now.
2020-03-30 14:10:58 -04:00
Tim McCormack
11fc118402 Remove django-celery from transitive deps; remove unused tasks (#23318)
The video thumbnail and transcript tasks were the only things using chord_task from
edx-celeryutils, which in turn was blocking django-celery removal. But, they're no longer used.

Co-authored-by: Diana Huang <diana.k.huang@gmail.com>
2020-03-05 21:57:12 +00:00
Feanil Patel
046feb0cf0 Merge pull request #22649 from edx/feanil/fix_pep8
Fix all E303 pep8 errors.
2019-12-30 13:32:26 -05:00
Feanil Patel
6e3fe00fff Fix all E303 pep8 errors. 2019-12-30 12:25:38 -05:00
Feanil Patel
2df8b8226b Merge pull request #22643 from edx/feanil/2to3_asserts
Run `2to3 -f asserts . -w` on edx-platform.
2019-12-30 12:13:42 -05:00
Feanil Patel
39d3a7ca17 Fix pep8 issues caused by the fixer. 2019-12-30 11:42:04 -05:00
Feanil Patel
9cf2f9f298 Run 2to3 -f future . -w
This will remove imports from __future__ that are no longer needed.

https://docs.python.org/3.5/library/2to3.html#2to3fixer-future
2019-12-30 10:35:30 -05:00
Feanil Patel
70294b0ad2 Run 2to3 -f asserts . -w on edx-platform.
https://docs.python.org/3.5/library/2to3.html#2to3fixer-asserts
2019-12-30 10:22:19 -05:00
Feanil Patel
9610aab0f8 Fix more cms contentstore tests. 2019-10-08 15:26:52 -04:00
Feanil Patel
2d766e7b5d Skip tests that can't succeed in Python3 + Django 1.11
There is a bug in the underlying management/base code that tries to make
all manageent command output be unicode.  This management command
outputs the binary tar file data and so breaks in python3.  In python2
the code is happy to pass bytes back and forth and in later versions of
django this is fixed.  Howevere it's not possible to get this test to
pass in Python3 and django 1.11
2019-10-08 15:26:52 -04:00
Awais Qureshi
9b39f91599 BOM-529
Python3 fix.
2019-09-30 12:02:54 +05:00
Ned Batchelder
efba6e45e8 Finally import courseware properly everywhere to get rid of a warning 2019-09-26 09:40:15 -04:00
Jeremy Bowman
fc57bf9763 Use bytes with GridFS when not specifying an encoding - BOM-157 2019-09-19 11:17:01 -04:00
Ayub khan
d7d1f9ddaf BOM-455
python3 compatibility
2019-09-11 17:01:25 +05:00
Ayub khan
52f9544b6a BOM-368
python3 compatibility
2019-09-05 12:40:39 +05:00
Ayub khan
f5d97260a5 Fixed all py3 tests with AssertionError: "Error: too few arguments" 2019-08-26 14:19:57 +05:00
Ayub khan
8a95a8e520 BOM-95
assertItemsEqual with six.assertCountEqual
2019-08-21 17:01:40 +05:00
aarif
61800c619f replaced StringIO imports with import from six
replaced cStringIO and StringIO imports with import from six

fixed StringIO imports

fixed StringIO imports

fixed XSS python-wrap warning
2019-08-08 00:09:16 +05:00
uzairr
80316c632d Update Help msgs of management commands
PROD-399
2019-08-02 15:20:26 +05:00
Usman Khalid
8dfc8e5fda Convert AboutModule, CourseInfoModule, HtmlModule and StaticTabModule to XBlocks. 2019-07-24 20:19:44 +05:00
Ayub
cc07c7c97f Merge pull request #20991 from edx/INCR-336
INCR-336 python3 compatibility
2019-07-16 11:39:10 +05:00
Aarif
8bdd2b29d2 INCR-337 Python 3 compatibility (#20993)
* ran python-modernize and isort on files specified in INCR-337

* changes made to comply with pylint
2019-07-15 14:12:52 +05:00
Ayub
56170466f1 Merge pull request #20988 from edx/INCR-335
INCR-335 python3 compatibility
2019-07-15 12:29:55 +05:00
Ayub
30aaeb7d4e Merge pull request #20986 from edx/INCR-334
INCR-334 python3 compatibility
2019-07-15 12:29:15 +05:00
arbisoft
32e89b48bc Fixing python-modernize issues. 2019-07-09 16:25:08 +05:00
Ayub khan
b06fcd27d2 INCR-334 python3 compatibility 2019-07-09 15:43:24 +05:00
Ayub khan
5613797aa8 INCR-336 python3 compatibility 2019-07-09 15:35:33 +05:00
Ayub khan
d7daac92e7 INCR-335 python3 compatibility 2019-07-09 14:43:10 +05:00
Zachary Hancock
b6b04a4551 sync courses command (#20946) 2019-07-08 10:45:36 -04:00
Usman Khalid
0b27c62ca6 Convert VideoModule to VideoBlock.
Some deprecated functionality has been removed:

- Reading data field and transforms being applied in the init() method.
- The source field.
- The source_visible attribute.
2019-06-20 22:12:39 +05:00
Nimisha Asthagiri
dfa3728edf Move django_comment_common from common to openedx 2019-05-03 12:10:18 -04:00
Nimisha Asthagiri
ef0e06cc0a Revert "Discussions consolidation" 2019-05-02 14:59:56 -04:00
Nimisha Asthagiri
df962a31b7 Move django_comment_common to openedx/core/djangoapps/discussion_common 2019-05-01 19:15:02 -04:00
Jeremy Bowman
d866fed2f1 Fix import order issues in contentstore 2019-02-14 14:20:47 -05:00