Commit Graph

82 Commits

Author SHA1 Message Date
Awais Jibran
1d39e431e3 Use allowed_xblocks settings (#27265)
* Use allowed_xblocks in olxcleaner validate.
2021-04-07 15:45:04 +05:00
Ahtisham Shahid
f3a62b5f89 Error message for proctor exam show to user (#27238)
* Error message for proctor exam visible to user

* Created exception class for Invalid Proctor settings
2021-04-07 10:46:19 +05:00
AsadAzam
c4da6c1fe4 Display import errors to user (#27147)
* Display import errors to user

* Refactored

* Refactored

* Refactored

* Fixed quality

* Fixed quality

* Refactored code

* Fixed message

* Refactored code
2021-04-02 18:53:39 +05:00
Awais Jibran
8023bbc1af Log Errors + Warnings. (#27210) 2021-04-02 16:02:46 +05:00
Awais Jibran
f898fc5d72 Short circuit, library import. (#27195) 2021-04-01 12:29:57 +05:00
Awais Jibran
ad7f6019fe Integrate "olxcleaner" with course import (#27164)
* Integrating olxcleaner in course import

* Adding toggle removal date and addressing pylint issues.
2021-03-31 13:26:14 +05:00
Awais Jibran
5f773d326d Add missing import logs. (#27066) 2021-03-22 15:28:13 +05:00
AsadAzam
1593b40702 Improved logging for import (#27024)
* Improved logging for import

* Refactored code

* Refactored code

* Refactored code

* Refactored code

* Refactored code

* refactored code

* Refactored code

* Updating log

Co-authored-by: Awais Jibran <awaisdar001@gmail.com>
2021-03-18 12:40:38 +05:00
David Ormsbee
f5b74fcf31 feat: Backfill and Django Admin for Learning Sequence Outline
* Adds the backfill_course_outlines management command to contentstore
* Adds a read-only Django admin interface to learning_sequences for the
  support team and debugging.
* Adds two new functions to the learning_sequences public API:
  key_supports_outlines and get_course_keys_with_outlines

The learning_sequences app isn't supposed to know about contentstore or
modulestore, as it's intended to be extracted out of edx-platform in the
long term. Therefore, the backfill_course_outlines command is in
contentstore, and not learning_sequences.

This work was tracked in TNL-7983, but it also fixes a bug where we were
trying to generate course outlines for libraries (TNL-7981).

All Open edX instances upgrading to Lilac should run the
backfill_course_outlines command as part of their upgrade process.
2021-03-09 09:05:57 -05:00
M. Zulqarnain
5562944e87 BOM-2369 (A): pyupgrade on api,contentstore and cms_user_tasks apps under CMS (#26676)
* pyupgrade on cms api,contentstore and cms_user_tasks apps
2021-03-04 15:06:07 +05:00
usamasadiq
7bbde8f0f5 Applied pylint-amnesty 2021-02-08 13:00:22 +05:00
David Ormsbee
4fde69441c Fix build: use @shared_task for update_outline_from_modulestore_task (#26216)
Update to make update_outline_from_modulestore_task use @shared_task
and @set_code_owner_attribute.
2021-01-28 11:22:39 -05:00
David Ormsbee
5353324c80 Merge pull request #26145 from edx/ormsbee/course-outline-publish-tnl-7733
Push Course Outlines to learning_sequences on publish.
2021-01-28 10:21:11 -05:00
David Ormsbee
669677c78a Push Course Outlines to learning_sequences on publish.
The learning_sequences app has its own model for Course Outlines.
Prior to this commit, these course outlines were only populated by
a management command in the learning_sequences app that queried
modulestore. This commit does a few things:

1. Move the update_course_outline command to live in contentstore
   (i.e. Studio). This makes learning_sequences unaware of
   modulestore, and makes it easier for us to extract it from
   edx-platform (or to plug in different kinds of course outlines).
2. Add tests.
3. Add performance and debug logging to course outline creation.
4. Make course outline creation happen every time a course publish
   happens.

This will allow us to start collecting data about how long building
course outlines takes, and get error reporting around any content
edge cases that break the course outline code.
2021-01-28 09:56:28 -05:00
Muhammad Soban Javed
d26a59eb67 import task decorator from celery APP instance in CMS (#25953)
* import task decorator from celery APP instance instead of celery package in CMS

* replaced task decorator with shared_task in cms and common

* Fixed import of shared_task

* Fixed import
2021-01-27 18:35:11 +05:00
stvn
a40feee9c7 Increase logging verbosity on update_search_index task
to gain insight into CR-3119/TNL-7866.
We know that one or more errors are occuring during this job, but it's
not obvious what these errors are.
2021-01-13 10:04:08 -08:00
Muhammad Soban Javed
bd601cf3a6 Update celery routing for celery 4+ (#25567)
* Update celery routing

- Used routing function instead of class
- Move task queues dictionary to Django settings
- Removed routing_key parameter
- Refactored routing for singleton celery instantiation

Co-authored-by: Awais Qureshi <awais.qureshi@arbisoft.com>
2020-12-16 13:40:47 +05:00
Kyle McCormick
4dda73d797 [BD-14] Migrate all environments to use database-backed organizations (#25153)
* Install `organizations` app into LMS and Studio non-optionally.
* Add toggle `ORGANIZATIONS_AUTOCREATE` to Studio.
* Remove the `FEATURES["ORGANIZATIONS_APP"]` toggle.
* Use the new `organizations.api.ensure_organization` function to
  either validate or get-or-create organizations, depending
  on the value of `ORGANIZATIONS_AUTOCREATE`,
  when creating course runs and V2 content libraries.
  We'll soon use it for V1 content libraries as well.
* Remove the `util.organizations_helpers` wrapper layer
  that had to exist because `organizations` was an optional app.
* Add `.get_library_keys()` method to the Split modulestore.
* Add Studio management command for backfilling organizations tables
  (`backfill_orgs_and_org_courses`).

For full details, see
https://github.com/edx/edx-organizations/blob/master/docs/decisions/0001-phase-in-db-backed-organizations-to-all.rst

TNL-7646
2020-12-02 13:58:40 -05:00
Robert Raposa
8eef18710d set code_owner for celery tasks
ARCHBOM-1260

Co-authored-by: Tim McCormack <tmccormack@edx.org>
2020-11-17 15:33:33 -05:00
Awais Qureshi
7201edb11d Revert "Update routing config" (#25536)" (#25549)" (#25553)" (#25561)
This reverts commit db4c3b1210.
2020-11-11 00:13:47 +05:00
Awais Qureshi
db4c3b1210 Revert "Revert ""Update routing config" (#25536)" (#25549)" (#25553)
This reverts commit c1fe3c3a93.
2020-11-10 23:23:09 +05:00
Kyle McCormick
151bd13666 Use full names for common.djangoapps imports; warn when using old style (#25477)
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
2020-11-10 07:02:01 -05:00
Muhammad Soban Javed
c1fe3c3a93 Revert ""Update routing config" (#25536)" (#25549)
This reverts commit 39a22734c1.
2020-11-09 23:43:47 +05:00
Awais Qureshi
39a22734c1 "Update routing config" (#25536)
* Revert "Revert "Update routing config"

* Removed settings from lms/celery.py and cms/celery.py

* Moved settings import from top-level to function's scopes

Co-authored-by: Soban Javed <iamsobanjaved@gmail.com>
2020-11-09 19:06:55 +05:00
Muhammad Soban Javed
5a2ea1f954 Revert "Update routing config" 2020-11-06 02:05:48 +05:00
Muhammad Soban Javed
bfa8f71f33 Update routing config #25419 2020-11-06 01:00:29 +05:00
Régis Behmo
4586002956 Import waffle classes from edx_toggles instead of waffle_utils
Those classes were ported to edx_toggles. The imports remain in
waffle_utils.__init__ for backward compatibility.
2020-11-03 19:25:37 +01:00
Soban Javed
3206d9cb9a Update celery routing
- Used routing function istead of class
- Move task queues to Djano settings
- Removed routing_key parameter
2020-11-02 15:03:53 +05:00
Kyle McCormick
a3deb6e317 Sort imports in cms/
Command: `isort --recursive cms/`
2020-10-19 09:34:31 -04:00
Kyle McCormick
538dbf696b Import CMS code using fully qualified module names
or, using proper the proper Python 3 relative imports
(e.g., 'from .views import x') which are standard and
unambiguous.
2020-10-19 09:34:31 -04:00
David Ormsbee
168c5deee8 Exclude CCX Courses from search indexing.
Search indexing is prohibitively slow for large CCX courses, even
taking hours in the case of some particularly large ones with
thousands of blocks. Temporarily disabling this functionality until
it can be made more performant (PSRE-288), so that we're not
blocking the workers from doing more latency-sensitive work.

There is a separate effort to put search indexing in its own set
of workers.
2020-09-28 13:58:13 -04:00
Aarif
98af9ce418 remove useless-supression warnings 2020-05-01 19:42:15 +05:00
Aarif
6ee2089077 fixed warnings for wrong-import-order 2020-04-08 23:43:06 +05:00
Tim McCormack
11fc118402 Remove django-celery from transitive deps; remove unused tasks (#23318)
The video thumbnail and transcript tasks were the only things using chord_task from
edx-celeryutils, which in turn was blocking django-celery removal. But, they're no longer used.

Co-authored-by: Diana Huang <diana.k.huang@gmail.com>
2020-03-05 21:57:12 +00:00
Tim McCormack
95d4acad7c Minimize dependency on django-celery (#23193)
- Reimplement `djcelery.common.respect_language` utility function so we
  can drop the dependency.
- Loosen `celery` pinning to a range, which brings in a fix for a
  possible crash bug

Also, pin `path` to 13.1.0 for now in order to continue supporting py35.
2020-02-26 00:46:36 +00:00
Ahtisham Shahid
7a026b7850 created const for update_search_index 2020-01-17 14:29:30 +05:00
Feanil Patel
9cf2f9f298 Run 2to3 -f future . -w
This will remove imports from __future__ that are no longer needed.

https://docs.python.org/3.5/library/2to3.html#2to3fixer-future
2019-12-30 10:35:30 -05:00
Feanil Patel
75eeee0f89 Correct the output file name for exports. 2019-12-19 12:48:13 -05:00
Feanil Patel
50ab035d49 Make the file name not have a 'b' prefix.
We make sure the temp file name is a string because we combine it with
other string components with path later.
2019-12-18 12:58:34 -05:00
Feanil Patel
ca705a2757 Encoding that was breaking import. 2019-10-07 13:49:25 -04:00
noraiz-anwar
67af4b8eb1 fix course import issue 2019-09-26 18:00:21 +05:00
Awais Qureshi
40d9a683e5 PROD-692
Fixing python3
2019-09-17 17:03:56 +05:00
Ayub khan
da9c71557b -Remove unicode and use six.text_type 2019-09-02 16:14:02 +05:00
arbisoft
84cfb8d1ec BOM-392
Fixing encode/decode issues for python3.
2019-08-22 16:45:25 +05:00
Waheed Ahmed
23bc5af0fc Fix course:org link creation upon rerun.
Upon rerun creation from publisher course:organization link
was not creating, due to which org logo was not appearing on
course certificates.

Fixed by creating course:org link upon rerun.

PROD-125
2019-08-05 13:48:50 +05:00
Feanil Patel
63f8167dfc Remove the task that is no longer being referenced.
Second phase for removing the task. The references to it were removed
earlier as a part of DEPR-41
2019-07-26 16:24:05 -04:00
Feanil Patel
1f7b12a38d Remove references to push notifications.
Push notifications was only ever setup to connect to Parse.com a service
that has been discontinued.  Since we haven't replaced the functionality
for a few years now, remove this dead code.

In cms/templates/js/course_info_update.underscore we're allowing content
to not be escaped because this is by design according to the tests.
Long term there should be a better fix for this but for now this is
intended behavior.
2019-07-24 11:45:46 -04:00
asadazam93
eb2422990c Reverted logs 2019-05-03 16:10:17 +05:00
Awais Jibran
3bfab9fd32 Adding more timing logs as course-rerun is slow for mitx-7.00x 2019-03-22 15:41:57 +05:00
Matthew Piatetsky
7df30938c4 fix unicode strings in cms/ 2019-02-04 13:01:51 -05:00