Commit Graph

104 Commits

Author SHA1 Message Date
Tim McCormack
83f6e560b7 fix: Add code_owner decorator to remaining Celery tasks (#31762)
This will ensure that errors raised by these tasks will alert the right
team. `send_course_enrollment_email` is the one I set out to fix, but I
discovered a few others.

I located tasks that were missing decorators by running the following
search and visually inspecting the results, although semgrep might be able
to do better:

```
ack '^@.*task|^@set_code_owner_attribute' cms lms common openedx xmodule --ignore-dir=tests --python
```

Also, add more detailed explanation of why a couple of tasks can't use the
decorator. This should only be an issue on tasks inheriting from
UserTaskMixin, which in practice is just CourseExportTask and
CourseImportTask (and the apparently unused EnrollmentReadTask and
EnrollmentWriteTask), via UserTask.
2023-02-15 18:20:50 +00:00
0x29a
d338f00e39 refactor: rename module (or item) -> block within cms 2023-01-30 18:15:23 +01:00
0x29a
a027f36724 refactor: rename module -> block within xmodule 2023-01-30 18:15:22 +01:00
0x29a
83396ffb07 refactor: convert course_module term to course_block 2022-12-19 17:48:49 +01:00
0x29a
0df1411636 refactor: xmodule/course_module.py -> xmodule/course_block.py 2022-12-19 17:48:49 +01:00
Zachary Hancock
2f3c93ed9f feat: sync with exam service on course publish (#31015)
Call into the exam service instead of the edx-proctoring plugin on course publish if the course_apps.exams_ida course waffle flag is enabled. This is an early step in moving away from edx-proctoring
2022-09-20 12:38:32 -04:00
Kshitij Sobti
a923cfc32e feat: Add mechanism to migrate discussion blocks to discussion flag (#30931)
The new mechanism for marking that a unit has discussions is to use the
discussion_enabled flag instead of adding a discussion block. This change
adds code that is run during the course rerun process to mark any existing
units that have a discussion block as discussible using the new mechanism.
It doesn't touch the existing discussion blocks.

If the new discussions configuration experience is globally enabled, this
will also switch from the legacy provider to the new provider. It analyses
the course for any discussion blocks that have been added to graded
subsections, and if that is the case, it also automatically enables
discussions in graded subsections (which are otherwise disabled for new
courses by default).
2022-09-19 16:56:50 +05:00
Dillon Dumesnil
61b93d953d feat: Add backfill course tabs management command
Previously, course tabs would only be created once and never try to
update the default tabs again. This leads to an issue if you ever want
to add a new tab. With this command, you can now update the default tabs
for all existing courses and new courses will pick it up upon creation
when CourseTabList.initialize_default is called.
2022-03-11 11:16:18 -05:00
Jawayria
0e63c20f64 chore: Applied lint-amnesty on cms/djangoapps/contentstore 2021-12-10 19:20:37 +05:00
Julia Eskew
a1914f6d31 fix: When running the task-creating task which regenerates all course outlines,
continue instead of returning when an unsupported course key is encountered.
2021-06-24 17:28:58 -04:00
Kyle McCormick
7a26ec6b92 refactor: move xml_importer-specific exceptions into xml_importer
This is a minor refactoring in order to remove
a code dependency of ./xmodule on ./cms
2021-06-08 10:46:52 -04:00
Julia Eskew
d83051f33b feat: Add Django admin interface to backfill course outlines. (#27749)
TNL-8080
2021-06-01 13:40:32 -04:00
=Awais Jibran
1b8195be60 Fix NR Dashbaord inconsistency with failed state 2021-05-28 11:04:52 +05:00
Usama Sadiq
4f4be6538a BOM-2477: pylint warnings lint-amnesty (#27585) 2021-05-11 17:22:40 +05:00
alangsto
50d57a64bb MST-757 Move exam registration to async task (#27398)
Synchronously registering proctored exams while saving content to studio is causing a significant slow down. The function that registers the exams has been moved to an async task. In addition, a signal handler on_course_publish has also been moved to the async task, as it relies on exam registration being complete before being executed.
2021-04-26 12:04:04 -04:00
Awais Jibran
e05d4a9c47 Monitor error in NR when OLX validation fails. (#27381) 2021-04-21 11:56:54 +05:00
Usama Sadiq
ab41a7daee refactor: Apply lint-amnesty on pylint violations 2021-04-20 21:43:59 +05:00
Awais Jibran
62c8805e3e Refactor + Tests: Course Import Feature (#27369)
* Code Refactoring
This PR bumps code coverage by adding unit tests & clean up some code for improving code quality and maintainability.
2021-04-19 23:49:42 +05:00
Awais Jibran
9722e37492 Enable Course Olx Validation during Course Import (#27300)
* Enable import failure and email with Errors/Warnings

This PR enables course import failure in case of olx validation errors. Here is the flow.
  * Course Import tries to import foo.tar.gz into their course shell
  * Course olx contains validation errors
  * During course import, olx is validated and import is failed with the error message "Course olx validation failed. Please check your email."
  * System generates an email with ERRORs & WARNINGs in the body of the email.

This PR also adds a waffle flag contentstore.bypass_olx_failure. The purpose of this test flag is to allow course teams to unblock by enabling them to bypass the
the olx-validation failure.
The workaround is shared on the ticket TNL8214.

* Disable olx validation out of the box.
2021-04-16 18:45:59 +05:00
Ahtisham Shahid
58231c4262 Moved Import error messages to single place (#27331) 2021-04-15 12:25:04 +05:00
Awais Jibran
30c80082c3 Use config settings for olxcleaner (#27294)
* Use config settings for olxcleaner

Use config settings instead of hardcoded values for olx validation. This would help in adding a great deal of control when you want to change these settings in the future. With this approch we would not need a redeploy.

* Use configs and deprecate waffleflag and also add / update tests
2021-04-12 16:27:42 +05:00
SaadYousaf
7242b5018a [TNL-8051] - bump olxcleaner version with latest update. 2021-04-08 23:46:50 +05:00
Awais Jibran
1d39e431e3 Use allowed_xblocks settings (#27265)
* Use allowed_xblocks in olxcleaner validate.
2021-04-07 15:45:04 +05:00
Ahtisham Shahid
f3a62b5f89 Error message for proctor exam show to user (#27238)
* Error message for proctor exam visible to user

* Created exception class for Invalid Proctor settings
2021-04-07 10:46:19 +05:00
AsadAzam
c4da6c1fe4 Display import errors to user (#27147)
* Display import errors to user

* Refactored

* Refactored

* Refactored

* Fixed quality

* Fixed quality

* Refactored code

* Fixed message

* Refactored code
2021-04-02 18:53:39 +05:00
Awais Jibran
8023bbc1af Log Errors + Warnings. (#27210) 2021-04-02 16:02:46 +05:00
Awais Jibran
f898fc5d72 Short circuit, library import. (#27195) 2021-04-01 12:29:57 +05:00
Awais Jibran
ad7f6019fe Integrate "olxcleaner" with course import (#27164)
* Integrating olxcleaner in course import

* Adding toggle removal date and addressing pylint issues.
2021-03-31 13:26:14 +05:00
Awais Jibran
5f773d326d Add missing import logs. (#27066) 2021-03-22 15:28:13 +05:00
AsadAzam
1593b40702 Improved logging for import (#27024)
* Improved logging for import

* Refactored code

* Refactored code

* Refactored code

* Refactored code

* Refactored code

* refactored code

* Refactored code

* Updating log

Co-authored-by: Awais Jibran <awaisdar001@gmail.com>
2021-03-18 12:40:38 +05:00
David Ormsbee
f5b74fcf31 feat: Backfill and Django Admin for Learning Sequence Outline
* Adds the backfill_course_outlines management command to contentstore
* Adds a read-only Django admin interface to learning_sequences for the
  support team and debugging.
* Adds two new functions to the learning_sequences public API:
  key_supports_outlines and get_course_keys_with_outlines

The learning_sequences app isn't supposed to know about contentstore or
modulestore, as it's intended to be extracted out of edx-platform in the
long term. Therefore, the backfill_course_outlines command is in
contentstore, and not learning_sequences.

This work was tracked in TNL-7983, but it also fixes a bug where we were
trying to generate course outlines for libraries (TNL-7981).

All Open edX instances upgrading to Lilac should run the
backfill_course_outlines command as part of their upgrade process.
2021-03-09 09:05:57 -05:00
M. Zulqarnain
5562944e87 BOM-2369 (A): pyupgrade on api,contentstore and cms_user_tasks apps under CMS (#26676)
* pyupgrade on cms api,contentstore and cms_user_tasks apps
2021-03-04 15:06:07 +05:00
usamasadiq
7bbde8f0f5 Applied pylint-amnesty 2021-02-08 13:00:22 +05:00
David Ormsbee
4fde69441c Fix build: use @shared_task for update_outline_from_modulestore_task (#26216)
Update to make update_outline_from_modulestore_task use @shared_task
and @set_code_owner_attribute.
2021-01-28 11:22:39 -05:00
David Ormsbee
5353324c80 Merge pull request #26145 from edx/ormsbee/course-outline-publish-tnl-7733
Push Course Outlines to learning_sequences on publish.
2021-01-28 10:21:11 -05:00
David Ormsbee
669677c78a Push Course Outlines to learning_sequences on publish.
The learning_sequences app has its own model for Course Outlines.
Prior to this commit, these course outlines were only populated by
a management command in the learning_sequences app that queried
modulestore. This commit does a few things:

1. Move the update_course_outline command to live in contentstore
   (i.e. Studio). This makes learning_sequences unaware of
   modulestore, and makes it easier for us to extract it from
   edx-platform (or to plug in different kinds of course outlines).
2. Add tests.
3. Add performance and debug logging to course outline creation.
4. Make course outline creation happen every time a course publish
   happens.

This will allow us to start collecting data about how long building
course outlines takes, and get error reporting around any content
edge cases that break the course outline code.
2021-01-28 09:56:28 -05:00
Muhammad Soban Javed
d26a59eb67 import task decorator from celery APP instance in CMS (#25953)
* import task decorator from celery APP instance instead of celery package in CMS

* replaced task decorator with shared_task in cms and common

* Fixed import of shared_task

* Fixed import
2021-01-27 18:35:11 +05:00
stvn
a40feee9c7 Increase logging verbosity on update_search_index task
to gain insight into CR-3119/TNL-7866.
We know that one or more errors are occuring during this job, but it's
not obvious what these errors are.
2021-01-13 10:04:08 -08:00
Muhammad Soban Javed
bd601cf3a6 Update celery routing for celery 4+ (#25567)
* Update celery routing

- Used routing function instead of class
- Move task queues dictionary to Django settings
- Removed routing_key parameter
- Refactored routing for singleton celery instantiation

Co-authored-by: Awais Qureshi <awais.qureshi@arbisoft.com>
2020-12-16 13:40:47 +05:00
Kyle McCormick
4dda73d797 [BD-14] Migrate all environments to use database-backed organizations (#25153)
* Install `organizations` app into LMS and Studio non-optionally.
* Add toggle `ORGANIZATIONS_AUTOCREATE` to Studio.
* Remove the `FEATURES["ORGANIZATIONS_APP"]` toggle.
* Use the new `organizations.api.ensure_organization` function to
  either validate or get-or-create organizations, depending
  on the value of `ORGANIZATIONS_AUTOCREATE`,
  when creating course runs and V2 content libraries.
  We'll soon use it for V1 content libraries as well.
* Remove the `util.organizations_helpers` wrapper layer
  that had to exist because `organizations` was an optional app.
* Add `.get_library_keys()` method to the Split modulestore.
* Add Studio management command for backfilling organizations tables
  (`backfill_orgs_and_org_courses`).

For full details, see
https://github.com/edx/edx-organizations/blob/master/docs/decisions/0001-phase-in-db-backed-organizations-to-all.rst

TNL-7646
2020-12-02 13:58:40 -05:00
Robert Raposa
8eef18710d set code_owner for celery tasks
ARCHBOM-1260

Co-authored-by: Tim McCormack <tmccormack@edx.org>
2020-11-17 15:33:33 -05:00
Awais Qureshi
7201edb11d Revert "Update routing config" (#25536)" (#25549)" (#25553)" (#25561)
This reverts commit db4c3b1210.
2020-11-11 00:13:47 +05:00
Awais Qureshi
db4c3b1210 Revert "Revert ""Update routing config" (#25536)" (#25549)" (#25553)
This reverts commit c1fe3c3a93.
2020-11-10 23:23:09 +05:00
Kyle McCormick
151bd13666 Use full names for common.djangoapps imports; warn when using old style (#25477)
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
2020-11-10 07:02:01 -05:00
Muhammad Soban Javed
c1fe3c3a93 Revert ""Update routing config" (#25536)" (#25549)
This reverts commit 39a22734c1.
2020-11-09 23:43:47 +05:00
Awais Qureshi
39a22734c1 "Update routing config" (#25536)
* Revert "Revert "Update routing config"

* Removed settings from lms/celery.py and cms/celery.py

* Moved settings import from top-level to function's scopes

Co-authored-by: Soban Javed <iamsobanjaved@gmail.com>
2020-11-09 19:06:55 +05:00
Muhammad Soban Javed
5a2ea1f954 Revert "Update routing config" 2020-11-06 02:05:48 +05:00
Muhammad Soban Javed
bfa8f71f33 Update routing config #25419 2020-11-06 01:00:29 +05:00
Régis Behmo
4586002956 Import waffle classes from edx_toggles instead of waffle_utils
Those classes were ported to edx_toggles. The imports remain in
waffle_utils.__init__ for backward compatibility.
2020-11-03 19:25:37 +01:00
Soban Javed
3206d9cb9a Update celery routing
- Used routing function istead of class
- Move task queues to Djano settings
- Removed routing_key parameter
2020-11-02 15:03:53 +05:00