Commit Graph

499 Commits

Author SHA1 Message Date
Robert Raposa
8eef18710d set code_owner for celery tasks
ARCHBOM-1260

Co-authored-by: Tim McCormack <tmccormack@edx.org>
2020-11-17 15:33:33 -05:00
Awais Qureshi
7201edb11d Revert "Update routing config" (#25536)" (#25549)" (#25553)" (#25561)
This reverts commit db4c3b1210.
2020-11-11 00:13:47 +05:00
Awais Qureshi
db4c3b1210 Revert "Revert ""Update routing config" (#25536)" (#25549)" (#25553)
This reverts commit c1fe3c3a93.
2020-11-10 23:23:09 +05:00
Kyle McCormick
151bd13666 Use full names for common.djangoapps imports; warn when using old style (#25477)
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
2020-11-10 07:02:01 -05:00
Muhammad Soban Javed
c1fe3c3a93 Revert ""Update routing config" (#25536)" (#25549)
This reverts commit 39a22734c1.
2020-11-09 23:43:47 +05:00
Awais Qureshi
39a22734c1 "Update routing config" (#25536)
* Revert "Revert "Update routing config"

* Removed settings from lms/celery.py and cms/celery.py

* Moved settings import from top-level to function's scopes

Co-authored-by: Soban Javed <iamsobanjaved@gmail.com>
2020-11-09 19:06:55 +05:00
Muhammad Soban Javed
5a2ea1f954 Revert "Update routing config" 2020-11-06 02:05:48 +05:00
Muhammad Soban Javed
bfa8f71f33 Update routing config #25419 2020-11-06 01:00:29 +05:00
Kyle McCormick
d1a775d3cd Use full names for lms.djangoapps imports (#25401)
* Use full LMS imports paths in LMS settings and urls modules
* Use full LMS import paths in Studio settings and urls modules
* Import from lms.djangoapps.badges instead of badges
* Import from lms.djangoapps.branding instead of branding
* Import from lms.djangoapps.bulk_email instead of bulk_email
* Import from lms.djangoapps.bulk_enroll instead of bulk_enroll
* Import from lms.djangoapps.ccx instead of ccx
* Import from lms.djangoapps.course_api instead of course_api
* Import from lms.djangoapps.course_blocks instead of course_blocks
* Import from lms.djangoapps.course_wiki instead of course_wiki
* Import from lms.djangoapps.courseware instead of courseware
* Import from lms.djangoapps.dashboard instead of dashboard
* Import from lms.djangoapps.discussion import discussion
* Import from lms.djangoapps.email_marketing instead of email_marketing
* Import from lms.djangoapps.experiments instead of experiments
* Import from lms.djangoapps.gating instead of gating
* Import from lms.djangoapps.grades instead of grades
* Import from lms.djangoapps.instructor_analytics instead of instructor_analytics
* Import form lms.djangoapps.lms_xblock instead of lms_xblock
* Import from lms.djangoapps.lti_provider instead of lti_provider
* Import from lms.djangoapps.mobile_api instead of mobile_api
* Import from lms.djangoapps.rss_proxy instead of rss_proxy
* Import from lms.djangoapps.static_template_view instead of static_template_view
* Import from lms.djangoapps.survey instead of survey
* Import from lms.djangoapps.verify_student instead of verify_student
* Stop suppressing EdxPlatformDeprecatedImportWarnings
2020-11-04 08:48:33 -05:00
Régis Behmo
4586002956 Import waffle classes from edx_toggles instead of waffle_utils
Those classes were ported to edx_toggles. The imports remain in
waffle_utils.__init__ for backward compatibility.
2020-11-03 19:25:37 +01:00
Soban Javed
3206d9cb9a Update celery routing
- Used routing function istead of class
- Move task queues to Djano settings
- Removed routing_key parameter
2020-11-02 15:03:53 +05:00
David Ormsbee
0ffd3699fc Add ability to manually fail instructor tasks in batches.
When an InstructorTask is stuck in QUEUING (say if there was a
problem with celery), the support team needs to manually intervene
and mark the task as "FAILED" so that new tasks of that type can
be created for that course. This is usually done one at a time,
but sometimes a bug or outage might cause many tasks to fail at
once, making recovery extremely cumbersome. This commit adds the
ability to do this process in batches.
2020-10-28 09:39:50 -04:00
Feanil Patel
0191f2f1bb Don't install instructor_task for common testing. (#25459)
* Import instructor tasks explicitly on workers.

Importing them via app config leads to an dependency chain nightmare as lots of other apps get pulled into the startup
and are not configured in situations like testing the common and lib unit tests.  This should register tasks to the celery
workers without messing with the startup dependency chains of the LMS, CMS and test setup.
2020-10-27 18:34:27 -04:00
Kyle McCormick
762b4fdf7f Use AppConfig in instuctor_task app to ensure tasks are discovered
This fixes an issue where instructor_task Celery tasks were not being
auto-discovered, thus leaving them unregistered and unable to be
completed by our Celery workers.

TNL-7652
2020-10-26 15:23:22 -04:00
Jeremy Bowman
d19368525b DEPR-43 Remove most of the shoppingcart app (#24692)
Removed most of the deprecated shoppingcart app, leaving just enough to allow us to cleanly remove the related database tables later. Also removed the relevant Django settings that weren't in use elsewhere.
2020-10-14 14:24:52 -04:00
Calen Pennington
2714da86a4 Always import task packages by absolute paths
This follows the recommendation in the Celery documentation so as to
not confuse automatic task name generation.

Docs: https://docs.celeryproject.org/en/stable/userguide/tasks.html#automatic-naming-and-relative-imports
2020-10-08 14:10:17 -04:00
Calen Pennington
4a120a9e26 Fix failures caused by removing lms/djangoapps from sys.path 2020-09-23 10:05:37 -04:00
Régis Behmo
307457a255 Simplify hack to obtain waffle module names
Instead of going up the stacktrace to find the module names of waffle
flags and switches, we manually pass the module __name__ whenever the
flag is created. This is similar to `logging.getLogger(__name__)`
standard behaviour.

As the waffle classes are used outside of edx-platform, we make the new
module_name argument an optional keyword argument. This will change once
we pull waffle_utils outside of edx-platform.

Note that the module name is normally only required to view the list of
existing waffle flags and switches. The module name should not be
necessary to verify if a flag is enabled. Thus, maybe it would make
sense to create a `add` class methor similar to:

    class WaffleFlag:
        @classmethod
        def add(cls, namespace, flag, module):
            instance = cls(namespace, flag)
            cls._class_instances.add((instance, module))
2020-09-14 09:30:24 +02:00
Kshitij Sobti
9adeeb0056 Remove report path from task output data
Use alternative method for getting report download link
2020-09-11 14:12:28 +02:00
0x29a
921875b70b Add endpoint and background task for downloading of submission files 2020-08-22 01:38:25 +02:00
Kshitij Sobti
2eff127e8d Add support for generating problem response reports for multiple blocks, or filtered block types.
This change adds support for specifying multiple root blocks while generating
problem response reports. It also allows specifying a block type filter so that
only blocks of the filtered types will be included in the report.

Finally, this change also consistenly uses absolute path for the location in the
report instead of relative paths.
2020-08-18 22:28:05 +05:30
David Ormsbee
5d33501c82 Merge pull request #19986 from open-craft/kshitij/report-status
Inline problem response report status
2020-08-13 12:51:55 -04:00
M. Zulqarnain
c8ea2c5fde Unpin psutil (#24745) 2020-08-13 01:20:04 +05:00
Kshitij Sobti
559ff54a06 Add UI to display report creation progress.
Adds an in-line status display for problem report generation progress.

Co-authored-by: Adam Kovari <adam@opencraft.com>
Co-authored-by: Eugeny Kolpakov <eugeny.kolpakov@gmail.com>
Co-authored-by: Kshitij Sobti <kshitij@sobti.in>
Co-authored-by: Braden MacDonald <braden@opencraft.com>
2020-08-12 19:05:31 +05:30
Robert Raposa
77e490f057 ARCHBOM-1305: remove deprecated flag_undefined_default (#24426)
This is the final step in removing the deprecated
flag_undefined_default as explained by the following ADR:
https://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/waffle_utils/docs/decisions/0001-refactor-waffle-flag-default.rst

Notes:

* All uses of flag_undefined_default=False were always
  supposed to have been no-ops.
* All uses of flag_undefined_default=True that are removed
  in this PR have been replaced by migrations in past PRs.
* The temporary metric temp_flag_default_used id no longer
  reporting any data.

ARCHBOM-1305
2020-07-09 09:31:31 -04:00
Diana Huang
1655ecd063 Remove final shoppingcart references from instructor. 2020-07-08 09:36:43 -04:00
Waheed Ahmed
a6a69224d1 Ratelimit login_user endpoint.
Ratelimited `login_user` endpoint using `django-ratelimit`, also
decreased default value of logistration rate limit to 100 requests
per five minutes per IP.

PROD-1877
2020-07-08 15:36:11 +05:00
Nimisha Asthagiri
80ea0a61f0 instructor_task: remove shoppingcart (DEPR-43) 2020-06-29 10:28:26 -04:00
Awais Jibran
5d4df5383b Change switch to waffle flag. 2020-06-15 15:15:10 +05:00
Diana Huang
c7bd5a8a15 Remove more references to fix tests. 2020-06-09 13:06:44 -04:00
Aarif
98af9ce418 remove useless-supression warnings 2020-05-01 19:42:15 +05:00
Alex Dusenbery
f40bb34ce7 For course grade and problem grade reports, clear course enrollment caches after each batch of users is processed. 2020-04-20 14:19:54 -04:00
SaadYousaf
297a114d6e remove unneccesary log. 2020-04-15 15:12:46 +05:00
Aarif
6ee2089077 fixed warnings for wrong-import-order 2020-04-08 23:43:06 +05:00
Awais Jibran
ac49d4fd99 Pass course structure to CourseGradeFactory.iter 2020-04-07 18:02:15 +05:00
SaadYousaf
738dbae563 add investigation logs. 2020-04-03 21:13:53 +05:00
SaadYousaf
3c1f7ec5e9 add logs to optimized ProblemGradeReport code. 2020-04-02 14:39:43 +05:00
SaadYousaf
8c4ae15766 remove flaky test. 2020-04-01 15:46:20 +05:00
SaadYousaf
6fb9d199a0 optimize code for problem_grade_report. 2020-03-30 20:33:49 +05:00
Saad Yousaf
a458491f04 Revert "[PROD-1287] - initial work to optimise problem_grade_report code." 2020-03-30 19:05:54 +05:00
SaadYousaf
4e4bffe404 initial work to optimise problem_grade_report code. 2020-03-30 13:14:01 +05:00
Awais Jibran
65094cbf35 Handle deprecation warn by importing right class.
2900k/ last 7 days
2020-03-13 21:50:40 +05:00
Dave St.Germain
a5b0f71108 Several optimizations for improving vertical rendering performance. 2020-02-20 11:41:21 -05:00
Awais Jibran
d7d258cd36 Generate grade report for verified learnes by default 2020-01-28 14:04:02 +05:00
Zulqarnain
644a1fc0a6 Change deprecated methods signatures 2020-01-21 16:33:37 +05:00
Awais Qureshi
220cab965f BOM-1126
Removing the moto from repo.
2020-01-15 22:34:13 +05:00
Calen Pennington
151b309225 Convert custom storage subclasses into mixins so that we can store to either the filesystem or S3 2020-01-02 14:49:19 -05:00
Calen Pennington
0ee1c2f05a Remove S3ReportStorage, because S3BotoStorage already handles a custom_domain argument 2020-01-02 10:25:53 -05:00
Feanil Patel
046feb0cf0 Merge pull request #22649 from edx/feanil/fix_pep8
Fix all E303 pep8 errors.
2019-12-30 13:32:26 -05:00
Feanil Patel
6e3fe00fff Fix all E303 pep8 errors. 2019-12-30 12:25:38 -05:00