Commit Graph

24 Commits

Author SHA1 Message Date
Sarina Canelake
19f1dacab2 s/pylint: disable=E1101/pylint: disable=no-member/ 2014-12-01 11:22:09 -05:00
stv
b215d771b7 Fix PEP8: E128 continuation line under-indented
for visual indent
2014-11-24 20:46:50 -05:00
Braden MacDonald
a8f622af7f Consistently ensure dog_stats_api tags are formatted correctly 2014-10-09 12:35:26 -07:00
Adam Palay
22a8ce39e2 add tracking for memory usage in creating subtasks (LMS-6488) 2014-08-27 11:12:52 -04:00
Adam Palay
5b769df452 No longer chunk queries when sending out bulk email (LMS-2718)
refactor _generate_items_for_subtask not to chunk queries

use django's iterator methor for querysets

remove bulk email query settings

use read_replica if available, since this is read-only

update changelog
2014-05-29 09:10:25 -04:00
Calen Pennington
cd746bf8e5 Make course ids and usage ids opaque to LMS and Studio [partial commit]
This commit adds the non-courseware lms/djangoapps and lms/lib.

These keys are now objects with a limited interface, and the particular
internal representation is managed by the data storage layer (the
modulestore).

For the LMS, there should be no outward-facing changes to the system.
The keys are, for now, a change to internal representation only. For
Studio, the new serialized form of the keys is used in urls, to allow
for further migration in the future.

Co-Author: Andy Armstrong <andya@edx.org>
Co-Author: Christina Roberts <christina@edx.org>
Co-Author: David Baumgold <db@edx.org>
Co-Author: Diana Huang <dkh@edx.org>
Co-Author: Don Mitchell <dmitchell@edx.org>
Co-Author: Julia Hansbrough <julia@edx.org>
Co-Author: Nimisha Asthagiri <nasthagiri@edx.org>
Co-Author: Sarina Canelake <sarina@edx.org>

[LMS-2370]
2014-05-08 12:09:23 -04:00
Usman Khalid
95a112a3dd instructor_task: Distribute extra items among subtasks of last query.
When creating an instructor task total_num_items may change between the
time it and the number of subtasks is calculated and the time the
subtasks are actually queued (all of this cannot happen in one transaction).
In such a case the extra items are distributed among the subtasks of the
last query.

LMS-2090
2014-02-04 23:30:02 +05:00
Brian Wilson
7bef11a7f1 Add retries when encountering DatabaseError when updating InstructorTask. 2013-11-05 11:18:08 -05:00
Brian Wilson
782a07e0ea Add datadog stats for duplicate subtask and subtask update errors. 2013-10-30 17:09:40 -04:00
Brian Wilson
5b48ed840b Refactor subtask creation logic to be less email-specific. 2013-10-28 12:58:11 -04:00
Brian Wilson
0f8f82c845 Define and use SubtaskStatus class. 2013-10-28 12:57:41 -04:00
Brian Wilson
ed4b954a53 Remove the use of celery.group from bulk email subtasks. 2013-10-28 12:47:11 -04:00
Brian Wilson
16b85c0c2a Check for requeued subtasks when in RETRY state. 2013-10-22 18:17:00 -04:00
Brian Wilson
ecb58e863f Add cache-based locking to subtasks to ensure that the same task is not running in two workers at the same time. 2013-10-22 14:32:52 -04:00
Brian Wilson
a699674015 Check that a subtask has not already completed before running. 2013-10-17 13:32:44 -04:00
Brian Wilson
b8239068cb Check that email subtasks are known to the InstructorTask before executing. 2013-10-16 13:02:41 -04:00
Brian Wilson
bc599a0662 Update tests with more complete coverage. 2013-10-16 13:02:40 -04:00
Brian Wilson
e75dd4654b Move subtask update logic that was only needed for tests into the tests that needed it. 2013-10-16 13:02:40 -04:00
Brian Wilson
04f90fee41 Fix subtask code to handle (tests) running in eager mode. 2013-10-16 13:01:59 -04:00
Brian Wilson
a4c35ac4ff Use separate retry count for calculating retry delay. 2013-10-16 13:01:59 -04:00
Brian Wilson
7b7afd472d Incorporate changes in max_retry logic, adding subtask_status as bulk_email arg. 2013-10-16 13:01:59 -04:00
Brian Wilson
08a08448ee Add some handling for SES exceptions. 2013-10-16 13:01:59 -04:00
Brian Wilson
0fd75181e3 Update handling of bulk-email retries to update InstructorTask before each retry. 2013-10-16 13:01:58 -04:00
Brian Wilson
5c29530db9 Factor out subtask-specific code into subtasks.py. 2013-10-16 13:01:58 -04:00