Adds a course_ids api that can filter by user role, since the courses
api could not perform well enough for this, and returned much more
data than we need.
Additionally, adds a LazyPageNumberPagination to provide more accurate
counts in the pagination response when using LazySequence with the
queryset.
BOM-897
BOM-1228
The api is already not very performant and trying to limit it to only
show courses where you are staff causes the code to iterate over almost
all the courses and times out before it returns any results to the user.
The plan is to build a different api for the thing we need that will
just provide the course IDs for courses where you are staff and sholud
be much faster.
made changes to test with previous version of django-waffle
updated the query count to test
testing with version 0.13
testing with version 0.14
testing with version 0.15
added version 0.14
updated the django-waffle version to use 0.18
updated the django-waffle version to use 0.16
updated the query counts to pass tests
ran make upgrade
updated the django-waffle to support django 2.2
made changes to test with previous version of django-waffle
updated the query count to test
testing with version 0.13
testing with version 0.14
testing with version 0.15
added version 0.14
updated the django-waffle version to use 0.18
updated the django-waffle version to use 0.16
updated the query counts to pass tests
removed the pdb statements
ran make upgrade
This adds middleware that will create custom parameter metrics in
New Relic to track the size of all the cookies being received for
our domain. The custom fields are "cookies_total_size" and a
separate named parameter for every cookie size, e.g.
"cookies.csrftoken.size".
This is intended to help us track cookie growth and better diagnose
issues where users lose their sessions. It is toggled by the
'request_utils.capture_cookie_sizes' Waffle Flag.
Prior to this commit, the course api (/api/courses/v1/courses/)
performed all the work necessary to return all courses available
to the user, and then only actually returned on page's worth of those
courses.
With this change, the api now does the work incrementally, computing
only the data needed to fetch the courses up to and including the page
being returned. This still increases approximately linearly as
the page number accessed being increases, but should be more cache-friendly.
One side effect of this is that the max_page reported by pagination
will be an overestimate (it will include pages that are removed due
to a users access restrictions).
This change also changes the sort-order of courses being returned by the
course_api. By sorting by course-id, rather than course-number, we
can sort in the database, rather than in Python, and defer loading data
from the end of the list until it is requested.
REVMI-90
Prior to this commit, the course api (/api/courses/v1/courses/)
performed all the work necessary to return all courses available
to the user, and then only actually returned on page's worth of those
courses.
With this change, the api now does the work incrementally, computing
only the data needed to fetch the courses up to and including the page
being returned. This still increases approximately linearly as
the page number accessed being increases, but should be more cache-friendly.
One side effect of this is that the max_page reported by pagination
will be an overestimate (it will include pages that are removed due
to a users access restrictions).
This change also changes the sort-order of courses being returned by the
course_api. By sorting by course-id, rather than course-number, we
can sort in the database, rather than in Python, and defer loading data
from the end of the list until it is requested.
REVMI-90
This endpoint is likely being inefficient with how it's querying various parts
of the code and can take courseware down, it needs to be rate limited until
optimized.
LEARNER-5527
This endpoint is likely being inefficient with how it's querying various parts of the code
and can take courseware down, it needs to be rate limited until optimized.
LEARNER-5527
There are a number of Django Signals that are on the modulestore's
SignalHandler class, such as SignalHandler.course_published. These
signals can trigger very expensive processes to occur, such as course
overview or block structures generation. Most of the time, the test
author doesn't care about these side-effects.
This commit does a few things:
* Converts the signals on SignalHandler to be instances of a new
SwitchedSignal class, that allows signal sending to be disabled.
* Creates a SignalIsolationMixin helper similar in spirit to the
CacheIsolationMixin, and adds it to the ModuleStoreIsolationMixin
(and thus to ModuleStoreTestCase and SharedModuleStoreTestCase).
* Converts our various tests to use this new mechanism. In some cases,
this means adjusting query counts downwards because they no longer
have to account for publishing listener actions.
Modulestore generated signals are now muted by default during test runs.
Calls to send() them will result in no-ops. You can choose to enable
specific signals for a given subclass of ModuleStoreTestCase or
SharedModuleStoreTestCase by specifying an ENABLED_SIGNALS class
attribute, like the following example:
from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase
class MyPublishTestCase(ModuleStoreTestCase):
ENABLED_SIGNALS = ['course_published', 'pre_publish']
You should take great care when disabling signals outside of a
ModuleStoreTestCase or SharedModuleStoreTestCase, since they can leak
out into other tests. Be sure to always clean up, and never disable
signals outside of testing. Because signals are essentially process
globals, it can have a lot of unpleasant side-effects if we start
mucking around with them during live requests.
Overall, this change has cut the total test execution time for
edx-platform by a bit over a third, though we still spend a lot in
pre-test setup during our test builds.
[PERF-413]
By default, disable all caching in tests, to preserve test independence.
In order to enable caching, inherit from CacheSetupMixin, and specify
which cache configuration is needed.
[EV-32]
