This is an attempt to fix a performance problem on the libraries home page. When you go to studio home and click on the libraries tab, on prod it will be quick for admins but extremely slow for course instructors (> 12 seconds) and leads to timeouts. It grows with the number of libraries that are assigned to the instructor.
The Python code for the request to load libraries for a particular user goes through all existing libraries and then checks all of the user's roles for each library, which results in a complexity of O(l*r), l=libraries, r=roles. This PR improves the complexity to O(l).
The BulkRoleCache and RoleCache classes were using a python set to store all roles for a particular user. A user can have a large number of roles, and lookup speed of iterating through a set is slow (O(n)). Most roles don't have the same course id, however. So if you have the course id of the role you're looking for, we can use a dict of course ids that contain related roles. The number of roles per course id is negligible, so we arrive at a lookup speed of O(1) when looking up a user's roles that belong to a specific course id.
The BulkRoleCache now caches and stores user roles in a data structure like this:
{
user_id_1: {
course_id_1: {role1, role2, role3}, # Set of roles associated with course_id_1
course_id_2: {role4, role5, role6}, # Set of roles associated with course_id_2
[ROLE_CACHE_UNGROUPED_ROLES_KEY]: {role7, role8} # Set of roles not tied to any specific course or library. For example, Global Staff roles.
},
user_id_2: { ... } # Similar structure for another user
}
While this changes the data structure used to store roles under the hood and adds the new property `roles_by_course_id` to the RoleCache,
when initializing the RoleCache will store roles additionally in the previous data structure - as a flat set - in the `_roles` property accessible via `all_roles_set`. This establishes
backwards compatibility.
We are now storing roles twice in the RoleCache (in each of the two data structures), which means this takes twice as much memory, but only in the scope of a request.
* feat: create DRF endpoint to get course index context
* refactor: update serializers location and added some tests
* refactor: move modulestore usage out of views
* fix: move course index call under bulk_operations
* test: add test to track db and mongo queries
---------
Co-authored-by: ruzniaievdm <ruzniaievdm@gmail.com>
* feat: create DRF endpoint to get course index context
* refactor: update serializers location and added some tests
* refactor: move modulestore usage out of views
---------
Co-authored-by: ruzniaievdm <ruzniaievdm@gmail.com>
* refactor: moves is_content_creator
from cms.djangoapps.contentstore.helpers to common.djangoapps.student.auth
* feat: adds content tagging app
Adds models and APIs to support tagging content objects (e.g. XBlocks,
content libraries) by content authors. Content tags can be thought of as
"name:value" fields, though underneath they are a bit more complicated.
* adds dependency on openedx-learning<=0.1.0
* adds tagging app to LMS and CMS
* adds content tagging models, api, rules, admin, and tests.
* content taxonomies and tags can be maintained per organization by
content creators for that organization.
Add ability to roll-forward ORA flex peer grading feature. Where enabled
for an Organization or course, flex peer grading will be turned on at the
course level for new course runs and course reruns. Where disabled,
a new course or course rerun will preserve existing / default setting
value.
This PR addresses the renaming of the contentstore/xblock_services folder to contentstore/xblock_storage_handlers as a follow-up to PR #32282. The renaming is done to prevent naming conflicts with xblock runtime services and to make the purpose of the files more understandable. The file xblock_service.py has been renamed to view_handlers.py to better reflect its functionality.
Justification and Future Refactoring Outlook:
The xblock_storage_handlers folder contains service methods that implement the business logic for view endpoints located in contentstore/views/block.py. It is renamed to xblock_storage_handlers to reflect its responsibility of handling storage-related operations of xblocks, such as creation, retrieval, and deletion.
The view_handlers.py file includes business methods called by the view endpoints. These methods, such as handle_xblock, delete_orphans, etc., interact with the required modulestore methods, handle any errors, and aggregate and serialize data for the response.
The term 'handler' in the context of 'view_handlers.py' represents methods that facilitate business logic for view endpoints. It is critical to note the distinction between these 'handler methods' and the xblock_handler method. The xblock_handler is a view endpoint itself, residing in contentstore/views/block.py, and is well known in this context. Although its name might suggest otherwise, it is not a handler in the sense of the 'handler methods' we've defined in 'view_handlers.py'. To maintain consistency with existing naming conventions, it remains as xblock_handler.
* feat: add xblock endpoint for updating an xblock
fix: remove debugger
feat: make function call more generic
refactor: just use request.json for request data as before
refactor: extract method
fix: revert wrong method change
fix: refactor correct method
feat: use handle_xblock method so that we can do more than update xblocks
fix: usage_key_string defaults to None
add all CRUD operations
fix usage key parameter
refactor: create /views folder
refactor: move xblock view functions to xblock_services
fix: tests
fix: tests
refactor: move xblock API endpoint to contentstore
* docs: add explanatory comment to new xblock_service
* feat: add feature flag for enabling content editing api
* feat: raise 404 if studio content api is disabled
* tests: test xblock endpoint
* test: make all post tests work
* test: check that xblock_handler receives correct args
* refactor: create util mixin for course factories with staff
* refactor: extract course staff authorization tests
* refactor: extract tests to api view testcase class
* test: add get tests
* test: fix tests
* test: fix tests
* test: fix tests
* test: add all crud tests
* fix: refactor to fix tests
* fix: merge conflict
* fix: merge conflict
* fix: tests after merge
* fix: json request decorator
* fix: lint
* fix: lint
* fix: lint
* fix: lint
* fix: new test files
* fix: lint
* fix: lint and apply PR suggestions
* fix: lint
* fix: lint
* fix: lint
* fix: lint
* fix: lint
* fix: lint
Adds a dropdown to select the organization. The dropdown will only be activated for users with CourseCreator permission to specific organizations in Studio.
Use cases:
When FEATURES['ENABLE_CREATOR_GROUP'] = True and the user has CourseCreator permission granted, a dropdown will appear with all specific organizations allowed. In case of all_organizations setting is enabled, all organizations will appear in the dropdown.
In case the user is staff, he can create organizations it will work as before
This removes user-facing Studio edit support for Old Mongo courses
(courses that have a CourseKey of the format {org}/{course}/{run}).
This does not affect our normal courses, which have CourseKeys
starting with "course-v1:".
After this commit:
* Old Mongo courses will continue to appear on the Studio course
listing page, but are not clickable.
* Any attempt to directly access an Old Mongo course in Studio via URL
fail with a 404 error.
* Course certificates will still be available for Old Mongo courses.
* Old Mongo courses will continue to be returned by CourseOverviews
and get_course_summaries() calls.
We decided against removing Old Mongo courses from the listing entirely
because that would require very expensive CourseOverviews query to
filter them out. Making that query more efficient would involve a
database migration to add appropriate indexing, which is something else
that we are looking to avoid. CourseOverviews are used everywhere in
the system, so we want to avoid changing how they work so that we can
minimize risk.
This is part of the Old Mongo Modulestore deprecation effort:
https://github.com/openedx/public-engineering/issues/62
Improving Studio homepage performance for users with course access role with no course_id
Fixing unit tests
Added create CourseOverviewFactory after creating course to course listing test
Fix order import for `from openedx.core.djangoapps.content.course_overviews.tests.factories import CourseOverviewFactory
`
(cherry picked from commit 997a0ff770744309f0ee84f3c0696a80310c5f2d)
This is a first stage for removing the LegacyWaffle* classes.
LegacyWaffleFlag usage replaced with WaffleFlag;
LegacyWaffleSwitche usage replaced with WaffleSwitch;
New CourseWaffleFlag added to the temporary module __future__ as FutureCourseWaffleFlag;
Updated all the imports to use CourseWaffleFlag from the __future__ module;
BREAKING CHANGE: A number of toggle related constants (e.g. ENABLE_ACCESSIBILITY_POLICY_PAGE)
changed types. They were strings, and are now toggle instances (e.g. WaffleSwitch). Although the entire
refactor should be self-contained in edx-platform, if any plugins or dependencies were directly
using these constants, they will break. If this is the case, try to find a better publicized way of
exposing those toggles.
