1. Upgraded Python dependency edx-enterprise
- Removed unencrypted credentials from SAP configuration model
2. Test updates
- Skipped `test_migrations_are_in_sync` for unencrypted credentials removal
- Updated related tests and requirements.
Commit generated by workflow `openedx/edx-platform/.github/workflows/upgrade-one-python-dependency.yml@refs/heads/master`
- Upgraded Python dependency for edx-enterprise
- Removed references to the char field `decrypted_secret`
- Updated the skip reason message for `test_migrations_are_in_sync`
Commit generated by workflow `openedx/edx-platform/.github/workflows/upgrade-one-python-dependency.yml@refs/heads/master`
The git-ignored target directory for LMS Sass compilation is:
lms/static/css
Unfortunately, that directory contains git-controlled directory of
vendored-in static assets:
lms/static/css/vendor
This is a problem for a couple reasons:
1. In Tutor, we would like to make lms/static/css a symlink to an
external location for the sake of build efficiency. This is
impossible to do without clobbering lms/static/css/vendor and
dirtying the git state.
2. More generally, when optimizing (or just understanding) a build
system, it adds complexity when git-controlled source directories are
mixed up inside git-ignored target directories.
The solution is to simply merge these vendored-in assets to another
existing git-controlled vendor directory:
common/static/css/vendor
LMS already reads assets from this folder, so no further changes need to
be made. common/static/css is fully git-controlled, so we avoid the
complexity described above.
Some models in third_party_auth used settings.SITE_ID as a field
default, which caused Django to say migrations were out of sync whenever
settings.SITE_ID happened to be anything other than 1 for any developer:
Your models in app(s): 'third_party_auth' have changes that are not
yet reflected in a migration, and so won't be applied. Run
'manage.py makemigrations' to make new migrations, and then re-run
'manage.py migrate' to apply them.
This could easily happen if a developer is testing out site
configuration or site-specific theming and ends up with a SITE_ID other
than 1.
The fix, inspired by a StackOverflow answer [1], is to simply create
a wrapper function for the dynamic default value. The wrapper function,
rather than the current value of SITE_ID, will be serialized to the
migraiton file.
This commit includes a migration file, but from a database perspective,
the migration is a no-op.
[1] https://stackoverflow.com/a/12654998
As of Python 3.3, the 3rd-party `mock` package has been subsumed into the
standard `unittest.mock` package. Refactoring tests to use the latter will
allow us to drop `mock` as a dependency, which is currently coming in
transitively through requirements/edx/paver.in.
We don't actually drop the `mock` dependency in this PR. That will happen
naturally in:
* https://github.com/openedx/edx-platform/pull/34830
This is an attempt to fix a performance problem on the libraries home page. When you go to studio home and click on the libraries tab, on prod it will be quick for admins but extremely slow for course instructors (> 12 seconds) and leads to timeouts. It grows with the number of libraries that are assigned to the instructor.
The Python code for the request to load libraries for a particular user goes through all existing libraries and then checks all of the user's roles for each library, which results in a complexity of O(l*r), l=libraries, r=roles. This PR improves the complexity to O(l).
The BulkRoleCache and RoleCache classes were using a python set to store all roles for a particular user. A user can have a large number of roles, and lookup speed of iterating through a set is slow (O(n)). Most roles don't have the same course id, however. So if you have the course id of the role you're looking for, we can use a dict of course ids that contain related roles. The number of roles per course id is negligible, so we arrive at a lookup speed of O(1) when looking up a user's roles that belong to a specific course id.
The BulkRoleCache now caches and stores user roles in a data structure like this:
{
user_id_1: {
course_id_1: {role1, role2, role3}, # Set of roles associated with course_id_1
course_id_2: {role4, role5, role6}, # Set of roles associated with course_id_2
[ROLE_CACHE_UNGROUPED_ROLES_KEY]: {role7, role8} # Set of roles not tied to any specific course or library. For example, Global Staff roles.
},
user_id_2: { ... } # Similar structure for another user
}
While this changes the data structure used to store roles under the hood and adds the new property `roles_by_course_id` to the RoleCache,
when initializing the RoleCache will store roles additionally in the previous data structure - as a flat set - in the `_roles` property accessible via `all_roles_set`. This establishes
backwards compatibility.
We are now storing roles twice in the RoleCache (in each of the two data structures), which means this takes twice as much memory, but only in the scope of a request.
Blockstore and all of its (experimental) functionality has been replaced with
openedx-learning, aka "Learning Core". This commit uninstalls the now-unused
openedx-blockstore package and removes all dangling references to it.
Note: This also removes the `copy_library_from_v1_to_v2` management command,
which has been broken ever since we switched from Blockstore to Learning Core.
Part of this DEPR: https://github.com/openedx/public-engineering/issues/238
* feat: adds SearchAccess model
Stores a numeric ID for each course + library, which will generally be
shorter than the full context_key, so we can pack more of them into the
the Meilisearch search filter.
Also:
* Adds data migration pre-populates the SearchAccess model from the existing
CourseOverview and ContentLibrary records
* Adds signal handlers to add/remove SearchAccess entries when content
is created or deleted.
* Adds get_access_ids_for_request() helper method for use in views.
* Adds tests.
* test: can't import content.search in lms tests
* feat: use SearchAccess in documents and views
* Adds an access_id field to the document, which stores the
SearchAccess.id for the block's context.
* Use the requesting user's allowed access_ids to filter search results
to documents with those access_ids.
* Since some users have a lot of individual access granted, limit the
number of access_ids in the filter to a large number (1_000)
* Updates tests to demonstrate.
* test: can't import content.search or content_staging in lms tests
* fix: make access_id field filterable
* fix: use SearchAccess.get_or_create in signal handlers
In theory, we shouldn't have to do this, because the CREATE and DELETE
events should keep the SearchAccess table up-to-date.
But in practice, signals can be missed (or in tests, they may be
disabled). So we assume that it's ok to re-use a SearchAccess.id created
for a given course or library context_key.
* refactor: refactors the view tests to make them clearer
Uses helper methods and decorators to wrap the settings and patches used
by multiple view tests.
* feat: adds org filters to meilisearch filter
* Uses content_tagging.rules.get_user_orgs to fetch the user's
content-related orgs for use in the meilisearch filter.
* Limits the number of orgs used to 1_000 to keep token size down
* refactor: removes data migration
Users should use the reindex_studio management command to populate SearchAccess.
* refactor: adds functions to common.djangoapps.student.role_helpers
to allow general access to the user's RoleCache without having to access
private attributes of User or RoleCache.
Related changes:
* Moves some functionality from openedx.core.djangoapps.enrollments.data.get_user_roles
to this new helper method.
* Use these new helper method in content_tagging.rules
* fix: get_access_ids_for_request only returns individual access
instead of all course keys that the user can read.
Org- and GlobalStaff access checks will handle the rest.
* fix: use org-level permissions when generating search filter
Also refactors tests to demonstrate this change for OrgStaff and
OrgInstructor users.
* refactor: remove SearchAccess creation signal handlers
Lets SearchAccess entries be created on demand during search indexing.
* feat: omit access_ids from the search filter that are covered by the user's org roles
---------
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
After testing the enterprise bulk enrollment flow with
force_enrollment=True, I'm finding that the enrollment is still not
forced. This extra logging will hopefully help shed light on where this
boolean might be accidentally ignored.
This pull request pulls translations via atlas for `studio-frontend` and refactor `load_sfe_i18n_messages` to load new translations into `conf/plugins-locale/studio-frontend` instead of relying on deprecated node.js package bundled translations.
* chore: update API endpoints to support default JWT auth
The default DRF Auth classes were recently updated to allow for both JWT and Session auth by default. Any endpoint that overrides the AUTHENTICATION_CLASSES but has just session, just JWT or just both of those should be updated to remove the override.
Details in https://github.com/openedx/edx-platform/issues/33662
