[APER-3241]
This PR updates the retirement pipeline to purge learners' names from certificate records when their account is being retired.
It also introduces a new management command that can be used by Open edX operators to purge the leftover name data (PII data) from the `certificates_generatedcertificate` table. This is designed as a one-time use data fixup, as the retirement functionality should clean this moving forward.
Note: I had originally implemented this as a `warnings.warn()` call
directly in lms/envs/devstack.py and cms/envs/devstack.py, but for
whatever reason, those warnings were getting swallowed. System checks
display more prominently, anyway.
Part of: https://github.com/openedx/public-engineering/issues/247
Blockstore and all of its (experimental) functionality has been replaced with
openedx-learning, aka "Learning Core". This commit uninstalls the now-unused
openedx-blockstore package and removes all dangling references to it.
Note: This also removes the `copy_library_from_v1_to_v2` management command,
which has been broken ever since we switched from Blockstore to Learning Core.
Part of this DEPR: https://github.com/openedx/public-engineering/issues/238
* feat: make tagging feature enabled by default
* fix: use the correct flag for tagging enabled
* fix: make compatible with other changes from master
* fix: more compatibility fixes
* fix: show tag counts at all levels of the outline, not just units
* chore: typo
* test: fix counts in test suite now that tagging is on by default
---------
Co-authored-by: Braden MacDonald <braden@opencraft.com>
Co-authored-by: Yusuf Musleh <yusuf@opencraft.com>
* fix: Multiple auto-tagging when creating a course
* Avoid tagging course-block and course-info blocks
* Tests
* feat: Avoid create index for course blocks
There was a logical error in the compile_sass management command:
instead of falling back to settings.COMPREHENSIVE_THEME_DIRS when
--theme-dirs was *None or missing*, we only fell back to it when
--theme-dirs was *missing*.
This caused theme compilation to be skipped when COMREHENSIVE_THEME_DIRS
*is not set* in the environment, even though
settings.COMPREHENSIVE_THEME_DIRS *is set* in Django settings, which
is currently the case for edx.org.
Reverts #34554, which causes compilation of edX.org's
legacy comprehensive theme to be skipped in their deployment pipeline.
We have not determined the precise cause yet, but it seems like the
compile_sass management command is not correctly getting the
list of comprehensive theme directories from Django settings.
Together, these changes make it so that all features of the Paver-based
asset compilation system are supported with drop-in Paver-free
replacements. The remaining Paver asset functions are trivial wrappers,
which can be comfortably deleted before Sumac.
* Turn `./manage.py ... compile_sass` into a simple wrapper around `npm
run compile-sass`
* Turn `paver webpack` into a simple wrapper around `npm run webpack`
* Turn `pavelib.assets:collect_assets` into a simple wrapper around
`./manage.py ... collectstatic`
* Add/improve deprecation warnings for all Paver asset commands.
* Load defaults for asset-related Django settings from environment
variables. This allows the build to work without Python. For the
settings which will be removed in Sumac, I've added deprecation
warnings.
* Change EDX_PLATFORM_THEME_DIRS env var to COMPREHENSIVE_THEME_DIRS.
This simplifies the migration instructions, because all the new env
vars now match their corresponding Django settings. This amends an
ADR, but it should not be a breaking change because the env var was
recently added (since Quince) and nobody should be using it yet.
* Future-proof the static assets ADR with links. The linked pages will
be kept up-to-date even if the ADR isn't.
Part of: https://github.com/openedx/edx-platform/issues/34467
Implements the connection from the teams feature to the content groups feature. This implementation uses the dynamic partition generator extension point to associate content groups with the users that belong to a Team.
This implementation was heavily inspired by the enrollment tracks dynamic partitions.
This PR removes the `clean_certificate_available_date` management command. Recently, a change was made to sync the end date of a course run with a display behavior of "end" as its certificate available date in the Credentials IDA. This change was made to resolve a long running issue with the Credentials IDA not displaying certificates to learners correctly on learner records when the display behavior is set to "end".
This management command, if run, would clear the certificate available date from the course certificate configurations in the Credentials IDA. This would be a regression based on the new functionality. Other improvements have been made to ensure that updates made in the LMS properly flow to Credentials.
TL;DR -- this management command is no longer needed because of feature changes in edx-platform. Allowing this management command to run would break functionality for learners.
* feat: Update search index when object tags updated
* feat: Update index when library block tags change
* refactor: Remove extra params
* docs: Add new event to hook events docs
* feat: Add and use upsert_block_tags_index_docs
This takes care of updating tags data in search index for both course and library blocks.
* chore: Update openedx-events
* fix: Update tests + include course block tags in reindex
* feat: Fix static-type issues + adjust tag_object
* fix: bug retrieving the Meilisearch API key UID (first time only)
* docs: Update comments
---------
Co-authored-by: Braden MacDonald <braden@opencraft.com>
The Hashable object was moved in python 3.3 and support for the old
location is dropped in python 3.10 the new location is available in
python 3.8 so we can just update this and it should work with both
python 3.8 and 3.11
https://docs.python.org/3.8/library/collections.html
Given code like the following
```
class Foo:
@process_cached
def bar(self):
pass
```
In Python 3.8 referencing `bar` would not call its `__get__` method.
```
x = Foo().bar
```
However in Python 3.11, making the same call would call the `__get__`
method, permanently replacing the underlying `process_cached` object
with the partial function that references it.
This meant that code to clear the cache would work in Python 3.8 but
would break in 3.11
```
Foo().bar.cache.clear() # Works in 3.8 but not in 3.11
```
In 3.11 this results in the following error:
```
E AttributeError: 'functools.partial' object has no attribute 'cache'
```
To make this compatible in both version, we just add the cache as an
accessible attribute on the partial we generate for our wrapped
function.
* feat: adds SearchAccess model
Stores a numeric ID for each course + library, which will generally be
shorter than the full context_key, so we can pack more of them into the
the Meilisearch search filter.
Also:
* Adds data migration pre-populates the SearchAccess model from the existing
CourseOverview and ContentLibrary records
* Adds signal handlers to add/remove SearchAccess entries when content
is created or deleted.
* Adds get_access_ids_for_request() helper method for use in views.
* Adds tests.
* test: can't import content.search in lms tests
* feat: use SearchAccess in documents and views
* Adds an access_id field to the document, which stores the
SearchAccess.id for the block's context.
* Use the requesting user's allowed access_ids to filter search results
to documents with those access_ids.
* Since some users have a lot of individual access granted, limit the
number of access_ids in the filter to a large number (1_000)
* Updates tests to demonstrate.
* test: can't import content.search or content_staging in lms tests
* fix: make access_id field filterable
* fix: use SearchAccess.get_or_create in signal handlers
In theory, we shouldn't have to do this, because the CREATE and DELETE
events should keep the SearchAccess table up-to-date.
But in practice, signals can be missed (or in tests, they may be
disabled). So we assume that it's ok to re-use a SearchAccess.id created
for a given course or library context_key.
* refactor: refactors the view tests to make them clearer
Uses helper methods and decorators to wrap the settings and patches used
by multiple view tests.
* feat: adds org filters to meilisearch filter
* Uses content_tagging.rules.get_user_orgs to fetch the user's
content-related orgs for use in the meilisearch filter.
* Limits the number of orgs used to 1_000 to keep token size down
* refactor: removes data migration
Users should use the reindex_studio management command to populate SearchAccess.
* refactor: adds functions to common.djangoapps.student.role_helpers
to allow general access to the user's RoleCache without having to access
private attributes of User or RoleCache.
Related changes:
* Moves some functionality from openedx.core.djangoapps.enrollments.data.get_user_roles
to this new helper method.
* Use these new helper method in content_tagging.rules
* fix: get_access_ids_for_request only returns individual access
instead of all course keys that the user can read.
Org- and GlobalStaff access checks will handle the rest.
* fix: use org-level permissions when generating search filter
Also refactors tests to demonstrate this change for OrgStaff and
OrgInstructor users.
* refactor: remove SearchAccess creation signal handlers
Lets SearchAccess entries be created on demand during search indexing.
* feat: omit access_ids from the search filter that are covered by the user's org roles
---------
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
After testing the enterprise bulk enrollment flow with
force_enrollment=True, I'm finding that the enrollment is still not
forced. This extra logging will hopefully help shed light on where this
boolean might be accidentally ignored.
