Adds the publish status field to the libraries v2 meilisearch index in order to support filtering by component publish status: published, modified, never.
Fixed component counter synchronization in these cases:
* When deleting a component inside a collection.
* With the library published, when adding a new component in a collection and reverting library changes.
* With the library published, when deleting a component inside a collection and reverting library changes.
Also adds a published > num_counts field in collections in the search index.
PII Annotations are very out of date, this commit adds most that were
missing in edx-platform, and some additional annotations to the
safelist. It is not comprehensive, several other upstream Open edX
packages also need to be updated. It also does not include removing
annotations that have been moved upstream, or been removed entirely.
Those are separate follow-on tasks.
We don't call the modulestore or update the cache here now that we are
backed by the database model. Previously the cache would change because
the `_encode_root_cache_key` function in `BlockStructureStore` class
used the `BlockstoreBlockData.VERSION` as a part of the cache key when
the data was not being cached in a DB model.
Previously, we were not caching BlockStructures to the database when we
were adding them to the store by default. Now that we are doing that,
the BlockStructureFactory test failed because it was taking a shortcut
that would no longer work. It was just creating a blockstructure that
had relations but not any block information.
Now that we're always persisting updates to the database, this broke
because to persist the structure to the database, we have to look up the
block information from the block_structure which now fails.
This change updates the test mixin to add more data so that the content
can be persisted to the database successfully as a part of this test.
This work is part of DEPR https://github.com/openedx/public-engineering/issues/32
Now that we've removed all uses for this switch remove the decleration
as well.
BREAKING CHANGE: The `block_structure.storage_backing_for_cache` will no
longer exist and its value will be ignored. If you have this switch set
in your instance you can remove it. The backing cache is now always ON.
Remove the `enable_storage_backing_for_cache_in_request` function and
its uses in the platform. The function is no longer needed because the
storage backing for the block_structure cache will be ON by default
moving forward.
BREAKING CHANGE: This `enable_storage_backing_for_cache_in_request`
function no longer exists and any calls to it should be removed. The
cache it enables is now always ON.
At one point, we envisioned having different kinds of libraries, e.g.
a "Video" library would be distinct from a "Problem" library. Later on,
we decided on a more generalized form of Libraries, where any given
library can hold any combination of content–which would then be
organized using collections and tagging.
Due to this shift in perspective, these values haven't actually been
used for a long time. This is just getting rid of them altogether.
* refactor: use django signals to trigger LIBRARY_COLLECTION events
* refactor: use collection usage_key as search document id
This change standardises the search document "id" to be a meilisearch ID
generated from the usage key, for all types of indexed objects.
This is important for collections so we can locate the collection
document in the search index solely from the data provided by the
LIBRARY_COLLECTION_DELETED event (library_key + collection_key), even if
the collection has been deleted from the database.
* refactor: avoid fetching more data than we have to.
* get_library_collection_usage_key and
searchable_doc_tags_for_collection do not need a Collection object;
the usage key can be created from the library_key and collection_key.
* updated searchable_doc_for_collection to require the parts of the
collection usage key + an optional collection. This allows us to
identify the collection's search document from its usage key without
requiring an existing Collection object (in case it's been deleted).
Also removes the edge case for indexing Collections not associated
with a ContentLibrary -- this won't ever really happen.
* feat: remove soft- and hard-deleted collections from search index
* feat: adds library_component_usage_key to content_libraries.api
* refactor: send CONTENT_OBJECT_ASSOCIATON_CHANGED on django model signals
so that added/removed collections are removed/re-added to component documents.
Special case: When a collection is soft-deleted/restored, we detect this
in the search index and update the collection's component documents
directly, without a CONTENT_OBJECT_ASSOCIATON_CHANGED signal.
* chore: bumps openedx-learning to 0.13.0
* feat: Add Library Collections REST endpoints
* test: Add tests for Collections REST APIs
* chore: Add missing __init__ files
* docs: Add warning about unstable REST APIs
* feat: emit CONTENT_OBJECT_ASSOCIATIONS_CHANGED
whenever a content object's tags or collections have changed,
and handle that event in content/search.
The deprecated CONTENT_OBJECT_TAGS_CHANGED event is still emitted when
tags change; to be removed after Sumac.
* docs: replaces CONTENT_OBJECT_TAGS_CHANGED with CONTENT_OBJECT_ASSOCIATIONS_CHANGED
* chore: updates openedx-events==9.14.0
* chore: updates openedx-learning==0.11.4
Co-authored-by: Yusuf Musleh <yusuf@opencraft.com>
Co-authored-by: Chris Chávez <xnpiochv@gmail.com>
Co-authored-by: Rômulo Penido <romulo@thinkdash.dev>
As of Python 3.3, the 3rd-party `mock` package has been subsumed into the
standard `unittest.mock` package. Refactoring tests to use the latter will
allow us to drop `mock` as a dependency, which is currently coming in
transitively through requirements/edx/paver.in.
We don't actually drop the `mock` dependency in this PR. That will happen
naturally in:
* https://github.com/openedx/edx-platform/pull/34830
