Meilisearch index documents were not synced properly when any block with children blocks like units, subsections, sections etc. were being deleted as the `XBLOCK_DELETED` is only triggered for the deleted block.
This PR fixes it by deleting all index documents that contain the deleted block in its `breadcrumbs` field as only blocks that are children of this block will have it its breadcrumbs field.
Similarly, the entity links that store links between course and library blocks was not synced properly due to children `ContainerLinks` not being deleted.
- `handle_update_xblock_upstream_link` is called asynchronously with celery. In `update_upstream_downstream_link_handler`, the xblock has the updated version, but when calling `handle_update_xblock_upstream_link` inside Celery, the xblock is outdated in a previous version, which is why the error occurs. This happens because `on_commit_changes_to` is executed before the MySQL transaction ends.
- Added `ImmediateOnCommitMixin` to be used in tests that need to call `on_commit_changes_to`. See https://github.com/openedx/edx-platform/pull/37485#issuecomment-3412979170 for more info
Index the newly created course after import and course re-run. Also recreate upstream links after course re-run.
We make use of newly created COURSE_RERUN_COMPLETED signal to run post re-run processes.
- Adds the subsections dict in the search index to be used by units
- Adds the sections dict in the search index to be used by sections.
- Rename `get_containers_contains_component` to `get_containers_contains_item` and add support to subsections/sections
- Call `CONTENT_OBJECT_ASSOCIATIONS_CHANGED` for sections and subsections in `update_container_children`
- Refactor of `test_containers.py` 8b9731dfa1
- Updates the `content_libraries/api/containers` and `content_libraries/rest_api` to support:
- Basic CRUD for sections and subsections as containers
- Basic CRUD support for section and subsection children
- Updates the `content/search` to support:
- Basic CRUD for sections and subsections as containers in the search index
- Basic CRUD support for section and subsection children in the search index
- Updates the `content_libraries/tests/test_containers.py` to test the new support
- Updates the `content/search/tests` to test the new support.
- I verified that the add/remove to collection works without major changes. Tests updated
* fix: "[created] received a naive datetime"
* fix: leaky "isolation" of events was causing test failures
* fix: make lib events more specific, emit them async, handle hierarchy correctly
* chore: bump openedx-events to 10.2.0 for new library PUBLISHED events
* feat: store content.child_usage_keys in Container search document
Stores the draft children + published children (if applicable)
Related fixes:
* fix: lib_api.get_container does not take a "user" arg
* refactor: fetch_customizable_fields_from_container does not need a "user" arg
* refactor: moves tags_count into LibraryItem
because anything that appears in a library may be tagged.
* refactor: remove get_container_from_key from public API
API users must use get_container and ContainerMetadata.
* refactor: made set_library_item_collections take an entity_key string instead of
a PublishableEntity instance, so we don't need to fetch a Container object to call it.
* refactor: changed ContainerLink.update_or_create to take the container PK
instead of a Container object, since this is enough.
Added container_pk to ContainerMetadata to support this.
* refactor: update sync model helper function docs
Adds some comments to explain certain confusing sections
* refactor: sync library content method
* feat: use edited_on block field
* test: modified field in course block index
* fix: extract uncommon methods to child class from base
* Added publish_display_name and last_published on container search document.
* This change is used in feat: Add unit from library in course [FC-0083] frontend-app-authoring#1829 to show only published units in the unit picker, also to show the published_display_name in the picker, and avoid empty titles.
* Which edX user roles will this change impact? "Developer".
efactors the content_libraries.api to use LibraryCollectionLocator and LibraryContainerLocator keys, instead of passing separate LibraryUsageKeyV2 keys along with the collection/container locators.
Renames misleading uses of collection_usage_key to collection_locator, including in the content_libraries.api and content.search.api method names and parameters.
This change impacts Developers, but should not affect end users.
This refactoring seems reasonable to do without going through deprecation, given the minimal use of these APIs.
* feat: Added get_containers_contains_component in containers api with tests
* feat: Add publish_status to containers search document
* feat: Add LIBRARY_CONTAINER_UPDATED whend deleted a component inside a container
* feat: Send LIBRARY_CONTAINER_UPDATED signal when updating component of container
* fix: Bugs sending LIBRARY_CONTAINER_UPDATED signal
* feat: Add publish_status of container as PublishStatus.Never by default
* refactor: ContentLibraryContainersTest to use update_container_children to add components
* style: Clean code after fix conflicts
* fix: Broken lint
* fix: lint
* feat: add components to container api
* feat: remove and replace components in container api
* refactor: container childern api
* chore: fix lint issues
* temp: install openedx-learning dev branch
* feat: update publish_status and children count in index
* chore: fix mypy issues
* test: fix reindex test
* refactor: rebase and fix conflicts
* test: update test to check signals
* docs: document can_stand_alone flag
* chore: bump openedx-learning version
* refactor: convert libraries API from attr.s to dataclass, fix types
* fix: make corresponding updates to 'search' code
* feat: use new version of openedx-learning with containers support
* temp: Use opencraft branch of opaquekeys
* refactor: Use LibraryElementKey instead of LibraryCollectionKey
* refactor: split libraries API & REST API up into smaller modules
* feat: new REST API for units in content libraries
* feat: python+REST API to get a unit
* feat: auto-generate slug/key/ID from title of units
* feat: generate search index documents for containers
* refactor: rename LibraryElementKey to LibraryItemKey
* fix: lint error
* feat: adds new units to search index on create/update
and when running reindex_studio.
Updates requirements for openedx-events and openedx-learning to support
these changes.
* fix: pylint
* fix: temp requirement
* fix: search index container events/tasks
* feat: add get_library_container_usage_key to libraries API
and use it when search indexing containers
* fix: index all containers during reindex_studio
* chore: bump openedx-events requirement
* fix: address review comments
* chore: bumps openedx-learning to 0.19.1
* fix: rename api method to library_container_locator
since container keys are locators, not usage keys
* chore: bumps opaque-keys dependency
* test: fix misnamed unit_usage_key
* feat: adds APIs to update or delete a container (#757)
* feat: adds python and REST APIs to update a container's display_name
* refactor: adds _get_container method to api to reduce code duplication
* feat: adds python and REST APIs to delete a container
* test: add container permission tests
---------
Co-authored-by: XnpioChV <xnpiochv@gmail.com>
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
Adds the publish status field to the libraries v2 meilisearch index in order to support filtering by component publish status: published, modified, never.
Fixed component counter synchronization in these cases:
* When deleting a component inside a collection.
* With the library published, when adding a new component in a collection and reverting library changes.
* With the library published, when deleting a component inside a collection and reverting library changes.
Also adds a published > num_counts field in collections in the search index.
At one point, we envisioned having different kinds of libraries, e.g.
a "Video" library would be distinct from a "Problem" library. Later on,
we decided on a more generalized form of Libraries, where any given
library can hold any combination of content–which would then be
organized using collections and tagging.
Due to this shift in perspective, these values haven't actually been
used for a long time. This is just getting rid of them altogether.
* refactor: use django signals to trigger LIBRARY_COLLECTION events
* refactor: use collection usage_key as search document id
This change standardises the search document "id" to be a meilisearch ID
generated from the usage key, for all types of indexed objects.
This is important for collections so we can locate the collection
document in the search index solely from the data provided by the
LIBRARY_COLLECTION_DELETED event (library_key + collection_key), even if
the collection has been deleted from the database.
* refactor: avoid fetching more data than we have to.
* get_library_collection_usage_key and
searchable_doc_tags_for_collection do not need a Collection object;
the usage key can be created from the library_key and collection_key.
* updated searchable_doc_for_collection to require the parts of the
collection usage key + an optional collection. This allows us to
identify the collection's search document from its usage key without
requiring an existing Collection object (in case it's been deleted).
Also removes the edge case for indexing Collections not associated
with a ContentLibrary -- this won't ever really happen.
* feat: remove soft- and hard-deleted collections from search index
* feat: adds library_component_usage_key to content_libraries.api
* refactor: send CONTENT_OBJECT_ASSOCIATON_CHANGED on django model signals
so that added/removed collections are removed/re-added to component documents.
Special case: When a collection is soft-deleted/restored, we detect this
in the search index and update the collection's component documents
directly, without a CONTENT_OBJECT_ASSOCIATON_CHANGED signal.
* chore: bumps openedx-learning to 0.13.0
* feat: Add Library Collections REST endpoints
* test: Add tests for Collections REST APIs
* chore: Add missing __init__ files
* docs: Add warning about unstable REST APIs
* feat: emit CONTENT_OBJECT_ASSOCIATIONS_CHANGED
whenever a content object's tags or collections have changed,
and handle that event in content/search.
The deprecated CONTENT_OBJECT_TAGS_CHANGED event is still emitted when
tags change; to be removed after Sumac.
* docs: replaces CONTENT_OBJECT_TAGS_CHANGED with CONTENT_OBJECT_ASSOCIATIONS_CHANGED
* chore: updates openedx-events==9.14.0
* chore: updates openedx-learning==0.11.4
Co-authored-by: Yusuf Musleh <yusuf@opencraft.com>
Co-authored-by: Chris Chávez <xnpiochv@gmail.com>
Co-authored-by: Rômulo Penido <romulo@thinkdash.dev>
