Update our calls to the openedx_content API to reflect 0.35.0's shift
in terminology from "contents" to "media".
---------
Co-authored-by: Kyle McCormick <kyle@axim.org>
build!: Switch to openedx-core (renamed from openedx-learning)
Instead of installing openedx-learning==0.32.0, we install openedx-core==0.34.1.
We update various class names, function names, docstrings, and comments to
represent the rename:
* We say "openedx-core" when referring to the whole repo or PyPI project
* or occasionally "Open edX Core" if we want it to look nice in the docs.
* We say "openedx_content" to refer to the Content API within openedx-core,
which is actually the thing we have been calling "Learning Core" all along.
* In snake-case code, it's `*_openedx_content_*`.
* In camel-case code, it's `*OpenedXContent*`
For consistency's sake we avoid anything else like oex_core, OeXCore,
OpenEdXCore, OexContent, openedx-content, OpenEdxContent, etc.
There should be no more references to learning_core, learning-core, Learning Core,
Learning-Core, LC, openedx-learning, openedx_learning, etc.
BREAKING CHANGE: for openedx-learning/openedx-core developers:
You may need to uninstall openedx-learning and re-install openedx-core
from your venv. If running tutor, you may need to un-mount openedx-learning,
rename the directory to openedx-core, re-mount it, and re-build.
The code APIs themselves are fully backwards-compatible.
Part of: https://github.com/openedx/openedx-core/issues/470
The openedx-learning repo was recently refactored to consolidate its
authoring apps into a single openedx_content app:
https://github.com/openedx/openedx-platform/pull/37924
This commit makes the following changes to accommodate this:
- Bumps the openedx-learning version to 0.31.0 to get the changes.
- Creates new migrations in content_libraries, contentstore, and
modulestore_migrator to foreign key references to authoring apps
to point to the new openedx_content app. This is done without
actually making database changes, since the openedx_content app
models are taking over the existing tables that the authoring apps
once pointed to.
- Creates new squashed migrations in these apps that create these
foreign keys to reference openedx_content app models from the start.
The full rationale for how and why this was done is in the following
openedx-learning ADR:
https://github.com/openedx/openedx-learning/blob/main/docs/decisions/0020-merge-authoring-apps-into-openedx-content.rst
These migrations should run fine from either a from-scratch scenario
(i.e. a new install or CI), or when upgrading from an Ulmo-or-later
database state. If you have a database state that comes from the middle
of the Ulmo development cycle (e.g. October 2025), you may encounter
migration errors in content_libraries, contentstore, or
modulestore_migrator, with an error message complaining about missing
tables. If you receive this message, run the following command:
python manage.py lms migrate openedx_content 0001
Then try to run the migrations for the app that failed. Repeat if
necessary for multiple apps.
There is no way to resume either the backup or restore library actions,
i.e. if you navigate away from it, you have to do it again. This is a
limitation of the current UI because we wanted to get something quick
and simple in for Ulmo, but it also reflects the fact that library
backup/restore should be much faster than course import/export has
historically been.
In any case, sending an email for a 5-10 second task is unnecessary and
distracting, so this commit suppresses the email.
Note: I'm using local imports to get around the fact that the
content_libraries public API is used by content_libraries/tasks.py
which defines the tasks. I can't import from content_libraries/tasks.py
directly, because that would violate import linter rules forbidding
other apps from importing things outside of api.py. This isn't ideal,
but it keeps the fix small and it keeps the logic in the
content_libraries app.
* Updates `ModulestoreBlockMigration` table to allow storing `null` values in `target` field for blocks that failed to migrate/import.
* Adds `unsupported_reason` field to store reason for failure.
* Add number of children blocks in failed block `unsupported_reason` field.
* Fixes issue with blocks like `openassessment` where `url_name` field is not included in its olx during serialization.
As part of the ongoing effort to deprecate and eventually remove xmodule,
we’ve started gradually migrating the necessary code files from xmodule
to more appropriate locations within the codebase.
Ticket: https://github.com/openedx/public-engineering/issues/445
Also: this tweaks importlinter ignores & add follow-up issue links
Co-authored-by: Kyle McCormick <kyle@axim.org>
* feat: add the authz check to the library api function
feat: add the authz publish check in rest_api blocks and containers
feat: add the authz checks in libraries and refactor
feat: add collections checks
feat: update enforcement in serializer file
refactor: refactor the permission check functions
fix: fix value error
fix: calling the queries twice
* test: add structure for test and apply feedback
refactor: refactor the tests and apply feedback
fix: apply feedback
Revert "refactor: refactor the tests and apply feedback"
This reverts commit aa0bd527dd7bc7dec4a7ad7adb41a3c932f4a587.
refactor: use constants and avoid mapping
test: fix the test to have them in order
docs: about we rely on bridgekeeper and the old check for two cases
docs: update openedx/core/djangoapps/content_libraries/api/libraries.py
Co-authored-by: Maria Grimaldi (Majo) <maria.grimaldi@edunext.co>
refactor: use global scope wildcard instead of *
refactor: allow receiving PermissionData objects
refactor: do not inherit from BaseRolesTestCase to favor CL setup methods
If both BaseRolesTestCase and ContentLibrariesRestApiTest define a method
with the same name (e.g., setUp()), Python will use the one found first
in the MRO, which is the one in BaseRolesTestCase because it is
listed first in the class definition leading to unexpected behavior.
refactor: remove unnecessary imports and indent
* chore: bump openedx-authz version
This pulls in publishing dependency changes from:
https://github.com/openedx/openedx-learning/pull/369
This fixes a bug where publishing a Content Library v2 container would
publish only its direct children instead of publishing all ancestors.
Co-authored-by: Kyle McCormick <kyle@axim.org>
Adds a library restore endpoint to restore a learning package from a
backup zip archive (/api/libraries/v2/restore/). The learning package
can then be used to create a content library.
The 'url' field on the GET /api/libraries/v2/{library_id}/backup/?task_id={task_id}
endpoint was returning realtive paths when the file was stored on the default
FileSystemStorage backend, which makes it inconsistent with other storage
backends and semantically incorrect.
This commit addresses this making sure it always returns an absolute url.
The ContentLibraryMetadata used to hold a version field that was meant
to represent the version of the library as a whole. This is a holdover
from v1 libraries, where all changes to the library resulted in a new
version of the content, and that version indicator was used by courses
to know whether or not an update was available.
This maps poorly to Learning Core backed libraries for a number of
reasons:
1. LC-backed libraries have Draft and Published branches, meaning that
a global "version" may be ambiguous.
2. LC-backed libraries have things like tagging and collections, where
modifications are explicitly *not* versioned at all, and do not show
up in either the publish log or the draft change log.
3. Courses that borrow content from LC-backed libraries track
versioning at the level of the individual thing being borrowed, e.g.
a single Component. This is in keeping with the goal to have very
large libraries with many small bits of content to search and use.
This commit removes the notion of a Library-global version entirely for
v2 (LC-backed) libraries. This does not affect legacy v1 libraries that
are backed by ModuleStore.
This introduces the modulestore_migrator app, which can be
used to copy content (courses and libraries) from modulestore
into Learning Core. It is currently aimed to work on the legacy
library -> v2 library migration, but it will be used in the future
for course->library and course->course migrations.
This includes an initial REST API, Django admin interface,
and Python API.
Closes: https://github.com/openedx/edx-platform/issues/37211
Requires some follow-up work before this is production-ready:
https://github.com/openedx/edx-platform/issues/37259
Co-authored-by: Andrii <andrii.hantkovskyi@raccoongang.com>
Co-authored-by: Maksim Sokolskiy <maksim.sokolskiy@raccoongang.com>
* test: Test for publish section/subsection
* test: published_by is now None for unpublished containers
* test: adds TODO comments to the tests
in anticipation of publishing container children
* feat: adds api to retrieve library block/container hierarchy
* test: adds query counts for hierarchy API tests
These are really high, but highlight the need for future optimizations.
* perf: reduce hierarchy API query counts
* perf: cut query counts in half
Required a refactor of the approach to avoid using the Metadata classes.
* chore: trigger ci
* chore: update openedx-learning constraint
* chore: compile requirements
* test: updating query count
* style: Add missing comment in kernel.in
* fix: get_container_from_key param and comments
* docs: mark api as UNSTABLE and add comment about get_library_object_hierarchy implementation
---------
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
* feat: copy endpoint for Library Containers
* fix: make source_usage_key optional and removing upstram info for xblock olx
* test: add tests
* refactor: remove unecessary changes to reduce diff
* fix: change assert
* feat: add `write_upstream` field to ContainerSerializer
* fix: remove comment
* refactor: change `source_usage_key` type and more
* fix: try to infer the source version
* fix: InvalidKeyError while copying container with assets
* fix: read source_version from OLX
* fix: remove store check
* fix: change ident
Co-authored-by: Braden MacDonald <mail@bradenm.com>
* feat: fill source_version and make get_component_version_from_block public
* refactor: rename `source_key` to `copied_from_block`
* test: add test to `write_copied_from=false`
* fix: removing unused fallback elif
* fix: remove `copied_from_block` param
---------
Co-authored-by: Braden MacDonald <mail@bradenm.com>
- Adds the subsections dict in the search index to be used by units
- Adds the sections dict in the search index to be used by sections.
- Rename `get_containers_contains_component` to `get_containers_contains_item` and add support to subsections/sections
- Call `CONTENT_OBJECT_ASSOCIATIONS_CHANGED` for sections and subsections in `update_container_children`
- Refactor of `test_containers.py` 8b9731dfa1
- Updates the `content_libraries/api/containers` and `content_libraries/rest_api` to support:
- Basic CRUD for sections and subsections as containers
- Basic CRUD support for section and subsection children
- Updates the `content/search` to support:
- Basic CRUD for sections and subsections as containers in the search index
- Basic CRUD support for section and subsection children in the search index
- Updates the `content_libraries/tests/test_containers.py` to test the new support
- Updates the `content/search/tests` to test the new support.
- I verified that the add/remove to collection works without major changes. Tests updated
* fix: "[created] received a naive datetime"
* fix: leaky "isolation" of events was causing test failures
* fix: make lib events more specific, emit them async, handle hierarchy correctly
* chore: bump openedx-events to 10.2.0 for new library PUBLISHED events
* feat: store content.child_usage_keys in Container search document
Stores the draft children + published children (if applicable)
Related fixes:
* fix: lib_api.get_container does not take a "user" arg
* refactor: fetch_customizable_fields_from_container does not need a "user" arg
* refactor: moves tags_count into LibraryItem
because anything that appears in a library may be tagged.
* refactor: remove get_container_from_key from public API
API users must use get_container and ContainerMetadata.
* refactor: made set_library_item_collections take an entity_key string instead of
a PublishableEntity instance, so we don't need to fetch a Container object to call it.
* refactor: changed ContainerLink.update_or_create to take the container PK
instead of a Container object, since this is enough.
Added container_pk to ContainerMetadata to support this.
* feat: library unit sync
* feat: create component link only for component xblocks
* feat: container link model
* feat: update downstream api views
* feat: delete extra components in container on sync (not working)
* fix: duplicate definitions of LibraryXBlockMetadata
* test: add a new integration test suite for syncing
* feat: partially implement container+child syncing
* fix: blockserializer wasn't always serializing all HTML block fields
* feat: handle reorder, addition and deletion of components in sync
Updates children components of unit in course based on upstream unit,
deletes removed component, adds new ones and updates order as per
upstream.
* feat: return unit upstreamInfo and disallow edits to units in courses that are sourced from a library (#773)
* feat: Add upstream_info to unit
* feat: disallow edits to units in courses that are sourced from a library (#774)
---------
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
* docs: capitalization of XBlock
Co-authored-by: David Ormsbee <dave@axim.org>
* refactor: (minor) change python property name to reflect type better
* fix: lots of "Tried to inspect a missing...upstream link" warnings
when viewing a unit in Studio
* docs: mention potential REST API for future refactor
* fix: check if upstream actually exists before making unit read-only
* chore: fix camel-case var
* fix: test failure when mocked XBlock doesn't have UpstreamSyncMixin
---------
Co-authored-by: Braden MacDonald <braden@opencraft.com>
Co-authored-by: Chris Chávez <xnpiochv@gmail.com>
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
Co-authored-by: Braden MacDonald <mail@bradenm.com>
Co-authored-by: David Ormsbee <dave@axim.org>
This is needed by several aspects of the Teak Libraries Overhaul
(https://github.com/orgs/openedx/projects/66) including:
* copy-paste of containers between courses and V2 libraries
* syncing of containers in courses from V2 libraries
* the import_from_modulestore API
* Added publish_display_name and last_published on container search document.
* This change is used in feat: Add unit from library in course [FC-0083] frontend-app-authoring#1829 to show only published units in the unit picker, also to show the published_display_name in the picker, and avoid empty titles.
* Which edX user roles will this change impact? "Developer".
