* Deprecate `get_block_draft_olx`
* Deprecate get olx view in content libraries
* Create `get_block_olx` in xblock API with support of versions
* Create get olx view in xblock
* fix: problem block could not be used with versioned handler URls
* refactor: simplify REST API handling of usage keys
* feat: add more version awareness and update tests
* fix: make the preview changes modal bigger as requested
* refactor: parse version at the urlconf layer too
This moves the Content Libraries V2 backend from Blockstore [1] over to
Learning Core [2] For high-level overview and rationale of this move, see
the Blockstore DEPR [3]. There are several follow-up tasks [4], most notably
adding support for static assets in libraries.
BREAKING CHANGE: Existing V2 libraries, backed by Blockstore, will stop
working. They will continue to be listed in Studio, but their content
will be unavailable. They need to be deleted (via Django admin) or manually
migrated to Learning Core. We do not expect production sites to be in
this situation, as the feature has never left "experimental" status.
[1] https://github.com/openedx-unsupported/blockstore
[2] https://github.com/openedx/openedx-learning/
[3] https://github.com/openedx/public-engineering/issues/238
[4] https://github.com/openedx/edx-platform/issues/34283
* test: warn about dependencies from cms->openedx->lms and vice versa
* test: warn about importing from package's internal implementation code
* chore: Update some imports to use public APIs only
* chore: Update 'bookmarks' app to have stricter public API
* fix: we are sharing 'adapters' from olx_rest_api to content_staging
when fetching block metadata and rendering blocks while maintaining the
original usage IDs/OLX.
This change is marked temporary because LabXchange need it during the
transition to a custom runtime, but it's not really useful to anyone
else. We will revert this change with a future PR.
The cache in the previous version of this code was unwittingly being
shared among all threads, and an occasional race condition would result
in the .children field of some XBlocks containing duplicate entries.
I tried to find other ways to keep the existing cache design and let it
be shared among all the threads (which would be more efficient), but I
couldn't find any clean way of doing it (and even then, this code was
not written with the intention of being used in a multi-threaded way).
So to keep the fix simple, I made the block data cache thread-local
instead of process-local. That eliminated the bug.
Technical details:
The big challenge with this code in the first place was due to the
parse_xml API of XBlocks, which instantiates the XBlock instance and
_then_ sets field data on it and returns it, as there is no mechanism
available to distinguish that from the case of instantiating an XBlock
(using previously loaded/cached field data) and then deliberately
changing its field values. In particular, parse_xml sets the 'children'
field just by calling self.children.append(...) but never explicitly
initializes self.children to [] first, so it's necessary for the field
data store to have a mechanism for setting self.children = [] when
parsing begins, but it's hard to know "when parsing begins" in general.
This is made more challenging since the BlockstoreFieldData design ties
field data changes to the XBlock instance itself (using a weakkey), but
it's impossible to get a reference to the XBlock instance before calling
parse_xml, and since the BlockstoreFieldData design uses a pass-through
approach where fields that aren't being actively changed are read from
the cache; since it doesn't know when children is being initialized vs.
being modified, it would sometimes pass-through and start one thread's
changes with the final result from another thread.
Anyhow, the bottom line is that avoiding unintentional multithreading
here solves the problem.
If we want the field data cache to be shared among threads, it might as
well be rewritten to use memcached and shared among all processes too.
That would be a very good performance boost but would take up a lot more
memory in memcached. Also the rewrite may be challenging due to the
aforementioned nuances of the XBlock parse_xml / construct_xblock /
add_node_as_child APIs. Perhaps modifying the runtime to use a
completely separate fielddata implementation for parsing vs. loading a
parsed+cached XBlock would do it.
Without this PR, there is no [reasonable] way to get the following data
for any XBlocks in the new runtime; now there is :)
* index_dictionary: data about the block content, for search indexing
* student_view_data: data-only equivalent of student_view, for use in
custom UIs/mobile
* children: list of child IDs
* editable_children: list of child IDs in the same bundle (use case:
when showing an OLX editor you want to allow editing the OLX of
children in the same bundle but not linked children)
Implementation details:
* Anonymous users are assigned a unique ID (like
`anon42c08f9996194e2a9339`) which gets stored in the django session.
`block.scope_ids.user_id` and `block.runtime.anonymous_student_id`
will both return this value.
* User state for anonymous users is stored in the django cache and
automatically expires as the cache gets pruned. Because user state is
stored, anonymous users can use interactive blocks like capa problems.
* There is no mechanism for upgrading to a registered account and
keeping user state since the user state store for anonymous users
(EphemeralKeyValueStore) is completely different than the one for
registered users (DjangoKeyValueStore/"CSM"), and has no "list all
keys" functionality.
* "User State Summary" field values are shared among [recently active]
anonymous users but are not shared with registered users.
* Anonymous users can only access the `public_view` of XBlocks, not the
regular `student_view`.
This commit introduces the changes needed for XBlocks in Blockstore to save
their user state into CSM. Before this commit, all student state for Blockstore
blocks was ephemeral (in-process dict store).
Notes:
* The main risk factor of this PR is that it adds non-course keys to the
course_id field in CSM. If any code (like analytics?) reads course keys
directly out of CSM and doesn't have graceful handling for key types it
doesn't recognize, it could cause an issue. With the included changes to
opaque-keys, calling CourseKey.from_string(...) on these values will raise
InvalidKeyError since they're not CourseKeys. (But calling
LearningContextKey.from_string(...) will work for both course and library
keys.)
* This commit introduces a slight regression for the Studio view of XBlocks in
Blockstore content libraries: their state is now lost from request to request.
I have a follow up PR to give them a proper studio-appropriate state store,
but I want to review it separately so it doesn't hold up this PR and we can
test this PR on its own.
https://github.com/edx/edx-platform/pull/20645
This introduces:
* A new XBlock runtime that can read and write XBlocks that are persisted using
Blockstore instead of Modulestore. The new runtime is currently isolated so
that it can be tested without risk to the current courseware/runtime.
* Content Libraries v2, which store XBlocks in Blockstore not modulestore
* An API Client for Blockstore
* "Learning Context" plugin API. A learning context is a more abstract concept
than a course; it's a collection of XBlocks that serves some learning purpose.
