xblock information was not being translated correctly when using the blockstore runtime. This makes the i18n_service block-specific so it can look for additional i18n files that can be installed with the XBlock
These changes were initially made to make it easier to do SECRET_KEY rotations. Along the way, we found it made sense to refractor the code as well.
Changes made:
- changed get_to_create to create because now the code should only get to this block when a write is necessary
- added a lookup for anonymous_user_id. This is to return an existing anonymous_user_id rather than calculating. This will mitigate the results of SECRET_KEY rotation.
- Added monitoring to help us make better decisions: should we not sue SECRET_KEY, performance considerations...
- put old function behind toggle in case something goes wrong in production with new code
- refractoring function structure for better understanding
This addresses a longstanding TODO item to make runtime.anonymous_student_id for content libraries v2 work the same way as it does for XBlocks in regular courses, persisting the "context ID" (equivalent to course ID) to the database. This way, if SECRET KEY is changed, existing anonymous IDs will continue to work unchanged.
This is a potentially breaking change, but should mostly affect capa problems using external code graders or Matlab code input, and I'm not aware of any such usage of the new runtime / libraries v2.
This is required because edx-platform uses the waffle API from
completion, which itself depends on edx-toggles. So if we change
edx-toggles import paths, we also need to upgrade our usage of
completion here.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
Also improve usefulness of some blockstore runtime logs for debugging
Context:
Sometimes when trying to load an XBlock's XML file from Amazon S3, AWS will return a 4xx or 5xx response along with error XML like:
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>foo/bar</Key>...</Error>
A bug in the get_bundle_file_data_with_cache method would cause this XML to be returned to the runtime anyways, as if it were the expected OLX. This would then (obviously) lead to strange parsing bugs, e.g. when trying to interpret <Code> as an <xblock-include>.
This fixes the bug and improves the logging, both to make this sort of issue easier to debug in the future and to return whatever detailed error code S3 provides (or Blockstore, if S3 is not being used).
Also improve usefulness of some blockstore runtime logs for debugging
Context:
Sometimes when trying to load an XBlock's XML file from Amazon S3, AWS will return a 4xx or 5xx response along with error XML like:
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>foo/bar</Key>...</Error>
A bug in the get_bundle_file_data_with_cache method would cause this XML to be returned to the runtime anyways, as if it were the expected OLX. This would then (obviously) lead to strange parsing bugs, e.g. when trying to interpret <Code> as an <xblock-include>.
This fixes the bug and improves the logging, both to make this sort of issue easier to debug in the future and to return whatever detailed error code S3 provides (or Blockstore, if S3 is not being used).
I needed this change because I found a bug:
1. Create a block with children in a content library
2. Delete that content library
3. Create a new identical block with children in a new content library.
4. If the OLX is identical to the original block, this new block will not load in the LMS.
The reason for the bug is that the .children field contains usage keys (which encode the library, for example), but the values were being stored in BlockstoreFieldData which caches really aggressively and caches based on the hash of the OLX. Since the OLX is identical, it assumes the .children values should be identical as well.
The fix was to move children to a children-specific field data store, and only store the part of the child data that is encoded by the OLX (the <xblock-include> data) in BlockstoreFieldData. This is a better match for the way the caching works and cleaned up a hacky part of the runtime (at least it's slightly less hacky now).
Implementation details:
* Anonymous users are assigned a unique ID (like
`anon42c08f9996194e2a9339`) which gets stored in the django session.
`block.scope_ids.user_id` and `block.runtime.anonymous_student_id`
will both return this value.
* User state for anonymous users is stored in the django cache and
automatically expires as the cache gets pruned. Because user state is
stored, anonymous users can use interactive blocks like capa problems.
* There is no mechanism for upgrading to a registered account and
keeping user state since the user state store for anonymous users
(EphemeralKeyValueStore) is completely different than the one for
registered users (DjangoKeyValueStore/"CSM"), and has no "list all
keys" functionality.
* "User State Summary" field values are shared among [recently active]
anonymous users but are not shared with registered users.
* Anonymous users can only access the `public_view` of XBlocks, not the
regular `student_view`.
This PR introduces some backend python + REST APIs for storing static
asset files along with an XBlock in a content library. It also updates
the new runtime to be able to load such static asset files.
Example use cases:
* Store an image file with an HTML block and then use the image inline
in the HTML block.
* Store a PDF file with an HTML block and provide a link in the HTML for
the learner to download the PDF.
* Store .srt files or even video .mp4 files that belong to a video
XBlock.
Within the bundle, these static asset files are stored in a "static/"
subfolder of the folder that contains the OLX file. Extending an
existing LMS/Studio convention, a static asset file such as "image.png"
is referenced within the OLX as "/static/image.png" and the URL will be
rewritten by the runtime.
This commit introduces the changes needed for XBlocks in Blockstore to save
their user state into CSM. Before this commit, all student state for Blockstore
blocks was ephemeral (in-process dict store).
Notes:
* The main risk factor of this PR is that it adds non-course keys to the
course_id field in CSM. If any code (like analytics?) reads course keys
directly out of CSM and doesn't have graceful handling for key types it
doesn't recognize, it could cause an issue. With the included changes to
opaque-keys, calling CourseKey.from_string(...) on these values will raise
InvalidKeyError since they're not CourseKeys. (But calling
LearningContextKey.from_string(...) will work for both course and library
keys.)
* This commit introduces a slight regression for the Studio view of XBlocks in
Blockstore content libraries: their state is now lost from request to request.
I have a follow up PR to give them a proper studio-appropriate state store,
but I want to review it separately so it doesn't hold up this PR and we can
test this PR on its own.
https://github.com/edx/edx-platform/pull/20645
This introduces:
* A new XBlock runtime that can read and write XBlocks that are persisted using
Blockstore instead of Modulestore. The new runtime is currently isolated so
that it can be tested without risk to the current courseware/runtime.
* Content Libraries v2, which store XBlocks in Blockstore not modulestore
* An API Client for Blockstore
* "Learning Context" plugin API. A learning context is a more abstract concept
than a course; it's a collection of XBlocks that serves some learning purpose.
