We migrated the source of truth for what the active draft and published
versions of course and v1 library content are to the
SplitModulestoreCourseIndex Django model. But the contentpruning
code (structures.py) that was developed in tubular and will be moved to
edx-platform is not aware of this newer model, and still only pulls its
source of truth from MongoDB. So we *must* continue to do writes to
MongoDB, or the pruning code will start pruning live versions.
The longer term fix for this is to make the pruning code aware of
SplitModulestoreCourseIndex, which will be easier once it's moved into
edx-platform.
Exposes the hide_from_toc xblock attribute so course authors can configure it as a section visibility option in Studio. Before this change, the Hide from TOC functionality was mainly used by OLX components. Hence, it wasn't available for configuration through the Studio UI. Still, its implementation existed in the platform and could be used by setting the attribute: hide_from_toc=true as part of the OLX definition.
Ref: https://openedx.atlassian.net/wiki/spaces/OEPM/pages/3853975595/Feature+Enhancement+Proposal+Hide+Sections+from+course+outline
This moves the Content Libraries V2 backend from Blockstore [1] over to
Learning Core [2] For high-level overview and rationale of this move, see
the Blockstore DEPR [3]. There are several follow-up tasks [4], most notably
adding support for static assets in libraries.
BREAKING CHANGE: Existing V2 libraries, backed by Blockstore, will stop
working. They will continue to be listed in Studio, but their content
will be unavailable. They need to be deleted (via Django admin) or manually
migrated to Learning Core. We do not expect production sites to be in
this situation, as the feature has never left "experimental" status.
[1] https://github.com/openedx-unsupported/blockstore
[2] https://github.com/openedx/openedx-learning/
[3] https://github.com/openedx/public-engineering/issues/238
[4] https://github.com/openedx/edx-platform/issues/34283
See discussion here: https://github.com/overhangio/tutor/pull/984
This is a breaking change that will explicitely set the timeout of
course structure cache entries to 1 week, instead of being unlimited. If
you wish to revert to the former behaviour, you should set
`course_structure_cache["TIMEOUT"] = None`.
The course structure cache keys were previously set with an explicit
timeout of `None` which was overriding the cache default timeout of 2h.
This was OK in environments where the cache is configured with a maximum
memory limit and an automatic key eviction strategy. But in others (such
as Tutor), the course structure cache could grow infinitely.
It was agreed that course structure cache keys should be long-lived but
should respect the default cache structure timeout. Thus, we set here
the TTL to 1 week.
We can also configure Tutor to use a cache eviction policy. But that
means we need to set a `maxmemory` value in Redis. It's not possible to
set a value that will be appropriate for everyone:
- if it's higher than the total memory (e.g: in small instances), server
will crash before the cache is filled.
- if it's too low (e.g: in large instances), the whole platform will abruptly
slow down as many cache entries are suddenly evicted.
That question of whether Tutor should define a cache eviction policy is
still under discussion, but it can be resolved independently of this
change.
`remove` method of Collection objects has been deprecated in favour
of either `delete_one` or `delete_many`.
This change will address 36 deprecation warnings that are generated from
test runs.
Co-authored-by: Lewis Kabui <lewisemm@users.noreply.github.com>
After we merged this PR: https://github.com/openedx/edx-platform/pull/33920
this error began popping up in logs:
Unable to load XBlock 'staffgradedxblock'
....
ImportError: cannot import name 'get_course_blocks' from
partially initialized module 'lms.djangoapps.course_blocks.api'
(most likely due to a circular import) ...
The root cause was the new imports of `derived_key` and `BlockKey` into
xmodule/library_content_block.py. Those new imports come from
xmodule/modulestore/store_utilities.py, which runs
`XBlock.load_classes()` at the module level, which fails because we are
still in the process of loading xmodule/library_content_block.
As a solution, we move both `derived_key` and `BlockKey` to
xmodule/util/keys.py. We could potentially move that file to opaque-keys
eventually, depending on how well we think that those concepts generalize.
Also:
* We rename the function from derived_key to derive_key, as
functions should be verbs.
* We combine the first to parameters of derive_key (a source ContextKey
and a source BlockKey) into a single parameter (a source UsageKey). In
my opinion, this makes the function call easier to understand.
Web scrapers do annoying stuff like visit urls they shouldn't know about and cause xblock handlers to break.
I tested this by:
Making sure video transcripts worked as normal while logged in
making sure that I got no 500s in my logs while attempting to view it logged out.
Datadog was removed 6+ years ago but some settings and related keys got
left behind. Get rid of them since we don't need them and also to
prevent people from assuming it's okay to hardcode datadog settings into
our codebase.
BREAKING CHANGE: Removes all code, tests, and settings related to
indexing of V2 (blockstore-backed) content libraries in elasticsearch.
This includes indexing of top-level library metadata as well as indexing
of library block metadata. Operators who enabled the experimental
Library Authoring MFE *and* the experimental ENABLE_CONTENT_LIBRARY_INDEX
feature may notice that sorting, filtering, and searching of V2
libraries and their blocks may now be slower and/or less powerful.
The ENABLE_CONTENT_LIBRARY_INDEX feature was already disabled by
default, so most/all operators (including edx.org) should not notice
any difference.
Removed settings include:
* FEATURES['ENABLE_CONTENT_LIBRARY_INDEX']
* ENABLE_ELASTICSEARCH_FOR_TESTS
* TEST_ELASTICSEARCH_USE_SSL
* TEST_ELASTICSEARCH_HOST
* TEST_ELASTICSEARCH_PORT
For rationale, see the updated "Status" section of:
./openedx/core/djangoapps/content_libraries/docs/decisions/0001-index-libraries-in-elasticsearch.rst
Refactors and reworks the LibraryContentBlock so that its
sync-from-library operations are asynchronous and work with
V2 content libraries. This also required us to make
library_content block duplication asynchronous, as that
involves syncing from the source library.
For the sake of clarity, this PR includes two major method renames:
* update_children(...) -> sync_from_library(...)
* refresh_library(...) -> sync_from_library(upgrade_to_latest=True, ...)
an an XBlock HTTP handler rename:
/refresh_children -> /upgrade_and_sync
There are still a couple issues with import or duplication
of library_content blocks referencing V2 libraries other than
latest. These will be resolved in an upcoming PR.
Part of: https://openedx.atlassian.net/wiki/spaces/COMM/pages/3820617729/Spec+Memo+Content+Library+Authoring+Experience+V2
Follow-up work: https://github.com/openedx/edx-platform/issues/33640
Co-authored-by: Connor Haugh <chaugh@2u.com>
Co-authored-by: Eugene Dyudyunov <evgen.dyudyunov@raccoongang.com>
Currently, ./xmodule/ unit tests are only run with LMS settings. However,
./common/ and ./xmodule/ are run twice: once with LMS settings and once with
CMS settings.
Just like ./common/ and ./openedx/, the unit tests in ./xmodule/ validate
behavior in both LMS and CMS. So, order to fully test ./xmodule/, we should to
run its tests with CMS settings too.
This will enable us to better validate certain LibraryContentBlocks behaviors
being touched by https://github.com/openedx/edx-platform/pull/33263 which can't
be expressed under LMS settings.
Also in this commit:
* refactor: rename the shards to be clear whether they're running under LMS or CMS
* docs: correct comments regarding conditions under which codejail's
test_cant_do_something_forbidden is skipped.
* test: update a unit test which was using the now-deleted library_sourced block to use
library_content block instead.
The Matlab-specific code is coupled with XQueue in places, making its
removal a little tricky. But we at least want to hide this now-useless
field from end-users before we cut the Quince release.
https://github.com/openedx/public-engineering/issues/192
Originally, we planned to add support for V2 libraries and for static
(hand-selected) library block reference via new block type: library_sourced.
We have since decided that it would be better to add those capabilities
in-place to the existing library_content block. This will ease V1->V2
library migration and make adoption of the new features easier for current
library users. It will also avoid duplication of logic between two block types,
we we fear would be error-prone. For details, see this ADR:
https://github.com/openedx/edx-platform/pull/33231
So, we are removing the library_sourced block.
This block has existed in edx-platform for a few years now, but was not
enabled by default and never officially supported. It was only usable via the
experimental V2 content library feature. Operators who added library_sourced
blocks to their course will now see them render as `HiddenBlock` instances, i.e.:
> ERROR: "library_sourced" is an unknown component type...
This should not impact other component types in such courses and should not
impact import/export.
This commit leaves behind just enough Old Mongo (DraftModulestore)
functionality to allow read-only access to static assets and the
root CourseBlock. It removes:
* create/update operations
* child/parent traversal
* inheritance related code
It also removes or converts tests for this functionality.
The ability to read from the root CourseBlock was maintained for
backwards compatibility, since top-level course settings are often
stored here, and this is used by various parts of the codebase,
like displaying dashboards and re-building CourseOverview models.
Any attempt to read the contents of a course by getting the
CourseBlock's children will return an empty list (i.e. it will look
empty).
This commit does _not_ delete content on MongoDB or run any sort of
data migration or cleanup.
