BREAKING CHANGE: Removes all code, tests, and settings related to
indexing of V2 (blockstore-backed) content libraries in elasticsearch.
This includes indexing of top-level library metadata as well as indexing
of library block metadata. Operators who enabled the experimental
Library Authoring MFE *and* the experimental ENABLE_CONTENT_LIBRARY_INDEX
feature may notice that sorting, filtering, and searching of V2
libraries and their blocks may now be slower and/or less powerful.
The ENABLE_CONTENT_LIBRARY_INDEX feature was already disabled by
default, so most/all operators (including edx.org) should not notice
any difference.
Removed settings include:
* FEATURES['ENABLE_CONTENT_LIBRARY_INDEX']
* ENABLE_ELASTICSEARCH_FOR_TESTS
* TEST_ELASTICSEARCH_USE_SSL
* TEST_ELASTICSEARCH_HOST
* TEST_ELASTICSEARCH_PORT
For rationale, see the updated "Status" section of:
./openedx/core/djangoapps/content_libraries/docs/decisions/0001-index-libraries-in-elasticsearch.rst
Originally, Blockstore was an independent micro-service, accessed via a REST API.
Then, we changed Blockstore so it could be installed as an in-process Django app.
To support both modes, there existed a blockstore_api wrapper library in edx-platform,
with toggles controlling whether the wrapper called out to the micro-service's REST API versus the
Django app's Python API. Now that the micro-service Blockstore implementation is deprecated,
though, this wrapper library and toggles are just unnecessary complexity.
As a first step towards cleanup, we:
* remove several toggles and settings (details below);
* remove the blocokstore_api wrapper methods which called the REST API and
marshalled them back into Python objects; and
* remove all test cases which relied on the Blockstore micro-service (and were skippped in CI).
In the future, we will remove the content libraries indexer,
clean up the remaining bits of blockstore_api, and flatten out all
the Blockstore-related test class hierarchies which are no longer nceessary.
BREAKING CHANGE:
* These Django settings are removed:
* BLOCKSTORE_PUBLIC_URL_ROOT
* BLOCKSTORE_API_URL
* BLOCKSTORE_API_AUTH_TOKEN
* BLOCKSTORE_USE_BLOCKSTORE_APP_API
* The blockstore.use_blockstore_app_api Waffle switch is removed.
* edx-platform will act as it did when the DJango setting BLOCKSTORE_USE_BLOCKSTORE_APP_API
or the Waffle switch blockstore.use_blockstore_app_api were enabled. That is, any running Blockstore
micro-service instance will be ignored, and the Blockstore package which is installed into edx-platform
will be used instead.
Ref: https://github.com/openedx/blockstore/issues/296
Refactors and reworks the LibraryContentBlock so that its
sync-from-library operations are asynchronous and work with
V2 content libraries. This also required us to make
library_content block duplication asynchronous, as that
involves syncing from the source library.
For the sake of clarity, this PR includes two major method renames:
* update_children(...) -> sync_from_library(...)
* refresh_library(...) -> sync_from_library(upgrade_to_latest=True, ...)
an an XBlock HTTP handler rename:
/refresh_children -> /upgrade_and_sync
There are still a couple issues with import or duplication
of library_content blocks referencing V2 libraries other than
latest. These will be resolved in an upcoming PR.
Part of: https://openedx.atlassian.net/wiki/spaces/COMM/pages/3820617729/Spec+Memo+Content+Library+Authoring+Experience+V2
Follow-up work: https://github.com/openedx/edx-platform/issues/33640
Co-authored-by: Connor Haugh <chaugh@2u.com>
Co-authored-by: Eugene Dyudyunov <evgen.dyudyunov@raccoongang.com>
* feat: TNL-11173 authoring API offered as v0, not v1
* docs: correct swaggerfile for authoring api
---------
Co-authored-by: Bernard Szabo <bszabo@edx.org>
Co-authored-by: Jesper Hodge <jhodge@outlook.de>
* feat!: `sha1` has been deprecated in django32 and removed in django42.
* test: fix quality failure
* fixup! update custom attribute tests (#33436)
I was wondering about all the cases, so I
updated the test to reflect this. I also
made some other minor adjustments.
---------
Co-authored-by: Muhammad Soban Javed <iamsobanjaved@gmai.com>
Co-authored-by: Robert Raposa <rraposa@edx.org>
Co-authored-by: Muhammad Soban Javed <58461728+iamsobanjaved@users.noreply.github.com>
This adds two new urls: `<studio-base>/cms-api/ui` and `<studio-base>/cms-api/schema` with swagger ui and swaggerfile only for the new CMS API using drf-spectacular
The S3BotoStorage backend was deprecated in favor of the S3Boto3Storage
backend. This change updates the default backend to use the latter.
style: This re-links and rewords a relevant comment that had been
disconnected during previous refactoring.
docs: ade79308a3/docs/backends/amazon-S3.rst (migrating-from-boto-to-boto3)
Fixes: FAL-3431
Fixes: public-engineering/128
* refactor: moves is_content_creator
from cms.djangoapps.contentstore.helpers to common.djangoapps.student.auth
* feat: adds content tagging app
Adds models and APIs to support tagging content objects (e.g. XBlocks,
content libraries) by content authors. Content tags can be thought of as
"name:value" fields, though underneath they are a bit more complicated.
* adds dependency on openedx-learning<=0.1.0
* adds tagging app to LMS and CMS
* adds content tagging models, api, rules, admin, and tests.
* content taxonomies and tags can be maintained per organization by
content creators for that organization.
`module-js` and `module-descriptor-js` are old JavaScript group
indicators, left over from when we managed XModule assets via Django
Pipeline. We would like to get rid of them in order to make it easier to
build XModule JS without using Python.
There is one single usage of `module-js` in the entire platform (the
rest have been replaced with Webpack references, which is the
less-outdated way of managing XModule assets :). The lone `module-js`
reference was added in 2013 [1] so that circuit diagrams would display
in the course wiki. However, the ability to render circuits in the wiki
was removed in 2015 [2], so it is safe to remove the reference.
There is also one single usage of `module-descriptor-js`. It's in the
legacy bulk email editor, which hackily cribs from the old HtmlBlock
editor. Fortunately, we are able to simply replace the Django Pipeline
reference with the equivalent XModule JS Webpack bundle. (Note: The old
email editor is currently still supported, but is currently being
replaced by frontend-app-communications, so this hack will be gone
eventually).
Finally, this commit also sneaks in one styling fix: it adds the
HtmlBlockEditor CSS back to the aforementioned legacy bulk email page.
The missing CSS was causing a read-only 1-line codemirror editor to
appear below the HTML editor [3]. This bug was introduced during the
original XModule SCSS decoupling [4], which removed builtin block CSS
from the LMS-wide bundle, thus removing the HTML editor CSS from the
bulk email page. We imagine that nobody noticed because the bug only
exists in master (not Palm) and frontend-app-communications seems to be
globally enabled on edx.org. As a simple fix, we add the new CSS link to
the legacy bulk email page, and it renders fine again [5].
References:
1. 3fc59b3da5
2. https://github.com/openedx/edx-platform/pull/10324
3. Before fix: https://github.com/openedx/edx-platform/assets/3628148/25fc41b2-403d-4339-8c49-0b04664dfa02
4. https://github.com/openedx/edx-platform/pull/32018
5. After fix: https://github.com/openedx/edx-platform/assets/3628148/9a5d74f1-cc83-4ebe-8f0c-ee270f7721b8
Part of: https://github.com/openedx/edx-platform/issues/32481
In ~Palm and earlier, all built-in XBlock Sass was included into LMS and CMS
styles before being compiled. The generated CSS was coupled together with
broader LMS/CMS CSS. This means that comprehensive themes have been able to
modify built-in XBlock appearance by setting certain Sass variables. We say that
built-in XBlock Sass was, and is expected to be, "theme-aware".
Shortly after Palm, we decoupled XBlock Sass from LMS and CMS Sass [1]. Each
built-in block's Sass is now compiled into two separate CSS targets, one for
block editing and one for block display. The CSS, now located at
`common/static/css/xmodule`, is injected into the running Webpack context with
the new `XModuleWebpackLoader`. Built-in XBlocks already used
`add_webpack_to_fragment` in order to add JS Webpack bundles to their view
fragments, so when CSS was added to Webpack, it Just Worked.
This unlocked a slieu of simplifications for static asset processing [2];
however, it accidentally made XBlock Sass theme-*unaware*, or perhaps
theme-confused, since the CSS was targeted at `common/static/css/xmodule`
regardless of the theme. The result of this is that **built-in XBlock views will
use CSS based on the Sass variables _last theme to be compiled._** Sass
variables are only used in a handful of places in XBlocks, so the bug is subtle,
but it is there for those running off of master. For example, using edX.org's
theme on master, we can see that there is a default blue underline in the Studio
sequence nav [3]. With this bugfix, it becomes the standard edX.org
greenish-black [4].
This commit makes several changes, firstly to fix the bug, and secondly to leave
ourselves with a more comprehensible asset setup in the `xmodule/` directory.
* We remove the `XModuleWebpackLoader`, thus taking built-in XBlock Sass back
out of Webpack.
* We compile XBlock Sass not to `common/static/css/xmodule`, but to:
* `[lms|cms]/static/css` for the default theme, and
* `<THEME_ROOT>/[lms|cms]/static/css`, for any custom theme.
This is where the comprehensive theming system expects to find themable
assets. Unfortunately, this does mean that the Sass is compiled twice, both
for LMS and CMS. We would have liked to compile it once to somewhere in the
`common/`, but comprehensive theming does not consider `common/` assets to be
themable.
* We split `add_webpack_to_fragment` into two more specialized functions:
* `add_webpack_js_to_fragment` , for adding *just* JS from a Webpack bundle,
and
* `add_sass_to_fragment`, for adding static links to CSS compiled themable
Sass (not Webpack). Both these functions are moved to a new module
`xmodule/util/builtin_assets.py`, since the original module
(`xmodule/util/xmodule_django.py`) didn't make a ton of sense.
* In an orthogonal bugfix, we merge Sass `CourseInfoBlock`, `StaticTabBlock`,
`AboutBlock` into the `HtmlBlock` Sass files. The first three were never used,
as their styling was handled by `HtmlBlock` (their shared parent class).
* As a refactoring, we change Webpack bundle names and Sass module names to be
less misleading:
* student_view, public_view, and author_view: was `<Name>BlockPreview`, is now
`<Name>BlockDisplay`.
* studio_view: was `<Name>BlockStudio`, is now `<Name>BlockEditor`.
* As a refactoring, we move the contents of `xmodule/static` into the existing
`xmodule/assets` directory, and adopt its simper structure. We now have:
* `xmodule/assets/*.scss`: Top-level compiled Sass modules. These could be
collapsed away in a future refactoring.
* `xmodule/assets/<blocktype>/*`: Resources for each block, including both JS
modules and Sass includes (underscore-prefixed so that they aren't
compiled). This structure maps closely with what externally-defined XBlocks
do.
* `xmodule/js` still exists, but it will soon be folded into the
`xmodule/assets`.
* We add a new README [4] to explain the new structure, and also update a
docstring in `openedx/lib/xblock/utils` which had fallen out of date with
reality.
* Side note: We avoid the term "XModule" in all of this, because that's
(thankfully) become a much less useful/accurate way to describe these blocks.
Instead, we say "built-in XBlocks".
Refs:
1. https://github.com/openedx/edx-platform/pull/32018
2. https://github.com/openedx/edx-platform/issues/32292
3. https://github.com/openedx/edx-platform/assets/3628148/8b44545d-0f71-4357-9385-69d6e1cca86f
4. https://github.com/openedx/edx-platform/assets/3628148/d0b7b309-b8a4-4697-920a-8a520e903e06
5. https://github.com/openedx/edx-platform/tree/master/xmodule/assets#readme
Part of: https://github.com/openedx/edx-platform/issues/32292
