Refactors and reworks the LibraryContentBlock so that its
sync-from-library operations are asynchronous and work with
V2 content libraries. This also required us to make
library_content block duplication asynchronous, as that
involves syncing from the source library.
For the sake of clarity, this PR includes two major method renames:
* update_children(...) -> sync_from_library(...)
* refresh_library(...) -> sync_from_library(upgrade_to_latest=True, ...)
an an XBlock HTTP handler rename:
/refresh_children -> /upgrade_and_sync
There are still a couple issues with import or duplication
of library_content blocks referencing V2 libraries other than
latest. These will be resolved in an upcoming PR.
Part of: https://openedx.atlassian.net/wiki/spaces/COMM/pages/3820617729/Spec+Memo+Content+Library+Authoring+Experience+V2
Follow-up work: https://github.com/openedx/edx-platform/issues/33640
Co-authored-by: Connor Haugh <chaugh@2u.com>
Co-authored-by: Eugene Dyudyunov <evgen.dyudyunov@raccoongang.com>
Originally, we planned to add support for V2 libraries and for static
(hand-selected) library block reference via new block type: library_sourced.
We have since decided that it would be better to add those capabilities
in-place to the existing library_content block. This will ease V1->V2
library migration and make adoption of the new features easier for current
library users. It will also avoid duplication of logic between two block types,
we we fear would be error-prone. For details, see this ADR:
https://github.com/openedx/edx-platform/pull/33231
So, we are removing the library_sourced block.
This block has existed in edx-platform for a few years now, but was not
enabled by default and never officially supported. It was only usable via the
experimental V2 content library feature. Operators who added library_sourced
blocks to their course will now see them render as `HiddenBlock` instances, i.e.:
> ERROR: "library_sourced" is an unknown component type...
This should not impact other component types in such courses and should not
impact import/export.
The Webpack configuration file for built-in XBlock JS used to be
generated at build time and git-ignored. It lived at
common/static/xmodule/webpack.xmodule.config.js. It was generated
because the JS that it referred to was also generated at build-time, and
the filenames of those JS modules were not static.
Now that its contents have been made entirely static [1], there is no
reason we need to continue generating this Webpack configuration file.
So, we check it into edx-platform under the name
./webpack.builtinblocks.config.js. We choose to put it in the repo's
root directory because the paths contained in the config file are
relative to the repo's root.
This allows us to behead both the xmodule/static_content.py
(`xmodule_assets`) script andthe `process_xmodule_assets` paver task, a
major step in removing the need for Python in the edx-platform asset
build [2]. It also allows us to delete the `HTMLSnippet` class and all
associated attributes, which were exclusively used by
xmodule/static_content.py..
We leave `xmodule_assets` and `process_xmodule_assets` in as stubs for
now in order to avoid breaking external code (like Tutor) which calls
Paver; the entire pavelib/assets.py function will be eventually removed
soon anyway [3]. Further, to avoid extraneous refactoring, we keep one
method of `HTMLSnippet` around on a few of its former subclasses:
`get_html`. This method was originally part of the XModule framework;
now, it is left over on a few classes as a simple internal helper
method.
References:
1. https://github.com/openedx/edx-platform/pull/32480
2. https://github.com/openedx/edx-platform/issues/31800
3. https://github.com/openedx/edx-platform/issues/31895
Part of: https://github.com/openedx/edx-platform/issues/32481
As part of the static asset build, JS modules for most built-in XBlocks were
unnecessarily copied from the original locations (under xmodule/js and
common/static/js) to a git-ignored location (under common/static/xmodule), and
then included into the Webpack builld via
common/static/xmodule/webpack.xmodule.config.js.
With this commit, we stop copying the JS modules. Instead, we have
common/static/xmodule/webpack.xmodule.config.js just reference the original
source under xmodule/js and common/static/js.
This lets us us radically simplify the xmodule/static_content.py build script.
It also sets the stage for the next change, in which we will check
webpack.xmodule.config.js into the repository, and delete
xmodule/static_content.py entirely.
common/static/xmodule/webpack.xmodule.config.js before:
module.exports = {
"entry": {
"AboutBlockDisplay": [
"./common/static/xmodule/modules/js/000-b82f6c436159f6bc7ca2513e29e82503.js",
"./common/static/xmodule/modules/js/001-3ed86006526f75d6c844739193a84c11.js",
"./common/static/xmodule/modules/js/002-3918b2d4f383c04fed8227cc9f523d6e.js",
"./common/static/xmodule/modules/js/003-b3206f2283964743c4772b9d72c67d64.js",
"./common/static/xmodule/modules/js/004-274b8109ca3426c2a6fde9ec2c56e969.js",
"./common/static/xmodule/modules/js/005-26caba6f71877f63a7dd4f6796109bf6.js"
],
"AboutBlockEditor": [
"./common/static/xmodule/descriptors/js/000-b82f6c436159f6bc7ca2513e29e82503.js",
"./common/static/xmodule/descriptors/js/001-19c4723cecaa5a5a46b8566b3544e732.js"
],
// etc
}
};
common/static/xmodule/webpack.xmodule.config.js after:
module.exports = {
"entry": {
"AboutBlockDisplay": [
"./xmodule/js/src/xmodule.js",
"./xmodule/js/src/html/display.js",
"./xmodule/js/src/javascript_loader.js",
"./xmodule/js/src/collapsible.js",
"./xmodule/js/src/html/imageModal.js",
"./xmodule/js/common_static/js/vendor/draggabilly.js"
],
"AboutBlockEditor": [
"./xmodule/js/src/xmodule.js",
"./xmodule/js/src/html/edit.js"
],
// etc
}
};
Part of: https://github.com/openedx/edx-platform/issues/32481
`module-js` and `module-descriptor-js` are old JavaScript group
indicators, left over from when we managed XModule assets via Django
Pipeline. We would like to get rid of them in order to make it easier to
build XModule JS without using Python.
There is one single usage of `module-js` in the entire platform (the
rest have been replaced with Webpack references, which is the
less-outdated way of managing XModule assets :). The lone `module-js`
reference was added in 2013 [1] so that circuit diagrams would display
in the course wiki. However, the ability to render circuits in the wiki
was removed in 2015 [2], so it is safe to remove the reference.
There is also one single usage of `module-descriptor-js`. It's in the
legacy bulk email editor, which hackily cribs from the old HtmlBlock
editor. Fortunately, we are able to simply replace the Django Pipeline
reference with the equivalent XModule JS Webpack bundle. (Note: The old
email editor is currently still supported, but is currently being
replaced by frontend-app-communications, so this hack will be gone
eventually).
Finally, this commit also sneaks in one styling fix: it adds the
HtmlBlockEditor CSS back to the aforementioned legacy bulk email page.
The missing CSS was causing a read-only 1-line codemirror editor to
appear below the HTML editor [3]. This bug was introduced during the
original XModule SCSS decoupling [4], which removed builtin block CSS
from the LMS-wide bundle, thus removing the HTML editor CSS from the
bulk email page. We imagine that nobody noticed because the bug only
exists in master (not Palm) and frontend-app-communications seems to be
globally enabled on edx.org. As a simple fix, we add the new CSS link to
the legacy bulk email page, and it renders fine again [5].
References:
1. 3fc59b3da5
2. https://github.com/openedx/edx-platform/pull/10324
3. Before fix: https://github.com/openedx/edx-platform/assets/3628148/25fc41b2-403d-4339-8c49-0b04664dfa02
4. https://github.com/openedx/edx-platform/pull/32018
5. After fix: https://github.com/openedx/edx-platform/assets/3628148/9a5d74f1-cc83-4ebe-8f0c-ee270f7721b8
Part of: https://github.com/openedx/edx-platform/issues/32481
In ~Palm and earlier, all built-in XBlock Sass was included into CMS
(and LMS) styles before being compiled. So, if a site theme was meant to
affect built-in XBlock styling, those changes would be manifested
directly in the base CMS CSS that is included into every single Studio
page. When the user provided the `?site_theme` querystring parameter,
which is intended to allow devs & admins to view Studio through a given
theme, CMS would look up the given theme and serve the corresponding
base CMS CSS, which would affect the built-in XBlocks views (as
expected).
After ~Palm, built-in XBlocks styles are handled more similarly to to
pure XBlock styles, in that they are only requested when CMS tries to
render the block. In Studio, blocks are not rendered by the original
request, but by a subsequent AJAX request to the `/container_preview`
enpoint. Thus, passing the `?site_theme` query parameter to the original
request will apply the given theme to Studio's chrome, but the theme
will _not_ apply to built-in XBlock views, whose CSS is now loaded via
async request.
To fix this, we simply pass Studio's querystring parameters (including
`?site_theme`) along to the `/container_view` AJAX request. This will
cause CMS to correctly serve the built-in XBlock CSS from the theme
specified by `?site_theme`, rather than whatever the current theme is.
Part of: https://github.com/openedx/edx-platform/issues/32292
In ~Palm and earlier, all built-in XBlock Sass was included into LMS and CMS
styles before being compiled. The generated CSS was coupled together with
broader LMS/CMS CSS. This means that comprehensive themes have been able to
modify built-in XBlock appearance by setting certain Sass variables. We say that
built-in XBlock Sass was, and is expected to be, "theme-aware".
Shortly after Palm, we decoupled XBlock Sass from LMS and CMS Sass [1]. Each
built-in block's Sass is now compiled into two separate CSS targets, one for
block editing and one for block display. The CSS, now located at
`common/static/css/xmodule`, is injected into the running Webpack context with
the new `XModuleWebpackLoader`. Built-in XBlocks already used
`add_webpack_to_fragment` in order to add JS Webpack bundles to their view
fragments, so when CSS was added to Webpack, it Just Worked.
This unlocked a slieu of simplifications for static asset processing [2];
however, it accidentally made XBlock Sass theme-*unaware*, or perhaps
theme-confused, since the CSS was targeted at `common/static/css/xmodule`
regardless of the theme. The result of this is that **built-in XBlock views will
use CSS based on the Sass variables _last theme to be compiled._** Sass
variables are only used in a handful of places in XBlocks, so the bug is subtle,
but it is there for those running off of master. For example, using edX.org's
theme on master, we can see that there is a default blue underline in the Studio
sequence nav [3]. With this bugfix, it becomes the standard edX.org
greenish-black [4].
This commit makes several changes, firstly to fix the bug, and secondly to leave
ourselves with a more comprehensible asset setup in the `xmodule/` directory.
* We remove the `XModuleWebpackLoader`, thus taking built-in XBlock Sass back
out of Webpack.
* We compile XBlock Sass not to `common/static/css/xmodule`, but to:
* `[lms|cms]/static/css` for the default theme, and
* `<THEME_ROOT>/[lms|cms]/static/css`, for any custom theme.
This is where the comprehensive theming system expects to find themable
assets. Unfortunately, this does mean that the Sass is compiled twice, both
for LMS and CMS. We would have liked to compile it once to somewhere in the
`common/`, but comprehensive theming does not consider `common/` assets to be
themable.
* We split `add_webpack_to_fragment` into two more specialized functions:
* `add_webpack_js_to_fragment` , for adding *just* JS from a Webpack bundle,
and
* `add_sass_to_fragment`, for adding static links to CSS compiled themable
Sass (not Webpack). Both these functions are moved to a new module
`xmodule/util/builtin_assets.py`, since the original module
(`xmodule/util/xmodule_django.py`) didn't make a ton of sense.
* In an orthogonal bugfix, we merge Sass `CourseInfoBlock`, `StaticTabBlock`,
`AboutBlock` into the `HtmlBlock` Sass files. The first three were never used,
as their styling was handled by `HtmlBlock` (their shared parent class).
* As a refactoring, we change Webpack bundle names and Sass module names to be
less misleading:
* student_view, public_view, and author_view: was `<Name>BlockPreview`, is now
`<Name>BlockDisplay`.
* studio_view: was `<Name>BlockStudio`, is now `<Name>BlockEditor`.
* As a refactoring, we move the contents of `xmodule/static` into the existing
`xmodule/assets` directory, and adopt its simper structure. We now have:
* `xmodule/assets/*.scss`: Top-level compiled Sass modules. These could be
collapsed away in a future refactoring.
* `xmodule/assets/<blocktype>/*`: Resources for each block, including both JS
modules and Sass includes (underscore-prefixed so that they aren't
compiled). This structure maps closely with what externally-defined XBlocks
do.
* `xmodule/js` still exists, but it will soon be folded into the
`xmodule/assets`.
* We add a new README [4] to explain the new structure, and also update a
docstring in `openedx/lib/xblock/utils` which had fallen out of date with
reality.
* Side note: We avoid the term "XModule" in all of this, because that's
(thankfully) become a much less useful/accurate way to describe these blocks.
Instead, we say "built-in XBlocks".
Refs:
1. https://github.com/openedx/edx-platform/pull/32018
2. https://github.com/openedx/edx-platform/issues/32292
3. https://github.com/openedx/edx-platform/assets/3628148/8b44545d-0f71-4357-9385-69d6e1cca86f
4. https://github.com/openedx/edx-platform/assets/3628148/d0b7b309-b8a4-4697-920a-8a520e903e06
5. https://github.com/openedx/edx-platform/tree/master/xmodule/assets#readme
Part of: https://github.com/openedx/edx-platform/issues/32292
YT: https://youtrack.raccoongang.com/issue/EDX_BLND_CLI-87
- V2 libraries are available for selection in the Random Block edit modal;
- selected V2 library blocks are copied to the modulestore and saved as children of the Random Block;
- V2 library version validation works the same as for the V1 libraries (with possibility to update block with the latest version);
- filtering by problem type can't be done for V2 the same as for V1 because the v2 library problems are not divided by types;
- the problem type field is hidden for v2 libraries in the edit mode;
- unit tests added/updated.
- Moving xmodule folder to root as we're dissolving sub-projects of common folder in edx-platform
- More info: https://openedx.atlassian.net/browse/BOM-2579
- -e common/lib/xmodule has been removed from the requirements as xmodule has itself become the part of edx-platform and not being installed through requirements
- The test files common/lib/xmodule/test_files/ have been removed as they are not being used anymore
