rewriting them.
In PERF-341, we adjusted the static_replace middleware to try and
exclude static XBlock resource URLs from being interpreted as the marker
URLs used to signify course assets in course content. Since they both
started with /static, this could, and did, cause issues where linking
directly to the assets of an XBlock within, say, one of its templates,
would lead to that link being rewritten and ultimately being incorrect.
The fix attempted to see if the link started with the prefix that all
static XBlock resource URLs start, and if so, it returned them
unmodified.
We incorrectly assumed that our testing captured all cases, and since
we're here, we know that this was wrong. We weren't accounting for cases
when the URLs being generated had the STATIC_URL configuration value
prefixed -- https://example.com/static/xblock/.... -- and so our direct
check of seeing if such a URL started with "/static/xblock" would always
fail, leading to the erroneous rewriting and nonsensical output.
This fix checks if the link either starts with the prefix OR if it
starts with the STATIC_URL value and contains the prefix overall. There
is a small overlap between the STATIC_URL and the prefix we check for,
so an inconsistency could arise down the line if we changed our
STATIC_URL to use a difference base directory, but our tests will at
least catch the issue now.
This version component reflects the "version" of the StaticContent
objects which we cache server-side. If the layout of those objects
changes between releases, errors occur when loading them from cache.
By using a separate version value, which can be incremented on its own
after a change has been made to the StaticContent class, we can avoid
loading older cached content and in turn take advantage of these changes
faster, without needing to intervene operationally.
When releasing the versioned assets work, we stumbled on a problem with old pickled
versions of the StaticContent objects residing in cache, which triggered a bug in the
code. Not wanting to blow away all cached items, we ended up having to revert and add
in some backwards-compatible helper code to ease the transition.
With this, we'll now utilize the version argument that Django's caching interface
allows, in conjunction with a constant value that can be modified when breaking changes
are being made, to let us gracefully ignore older cached course assets.
2. Update COMPREHNSIVE_THEME_DIR to COMPREHENSIVE_THEME_DIRS
3. Update paver commands to support multi theme dirs
4. Updating template loaders
5. Add ENABLE_COMPREHENSIVE_THEMING flag to enable or disable theming via settings
6. Update tests
7. Add backward compatibility for COMPREHEHNSIVE_THEME_DIR
For XBlocks that used their static.public resources in the rendered
output -- for example, a link to a bundled image -- those URLs would be
treated as course assets using the '/static/' prefix trick, and thus,
rewritten. These rewritten URLs don't work because they aren't course
assets.
When course content authors are creating their courses, we provide them
a special shorthand way of writing URLs that reference assets they have
uploaded to the course. If they uploaded a file called my-lil-pony.mp4,
and they wanted to provide a link to it when users view the course, they
would use /static/my-lil-pony.mp4. This special prefix -- /static/ --
signals to the static_replace middleware that it's a course asset and we
should write /static/my-lil-pony.mp4 to a URL that properly references
the asset based on the course ID, and things like the configured asset
CDN, etc.
Thus, the URL /static/my-lil-pony.mp4 gets turned into something like:
/assets/courseware/<md5hash>/asset-v1:edX+Demo+2016T1+type@asset+block/my-lil-pony.mp4
when viewed in the courseware.
Now, we also serve actual static assets from a prefix of /static/. This
is stuff like our JavaScript and CSS, and the JS/CSS/etc of
XBlocks/XModules. These paths look like:
/static/js/lms-main_vendor.46d6a8c02600.js
or
/static/xblock/resources/xmodule.vertical_block/public/js/vertical_student_view.43727a907769.js
Normally, these paths are caught by nginx, before they reach the LMS,
and are served straight from the filesystem. However, if you were to
have one of these paths in your course content, the static_replace
middleware would see the /static/ at the front and immediately think
it's a course asset, and would dutifully rewrite the URL to something
like:
/assets/courseware/<md5hash>/asset-v1:edX+Demo+2016T1+type@asset+block/static_xblock_resources_xmodule.vertical_block_public_js_vertical_student_view.43727a907769.js
which is not a course asset, and so it will always fail to load.
Long story short, I changed the static_replace middleware to
specifically check to see if the path being matched starts with
/static/xblock/, and if so, it keeps the original instead of rewriting
it.
* mattdrayer: Add helpers.get_value test
* mattdrayer: Change to simpler implementation, per @douglashall
* mattdrayer: Address quality violations and test failures
We're seeing errors in NR from objects read out of the cache lacking the
'StaticContent' object has no attribute 'content_digest'
File "/edx/app/edxapp/edx-platform/common/djangoapps/contentserver/middleware.py",
line 70, in process_request
This reverts commit 849ebc5f22.
* mattdrayer: Add helpers.get_value test
* mattdrayer: Change to simpler implementation, per @douglashall
* mattdrayer: Address quality violations and test failures
When program detail pages are enabled, cards on the listing page will link to their respective detail pages. Includes extensive cleanup of program listing tests. ECOM-4227.
