XFF is just part of the chain; record the length of the whole chain instead
(which is always one larger).
Also include junk in one of the test values for realism.
Processing cookies at response time included cookies
that were temporary, like the JWT cookie that is
created by the server by combining the JWT header-payload
and JWT signature cookies. Since we are trying to monitor
the cookie header, we do not want to process this cookie.
However, since we want to include the user id in the logging
message, we delay the logging until response time.
Also, fixed docstring which mislabeled a custom attribute.
ARCHBOM-2055
Deprecates the following attributes from ModuleSystem:
* replace_urls
* replace_course_urls
* replace_jump_to_id_urls
A new ReplaceURLService is created as replacement with a unified replace_urls method
In case of unusual cookie headers containing "Cookie ",
add custom attributes for monitoring:
- cookies.header.corrupt_count
- cookies.header.corrupt_key_count
See annotation documentation for more details.
Separately, updated to skip cookie log sampling for
0 size cookie header.
ARCHBOM-2055
Contains a number of cookie monitoring changes.
Enhancements:
- Add sampling capability for cookie logging on headers
smaller than the threshold. For details, see
COOKIE_SAMPLING_REQUEST_COUNT.
- Add cookie header size to log message.
- Sort logged cookies starting with largest cookie.
- Move logging from Middleware request processing
to response processing to ensure the user id is
available for logging for authenticated calls.
- Added cookies.header.size.computed to check
if there are any large hidden duplicate cookies.
Can be compared against the cookies.header.size
custom attribute.
- Add delimiters into logs to make it simpler to parse
when the logging tools accidentally exports multiple
log lines together.
Removed:
- Legacy cookie capture code. This code was dangerous to
to enable and provided more limited insight than the
newer logging, so this was removed to simplify the code.
Other refactors:
- Switched Middleware to use new Django format, rather
than the Mixin.
- Moved tests to its own test class. Note: this
middleware is likely to move to a separate
library.
ARCHBOM-2055
This should really be all we need for most cases, and we don't want to
emit sensitive data more than necessary, even encrypted. If we need to
inspect one cookie in particular, we can add special logging for that.
Also, change to greater-than-or-equal for threshold to match setting docs.
ref: ARCHBOM-2042
It's long past time that the default test modulestore was Split,
instead of Old Mongo. This commit switches the default store and
fixes some tests that now fail:
- Tests that didn't expect MFE to be enabled (because we don't
enable MFE for Old Mongo) - opt out of MFE for those
- Tests that hardcoded old key string formats
- Lots of other random little differences
In many places, I didn't spend much time trying to figure out how to
properly fix the test, and instead just set the modulestore to Old
Mongo.
For those tests that I didn't spend time investigating, I've set
the modulestore to TEST_DATA_MONGO_AMNESTY_MODULESTORE - search for
that string to find further work.
* Deprecates ModuleSystem can_execute_unsafe_code, get_python_lib_zip and cache properties
* Adds a new CacheService and SandboxService to provide the deprecated property
* Adds tests for the added CacheService and SandboxService
* Updates the ModuleSystemShim tests in Lms and Studio
Suppress them both in tests (via setup.py and pytest.ini)
and in management command & application runs
(via logsettings.py).
Developers aren't looking at these warnings; they'll be dealt with in a
formal process for upgrading Django. Suppress them for now so that
important information isn't lost in the noise.
* refactor: deprecates ModuleSystem.render_template
in favor of the added MakoSystem render_template method.
Related changes:
* Adds the MakoService to the StudioEditModuleRuntime,
PreviewModuleSystem, LmsModuleSystem, and XBlockRuntime
* MakoService constructor takes a `namespace_prefix` string, so that the
CMS PreviewModuleSystem can render to LMS templates, without needing
the special render_from_lms helper method.
* ModuleSystem.render_template becomes a read-only property, so the
constructor calls and test module systems are updated accordingly.
* Adds tests for the MakoService and module system shims.
(cherry picked from commit 457f959356)
* refactor: use MakoService.render_template to remove deprecation warnings
from block code.
(cherry picked from commit 8d62d337f5)
* refactor: use MakoService.render_template to remove deprecation warnings
from test code.
(cherry picked from commit 26b43465a4)
* test: Adds a test to verify the bug introduced by the previous changes
The AuthoringMixin is automatically added to all XBlocks (see
settings.XBLOCK_MIXINS), and AuthoringMixin.visibility_view expects the
"mako" service.
This test verifies the bug by testing the PureXBlock, which does not
require the "mako" service, and so fails when the visibility_view is
rendered.
* fix: AuthoringMixin needs mako service
which fixes the visibility_view for XBlocks which don't explicitly
require the mako service.
Also removes the unneeded class property _services_requested from
AuthoringMixin and StudioEditableBlock. This property is better provided
by the XBlockMixin class.
* feat: Add support for using the discussions MFE UI instead of existing UI
Adds a new course waffle flag that when set along with the discussions MFE URL shows the discussions MFE UI instead of the regular UI.
* test: add tests
* squash!: more consistent url name
* chore: update deprecated import from collections
* chore: remove outdated imports from markdown library
as it hasn't been supported since 2.0.3 and we're on 3.x.
This was deprecated at least as early as 2012!
* docs: add docstring and remove lint-amnesty to markdown plugin
* chore: remove deprecated etree import
* style: remove unnecessary-comprehension for sets
* style: resolve a number of amnestied pylint complaints
Co-authored-by: stvn <stvn@mit.edu>
* feat: Add a new way to enable/disable teams
Adds a new mechanism for enabling/disabling the team feature in a course using an 'enabled' field to the teams config.
If this field is set to true, teams is enabled (team sets/groups) still need to be defined. If this is set to false then teams is disabled whether or not team sets are defined.
* fix: review feedback
This:
1. Introduces a variable for the Course Outline view in Studio.
A custom theme can override it to add new editors.
2. Exports a function for creating new editor modals.
A custom theme can use it to create editors without adding boilerplate code.
3. Adds a pluggable override for XBlock fields that are passed to the Studio.
Without this, custom editors in Studio cannot retrieve values of XBlock fields.
The learning MFE paths include /course/{course_id} which doesn't match /courses/{course_id} which is what the regex expects. This causes issues with the Wiki when when accessed from the learning MFE doesn't detect that course it's related to in the middleware.
rest-condition has not been updated for a while, and with DRF 3.9 it is no longer needed since the ability to combine Permission classes using boolean operators is now provided by DRF directly.
* fix: adding more parameters for cookie monitoring
Added: cookies_total_num, cookies_unaccounted_size.
-Both are to help us gauge how many cookies we are not collecting data for.
Increased: # of cookies data collected
* feat: Course Apps API
This adds a new concept called course apps. These are exposed via a new
"openedx.course_app" entrypoint, which helps the LMS and studio discover such
apps and list them in a new rest api for the same.
These course apps will drive the pages and resources view in the course authoring
MFE. This system will track which apps are enabled and which are disabled. It
also allows third-party apps to be listed here by using the plugin entrypoint.
* Apply feedback from review
