In case of unusual cookie headers containing "Cookie ",
add custom attributes for monitoring:
- cookies.header.corrupt_count
- cookies.header.corrupt_key_count
See annotation documentation for more details.
Separately, updated to skip cookie log sampling for
0 size cookie header.
ARCHBOM-2055
Previously, course tabs would only be created once and never try to
update the default tabs again. This leads to an issue if you ever want
to add a new tab. With this command, you can now update the default tabs
for all existing courses and new courses will pick it up upon creation
when CourseTabList.initialize_default is called.
* feat: [AA-1207] remove redundant Tabs fields from courseware API
All the tab information now goes through the course home metadata tab fields. This field is redundant.
Contains a number of cookie monitoring changes.
Enhancements:
- Add sampling capability for cookie logging on headers
smaller than the threshold. For details, see
COOKIE_SAMPLING_REQUEST_COUNT.
- Add cookie header size to log message.
- Sort logged cookies starting with largest cookie.
- Move logging from Middleware request processing
to response processing to ensure the user id is
available for logging for authenticated calls.
- Added cookies.header.size.computed to check
if there are any large hidden duplicate cookies.
Can be compared against the cookies.header.size
custom attribute.
- Add delimiters into logs to make it simpler to parse
when the logging tools accidentally exports multiple
log lines together.
Removed:
- Legacy cookie capture code. This code was dangerous to
to enable and provided more limited insight than the
newer logging, so this was removed to simplify the code.
Other refactors:
- Switched Middleware to use new Django format, rather
than the Mixin.
- Moved tests to its own test class. Note: this
middleware is likely to move to a separate
library.
ARCHBOM-2055
Description
This is a follow up to #29058 and #29413. This is the next step in moving part of the modulestore data (the course indexes / "active versions" table) from MongoDB to MySQL.
There are four steps planned in moving course index data to MySQL:
Step 1: create the tables in MySQL, start writing to MySQL + MongoDB ✅ done
Step 2: migrate all remaining courses to MySQL ✅ done
Step 3: switch reads from MongoDB to MySQL (this PR)
Step 4 (much later, once we know this is working well): stop writing to MongoDB altogether.
Supporting information
OpenCraft Jira ticket: MNG-2557
Status
✅ Tested with a large Open edX instance is in progress.
Testing instructions
Try making changes in Studio and verify that they work fine.
Deadline
None
* refactor: modified the imports to use openedx-events library
* chore: upgrade openedx-events to latest version
* fix: ran isort to avoid style errors
Co-authored-by: Maria Grimaldi <maria.grimaldi@edunext.co>
Discussion topic links don't contain enough useful information if the unit they
are linked to is deleted. This ADR proposes storing more information for each
discussion topic to overcome this limitation.
* feat: [AA-1206] remove redundant fields from API
Part of a larger effort to clean up the MFE BFF endpoints.
Remove the redundant fields username and course_access, both of which are also available in the course home metadata call.
This should really be all we need for most cases, and we don't want to
emit sensitive data more than necessary, even encrypted. If we need to
inspect one cookie in particular, we can add special logging for that.
Also, change to greater-than-or-equal for threshold to match setting docs.
ref: ARCHBOM-2042
Convert more tests from MONGO_AMNESTY to SPLIT modulestores.
This is in preparation for just wholesale denying access to Old
Mongo, so I either converted tests to split or just deleted some
test variants that were Old Mongo specific. (e.g. ddt lines)
It's likely that someone will at some point enable encrypted logging but
forget to deploy the config change that sets the key; if this happens, we
should gracefully return a warning rather than raise an exception.
Along the same lines, make sure that safe-sessions won't raise an exception
if the setting is missing, and document the suggested use of getattr.
It's long past time that the default test modulestore was Split,
instead of Old Mongo. This commit switches the default store and
fixes some tests that now fail:
- Tests that didn't expect MFE to be enabled (because we don't
enable MFE for Old Mongo) - opt out of MFE for those
- Tests that hardcoded old key string formats
- Lots of other random little differences
In many places, I didn't spend much time trying to figure out how to
properly fix the test, and instead just set the modulestore to Old
Mongo.
For those tests that I didn't spend time investigating, I've set
the modulestore to TEST_DATA_MONGO_AMNESTY_MODULESTORE - search for
that string to find further work.
