JIRA: https://openedx.atlassian.net/browse/BOM-2580
This PR aims at refactoring the discussion xblock sub project and moving it within the xmodule directory effectively removing its position as a sub project within edx-platform
Deprecates the following attributes from ModuleSystem:
* replace_urls
* replace_course_urls
* replace_jump_to_id_urls
A new ReplaceURLService is created as replacement with a unified replace_urls method
In case of unusual cookie headers containing "Cookie ",
add custom attributes for monitoring:
- cookies.header.corrupt_count
- cookies.header.corrupt_key_count
See annotation documentation for more details.
Separately, updated to skip cookie log sampling for
0 size cookie header.
ARCHBOM-2055
Contains a number of cookie monitoring changes.
Enhancements:
- Add sampling capability for cookie logging on headers
smaller than the threshold. For details, see
COOKIE_SAMPLING_REQUEST_COUNT.
- Add cookie header size to log message.
- Sort logged cookies starting with largest cookie.
- Move logging from Middleware request processing
to response processing to ensure the user id is
available for logging for authenticated calls.
- Added cookies.header.size.computed to check
if there are any large hidden duplicate cookies.
Can be compared against the cookies.header.size
custom attribute.
- Add delimiters into logs to make it simpler to parse
when the logging tools accidentally exports multiple
log lines together.
Removed:
- Legacy cookie capture code. This code was dangerous to
to enable and provided more limited insight than the
newer logging, so this was removed to simplify the code.
Other refactors:
- Switched Middleware to use new Django format, rather
than the Mixin.
- Moved tests to its own test class. Note: this
middleware is likely to move to a separate
library.
ARCHBOM-2055
This should really be all we need for most cases, and we don't want to
emit sensitive data more than necessary, even encrypted. If we need to
inspect one cookie in particular, we can add special logging for that.
Also, change to greater-than-or-equal for threshold to match setting docs.
ref: ARCHBOM-2042
It's long past time that the default test modulestore was Split,
instead of Old Mongo. This commit switches the default store and
fixes some tests that now fail:
- Tests that didn't expect MFE to be enabled (because we don't
enable MFE for Old Mongo) - opt out of MFE for those
- Tests that hardcoded old key string formats
- Lots of other random little differences
In many places, I didn't spend much time trying to figure out how to
properly fix the test, and instead just set the modulestore to Old
Mongo.
For those tests that I didn't spend time investigating, I've set
the modulestore to TEST_DATA_MONGO_AMNESTY_MODULESTORE - search for
that string to find further work.
* Deprecates ModuleSystem can_execute_unsafe_code, get_python_lib_zip and cache properties
* Adds a new CacheService and SandboxService to provide the deprecated property
* Adds tests for the added CacheService and SandboxService
* Updates the ModuleSystemShim tests in Lms and Studio
* feat: Add a new way to enable/disable teams
Adds a new mechanism for enabling/disabling the team feature in a course using an 'enabled' field to the teams config.
If this field is set to true, teams is enabled (team sets/groups) still need to be defined. If this is set to false then teams is disabled whether or not team sets are defined.
* fix: review feedback
The learning MFE paths include /course/{course_id} which doesn't match /courses/{course_id} which is what the regex expects. This causes issues with the Wiki when when accessed from the learning MFE doesn't detect that course it's related to in the middleware.
* fix: adding more parameters for cookie monitoring
Added: cookies_total_num, cookies_unaccounted_size.
-Both are to help us gauge how many cookies we are not collecting data for.
Increased: # of cookies data collected
The `error_expected` custom attribute used to contain
both the class name and the error message. This had
the following issues:
* Combining data in the same custom attribute limits
the ability to query.
* The additional error class and message data is only
needed for ignored errors, since this data isn't
available elsewhere.
The following changes were made:
* `error_expected` will always have the value True
if present.
* `error_ignored` no longer exists.
* `error_ignored_class` will contain the error module
and class for ignored errors.
* `error_ignored_message` will contain the error message
for ignored errors.
ARCHBOM-1708
Adds logging and monitoring capabilities for expected
errors. See the ADR and how-to documentation for
details of how to configure and use the EXPECTED_ERRORS
setting and new monitoring and logging.
ARCHBOM-1708
Co-authored-by: Tim McCormack <tmccormack@edx.org>
Instead of adding new attributes for each cookie name we create
consistent attribute names. This should prevent any issues where we
have too many different unique attribute names because the cookie names
are unique to the user.
We added two new settings to make the number of cookies and groups
capture configurable:
- TOP_N_COOKIES_CAPTURED
- TOP_N_COOKIE_GROUPS_CAPTURED
Setting a new metric per cookie name resulted in a lot of metrics
getting added to New Relic. In some cases, this was causing other
more important metrics to not get registered.
We want to be able to easily figure out what our biggest cookies are and we
want to also group cookies by prefix because certain services create multiple
cookies and then put unique identifiers in the cookie name.
For example braze cookie names use the following pattern:
ab.storage.<userId>
ab.storage.<deviceId>
ab.storage.<sessionId>
In this case we want to group all the `ab` cookies together so we can see
their total size.
New attributes:
cookies.<group_prefix>.group.size: The size of a group of cookies. For example
the sum of the size of all braze cookies would be the value of the
`cookies.ab.group.size` attribute.
cookies.max.name: The name of the largest cookie sent by the user.
cookies.max.size: The size of the largest cookie sent by the user.
cookies.max.group.name: The name of the largest group of cookies. A single cookie
counts as a group of one for this calculation.
cookies.max.group.size: The sum total size of all the cookies in the largest group.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
* Moving plugins infrastructure to edx-django-utils
This PR extracts the code that enables plugins in edx-platform and puts it in edx-django-utils. This is done to allow other IDAS to add plugin functionality.
* Replace track calls with eventtracking in task_track and server_track
* Making events compatible with events emitted via track app
* Fixed broken tests and quality violation
* Removed useless changes in shim. using in to have and keys in event context
Co-authored-by: zia.fazal@arbisoft.com <zia.fazal@arbisoft.com>
* Revert "Ran make migration on third_party_auth (#23253)"
This reverts commit 49be65cc58.
* Removing provider.util import
* Removing further provider things
* Adding hash tests
This is a follow up from MST-16, which was commited
in 3858036a4e.
Changes:
* Enrich course teams_configuration from a plain Dict
to a custom XBlock field that uses the new TeamsConfig
wrapper class.
* Remove teams_conf property from course, as the previous
change made it redundant.
* Update teams_enabled implementation.
* Remove teams_max_size field from course, which is
no longer semantically correct, as max team size
is now defined on a teamset level.
* Remove teams_topics in order to discourage use of raw
teams config dict.
* Add convenience properties teamsets and teamsets_by_id
to course.
* Allow periods and spaces in teamset IDs to avoid breaking
existing course teams.
Some parts of the code still use the old raw config data
(identifiable by searching "cleaned_data_old_format"),
which we expect to be slowly factored away as we build
new teams features. MST-40 has been created to remove any
remaining references if necessary.
MST-18
* fix: bokchoy test
* fix: remove pdb break
Python 3 has no way to access this implementation of rounding in the
standard library. We implement it here so that we can continue to use
it for grades calculation to keep regrading consistent.
Currently we can't be confident that if we change the rounding behaviour
we won't impact students. We can't be sure that students that were
previously passing wouldn't suddenly no longer be passing. Given this,
it's lower risk to just implement the old rounding strategy here and use
it when we are rounding to calculate grades.
Wrap CourseModule.teams_configuration in TeamsConfig
class, centralizing parsing, validation, error handling,
etc. Wrapped object is exposed on 'teams_conf' field.
Old code still uses 'teams_configuration' dict;
we should change this in the future (MST-18).
MST-16
