The `dump_settings` command currently prints out the raw `repr(...)`s for
defined functions, e.g.:
"WIKI_CAN_ASSIGN": "<function CAN_ASSIGN at 0x74ce5e9b2020>",
In addition to being uninformative, these `at 0x74ce...` hashes change every
run, so they appear in the diff as having "changed" every time. With this
commit, here's what `dump_settings` will print out for a function instead:
"WIKI_CAN_ASSIGN": {
"module": "lms.djangoapps.course_wiki.settings",
"qualname": "CAN_ASSIGN"
},
This command dumps the current Django settings to JSON for
debugging/diagnostics. The output of this command is for *humans*... it
is NOT suitable for consumption by production systems.
In particular, we are introducing this command as part of a series of
refactorings to the Django settings files lms/envs/* and cms/envs/*.
We want to ensure that these refactorings do not introduce any
unexpected breaking changes, so the dump_settings command will both help
us manually verify our refactorings and help operators verify that our
refactorings behave expectedly when using their custom python/yaml
settings files.
Related to: https://github.com/openedx/edx-platform/pull/36131
BREAKING CHANGE: Removes all remaining Paver commands including
`pavelib/prereqs.py:*` and `pavelib/assets.py:*`.
BREAKING CHANGE: Removes `./manage.py [lms|cms] compile_sass`, which
was just a wrapper around Paver commands.
BREAKING CHANGE: Removes paver.txt. Operators should install testing.txt
instead.
Part of: https://github.com/openedx/edx-platform/issues/34467
Note: I had originally implemented this as a `warnings.warn()` call
directly in lms/envs/devstack.py and cms/envs/devstack.py, but for
whatever reason, those warnings were getting swallowed. System checks
display more prominently, anyway.
Part of: https://github.com/openedx/public-engineering/issues/247
This switch has been kept disabled in edx.org for well over a year with no
trouble, and the migration to `CLOSEST_CLIENT_IP_FROM_HEADERS`
was introduced in Nutmeg.
DEPR issue: https://github.com/openedx/edx-platform/issues/33733
The IP chain code has moved to edx-django-utils—except for the legacy-IP code,
which is now in a new module. This will allow other IDAs to use the IP code.
Commit includes some adjustments to the toggle annotation.
Part of <https://github.com/openedx/edx-django-utils/issues/241>.
The username was allowed in the login endpoint alongside the email address
but rate-limiting logic was not updated to rate limit on the new POST
param `email_or_username`.
VAN-1003
Previously, our rate-limiting code trusted the entire `X-Forwarded-For`
header, allowing a malicious client to spoof that header and evade
rate-limiting. This commit introduces a new module and setting
allowing us to make a more conservative choice of IPs.
- Create new `openedx.core.djangoapps.util.ip` module for producing
the IP "external chain" for requests based on the XFF header and the
REMOTE_ADDR.
- Include a function that gives the safest choice of IPs.
- Add new setting `CLOSEST_CLIENT_IP_FROM_HEADERS` for configuring how
the external chain is derived (i.e. setting the trust
boundary). Currently has a default, but we may want to make it
mandatory in the future.
- Change `django-ratelimit` code to use the proximate IP in the external
chain -- the one just outside the trust boundary.
Also:
- Change `XForwardedForMiddleware` to use more conservative choice for
its `REMOTE_ADDR` override
- Other adjustments to `XForwardedForMiddleware` as needed in order to
initialize new module and support code that needs the real
`REMOTE_ADDR` value
- Metrics for observability into the change (and XFF composition)
- Feature switch to restore legacy mode if needed
This also gives us a path forward to removing use of the django-ipware
package, which is no longer maintained and has a handful of bugs that make it
difficult to use safely.
Internal ticket: ARCHBOM-2056
* build: Removed the diff-quality step
Applied lint-amnesty on all the warnings
Removed pylint thresholds comparison code and related tests
Co-authored-by: Usama Sadiq <usama.sadiq@arbisoft.com>
We use django-ratelimit to limit per IP login attempts, and then we use
django-ratelimit-backend to limit per username login attempts. This
change replaces the usage of django-ratelimit-backend with another
instance of django-ratelimit so that both limits can be managed by one
library.
This is the first step in being able to fully excise
django-ratelimit-backend from edx-platform. Note that we're still using
the `RateLimitMixin` in openedx/core/djangoapps/oauth_dispatch/dot_overrides/backends.py
because studio and the admin UI still relies on that for rate limiting.
Those login paths will have to be updated before we can remove the mixin
from our auth backend.
By explicitly importing the legacy namespace classes, we make it clear
that we are using soon-to-be-deprecated classes. We will then be able to
start removing the legacy classes, one module at a time.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
The UserIdFilter and RemoteIpFilter logging filter classes
were moved from edx-platform to edx-django-utils for more
distributed use. This updated removes those classes from
edx-platform, and references their counterparts in
edx-django-utils.
ENT-3494
* Add remote IP to logging config
Add a new filter to get the remote IP for the current
request and include it in log statements
SEG-34
* Added line for formatting
Added line for formatting
SEG-34
Fix the issue that was preventing us from upgrading pytest. pytest does some manipulation of test packages that prevents `pkg_resources` from loading resources from them, but used to contain a workaround for the problem. That workaround was [removed](https://github.com/pytest-dev/pytest/issues/5392) in 4.6.0 as a performance enhancement when pytest switched from `pkg_resources` to `importlib-metadata` for its own entrypoint handling. This tripped up one of our test modules which defined classes that loaded templates from inside a test package. Moving these resources to the parent package fixes the problem.
More and more, `pkg_resources` is being abandoned in favor of `importlib-metadata` and `importlib_resources` as they have a simpler design with much better performance. However, `importlib_resources` doesn't support loading files from any directory which isn't itself a Python package (and doesn't allow direct use of paths including directories within the package). Jinja2 chose a [different approach](https://github.com/pallets/jinja/pull/1082) that we may want to emulate in our resource handling.
Also fixed usage of a removed `pytest.raises()` parameter and a bug in our configuration of the `common/lib` tests that became a problem after the upgrade.
