The username was allowed in the login endpoint alongside the email address
but rate-limiting logic was not updated to rate limit on the new POST
param `email_or_username`.
VAN-1003
Previously, our rate-limiting code trusted the entire `X-Forwarded-For`
header, allowing a malicious client to spoof that header and evade
rate-limiting. This commit introduces a new module and setting
allowing us to make a more conservative choice of IPs.
- Create new `openedx.core.djangoapps.util.ip` module for producing
the IP "external chain" for requests based on the XFF header and the
REMOTE_ADDR.
- Include a function that gives the safest choice of IPs.
- Add new setting `CLOSEST_CLIENT_IP_FROM_HEADERS` for configuring how
the external chain is derived (i.e. setting the trust
boundary). Currently has a default, but we may want to make it
mandatory in the future.
- Change `django-ratelimit` code to use the proximate IP in the external
chain -- the one just outside the trust boundary.
Also:
- Change `XForwardedForMiddleware` to use more conservative choice for
its `REMOTE_ADDR` override
- Other adjustments to `XForwardedForMiddleware` as needed in order to
initialize new module and support code that needs the real
`REMOTE_ADDR` value
- Metrics for observability into the change (and XFF composition)
- Feature switch to restore legacy mode if needed
This also gives us a path forward to removing use of the django-ipware
package, which is no longer maintained and has a handful of bugs that make it
difficult to use safely.
Internal ticket: ARCHBOM-2056
* build: Removed the diff-quality step
Applied lint-amnesty on all the warnings
Removed pylint thresholds comparison code and related tests
Co-authored-by: Usama Sadiq <usama.sadiq@arbisoft.com>
We use django-ratelimit to limit per IP login attempts, and then we use
django-ratelimit-backend to limit per username login attempts. This
change replaces the usage of django-ratelimit-backend with another
instance of django-ratelimit so that both limits can be managed by one
library.
This is the first step in being able to fully excise
django-ratelimit-backend from edx-platform. Note that we're still using
the `RateLimitMixin` in openedx/core/djangoapps/oauth_dispatch/dot_overrides/backends.py
because studio and the admin UI still relies on that for rate limiting.
Those login paths will have to be updated before we can remove the mixin
from our auth backend.
By explicitly importing the legacy namespace classes, we make it clear
that we are using soon-to-be-deprecated classes. We will then be able to
start removing the legacy classes, one module at a time.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
The UserIdFilter and RemoteIpFilter logging filter classes
were moved from edx-platform to edx-django-utils for more
distributed use. This updated removes those classes from
edx-platform, and references their counterparts in
edx-django-utils.
ENT-3494
* Add remote IP to logging config
Add a new filter to get the remote IP for the current
request and include it in log statements
SEG-34
* Added line for formatting
Added line for formatting
SEG-34
Fix the issue that was preventing us from upgrading pytest. pytest does some manipulation of test packages that prevents `pkg_resources` from loading resources from them, but used to contain a workaround for the problem. That workaround was [removed](https://github.com/pytest-dev/pytest/issues/5392) in 4.6.0 as a performance enhancement when pytest switched from `pkg_resources` to `importlib-metadata` for its own entrypoint handling. This tripped up one of our test modules which defined classes that loaded templates from inside a test package. Moving these resources to the parent package fixes the problem.
More and more, `pkg_resources` is being abandoned in favor of `importlib-metadata` and `importlib_resources` as they have a simpler design with much better performance. However, `importlib_resources` doesn't support loading files from any directory which isn't itself a Python package (and doesn't allow direct use of paths including directories within the package). Jinja2 chose a [different approach](https://github.com/pallets/jinja/pull/1082) that we may want to emulate in our resource handling.
Also fixed usage of a removed `pytest.raises()` parameter and a bug in our configuration of the `common/lib` tests that became a problem after the upgrade.
