* refactor: enterprise dependencies for EdxRestAPIClient replacement
This is a part of https://github.com/openedx/public-engineering/issues/42
- add settings for enterprise-backend-service DOT application
- update utils used by enterprise to get rid of EdxRestAPIClient
- original utils stays in the code (to keep edx-platform api
clients working) till the
https://github.com/openedx/public-engineering/issues/39 deprecation
work will be done
* fix: fix typo in the docstring
Previously, our rate-limiting code trusted the entire `X-Forwarded-For`
header, allowing a malicious client to spoof that header and evade
rate-limiting. This commit introduces a new module and setting
allowing us to make a more conservative choice of IPs.
- Create new `openedx.core.djangoapps.util.ip` module for producing
the IP "external chain" for requests based on the XFF header and the
REMOTE_ADDR.
- Include a function that gives the safest choice of IPs.
- Add new setting `CLOSEST_CLIENT_IP_FROM_HEADERS` for configuring how
the external chain is derived (i.e. setting the trust
boundary). Currently has a default, but we may want to make it
mandatory in the future.
- Change `django-ratelimit` code to use the proximate IP in the external
chain -- the one just outside the trust boundary.
Also:
- Change `XForwardedForMiddleware` to use more conservative choice for
its `REMOTE_ADDR` override
- Other adjustments to `XForwardedForMiddleware` as needed in order to
initialize new module and support code that needs the real
`REMOTE_ADDR` value
- Metrics for observability into the change (and XFF composition)
- Feature switch to restore legacy mode if needed
This also gives us a path forward to removing use of the django-ipware
package, which is no longer maintained and has a handful of bugs that make it
difficult to use safely.
Internal ticket: ARCHBOM-2056
Introduce a new CMS settings COURSEGRAPH_CONNECTION,
which allows operators to specify default connection paramters
for a Neo4j instance.
This has three purposes:
* The `./manage.py cms dump_to_neo4j` management command will be
much easier for developers and operators to type out because connection
arguments can now be omitted. Note that connection arguments, if
supplied, will override the arguments specified in CMS settings.
* The automatic push-to-coursegraph-on-publish-signal introduced in
subsequent commits can use these connection settings.
* The CourseGraph Django admin actions introduced in subsequent
commits can use these connection settings.
* feat: Add support for using the discussions MFE UI instead of existing UI
Adds a new course waffle flag that when set along with the discussions MFE URL shows the discussions MFE UI instead of the regular UI.
* test: add tests
* squash!: more consistent url name
- Update migration instructions
- Changes regarding redirect URLs and cookie domain are to permit the
site to run on multiple domains.
- Set LOGIN_URL in common so that it can be unset in environment overrides
This bypasses the "redirect to LMS" login/signup code, but does not yet
remove it; removal is covered by DEPR-166 so that this remains a
configuration-only change for now.
There should have no user-visible effect.
ref: ARCHBOM-1890
This changes the "Sign out" link on Studio to point to Studio's own logout
view, which clears the session and then redirects to LMS's logout page. The
LMS logout page then skips loading the Studio logout because it is seen in
the Referer header.
This change also brings Studio better into line with how other IDAs perform
their logouts.
Background:
After the rollout of Studio OAuth, logouts initiated on Studio failed to
actually log out Studio (but all other IDAs were logged out). This was
because the LMS logout view loads the logout pages of other IDAs but skips
any that is a *prefix* match on the Referer header, and browsers now often
send a truncated version of the Referer for privacy. Therefore, Studio was
always skipped when coming from Studio.
The fix is to make sure that Studio has already performed its logout by the
time the LMS logout page is loaded.
One wrinkle here is that the LMS logout view is activated by `/logout`, but
the correct logout view (provided by auth_backends) is activated by
`/logout/` -- with a trailing slash. This is fragile and unfortunate, but
can be cleaned up when we later remove other leftovers of Studio's previous
ability to handle logistration.
ref: ARCHBOM-1897
This also requires using different session cookie names in devstack for
LMS and Studio so that we can properly use the OAuth flow locally, rather
than just sharing a session cookie on the localhost domain. (Note that
ports are not used for cookie segmentation in browsers.)
This depends on the following provisioning PR:
https://github.com/edx/devstack/pull/825
Login redirect was using wrong url, thus making it impossible to login from studio. This settings change should fix that.
Useful information to include:
Which edX user roles will this change impact? "Developer"
Supporting information
https://openedx.atlassian.net/browse/ARCHBOM-1687
Testing instructions
Bring up devstack studio and press sign in. If it redirects you correctly to lms login page, this works!
Deprecated edx-platform import paths
(for example, `student` instead of
`common.djangoapps.student`) currently raise
warnings when used. We want to fully remove
support for those paths.
As an easily reversible way to initially remove
support, we add a new setting to LMS and Studio
called `ERROR_ON_DEPRECATED_EDX_PLATFORM_IMPORTS`,
defaulting to False. We set it to True for devstack
and will set it to True in Stage and Production
soon. If critical errors occur, we can easily
flip the setting back to False.
* Install `organizations` app into LMS and Studio non-optionally.
* Add toggle `ORGANIZATIONS_AUTOCREATE` to Studio.
* Remove the `FEATURES["ORGANIZATIONS_APP"]` toggle.
* Use the new `organizations.api.ensure_organization` function to
either validate or get-or-create organizations, depending
on the value of `ORGANIZATIONS_AUTOCREATE`,
when creating course runs and V2 content libraries.
We'll soon use it for V1 content libraries as well.
* Remove the `util.organizations_helpers` wrapper layer
that had to exist because `organizations` was an optional app.
* Add `.get_library_keys()` method to the Split modulestore.
* Add Studio management command for backfilling organizations tables
(`backfill_orgs_and_org_courses`).
For full details, see
https://github.com/edx/edx-organizations/blob/master/docs/decisions/0001-phase-in-db-backed-organizations-to-all.rst
TNL-7646
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
This change was originally made in preparation for a BD-14
change that would enable database-backed organizations
across Open edX. Since then, we've figured out a way
of rolling out database-backed organizations without
mandating that organization slugs in new courses are
validated. So, this puts devstack back to where it was before,
with ORGANIZATIONS_APP==True for LMS on devstack and
ORGANIZATIONS_APP==False for Studio on devstack.
From a developer perspective, this means that course
runs can again be created in Studio with any org slug.
TNL-7425
It is already enabled in devstack LMS, stage LMS/Studio,
and prod LMS/Studio.
However, it is currently disabled in edge LMS/Studio,
and as far as I know, sandbox LMS/Studio as well as the
default Open edX LMS/Studio.
We would like to move towards enabling it globally by
default, and enabling it in devstack Studio would be
a first step towards that.
* Moving plugins infrastructure to edx-django-utils
This PR extracts the code that enables plugins in edx-platform and puts it in edx-django-utils. This is done to allow other IDAS to add plugin functionality.
Background: I use private.py a lot in my devstack for my private configurations. But issues are raised when I need to change/override a setting that is defined in devstack_docker.py file. Since that file's contents are loaded after private.py, my private.py is useless against the settings in devstack_docker.py.
Description: Now that the devstack is dockerized, I think it makes more sense to import the private.py in the devstack_docker.py files i.e. at the end of configuration files hierarchy.
- This PR removes all imports from provider by either bringing them into edx-platform or finding dot replacement. Removing tests that tested dop parts of code.
- Skipping some tests and removing dop
The tests are difficult to fix due to its entanglement with dop use in third_party_auth.
These tests should be restarted once dop has been removed from third_party_auth and its tests.
- set ENABLE_DOP_ADAPTER = False for devstack
-Updated middleware setting to use use middleware insted of
middleware_classes
-github install of django-method-override fork to support
new style middleware in django1.11
We are currently running this plugin from a has directly off of master
to get support for a newer version of DJDT. However even on master
there is not yet support for python 3. We're running into this when
running runserver and when collecting static assets.
A quick survey of the development team at edX shows that people are no
longer actively using this debug panel.(14 responders all saying no to
the question "In the last year have you used the mongo portion of the
django-debug-toolbar in edx-platform?")
This makes sense since the module store and courseware related
developement is not very active right now. Given all this, I'm removing
this toolbar for now and if we need it again we can survey what's
available at that time to choose the best tool.
https://github.com/edx/edx-platform/pull/20645
This introduces:
* A new XBlock runtime that can read and write XBlocks that are persisted using
Blockstore instead of Modulestore. The new runtime is currently isolated so
that it can be tested without risk to the current courseware/runtime.
* Content Libraries v2, which store XBlocks in Blockstore not modulestore
* An API Client for Blockstore
* "Learning Context" plugin API. A learning context is a more abstract concept
than a course; it's a collection of XBlocks that serves some learning purpose.
* Install drf-yasg
* Add drf-yasg settings and urls
* Pin drf to make drf-yasg work
* Adjust config-models version to be compatible
* Remove django-rest-swagger (the old way)
Some deprecated functionality has been removed:
- Reading data field and transforms being applied in the init() method.
- The source field.
- The source_visible attribute.
