Improving Studio homepage performance for users with course access role with no course_id
Fixing unit tests
Added create CourseOverviewFactory after creating course to course listing test
Fix order import for `from openedx.core.djangoapps.content.course_overviews.tests.factories import CourseOverviewFactory
`
(cherry picked from commit 997a0ff770744309f0ee84f3c0696a80310c5f2d)
Unfortunately, some code in edx-platform is imported
relative to sub-projects instead of the repository root.
The only three remaining instances of this are:
* common/lib/xmodule/xmodule (imported as just 'xmodule')
* common/lib/capa/capa (imported as just 'capa')
* openedx/core/lib/xblock_builtin/xblock_discussion
(imported as just 'xblock_discussion')
For more details on the situation, see:
https://openedx.atlassian.net/browse/BOM-2579
(public, but requires Atlassian account creation).
We would like to get to a point where all edx-platform
import paths match their folder paths, relative to the repo
root. For now, though, all common/lib/capa and common/lib/xmodule
code should be imported as just `from capa` and `from xmodule`,
respectively. Importing using the full `common.lib.xmodule.xmodule...`
path will often work, but it instantiates a second instance of all
modules imported this way, which in the past has led to very
difficult-to-diagnose bugs. It also confuses tooling such as
import-linter, which we are trying to add to edx-platform
(see https://openedx.atlassian.net/browse/BOM-2576)
* feat: test out individualized anonymouse_user_id for studio preview on xblocks
* test: adds tests for the PreviewModuleSystem anonymous_user_id (#30400)
and the INDIVIDUALIZE_ANONYMOUS_USER_ID flag.
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Simon Chen <schen@edX-C02FW0GUML85.local>
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Remove temporary FutureCourseWaffleFlag class;
Update ora2 and edx-toggles to versions cleaned from the
LegacyWaffle* classes;
Replace `override_flag`s with `override_waffle_flag`;
Replace `override_switch`s with `override_waffle_switch` (where it's possible).
BREAKING: get rid of the LegacyWaffle-based CourseWaffleFlag.
Both CourseWaffleFlag and FutureCourseWaffleFlag now use the modern
WaffleFlag as parent class. FutureCourseWaffleFlag left to support ORA
transition to modern waffle.
Switch to the ORA version which supporting new Waffles.
This is a first stage for removing the LegacyWaffle* classes.
LegacyWaffleFlag usage replaced with WaffleFlag;
LegacyWaffleSwitche usage replaced with WaffleSwitch;
New CourseWaffleFlag added to the temporary module __future__ as FutureCourseWaffleFlag;
Updated all the imports to use CourseWaffleFlag from the __future__ module;
BREAKING CHANGE: A number of toggle related constants (e.g. ENABLE_ACCESSIBILITY_POLICY_PAGE)
changed types. They were strings, and are now toggle instances (e.g. WaffleSwitch). Although the entire
refactor should be self-contained in edx-platform, if any plugins or dependencies were directly
using these constants, they will break. If this is the case, try to find a better publicized way of
exposing those toggles.
Implements a feature flag DISABLE_UNENROLLMENT that is used to disable students un-enrollment for all courses. The Unenrollment option should be disabled when this feature is set to True.
ref: BB-4951
Co-authored-by: tinumide <tinuade@opencraft.com>
Co-authored-by: Tim McCormack <tmccormack@edx.org>
* refactor: enterprise dependencies for EdxRestAPIClient replacement
This is a part of https://github.com/openedx/public-engineering/issues/42
- add settings for enterprise-backend-service DOT application
- update utils used by enterprise to get rid of EdxRestAPIClient
- original utils stays in the code (to keep edx-platform api
clients working) till the
https://github.com/openedx/public-engineering/issues/39 deprecation
work will be done
* fix: fix typo in the docstring
Tests which @requires_blockstore (i.e. the Blockstore service) have
been made to run as a unit test using the installed Blockstore app, and
will be run by the platform CI.
The Blockstore service tests can still be run manually by setting
EDXAPP_RUN_BLOCKSTORE_TESTS=1
Related fixes:
* adds blockstore bundle storage settings
* let the studio devstack and test servers serve static files from
the /media URL This allows the blockstore/content libraries API to
serve blockstore assets in dev.
* Wrap ContentLibrary creation in an atomic transaction, so that if it
fails, the related bundle can be deleted directly from the database
during the exception handler. (Previously, we called a REST API which
deleted it as part of a separate service.)
* adds blockstore as a requirement and an installed app, with
configurable bundle storage settings.
* adds waffle switch and setting to allow use of blockstore's python API
instead of REST API in live testservers and in production.
* adds database router which, when a `blockstore` DATABASE connection is
configured, allows the platform to use the blockstore service's
database instead of the default edxapp database.
* replaces blockstore_api exceptions and models with blockstore.app.api classes
* minor fixes to the blockstore_api to make the Blockstore REST API
return data packaged the same as the Blockstore Python API.
Previously, our rate-limiting code trusted the entire `X-Forwarded-For`
header, allowing a malicious client to spoof that header and evade
rate-limiting. This commit introduces a new module and setting
allowing us to make a more conservative choice of IPs.
- Create new `openedx.core.djangoapps.util.ip` module for producing
the IP "external chain" for requests based on the XFF header and the
REMOTE_ADDR.
- Include a function that gives the safest choice of IPs.
- Add new setting `CLOSEST_CLIENT_IP_FROM_HEADERS` for configuring how
the external chain is derived (i.e. setting the trust
boundary). Currently has a default, but we may want to make it
mandatory in the future.
- Change `django-ratelimit` code to use the proximate IP in the external
chain -- the one just outside the trust boundary.
Also:
- Change `XForwardedForMiddleware` to use more conservative choice for
its `REMOTE_ADDR` override
- Other adjustments to `XForwardedForMiddleware` as needed in order to
initialize new module and support code that needs the real
`REMOTE_ADDR` value
- Metrics for observability into the change (and XFF composition)
- Feature switch to restore legacy mode if needed
This also gives us a path forward to removing use of the django-ipware
package, which is no longer maintained and has a handful of bugs that make it
difficult to use safely.
Internal ticket: ARCHBOM-2056
This:
1. Removes the `filestore` property from the `ModuleSystem` in favor of
the `runtime.resources_fs` property.
In the original code, `filestore` is equal to
`DescriptorSystem.runtime.resources_fs`. It's safe to replace it with
`ModuleSystem.runtime.resources_fs` because both runtimes are combined
using the `CachingDescriptorSystem`. It provides the `resources_fs` property
that uses the same file storage.
2. Renames `filestore` argument to `resources_fs` in the `LoncapaSystem`
constructor.
3. Adds the deprecated `filestore` property to the `ModuleSystemShim`
and `RuntimeShim`.
Update the README of the CMS's CourseGraph support app:
* Point to the newly-developed CourseGraph plugin for Tutor,
and remove some prose that's now redundant with the Tutor
plugin's README.
* Add a link to the now-public CourseGraph Queries wiki page.
* Capitalize the G in CourseGraph.
* Fix a couple misc. formatting things.
This introduces two admin actions:
* Dump to CourseGraph (respect cache), and
* Dump to CourseGraph (override cache)
which allow admins to select a collection of courses from Django
admin and dump them to the Neo4j instance specified by
settings.COURSEGRAPH_CONNECTION, with or without respecting
the cache (that is: whether the course has already been dumped
since its last publishing).
The `get_course_last_published` function is used by CourseGraph to
determine whether or not a course should be dumped to Neo4j.
If the course hasn't been published since it was last dumped to
Neo4j, then it can be skipped (unless the override_cache option
is enabled).
The function was previously built using the BlockStructure
data model. While this worked fine in Production instances that
enable `block_structure.storage_backing_for_cache`, this
implementation did NOT work in development environments,
which do not use the BlockStrcture model.
Instead, we switch to using CourseOverview.modified to
approximate when a course was last published. This is method
has fewer moving parts and is universally available across
instances.
Previously, CourseGraph needed to be kept up-to-date by
running `./manage.py dump_to_neo4j ...` manually or on a cron timer.
This introduces a CMS new setting: COURSEGRAPH_DUMP_COURSE_ON_PUBLISH.
When enabled, the CMS course_published signal handler will
asynchronously dump each individual course to CourseGraph when it
is published.
This follows a pattern established by other subsystems like
learning_sequences and special exam registration, both of which
fire off asynchronous post-processing tasks from the course-
publish handler.
Introduce a new CMS settings COURSEGRAPH_CONNECTION,
which allows operators to specify default connection paramters
for a Neo4j instance.
This has three purposes:
* The `./manage.py cms dump_to_neo4j` management command will be
much easier for developers and operators to type out because connection
arguments can now be omitted. Note that connection arguments, if
supplied, will override the arguments specified in CMS settings.
* The automatic push-to-coursegraph-on-publish-signal introduced in
subsequent commits can use these connection settings.
* The CourseGraph Django admin actions introduced in subsequent
commits can use these connection settings.
