Currently, LMS uses 3 Celery workers: lms_default_1, lms_high_1 and
lms_high_mem_1. Each Celery worker sends messages to a single queue:
edx.core.default, edx.core.high and edx.core.high_mem, respectively.
The number of child processes per Celery worker is set to 1. Due to
this configuration, any task in a queue blocks all other tasks.
Currently, the Celery check task submitted by the /heartbeat?extended
LMS HTTP API endpoint runs in the default queue. When some slow task
(eg course grades creation) is sent to the default queue, it will
block the Celery check task, which will expire and the heartbeat endpoint
will fail. This patch moves the task to another queue which has
only shorter tasks and in which this problem will not occur.
Use a Django setting for the Celery check task routing key so that
it can be overriden by individual OpenEDX instances via JSON
env files.
From what I've gleaned, CONFIG_FILE (and LMS_CFG and STUDIO_CFG
environment variables) supercedes CONFIG_ROOT. There's no code anymore
that reads the value of the CONFIG_ROOT django setting, and in turn the
CONFIG_ROOT environment variable.
Tangentially related to DENG-369
* Moving plugins infrastructure to edx-django-utils
This PR extracts the code that enables plugins in edx-platform and puts it in edx-django-utils. This is done to allow other IDAS to add plugin functionality.
* Enforce limit on number of blocks allowed in library (blockstore)
* Enforce limit on number of blocks allowed in library (modulestore)
* Changes from review feedback
We shouldn't be explicitly setting this here. It's already set in all of
our override files and not everyone who wants to use production.py wants
to use SES so we shouldn't hardcode it here.
Also move the other settings to above the config loading so that all
config can be overridden by the config file.
Now that we autoload most keys directly from the yaml file into the
settings context. We don't need boilerplate of the form:
KEY = ENV_TOKENS.get('KEY', KEY)
KEY = AUTH_TOKENS.get('KEY', KEY)
which tries to get the value of KEY from the config file but if it's not
there it pulls it from the defaults found in common.py
This should allow us to remove all the boilerplate code in this file
where a name is pulled from the config dict and put into the top level
namespace of the settings module.
We do this first so that any logic that adds more complex or dynamic
keys will still run and is safe.
Now that this is here we can start removing any simple boilerplate.
1. Created a new celery queue with key `SOFTWARE_SECURE_VERIFICATION_ROUTING_KEY`.
2. Added a celery task with retry logic.
3. sorted imports with isort.
4. Changed deprecated `log.warn` => `log.warning`.
BROKER_HEARTBEAT appears to interfere with the celery worker
configuration. If we want to disable or change the heartbeat interval,
It must be from these configuration options.
Just setting --without-heartbeat and/or --heartbeat-interval on the
workers does not fully work. Eg. --without-heartbeat disables sending
heartbeats, but BROKER_HEARTBEAT = 60 means that the connection will get
a 60s heartbeat interval negotiated.
This commit adds a feature that allows overriding any marketing URL, either by configuration on lms.env.json or SiteConfiguration, regardless if a marketing website is set up.
The links can be set by passing MKTG_URL_OVERRIDES to either `lms.env.json` or `SiteConfiguration`.
-Updated middleware setting to use use middleware insted of
middleware_classes
-github install of django-method-override fork to support
new style middleware in django1.11
This completes the work started in https://github.com/edx/edx-platform/pull/19453
to use the LMS login and registration for Studio, rather than Studio
providing its own implementation.
LMS login/registration are being used for the following reasons:
1. LMS logistration properly handles all SSO integrations.
2. A single logistration is simpler to maintain and understand.
3. Allows Studio to work more like all other IDAs that use LMS
logistration.
The original switch to use LMS logistration for Studio also added the
toggle `DISABLE_STUDIO_SSO_OVER_LMS` to provide the community some
additional time for switching. This commit removes this toggle, which
at this point means all deployments will use the LMS logistration.
This change requires sharing cookies across LMS and Studio. Should that
prove to be a problem for certain Open edX instances, there are
discussions of possible alternative solutions.
See https://github.com/edx/edx-platform/pull/19845#issuecomment-559154256
Detailed changes:
* Fix some Studio links that still went to old Studio signin and signup.
* Remove DISABLE_STUDIO_SSO_OVER_LMS feature toggle.
* Remove old studio signin and signup pages and templates.
* Fix url name "login", which had different meanings for Studio and LMS.
* Use the following settings: LOGIN_URL, FRONTEND_LOGIN_URL,
FRONTEND_LOGOUT_URL, and FRONTEND_REGISTER_URL.
* Redirect /signin and /signup to the LMS logistration.
* Add custom metric `uses_pattern_library`.
* Add custom metric `student_activate_account`.
* Add Django Settings to allow /signin, /signup, and /login_post to be
disabled once ready.
This work also relates to ARCH-218 and DEPR-6.
ARCH-1253
- The fields need to be unicode not bytes. In python 2 it was auto
converting.
- The static url base has been allowed to be a unicode string for a
while.
https://github.com/edx/edx-platform/pull/20645
This introduces:
* A new XBlock runtime that can read and write XBlocks that are persisted using
Blockstore instead of Modulestore. The new runtime is currently isolated so
that it can be tested without risk to the current courseware/runtime.
* Content Libraries v2, which store XBlocks in Blockstore not modulestore
* An API Client for Blockstore
* "Learning Context" plugin API. A learning context is a more abstract concept
than a course; it's a collection of XBlocks that serves some learning purpose.
