OeX_ES-44: Update edx-search elasticsearch.
* removes redundant doc_types from the lms/cms, replaces courseware
index with doc_types to two indices
* code polishing
* Use a new config variable for ES7 deployment. (#2226)
Co-authored-by: Golub-Sergey <1.golub.sergey.1@gmail.com>
Co-authored-by: Diana Huang <diana.k.huang@gmail.com>
Currently, LMS uses 3 Celery workers: lms_default_1, lms_high_1 and
lms_high_mem_1. Each Celery worker sends messages to a single queue:
edx.core.default, edx.core.high and edx.core.high_mem, respectively.
The number of child processes per Celery worker is set to 1. Due to
this configuration, any task in a queue blocks all other tasks.
Currently, the Celery check task submitted by the /heartbeat?extended
LMS HTTP API endpoint runs in the default queue. When some slow task
(eg course grades creation) is sent to the default queue, it will
block the Celery check task, which will expire and the heartbeat endpoint
will fail. This patch moves the task to another queue which has
only shorter tasks and in which this problem will not occur.
Use a Django setting for the Celery check task routing key so that
it can be overriden by individual OpenEDX instances via JSON
env files.
From what I've gleaned, CONFIG_FILE (and LMS_CFG and STUDIO_CFG
environment variables) supercedes CONFIG_ROOT. There's no code anymore
that reads the value of the CONFIG_ROOT django setting, and in turn the
CONFIG_ROOT environment variable.
Tangentially related to DENG-369
* Moving plugins infrastructure to edx-django-utils
This PR extracts the code that enables plugins in edx-platform and puts it in edx-django-utils. This is done to allow other IDAS to add plugin functionality.
* Enforce limit on number of blocks allowed in library (blockstore)
* Enforce limit on number of blocks allowed in library (modulestore)
* Changes from review feedback
We shouldn't be explicitly setting this here. It's already set in all of
our override files and not everyone who wants to use production.py wants
to use SES so we shouldn't hardcode it here.
Also move the other settings to above the config loading so that all
config can be overridden by the config file.
Now that we autoload most keys directly from the yaml file into the
settings context. We don't need boilerplate of the form:
KEY = ENV_TOKENS.get('KEY', KEY)
KEY = AUTH_TOKENS.get('KEY', KEY)
which tries to get the value of KEY from the config file but if it's not
there it pulls it from the defaults found in common.py
This should allow us to remove all the boilerplate code in this file
where a name is pulled from the config dict and put into the top level
namespace of the settings module.
We do this first so that any logic that adds more complex or dynamic
keys will still run and is safe.
Now that this is here we can start removing any simple boilerplate.
1. Created a new celery queue with key `SOFTWARE_SECURE_VERIFICATION_ROUTING_KEY`.
2. Added a celery task with retry logic.
3. sorted imports with isort.
4. Changed deprecated `log.warn` => `log.warning`.
BROKER_HEARTBEAT appears to interfere with the celery worker
configuration. If we want to disable or change the heartbeat interval,
It must be from these configuration options.
Just setting --without-heartbeat and/or --heartbeat-interval on the
workers does not fully work. Eg. --without-heartbeat disables sending
heartbeats, but BROKER_HEARTBEAT = 60 means that the connection will get
a 60s heartbeat interval negotiated.
This commit adds a feature that allows overriding any marketing URL, either by configuration on lms.env.json or SiteConfiguration, regardless if a marketing website is set up.
The links can be set by passing MKTG_URL_OVERRIDES to either `lms.env.json` or `SiteConfiguration`.
-Updated middleware setting to use use middleware insted of
middleware_classes
-github install of django-method-override fork to support
new style middleware in django1.11
