At some point something changed in upstream config or in celery and now
a black transport is no longer valid. You have to supply some transport
even if you are running in always eager mode. Add enough config so that
celery tasks will execute without issues when running `runserver` on
baremetal.
Fixes the "No such transport" error:
```
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/task.py", line 575, in apply_async
with app.producer_or_acquire(producer) as eager_producer: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 932, in producer_or_acquire producer, self.producer_pool.acquire, block=True,
^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 1354, in producer_pool
return self.amqp.producer_pool ^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/amqp.py", line 591, in producer_pool
self.app.connection_for_write()]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 829, in connection_for_write
return self._connection(url or self.conf.broker_write_url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 880, in _connection
return self.amqp.Connection(
^^^^^^^^^^^^^^^^^^^^^ File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/kombu/connection.py", line 201, in __init__
if not get_transport_cls(transport).can_parse_url: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/kombu/transport/__init__.py", line 91, in get_transport_cls _transport_cache[transport] = resolve_transport(transport)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/kombu/transport/__init__.py", line 72, in resolve_transport
raise KeyError(f'No such transport: {transport}') KeyError: 'No such transport: '
```
This package is no longer supported (https://www.npmjs.com/package/picturefill)
and it was a polyfill for the Picture element which is now widely
supported (https://caniuse.com/?search=Picture) and we only used it in
one spot to make things work correctly in IE. I think it's safe to drop
as we and the world has stopped supporting IE a while ago.
BREAKING CHANGE: Program Card images on the legacy learner dashboard may
no longer load properly on IE.
This is a pure refactoring of cms/envs/production.py, removing several
redundant statements that have accrued over the years as the platform moved
from python-only, to python+json, to python+json+yaml, to today's python+yaml
setup.
This is the CMS version of:
* a81493ce7f
* (originally 15939232d5)
Also included:
* Add some more explicit structure to the both LMS's and CMS's
production.py using big comments.
* In both LMS and CMS settings, alphabetize the production overrides,
and remove the extraneous comments. Separate out the handful of settings
which have useful comments. The rest of the settings' comments were not
helpful--they were either just stating the obvious, or they were duplicative
of what's documented in common.py.
Co-Authored-By: Feanil Patel <feanil@axim.org>
Part of: https://github.com/openedx/edx-platform/issues/36215
The values are nonsensical but this should be a fairly complete collection
of settings, both at the top level and in sub-dictionaries.
While edxapp will not work properly with these settings, the following
commands should work:
```
DJANGO_SETTINGS_MODULE=lms.envs.production LMS_CFG=lms/envs/mock.yml ./manage.py lms dump_settings
DJANGO_SETTINGS_MODULE=cms.envs.production CMS_CFG=cms/envs/mock.yml ./manage.py cms dump_settings
```
Some of our settings depend on the values of other settings. Rather
than explicitly looking up each one in the YAML settings file, we can
simply derive them based on the setting in the YAML file after all the
YAML settings have been loaded.
Part of: https://github.com/openedx/edx-platform/issues/36215
Co-Authored-By: Feanil Patel <feanil@axim.org>
The Python API for declaring derived settings was confusing to the uninitiated
reader, and also prone to spelling mistakes. This replaces the API with one
that is more readable and more concise, and updates the implementation of
`derive_settings` to properly derive settings declared using the new API.
BREAKING CHANGE: The `derived` and `derived_collection_entry` function are
replaced with the `Derived` class. We do not expect those functions to have
been used outside of edx-platform, but if they are, this commit will cause them
to loudly ImportError.
Note that there should be NO change in behavior to the `derive_settings`
function, which we DO know to be used by some external edx-platform plugins.
Part of: https://github.com/openedx/edx-platform/issues/36215
Quoting the README:
> If you are reading this and the 'Maple' Open edX release is out, you
> can safely delete this whole djangoapp. It only exists to hold old
> model migrations and any post-Maple installation will no longer have
> any model in the database for this app.
>
> But for some minor historical context, this djangoapp used to hold
> some integration with sailthru that we no longer needed.
We've released Sumac. This is overdue.
In the near term, we wish to precisely preserve the existing values of
all Django settings exposed by lms/envs/production.py in order to avoid
breaking legacy Django plugins without a proper announcement.
That includes preserving the behavior of these old, redundant dicts:
* ENV_TOKENS
* AUTH_TOKENS
* ENV_FEATURES
* ENV_CELERY_QUEUES
* ALTERNATE_QUEUE_ENVS
Particularly, it means we need to ensure that updates to Django
settings are reflected in these dicts. The most reliable way to do that
is to change the yaml-loading logic so that these values are aliased
to the corresponding values in the global namespace rather than
deep-copied.
Finally, we remove KEYS_WITH_MERGED_VALUES from the global namespace,
and inline the remaining list. We have modified the list (specifically,
we dropped the no-op MKTG_URL_OVERRIDES). Plugins should not be counting
on the value of the list, so we remove it.
This reintroduces commit 15939232d5,
which was reverted due to a typo.
The typo is fixed in the commit immediately following this one.
Co-Authored-By: Feanil Patel <feanil@axim.org>
`notify_credentials` has 2 ways of running.
1. The manual, one-off method which uses `--args_from_database` to specify what should be sent.
2. [The automated method](7316111b35/openedx/core/djangoapps/credentials/management/commands/notify_credentials.py (L157-L159)), which runs on a regular schedule, to catch anything which fell through the cracks.
The automated method does a certain amount of time/date math in order to calculate the entry point of the window based on the current runtime. This is, I assume, why it has some hardcoded logic; it's not at all simple to have a `cron`-run management command running on a regular cadence that can do the same time logic.
```py
if options['auto']:
options['end_date'] = datetime.now().replace(minute=0, second=0, microsecond=0)
options['start_date'] = options['end_date'] - timedelta(hours=4)
```
However, it is not ideal that the actual time window of 4 hours is hardcoded directly into `edx-platform`.
This fix
* creates an overridable `NOTIFY_CREDENTIALS_FREQUENCY` that defaults to 14400 seconds (4 hours).
* pulls that frequency into the quoted code
Using seconds allows maximum flexibility.
FIXES: APER-3383
We would like new devstack instances to better support the user retirement pipeline out of the box after provisioning. The first step is to update the default list of retirement states devstack is configured to add/use.
This change disables the profiling panel for performance reasons. It's
rarely useful anyway, given the lack of granularity in the data it
displays.
This commit also enables the Cache panel, which is import for tracking
where we're making excessive calls to redis/memcached.
This is a pure refactoring of lms/envs/production.py, removing several
redundant statements that have accrued over the years as the platform moved
from python-only, to python+json, to python+json+yaml, to today's python+yaml
setup.
Notes on some of the more involved refactorings:
* AWS Locals Load block is handled by the YAML loading at the very top, we
don't need to re-load it since there were no changes to those settings
between the YAML loading at the top and this section.
* MKTG_URL_OVERRIDES, we drop doing any overrides and remove it from the merge
list beacuse the default value in
lms/envs/common.py is empty. So the update is a no-op and is the same as just
loading this data directly from the YAML config.
* CODE_JAIL block, we've been overriding the entire dict if it is in your YAML
config, so then going through and updating the individual values is not necessary.
* SSL_AUTH_EMAIL_DOMAIN and SSL_AUTH_DN_FORMAT_STRING are not used anywhere in
the openedx org, looks like they were used by the old `dashboard` djangoapp
and can probably be deleted but might be used by plugins so not removing for
now to keep the change backward compatible.
* DEFAULT_FILE_STORAGE, previously two of the braches were no-ops so we only
keep the one branch we need for when we want to update DEFAULT_FILE_STORAGE
automatically if AWS keys are set.
Somewhat related to: https://github.com/openedx/open-edx-proposals/issues/587
Co-Authored-By: Feanil Patel <feanil@axim.org>
We've been keeping this pinned to 1, but since we're well past the first
version of Celery that can understand both protocol versions, we can just
remove the pin. Celery should then default to protocol 2.
No action should be required from operators.
This will allow us to test protocol 2 in a stage environment before
removing the override to make 2 the default.
We may have seen a bug where something in celery (or an
associated library) was adding headers to a v1 message as
if it were a v2 message, which caused a bug in ddtrace; such
things may become more likely over time as code is written
with the assumption of v2 messages. Moving to v2 will avoid
those issues.
See https://github.com/edx/edx-arch-experiments/issues/800 for further details.
* feat: waffle based switch to ses for goal reminder email
* test: added test cases for ace message parameters
* feat: updated logic to specify channel name
* chore: update edx-ace version
* feat!: removes deprecated v1 certificate behavior
this removes the long-deprecated v1 certificate behavior. This removes
the old-style date selection behavior (ie., not a choice between
*Immediately upon passing*, *End date of course*, *A date after the course
end date*), which is no longer reliably maintained or supported in
Studio or Credentials.
FIXES: #35399
* feat: Be able to login to bare-metal studio easily.
Updating the documentation and the devstack.py files so that if you're
running bare-metal you can easily setup studio login via the LMS.
I also added the Ports that the various MFEs expect to the runserver
scripts so that it's easier to run those locally as well.
Co-authored-by: Kyle McCormick <kyle@axim.org>
Final planned portion of https://github.com/openedx/edx-platform/issues/34702 -- waffle flag and middleware are removed.
Commits:
1. Feature change
- Delete `content_server.use_view` waffle flag in favor of always using view
- Delete the husk of `StaticContentServerMiddleware` and references to it
- Update views module docstring
2. Refactor (move)
- Move contentserver implementation into views.py
3. Lint cleanup
- Fix import ordering (from refactor debris + amnestied lint)
This repo is no longer using USE-JWT-COOKIE header,
since it has the required edx-drf-extensions>10.2.0,
where it was fully removed.
This is final clean-up for this repo.
See "[DEPR]: USE-JWT-COOKIE header" for more details:
- https://github.com/openedx/edx-drf-extensions/issues/371
