At some point something changed in upstream config or in celery and now
a black transport is no longer valid. You have to supply some transport
even if you are running in always eager mode. Add enough config so that
celery tasks will execute without issues when running `runserver` on
baremetal.
Fixes the "No such transport" error:
```
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/task.py", line 575, in apply_async
with app.producer_or_acquire(producer) as eager_producer: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 932, in producer_or_acquire producer, self.producer_pool.acquire, block=True,
^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 1354, in producer_pool
return self.amqp.producer_pool ^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/amqp.py", line 591, in producer_pool
self.app.connection_for_write()]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 829, in connection_for_write
return self._connection(url or self.conf.broker_write_url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/celery/app/base.py", line 880, in _connection
return self.amqp.Connection(
^^^^^^^^^^^^^^^^^^^^^ File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/kombu/connection.py", line 201, in __init__
if not get_transport_cls(transport).can_parse_url: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/kombu/transport/__init__.py", line 91, in get_transport_cls _transport_cache[transport] = resolve_transport(transport)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/feanil/.virtualenvs/edx-platform/lib/python3.11/site-packages/kombu/transport/__init__.py", line 72, in resolve_transport
raise KeyError(f'No such transport: {transport}') KeyError: 'No such transport: '
```
This package is no longer supported (https://www.npmjs.com/package/picturefill)
and it was a polyfill for the Picture element which is now widely
supported (https://caniuse.com/?search=Picture) and we only used it in
one spot to make things work correctly in IE. I think it's safe to drop
as we and the world has stopped supporting IE a while ago.
BREAKING CHANGE: Program Card images on the legacy learner dashboard may
no longer load properly on IE.
This is a pure refactoring of cms/envs/production.py, removing several
redundant statements that have accrued over the years as the platform moved
from python-only, to python+json, to python+json+yaml, to today's python+yaml
setup.
This is the CMS version of:
* a81493ce7f
* (originally 15939232d5)
Also included:
* Add some more explicit structure to the both LMS's and CMS's
production.py using big comments.
* In both LMS and CMS settings, alphabetize the production overrides,
and remove the extraneous comments. Separate out the handful of settings
which have useful comments. The rest of the settings' comments were not
helpful--they were either just stating the obvious, or they were duplicative
of what's documented in common.py.
Co-Authored-By: Feanil Patel <feanil@axim.org>
Part of: https://github.com/openedx/edx-platform/issues/36215
The cms/startup.py and lms/startup.py files were created to
allow us to do a lot of custom initialization around things
like the ModuleStore, monkey-patching, adding MIME types to
our process, etc. As far back as 2017, we recognized that
this was a bad thing, marked these modules as "deprecated",
and started removing things or putting them in the standard
Django locations for them (0279181).
In its current state, these startup modules no longer do any
custom work, and just invoke django.startup(). But this is
meant for running Django code in "standalone" usage, e.g. if
you have a script that isn't a management command but needs
some Django functionality.
The "runserver" command used during development normally
launches a child process to serve requests and knows how to
kill and respawn that process when files are modified, so
that changes are reflected. It can also normally handle the
case where there's a SyntaxError in the child process, and
fixing that error will reload the code again.
Something about running django.startup() manually interferes
with this functionality in "runserver". It still reloads the
code in response to changes, but if the code gets into a
broken state for any reason (like a syntax error), the master
process itself dies. That causes the container to restart,
only to die again shortly afterwards in a loop until the
error is fixed. The container restarts will break any shell
you had opened into the container, as well as any IDE
integrations that connected to that container to access the
files and Python instance.
Getting rid of the custom startup code fixes this and moves
us one small step closer to being a more normal Django
project.
for Open edX operators who still have users with legacy PDF certificates, retirement requires first extracting information from the user's GeneratedCertificate record in order to delete the 4 associated files for each PDF certificate, and then removing the links to the relevant files. this creates a management command to do that work.
After thinking about it, I have removed the update to `status` from this management command, as per the original specification of the ticket. I added it for completeness originally, but was already uncomfortable, because it's not exactly accurate. The `CertificateStatuses` enum does define a `deleted` status:
```
deleted - The PDF certificate has been deleted.
```
but I think it's inappropriate to use here.
#### Why not use `CertificateStatuses.deleted` in the first place
There are multiple places in the code where it's clear that almost all of the statuses are legacy and unused (eg. [Example 1](6c6fd84e53/lms/djangoapps/certificates/data.py (L12-L34)), [Example 2](1029de5537/common/djangoapps/student/helpers.py (L491-L492))). There are innumerable APIs in the system that have expectations about what might possibly be returned from a `GeneratedCertificate.status` object, and none of them is expecting `deleted`
#### Why not revoke the certificate
Ultimately, the certificate isn't revoked, which has a specific meaning around saying it was unearned. The certificate was earned; it has simply been deleted. We should not be kicking off program certificate invalidation, because that's not what's happening. We should be trusting the normal user retirement process to remove/purge PII from any program certificates that might exist. The nature of web certificates simply means that we are going through this process outside of the normal retirement flow. The normal retirement flow can be trusted to implement any certificate object revocation/removal/PII-purging, and doing an extra step outside of that flow is counterproductive.
#### Why not robustly add a flow for `CertificateStatuses.deleted`
When PDF certificates were removed from the system, they weren't removed in their entirety. Instead, we have this vestigial remains of PDF certificates code, just enough to allow learners to display and use the ones that they already have, without any of the support systems for modifying them. Adding a `deleted` status, verifying that all other APIs wouldn't break in the presence of a certificate with that status, adding the signals to process and propagate the change: all of this would be adding more tech debt upon the already existing technical debt which is the PDF certs code. Better to simply add this one necessary data integrity change, and focus on a process which might allow us to eventually remove the web certificates code.
#### Why it is good enough to ignore the status
The original ask was simply to enforce data integrity: to remove links to files that have been deleted, as an indication that they've been deleted. I only added `status` update out of a (misplaced but well-intentioned) completionist urge.
FIXES: APER-3889
The values are nonsensical but this should be a fairly complete collection
of settings, both at the top level and in sub-dictionaries.
While edxapp will not work properly with these settings, the following
commands should work:
```
DJANGO_SETTINGS_MODULE=lms.envs.production LMS_CFG=lms/envs/mock.yml ./manage.py lms dump_settings
DJANGO_SETTINGS_MODULE=cms.envs.production CMS_CFG=cms/envs/mock.yml ./manage.py cms dump_settings
```
Some of our settings depend on the values of other settings. Rather
than explicitly looking up each one in the YAML settings file, we can
simply derive them based on the setting in the YAML file after all the
YAML settings have been loaded.
Part of: https://github.com/openedx/edx-platform/issues/36215
Co-Authored-By: Feanil Patel <feanil@axim.org>
The Python API for declaring derived settings was confusing to the uninitiated
reader, and also prone to spelling mistakes. This replaces the API with one
that is more readable and more concise, and updates the implementation of
`derive_settings` to properly derive settings declared using the new API.
BREAKING CHANGE: The `derived` and `derived_collection_entry` function are
replaced with the `Derived` class. We do not expect those functions to have
been used outside of edx-platform, but if they are, this commit will cause them
to loudly ImportError.
Note that there should be NO change in behavior to the `derive_settings`
function, which we DO know to be used by some external edx-platform plugins.
Part of: https://github.com/openedx/edx-platform/issues/36215
Quoting the README:
> If you are reading this and the 'Maple' Open edX release is out, you
> can safely delete this whole djangoapp. It only exists to hold old
> model migrations and any post-Maple installation will no longer have
> any model in the database for this app.
>
> But for some minor historical context, this djangoapp used to hold
> some integration with sailthru that we no longer needed.
We've released Sumac. This is overdue.
In the near term, we wish to precisely preserve the existing values of
all Django settings exposed by lms/envs/production.py in order to avoid
breaking legacy Django plugins without a proper announcement.
That includes preserving the behavior of these old, redundant dicts:
* ENV_TOKENS
* AUTH_TOKENS
* ENV_FEATURES
* ENV_CELERY_QUEUES
* ALTERNATE_QUEUE_ENVS
Particularly, it means we need to ensure that updates to Django
settings are reflected in these dicts. The most reliable way to do that
is to change the yaml-loading logic so that these values are aliased
to the corresponding values in the global namespace rather than
deep-copied.
Finally, we remove KEYS_WITH_MERGED_VALUES from the global namespace,
and inline the remaining list. We have modified the list (specifically,
we dropped the no-op MKTG_URL_OVERRIDES). Plugins should not be counting
on the value of the list, so we remove it.
This reintroduces commit 15939232d5,
which was reverted due to a typo.
The typo is fixed in the commit immediately following this one.
Co-Authored-By: Feanil Patel <feanil@axim.org>
`notify_credentials` has 2 ways of running.
1. The manual, one-off method which uses `--args_from_database` to specify what should be sent.
2. [The automated method](7316111b35/openedx/core/djangoapps/credentials/management/commands/notify_credentials.py (L157-L159)), which runs on a regular schedule, to catch anything which fell through the cracks.
The automated method does a certain amount of time/date math in order to calculate the entry point of the window based on the current runtime. This is, I assume, why it has some hardcoded logic; it's not at all simple to have a `cron`-run management command running on a regular cadence that can do the same time logic.
```py
if options['auto']:
options['end_date'] = datetime.now().replace(minute=0, second=0, microsecond=0)
options['start_date'] = options['end_date'] - timedelta(hours=4)
```
However, it is not ideal that the actual time window of 4 hours is hardcoded directly into `edx-platform`.
This fix
* creates an overridable `NOTIFY_CREDENTIALS_FREQUENCY` that defaults to 14400 seconds (4 hours).
* pulls that frequency into the quoted code
Using seconds allows maximum flexibility.
FIXES: APER-3383
* fix: catch a possible exception in beta course configuration
when the learner is a beta tester, and the beta test has been set up with a large duration, the courseware djangoapp can return an overflow error on a timedelta call. In those circumstances, this defaults to returning an unmodified date.
FIXES: APER-3848
* feat!: Removing the long-deprecated legacy course_modes chooser
`course_modes/choose.html` (and its corresponding
`_upgrade_button.html`) were specifically only used for the edge case
where an enterprise user found themselves in the non-enterprise learner
dashboard, and attempted to enroll in a course outside of the
enterprise flow. Once upon a time, in a 2U-only workflow, the commerce
system would apply specific discounts for users within the said case.
That's no longer true, and it has never been true outside of this one
company.
Removing this template cleans up a legacy version of a legacy page that
was, realistically, exclusively seen by employees of 2U,
and nobody else.
Removes:
* The corresponding testsfor behavior only seen in the legacy page.
* A waffle flag since all cases route as if the flag is set: `VALUE_PROP_TRACK_SELECTION_FLAG`: `course_modes.use_new_track_selection`
* Some variables set in `CourseModeView` which were only ever rendered in the legacy template (`title_content`, `has_credit_upsell`) have been removed from the class.
* There is a high likelihood that the class is still a target for re-factoring now that the legacy view is gone, but I'm hesitant to touch something which is not covered by previously existing tests, because the logic around what template gets rendered when is complex.
FIXES: APER-3779
FIXES: #36090
