* refactor: moved remaining feature dicts settings into top-level settings.
* refactor: moved remaining feature dicts settings into top-level settings.
* fix: fixed the test files
* fix: fixed tehe pylint errors
* fix: fixation of the cms ci failure
* fix: fixed remaining feature settings for cms
* fix: added fix for requirements
* fix: added fix for lms tests
* fix: resolved the test views issue
* fix: configured views file and test_views
* fix: fixed lint errors and assertion issues
* fix: added fix for base url issue in test view
* fix: added fix for base_url and assertion issue
* fix: added configurations for base utl fix
* fix: handled none issue for mfe config
* fix: corrected override settings in test views
* fix: added getattr defensive technique for view settings
* fix: reverted views and test_views file
* fix: added settings in views file
* fix: added with patch within functions in test view
* fix: rearranged the features in default_legacy_config
* fix: fixing the tests with clearing cache
* fix: reverted test views to verify the CI check
* fix: added cache clear in mfe config test
* fix: fixed the patch toggles to override settings
* fix: fixed the lint errors
* fix: changed patch toggle to override settings
* refactor(certificates): replace direct model imports with data classes and APIs
* fix: use Certificates API to create certificates
* docs: update docstring for get_certificate_for_user
* fix: remove trailing whitespace
---------
Co-authored-by: coder1918 <ram.chandra@wgu.edu>
Co-authored-by: Deborah Kaplan <deborahgu@users.noreply.github.com>
First PR to replace pytz with zoneinfo for UTC handling across codebase.
This PR migrates all UTC timezone handling from pytz to Python’s standard
library zoneinfo. The pytz library is now deprecated, and its documentation
recommends using zoneinfo for all new code. This update modernizes our
codebase, removes legacy pytz usage, and ensures compatibility with
current best practices for timezone management in Python 3.9+. No functional
changes to timezone logic - just a direct replacement for UTC handling.
https://github.com/openedx/edx-platform/issues/33980
In the context of edx-platform django settings,
devstack_docker is an alias to devstack. Both
are deprecated, technically, but we currently
want to remove all devstack-related settings files
except devstack.py (which tutor still uses). So,
in order to remove devstack_docker.py, we update
its references to devstack.py
`notify_credentials` has 2 ways of running.
1. The manual, one-off method which uses `--args_from_database` to specify what should be sent.
2. [The automated method](7316111b35/openedx/core/djangoapps/credentials/management/commands/notify_credentials.py (L157-L159)), which runs on a regular schedule, to catch anything which fell through the cracks.
The automated method does a certain amount of time/date math in order to calculate the entry point of the window based on the current runtime. This is, I assume, why it has some hardcoded logic; it's not at all simple to have a `cron`-run management command running on a regular cadence that can do the same time logic.
```py
if options['auto']:
options['end_date'] = datetime.now().replace(minute=0, second=0, microsecond=0)
options['start_date'] = options['end_date'] - timedelta(hours=4)
```
However, it is not ideal that the actual time window of 4 hours is hardcoded directly into `edx-platform`.
This fix
* creates an overridable `NOTIFY_CREDENTIALS_FREQUENCY` that defaults to 14400 seconds (4 hours).
* pulls that frequency into the quoted code
Using seconds allows maximum flexibility.
FIXES: APER-3383
* feat: removing visible_date-to-creds updates per-cert
The credentials IDA now relies on the course certificate configuration
and (if present) `certificate_available_date` for displayability. We no
longer need to send `visible_date` updates for every awarded certificate
when a course overview changes.
* feat: linting before touching all these files
All these files are old enough, relative either to our current linting
rules or our current linter automation, that modifying anything in them
either makes the linter cranky or wants to reformat the entire file.
Rather than mixing cleanup with code changes, this commit just lints
this set of files to our current standards.
The LMS `certificates` app `README` has been, and the
openedx core `credentials` and `programs` READMEs have been created.
This will clarify not only what the limited responsibilities
of each of these django apps is, but also the way they interact with
each other.
FIXES: APER-2929
[APER-3385]
This PR fixes an existing management command that now has incorrect logic. We have recently done a lot of work to
improve certificate-related date business logic to fix data inconsistencies between systems. Instead of maintaining
separate and duplicated logic for sending date data to Credentials, instead we can use an existing (and tested)
Celery task that will determine and send the correct date to the Credentials IDA.
Additionally, the original version of this management command skipped self-paced courses completely. This is no
longer the case as we _know_ that there are self-paced courses that have been associated with a certificate available
date because of bugs in the product. This management command will serve as a means to do a mass data fixup for data stored by the Credentials IDA.
This PR removes the `clean_certificate_available_date` management command. Recently, a change was made to sync the end date of a course run with a display behavior of "end" as its certificate available date in the Credentials IDA. This change was made to resolve a long running issue with the Credentials IDA not displaying certificates to learners correctly on learner records when the display behavior is set to "end".
This management command, if run, would clear the certificate available date from the course certificate configurations in the Credentials IDA. This would be a regression based on the new functionality. Other improvements have been made to ensure that updates made in the LMS properly flow to Credentials.
TL;DR -- this management command is no longer needed because of feature changes in edx-platform. Allowing this management command to run would break functionality for learners.
[APER-3229]
In a previous PR, I created a new utility function named `is_credentials_enabled()` that can be used to determine if use of the Credentials IDA is enabled by config in an Open edX instance.
This PR is some additional cleanup that replaces the direct import and use of the `CredentialsApiConfig` model with the new utility function.
I took some additional time to update some existing log messages to include more info while reducing our need to log PII. I've removed as much use of a learner's username as possible, replacing it with logging the learner's LMS User Id instead.
[APER-3229]
The monolith and the Credentials IDA keep independent records of a course runs certificate availability/visibility preferences. This PR aims to improve the communication between the monolith and the Credentiala IDA to keep the availability date/preference in sync with the monoliths records.
The current code is too strict and actually prevents valid updates in some configurations.
Additionally, the Credentials IDA doesn't understand the concept of "course pacing" (instructor-paced vs self-paced) and has troubles with courses with an availability date of "end". Instead of having to add the concept of course pacing (and syncing more data between the two systems), this PR proposes sending the end date of a course as the "certificate available date" to Credentials.
This way, the Credentials IDA can manage the visibility of awarded credentials in a course run with a display behavior of "end" using the existing feature set and models of the Credentials service.
To determine whether or not we need to revoke any program certificates, we first run get_revokable_program_uuids. This calls get_certified_programs, which calls get_credentials, which uses the OpenEdx Core utility method get_api_data. get_api_data makes the API call inside a try block, does raise_for_status also inside the try block, and then catches the exception, logs it, and returns an empty result to the caller.
This means that on a failure to call the credentials API, get_credentials can’t tell the difference between a failure to hit the API (because credentials is, as it sometimes is during a notify_programs run, overloaded), or a learner with no program certificates. In this particular case, this is absolute failure, incorrect behavior.
* Adds a new flag, `raise_on_error` which will make `get_api_data` log the exception and then re-raise the HTTPError, defaulting to false in order to avoid changing the behavior on any other callers
* Also: my editor reformatted all of the touched files to our modern code standards, and it seemed appropriate to let it do that.
* Also: added type hints in some cases, because they helped me write the code and debug. Our test suite definitely reports mypy results on type errors so we are verifying that hints are correct.
FIXES: APER-3146
[APER-2504]
This is a companion to PR #32458. This updates the `notify_credentials` management command and adds an additional argument/switch (`--revoke_program_certs`).
If included, this option will be converted to a boolean and passed as a script option. Eventually, the `send_notifications` function (updated in the previously mentioned PR) will determine if we should fire a signal that checks if any program certs need to revoked.
[APER-2504]
This PR adds additional functionality to the tasks kicked off when the `notify_credentials` management command is run.
I have added a new keyword arg (revoke_program_certs) that, if True, will check to see if we need to revoke a program certificate. This functionality was introduced to help automate fixing cases where a learner still has access to a Program Certificate even if they have been awarded all of the course certificates in a Program (see APER-2499).
The functionality has to be committed in two separate PRs because of our blue/green deployments. The task changes will come first, then we will update the management command to be able to set/pass the new settings. New settings were added as keyword args (defaulting to False) in order to ensure that we won't trip up our workers.
[APER-2240]
The monolith no longer needs to understand how to build URLs to the Learner Record MFE. The Credentials IDA has logic to determine if (and how) a request should be redirected to this MFE. This PR remove unused settings from edx-platform.
We have already removed the use of these settings in #31672.
[APER-2240]
The monolith no longer needs to understand how to build URLs to the Learner Record MFE. The Credentials IDA has logic to determine if (and how) a request should be redirected to the MFE so we can remove these changes. Another PR later will remove the unused settings.
[APER-1968]
We don't have a good way to understand if grade data in Credentials is out of sync with the LMS. Grades are sent to Credentials via a REST API call originating from an asynchronous Celery task on the LMS side. This PR updates our Celery task `send_grade_to_credentials` to include sending the `modified` DateTime value of a grade record to the Credentials IDA. Updates will be made on the Credentials side to accept and store this data as part of the UserGrade instance.
* Updates the `send_grade_to_credentials` task to include passing the grade's `modified` DateTime info as part of the request data to Credentials
* Updates the `CourseGradeBase` class to include an optional `last_updated` field. This will store the `modified` date of a PersistentCourseGrade instance when a grade is read through the CourseGradeFactory.
* Update existing log statement to use format strings where possible.
[APER-1922]
We are converting the legacy UI of the `records` app in the Credentials IDA (credentials.edx.org/records/, credentials.edx.org/records/programs/{uuid}, etc.) to a new MFE. Today, the Program Dashboard and the legacy (non-MFE) profile page have buttons that route learners to the Credentials IDA pages. We need to (optionally) introduce a way to route learner's to the new MFE instead.
- Introduces a new configuration setting called `LEARNER_RECORD_MICROFRONTEND_URL` (defaulting to None). This will be used by the LMS to store the base URL of the new MFE (e.g. records.stage.edx.org).
- Introduces a new waffle switch named `USE_LEARNER_RECORD_MFE`. This will be used to control whether routing learner's to the new MFE is enabled from the LMS's side.
- Updates the existing `get_credentials_records_url` function to add additional logic that will determine if we need to build a link to the legacy FE or the new MFE
- Adds tests for new and existing behavior. There were no existing unit tests for the utility function that I updated.
Now that we're actively using the LMS_USER_ID inside credentials to
identify users, we need to make sure that users created by
notify_credentials are including it in the jwt scopes when authenticated
with credentials.
We use the `notify_credentials` management command to keep certificate-
related data in the LMS and Credentials service in sync. We can run it
with specific arguments (user_ids, course_keys, etc.) when we notice a
data discrepancy; and it is run regularly by a Jenkins job with the
`--auto` flag every ~4 hours to keep things up-to-date.
Because we probably never want to notify credentials of of ALL the
GeneratedCertificates, the celery task must be given some arguments
to filter down to the relevant certificates. Running the management
command with the `--auto` flag (as the Jenkins job does) adds
`start_date` and `end_date` arguments of 4 hours ago and now,
respectively.
The handle_notify_credentials celery task then takes those arguments and
looks for any GeneratedCertificates that have been modified within the
given time range by checking the GeneratedCertificate modified_date.
It will send the current data for those certificates to credentials.
However, we also want to notify credentials about certificates that have
an associated CertificateDateOverride that has changed within that time
range: added, updated, or deleted. But changes to a
CertificateDateOverride won’t affect the GeneratedCertificate’s modified
date, and therefore wouldn’t be included in the list of certs cent to
credentials.
This commit adds a check for changed CertificateDateOverrides and
includes their associated GeneratedCertificates in the list of certs. We
use the CertificateDateOverride’s history model for this check so that
we can include certificates whose override was deleted.
MICROBA-1489
* [fix] Certificate availability date cleanup
This task will clean out the misconfigured certificate available date. When courses Change their
certificates_display_behavior, the certificate_available_date was not updating properly. This is
command is meant to be ran one time to clean up any courses that were not supposed to have
certificate_available_date
* build: Removed the diff-quality step
Applied lint-amnesty on all the warnings
Removed pylint thresholds comparison code and related tests
Co-authored-by: Usama Sadiq <usama.sadiq@arbisoft.com>
[refactor] change backfill date task to long running task
Change from a task that spins off multiple other tasks to a single long
running task. This will hopefully be way less heavy when this is ran in
production.
* [refactor] Move backfill_date_for_all_course_runs to Tasks.py
In order for celery to register this task, it needs to be in the right
file. duh!
* fix imports
* fix more imports
* [feat] Management command to populate credentials availability date
This is a command to populate the new CredentialsCertificate model's
available_date for every existing course_run.
