* feat: removing visible_date-to-creds updates per-cert
The credentials IDA now relies on the course certificate configuration
and (if present) `certificate_available_date` for displayability. We no
longer need to send `visible_date` updates for every awarded certificate
when a course overview changes.
* feat: linting before touching all these files
All these files are old enough, relative either to our current linting
rules or our current linter automation, that modifying anything in them
either makes the linter cranky or wants to reformat the entire file.
Rather than mixing cleanup with code changes, this commit just lints
this set of files to our current standards.
[APER-3241]
This PR updates the retirement pipeline to purge learners' names from certificate records when their account is being retired.
It also introduces a new management command that can be used by Open edX operators to purge the leftover name data (PII data) from the `certificates_generatedcertificate` table. This is designed as a one-time use data fixup, as the retirement functionality should clean this moving forward.
* feat: receiver for invalidate certificate
- consumes event of exam attempt rejected
- initial commit, need to make tests
* temp: moving consumer from signals to handlers.py
- Still need to make this work
- Need to make tests work too
* feat: refactored underlying code to api.py
- tests still need to be tweaked
* fix: commit history
* fix: improve api func name + add source param
**Previously**
When a course administrator changed the `certificates_display_behavior` (presumably to `end_with_date`) AND set the `certificate_available_date` in Studio, the `certificate_available_date` was not syncing to Credentials.
This was because we chose to send the `certificate_available_date` only if the course is self-paced and the `certificate_display_behavior` is set to `end_with_date`. [See PR #28275](https://github.com/openedx/edx-platform/pull/28275). However, we were checking those two conditions by looking at the relevant `CourseOverview`, which was not yet truly saved to reflect the updated display behavior at the time of the check due to atomic requests. [Read more about atomic requests and transactions here](https://docs.djangoproject.com/en/4.0/topics/db/transactions/#tying-transactions-to-http-requests-1); we have `ATOMIC_REQUESTS` set to `TRUE` in our codebase. Because the `certificate_display_behavior` was not (yet) `end_with_date`, the post to Credentials was not being fired.
**Solution**
To fix, this commit sends the `COURSE_CERT_DATE_CHANGE` signal `on_commit` instead, which waits until the transaction has completed and the update to the `CourseOverview` has been truly applied to the database. [Read more about `on_commit` here](https://docs.djangoproject.com/en/4.0/topics/db/transactions/#django.db.transaction.on_commit). Now, when the relevant `CourseOverview` is read, it will have the updated `certificate_display_behavior`.
See the [Django docs for how to test on_commit callbacks here](https://docs.djangoproject.com/en/3.2/topics/testing/tools/#django.test.TestCase.captureOnCommitCallbacks); this seems to be our first time using the built-in method.
This commit also cleans up some previous code that was meant to get around the problem caused by atomic requests, that is now unneccessary with this fix. It essentially reverses the work done in [PR #26991](https://github.com/openedx/edx-platform/pull/26991): we no longer need to explicitly pass the `certificate_available_date` since we can trust the `CourseOverview` to be properly updated.
**Rejected Solutions**
A. Simply publish the `COURSE_CERT_DATE_CHANGE` signal `on_commit`; no other changes. Rejected because: This would fix the problem, but leaves a lot of unnecessary code and some puzzling inconsistencies. I prefer the solution above because we are cleaning up behind ourselves.
B. Pass the new `certificate_display_behavior` along with the `certificate_available_date`; read those direclty instead of checking the (not-yet-properly-updated) `CourseOverview`. Rejected because: The pattern of passing the new `certificate_available_date` down through all these methods was put in place to get around the atomic requests problem. I believe `on_commit` to be a better solution to getting around that problem. I’d like to move away from passing data down through several functions / methods.
C. Start the celery task `on_commit` (rather than send the signal `on_commit`). Rejected because: The signal receiver basically only starts the celery task, and I find the break to be a bit more readable when sending the signal. No need to split hairs here.
D. Remove the check for pacing and display behavior; send the updated `certificate_available_date` every time there is a change, no matter what the current display behavior is. Rejected because: We intentionally added this check in [PR #28275](https://github.com/openedx/edx-platform/pull/28275) because the task was not behaving as expected without it (specifically around self-paced courses). I assume this is still necessary.
**Relevant Prior Work**
The following PRs--in order--show how this section (and other relevant sections) of the code have been changed over time:
1. [Move cert date signals to avoid race conditions #26841](https://github.com/openedx/edx-platform/pull/26841)
2. [feat: Pass date in cert date update signal #26991](https://github.com/openedx/edx-platform/pull/26991)
3. [Fix certificate available date sync #28275](https://github.com/openedx/edx-platform/pull/28275)
4. [fix: Correct an issue where cert available date was not sent to Crede… #28524](https://github.com/openedx/edx-platform/pull/28524)
MICROBA-1818
We use the `notify_credentials` management command to keep certificate-
related data in the LMS and Credentials service in sync. We can run it
with specific arguments (user_ids, course_keys, etc.) when we notice a
data discrepancy; and it is run regularly by a Jenkins job with the
`--auto` flag every ~4 hours to keep things up-to-date.
Because we probably never want to notify credentials of of ALL the
GeneratedCertificates, the celery task must be given some arguments
to filter down to the relevant certificates. Running the management
command with the `--auto` flag (as the Jenkins job does) adds
`start_date` and `end_date` arguments of 4 hours ago and now,
respectively.
The handle_notify_credentials celery task then takes those arguments and
looks for any GeneratedCertificates that have been modified within the
given time range by checking the GeneratedCertificate modified_date.
It will send the current data for those certificates to credentials.
However, we also want to notify credentials about certificates that have
an associated CertificateDateOverride that has changed within that time
range: added, updated, or deleted. But changes to a
CertificateDateOverride won’t affect the GeneratedCertificate’s modified
date, and therefore wouldn’t be included in the list of certs cent to
credentials.
This commit adds a check for changed CertificateDateOverrides and
includes their associated GeneratedCertificates in the list of certs. We
use the CertificateDateOverride’s history model for this check so that
we can include certificates whose override was deleted.
MICROBA-1489
If the certificate has an associated certificate date override, display
that date on the certificate instead of any other date. The date
override should not affect whether or not the certificate is visible /
available; only the date displayed on the certificate.
* refactor: Merge the openedx certs app with lms one
Move the certs API from openedx into the lms certificates app.
Functionally, this is a no-op. Cleanup will happen in a subsequent
commit. This is simply a move.
[MICROBA-1354]
* Update Python API function named `cert_generated_enabled` to `has_self_generated_certificates_enabled` to more accurately reflect the purpose of the function
[MICROBA-1087]
[DEPR-155]
* Remove `generate_example_certificates` functionality
* Adjust Instructor dashboard slightly to prevent people from clicking the `Generate Example Certificates` button, remove form/code that called the `generate_example_certificates` endpoint.
feat: reimagine certificate display settings
The course settings `certificate_available_date` (CAD) and
`certificates_display_behavior` (CDB) were previously
acting indedependantly of one another. They now work in
tandem. This change:
- limits CDB to a dropdown
- removes "early_with_info" and adds "end_with_date"
- only takes CAD into account if "end_with_date" is selected
- Moves CDB to the main course schedule settings page
- updates CourseOverview model and CourseDetails objects to
validate these fields and choose sane defaults if they aren't
expected values
This work was previously done in bd9e7dd (complete with bugs), so this
version is toggleable via the ENABLE_V2_CERT_DISPLAY_SETTINGS setting
[MICROBA-1227]
[DEPR-155]
* remove queue.py (and associated unit tests)
* remove references to `queue.py` from functions in api.py (also update unit tests)
The `may_certify` function is used to determine whether a learner should
be able to see that their certificate is ready. It is therefore entirely
in the certificates domain. This remove may_certifiy and
may_certify_for_course from the xmodule and course_overview
apps and into the certificates app. The xmodule may_certify was not called
outside of tests prior to this, and the other's were easily moved.
In order to avoid circular imports, this also moved
certificate_info_for_user out of the model code and into an API
[MICROBA-1289]
* Remove use of the CourseOverview model directly in the Certificates app
* Introduce a few new Python API functions in the `course_overview` and `catalog` django apps to replace functionality in the Certificates app.
The course settings `certificate_available_date` (CAD) and
`certificates_display_behavior` (CDB) were previously acting indedependantly
of one another. They now work in tandem. This change:
- limits CDB to a dropdown
- removes "early_with_info" and adds "end_with_date"
- only takes CAD into account if "end_with_date" is selected
- Moves CDB to the main course schedule settings page
- updates CourseOverview to validate these fields and choose sane
defaults if they aren't expected values
Certificates will now show under the following circumstances:
"Immediately upon passing"
certificate_availability_date = null
certificates_display_behavior = "early_no_info"
"End date of course"
certificate_availability_date = null
certificates_display_behavior = "end"
"A date after the course end date"
certificate_availability_date = <date>
certificates_display_behavior = "end_with_date"
[MICROBA-1307]
Before this change a user would not be auto refunded if they had a
certificate in a course with any status. This had unintended
consequences. This change updates the logic to only block auto refund
for statuses that we do not want to refund on such as downloadable.
[MICROBA-1075]
- Adds a new `edx.certificate.revoked` event to the LMS.
- Refactor of our certificate revocation functions in the GeneratedCertificate model.
This new event will be emit when a GeneratedCertificate with the status of `downloadable` is revoked (through the `invalidate(), mark_notpassing(), or mark_unverified() functions). Event will have a `source` field that will allow us how our certificates are being revoked from learners and can be broken down in the following way:
*Invalidate*
- allowlist_removal
- certificate_generation
- certificate_regeneration
- certificate_service
- certificate_invalidation_list
- bulk_certificate_regeneration
*mark_notpassing*
- certificate_generation
- notpassing_signal
*unverified*
- certificate_generation
[MICROBA-678]
Added cert availability date to the API used to get certificate status
by the learning MFE to support updated messaging.
Updated the cert availability messaging in the Coruse Dashboard,
including a text color change.
[MICROBA-1238]
* remove unused `course` argument from `generate_user_certificates` function in the certificates app
* remove unused `course` argument from `regenerate_user_certificates` functioni n the certificates app
* remove `course` argument if passed in edx-platform apps outside of the certificates app
[MICROBA-1238]
- Remove unused `course` arguments from certificate generation functions
Part of our cleanup after removing use of the modulestore from the Certificates app.
[MICROBA-1208]
* consolidate `has_html_certificates_enabled` and `has_html_certificates_enabled_from_course_overview`, the latter of the two functions was created for use during our transition away from using the modulestore in the certificate app.
[MICROBA-1077]
* Skip bulk certificate invalidation during bulk regeneration. I couldn't come up with a reason to continue to invalidate certificates right before we attempted regeneration.
[MICROBA-1025]
- Update management command to use the same logic that the Instructor Dashboard uses
- Fix bug in management command where processing stopped when encountering a user that did not exist
- Add more logging
- Add and update tests where needed
[MICROBA-1024]
- Move some of the recently added logic from the instructor app to the certificates app
- Attempt to not use other certificate models directly in the code I am touching, moving this logic to certificates as well.
