[APER-3385]
This PR fixes an existing management command that now has incorrect logic. We have recently done a lot of work to
improve certificate-related date business logic to fix data inconsistencies between systems. Instead of maintaining
separate and duplicated logic for sending date data to Credentials, instead we can use an existing (and tested)
Celery task that will determine and send the correct date to the Credentials IDA.
Additionally, the original version of this management command skipped self-paced courses completely. This is no
longer the case as we _know_ that there are self-paced courses that have been associated with a certificate available
date because of bugs in the product. This management command will serve as a means to do a mass data fixup for data stored by the Credentials IDA.
This PR removes the `clean_certificate_available_date` management command. Recently, a change was made to sync the end date of a course run with a display behavior of "end" as its certificate available date in the Credentials IDA. This change was made to resolve a long running issue with the Credentials IDA not displaying certificates to learners correctly on learner records when the display behavior is set to "end".
This management command, if run, would clear the certificate available date from the course certificate configurations in the Credentials IDA. This would be a regression based on the new functionality. Other improvements have been made to ensure that updates made in the LMS properly flow to Credentials.
TL;DR -- this management command is no longer needed because of feature changes in edx-platform. Allowing this management command to run would break functionality for learners.
[APER-2504]
This is a companion to PR #32458. This updates the `notify_credentials` management command and adds an additional argument/switch (`--revoke_program_certs`).
If included, this option will be converted to a boolean and passed as a script option. Eventually, the `send_notifications` function (updated in the previously mentioned PR) will determine if we should fire a signal that checks if any program certs need to revoked.
* [fix] Certificate availability date cleanup
This task will clean out the misconfigured certificate available date. When courses Change their
certificates_display_behavior, the certificate_available_date was not updating properly. This is
command is meant to be ran one time to clean up any courses that were not supposed to have
certificate_available_date
* build: Removed the diff-quality step
Applied lint-amnesty on all the warnings
Removed pylint thresholds comparison code and related tests
Co-authored-by: Usama Sadiq <usama.sadiq@arbisoft.com>
* [refactor] Move backfill_date_for_all_course_runs to Tasks.py
In order for celery to register this task, it needs to be in the right
file. duh!
* fix imports
* fix more imports
* [feat] Management command to populate credentials availability date
This is a command to populate the new CredentialsCertificate model's
available_date for every existing course_run.
* [feat] Added program_uuids to notify_credentials mgmt cmd
[MICROBA-951]
To support updating a users credentials in the Credentials services for
all users enrolled in a program we have added a command line argument to
the notify_credentials command called program_uuids. This supports a
list of program uuids. It will retrieve all course runs in the listed
programs and update the related credentials data.
* updated comments
Before this change, the notify_credentials management command ran for
about 10-15 minutes and occasionally overlapped with an edxapp
deployment which would cause the machine the command was running
on to die. The command ran through a bunch of certs and grades, and then
spun off celery tasks to send the relevant data to credentials.
Now, most of the logic of the notify_credentials management command has
been moved into a new celery task, so that combing through longs lists
of certs and grades is done asynchronously. This task then spins off the
original data-sending tasks as before.
Note that this requires a change in alerting: where the jenkins job
would previously notify us about failures, we need to know when this
new celery task fails for any reason. We’ve made this a LoggedTask
so that failures will be logged to Splunk and alerting can be built off
of the error messages.
This commit also moves the relevant tests.
MICROBA-963
[MICROBA-951]
When running the notify_credentials command locally there is an
OperationalError thrown by MySQL when there are no more results
available for the queryset iterator. This change catches that exception,
checks state, and then logs according to that state. This code runs in
production without issue. This changes allows for the code to be run
locally without a separate code path.
Adds a command to create an API connection to credentials for testing
program certificates on devstack. This command is not meant to be ran
manually, and will be included in a provisioning type script that will
be added later.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
** This removes some of the extra
celery calls to credential service for
updating grade when a certificate is updated.
** This removes extra db calls
by reusing existing queryset values.
PROD-1363
This stage does the following:
- Includes a data migration to copy the values from old to new field.
- Changes business logic to switch to using new field.
- Deletes all code references of the old field.
This reverts commit 84de6bc6de
which reverts commit cebeab4348
which implements the first stage of the `values` column rename in
SiteConfiguration. However, I included a small change:
This time, we set a default value on the new `site_values` column so
that the ORM will happily deserialize the JSONField without throwing a
JSONDecodeError.
notify_credentials job when executed without
any restriction consumes a lot of memory
because of huge dataset that makes
it unsuable to be executed periodically.
This patch would time bound it to only work
on last day dataset.
PROD-1190
To save the memory get cert and grade object directly from
the db because by doing that we'll override the queryset
evaluation and object won't be cached once it has gone out
of scope.
PROD-987
