Fixes an issue where bulk create was only batching writes. With a sufficiently large input the users queryset would fail to evaluate due to the size of the filter list sent to the db.
The verify_student Django app contains a Signal receiver that receives the SoftwareSecurePhotoVerification post_save signal. It then emits an idv_update_signal to communicate that a change to IDV has occured. This Signal is received by the nameaffirmation app, which uses it to start a Celery task to create or update VerifiedName records based on the changes to the SoftwareSecurePhotoVerification model.
During the phased roll out of the Verified Name feature, due to the way percentage rollout is implemented by django-waffle and the way SoftwareSecurePhotoVerifications are updated, a mismatch of states occured where created VerifiedNames stayed in the "pending" state, even as the corresponding SoftwareSecurePhotoVerifications moved into "submitted", "approved", or "denied". Please see below for additional details.
This management commands takes as an argument a list of SoftwareSecurePhotoVerification IDs verification-ids, as well as a batch-size and sleep-time. In batches of batch-size, the command saves the SoftwareSecurePhotoVerification associated with the IDs argument. Each batch is separated by a pause of sleep_time in seconds.
By saving each SoftwareSecurePhotoVerification, the post_save signal is re-emitted, thereby re-emitting the idv_update_signal. Now that the aforementioned bug has been fixed, this will correctly trigger the Celery task and sync the SoftwareSecurePhotoVerification and VerifiedName objects.
Please find additional details about the bug below.
The release reached a percentage of 50% before it was observed that, due to the way percentage roll out works in django-waffle, the code to create or update VerifiedName records was not working properly. The code was written such that any change to a SoftwareSecurePhotoVerification model instance sent a signal, which was received and handled by the Name Affirmation application. If the VERIFIED_NAME_FLAG was on for the requesting user, a Celery task was launched from the Name Affirmation application to perform the creation of or update to the appropriate VerifiedName model instances based on the verify_student application signal. However, we observed that when SoftwareSecurePhotoVerification records were moved into the "created" or "ready" status, a Celery task in Name Affirmation was created, but when SoftwareSecurePhotoVerification records were moved into the "submitted" status, the corresponding Celery task in Name Affirmation was not created. This caused VerifiedName records to stay in the "pending" state.
The django-waffle waffle flag used by the edx-toggle library implements percentage rollout by setting a cookie in a learner's browser session to assign them to the enabled or disabled group.
It turns out that the code that submits a SoftwareSecurePhotoVerification record, which moves it into the "submitted" state, happens as part of a Celery task in the verify_student application in the edx-platform. Therefore, we believe that because there is no request object in a Celery task, the edx-toggle code is defaulting to the case where there is no request object. In this case, the code checks whether the flag is enabled for everyone when determining whether the flag is enabled. Because of the percentage rollout (i.e. waffle flag not enabled for everyone), the Celery task in Name Affirmation is not created. This behavior was confirmed by logging added as part of https://github.com/edx/edx-name-affirmation/pull/62.
[MST-1130](https://openedx.atlassian.net/browse/MST-1130)
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
Created helper function to render url dynamically
Modify helper function to include reverify links
Fix code based on failing tests
Fixed query string in redirect URL
1. Created a new celery queue with key `SOFTWARE_SECURE_VERIFICATION_ROUTING_KEY`.
2. Added a celery task with retry logic.
3. sorted imports with isort.
4. Changed deprecated `log.warn` => `log.warning`.
We are seeing an error in the send_verification_expiry_email job
because one verification model instance had a null expiry_date but
a non-null expery_email_date.
This makes us more robust to that odd data and makes the job more
robust by having it still send other emails out even if one fails.
AA-70
We sometimes update preexisting SAML SSO providers to configure them
to automatically create SSO identity verification (IdV) records when a
learner links an account via that provider. Turning that configuration
from off to on does make it such that when learners log back in via
their linked account, a new IdV record will be created for them. But
it's possible we'd want this process to happen more automatically and
seamlessly, for which this management command will be helpful.
Note that this does not help with removing SSO verification records
for a provider for which this configuration has been turned off.
JIRA:EDUCATOR-4947
The MockS3Mixin prevents the correct setup of the ModuleStoreTestCase
and made this test fail. Since the fix for this wasn't trivial, this
test was skipped on python 3, and now is removed.
There were cases where we needed to encode things to codecs other than
ascii. In these cases, python returns byte strings but we needed them
to be unicode so that they serialize correctly later when we combine
them with other unicode strings.
If already DEFAULT Number of emails are sent, then verify that user
is enrolled in at least one verified course run for which the course
has not ended else stop sending emails
LEARNER-7313
Move command parameter arguments, resend-days and days-range, to
common settings. This will help in creating a consistency when the
default values are changed in the future.
LEARNER-7313
datetime.now(UTC) gives unexpected results. Using "now()" function
from django.utils.timezone tends to be a much more reliable way to
get a time zone aware timestamp for the current time.
PROD-190
Template for email added (email/verification_expiry_email)
The management command filters the SoftwareSecurePhotoVerification model on the basis of following criteria :
-- the verification is approved
-- the start_date < expiry_date < today or specified days have passed to resend email
After the basic filtering batches are created to send email. For each verification in a batch email is sent and
email_expiry_date is set to 15 days from today (default days are 15, it can be changed too)
Between each batch there is a delay of sleep_time seconds
Method approve() is overridden to accommodate new users that will have their verification approved in the future.
The approve() method is called from software secure results callback every time a Software Secure Photo Verification
is approved.The method overridden now updates expiry_date and then calls super to perform task it was performing
before this change.The expiry_date set is the same as the one sent in verification approval email.
In results_callback for sspv the expiry_date and expiry_email_date for the most recent previous verification if exists
are also updated to NULL.
