Commit Graph

23342 Commits

Author SHA1 Message Date
Zachary Hancock
abf1f21668 fix: bulk verification creation failure for large input (#29197)
Fixes an issue where bulk create was only batching writes. With a sufficiently large input the users queryset would fail to evaluate due to the size of the filter list sent to the db.
2021-11-02 17:45:43 -04:00
M. Zulqarnain
1003f262ad feat: Django codemods on LMS (1) (#28851) 2021-11-02 17:00:46 +05:00
Maxim Beder
eabd6e8019 feat: add anonymous settings to discussions api (#28981)
If course allows anonymous posts, and user is an author of a post, add
`anonymous` to posts' editable fields.
If course allows anonymous to peers posts, and user is an author of a
post, add `anonymous_to_peers` to posts' editable fields.

Course endpoint response to get request will now include course's
`allow_anonymous` and `allow_anonymous_to_peers` settings.

Added `anonymous` and `anonymous_to_peers` fields to the content
serializer, from which ThreadSerializer and CommentSerializer inherit.
Both fields have a default value of False, because there are cases where
course configuration doesn't allow them to be initialized (if a course
doesn't allow anonymous posts, the fields won't be included in the list
of initializable/editable fields).
2021-11-02 11:25:40 +05:00
Binod Pant
60da5db4b6 feat: degreed2 integrated channels (#29183)
* feat: degreed2 integrated channels

ENT-2789

* feat:  New integrated channel via edx-enterprise

* fix: pull in edx-enterprise 3.33.1

fixes db_overrides check failure by renaming field `key` to `client_id`
2021-11-01 21:18:59 -04:00
jawad khan
9c1fc4cbba feat: LEARNER-8493 Remove upgrade banner from mobile upgrade banner (#29160)
* feat: remove upgrade banner from mobile upgrade banner

After investigating the best way to keep a user logged in, and solve rendering issues, we decided to show the unit page in a full webview within the app. However, in order to comply with app store guidelines, we need to remove the upgrade banner from this page.

LEARNER-8493
2021-11-02 04:47:48 +05:00
Michael Roytman
b953ef044e Merge pull request #29155 from edx/mroytman/MST-1130-management-command-cleanup
feat: Add Management Command to Re-Emit SoftwareSecurePhotoVerification post_save Signal
2021-11-01 15:01:49 -04:00
Michael Roytman
7ae8ed37a3 Merge pull request #29136 from edx/mroytman/MST-1130-remove-verified-name-waffle-flag
Remove Use of VERIFIED_NAME_FLAG Waffle Flag and is_verified_enabled Utility
2021-11-01 14:29:51 -04:00
kenclary
a9b0570d95 Merge pull request #29172 from edx/kclary/coursegraph
fix: move coursegraph celery tasks to cms (from lms), and on a non-default, non-high-priority queue.
2021-11-01 14:06:45 -04:00
michaelroytman
bb299c9521 feat: Remove Use of VERIFIED_NAME_FLAG Waffle Flag and is_verified_enabled Utility
The VERIFIED_NAME_FLAG, the VerifiedNameEnabledView, and the verified_name_enabled key removed from responses for both VerifiedNameView view and VerifiedNameHistoryView
were removed as part https://github.com/edx/edx-name-affirmation/pull/12. This was released in version 2.0.0 of the edx-name-affirmation PyPI package. Please see below for additional context for the removal, copied from the name-affirmation commit message.

The VERIFIED_NAME_FLAG was added as part https://github.com/edx/edx-name-affirmation/pull/12, [MST-801](https://openedx.atlassian.net/browse/MST-801) in order to control the release of the Verified Name project. It was used for a phased roll out by percentage of users.

The release reached a percentage of 50% before it was observed that, due to the way percentage roll out works in django-waffle, the code to create or update VerifiedName records was not working properly. The code was written such that any change to a SoftwareSecurePhotoVerification model instance sent a signal, which was received and handled by the Name Affirmation application. If the VERIFIED_NAME_FLAG was on for the requesting user, a Celery task was launched from the Name Affirmation application to perform the creation of or update to the appropriate VerifiedName model instances based on the verify_student application signal. However, we observed that when SoftwareSecurePhotoVerification records were moved into the "created" or "ready" status, a Celery task in Name Affirmation was created, but when SoftwareSecurePhotoVerification records were moved into the "submitted" status, the corresponding Celery task in Name Affirmation was not created. This caused VerifiedName records to stay in the "pending" state.

The django-waffle waffle flag used by the edx-toggle library implements percentage rollout by setting a cookie in a learner's browser session to assign them to the enabled or disabled group.
It turns out that the code that submits a SoftwareSecurePhotoVerification record, which moves it into the "submitted" state, happens as part of a Celery task in the verify_student application in the edx-platform. Therefore, we believe that because there is no request object in a Celery task, the edx-toggle code is defaulting to the case where there is no request object. In this case, the code checks whether the flag is enabled for everyone when determining whether the flag is enabled. Because of the percentage rollout (i.e. waffle flag not enabled for everyone), the Celery task in Name Affirmation is not created. This behavior was confirmed by logging added as part of https://github.com/edx/edx-name-affirmation/pull/62.

We have determined that we do not need the waffle flag, as we are comfortable that enabling the waffle flag for everyone will fix the issue and are comfortable releasing the feature to all users. For this reason, we are removing references to the flag.

[MST-1130](https://openedx.atlassian.net/browse/MST-1130)
2021-11-01 13:33:55 -04:00
michaelroytman
7ffdae90be feat: Add Management Command to Re-Emit SoftwareSecurePhotoVerification post_save Signal
The verify_student Django app contains a Signal receiver that receives the SoftwareSecurePhotoVerification post_save signal. It then emits an idv_update_signal to communicate that a change to IDV has occured. This Signal is received by the nameaffirmation app, which uses it to start a Celery task to create or update VerifiedName records based on the changes to the SoftwareSecurePhotoVerification model.

During the phased roll out of the Verified Name feature, due to the way percentage rollout is implemented by django-waffle and the way SoftwareSecurePhotoVerifications are updated, a mismatch of states occured where created VerifiedNames stayed in the "pending" state, even as the corresponding SoftwareSecurePhotoVerifications moved into "submitted", "approved", or "denied". Please see below for additional details.

This management commands takes as an argument a list of SoftwareSecurePhotoVerification IDs verification-ids, as well as a batch-size and sleep-time. In batches of batch-size, the command saves the SoftwareSecurePhotoVerification associated with the IDs argument. Each batch is separated by a pause of sleep_time in seconds.

By saving each SoftwareSecurePhotoVerification, the post_save signal is re-emitted, thereby re-emitting the idv_update_signal. Now that the aforementioned bug has been fixed, this will correctly trigger the Celery task and sync the SoftwareSecurePhotoVerification and VerifiedName objects.

Please find additional details about the bug below.

The release reached a percentage of 50% before it was observed that, due to the way percentage roll out works in django-waffle, the code to create or update VerifiedName records was not working properly. The code was written such that any change to a SoftwareSecurePhotoVerification model instance sent a signal, which was received and handled by the Name Affirmation application. If the VERIFIED_NAME_FLAG was on for the requesting user, a Celery task was launched from the Name Affirmation application to perform the creation of or update to the appropriate VerifiedName model instances based on the verify_student application signal. However, we observed that when SoftwareSecurePhotoVerification records were moved into the "created" or "ready" status, a Celery task in Name Affirmation was created, but when SoftwareSecurePhotoVerification records were moved into the "submitted" status, the corresponding Celery task in Name Affirmation was not created. This caused VerifiedName records to stay in the "pending" state.

The django-waffle waffle flag used by the edx-toggle library implements percentage rollout by setting a cookie in a learner's browser session to assign them to the enabled or disabled group.
It turns out that the code that submits a SoftwareSecurePhotoVerification record, which moves it into the "submitted" state, happens as part of a Celery task in the verify_student application in the edx-platform. Therefore, we believe that because there is no request object in a Celery task, the edx-toggle code is defaulting to the case where there is no request object. In this case, the code checks whether the flag is enabled for everyone when determining whether the flag is enabled. Because of the percentage rollout (i.e. waffle flag not enabled for everyone), the Celery task in Name Affirmation is not created. This behavior was confirmed by logging added as part of https://github.com/edx/edx-name-affirmation/pull/62.

[MST-1130](https://openedx.atlassian.net/browse/MST-1130)
2021-11-01 13:20:35 -04:00
Ken Clary
c783684a09 fix: move coursegraph celery tasks to cms (from lms), and on a non-default, non-high-priority queue.
This is so that the lms default celery queue does not get backed up
when coursegraph is hosed (which is likely when coursegraph has been
redeployed and needs to get the full set of courses).

TNL-8386
2021-11-01 10:05:22 -04:00
alangsto
d70ebe6343 chore: update edx-proctoring version (#29179) 2021-11-01 09:34:48 -04:00
Kshitij Sobti
ff61434893 feat: Add a new way to enable/disable teams [BD-38] [TNL-9175] [BB-5066] (#29099)
* feat: Add a new way to enable/disable teams

Adds a new mechanism for enabling/disabling the team feature in a course using an 'enabled' field to the teams config.
If this field is set to true, teams is enabled (team sets/groups) still need to be defined. If this is set to false then teams is disabled whether or not team sets are defined.

* fix: review feedback
2021-11-01 17:12:06 +05:00
Matthew Piatetsky
4583b639e6 fix: change svg to png and other email fixes (#29145) 2021-10-29 15:12:36 -04:00
Matthew Piatetsky
1a8d322e71 fix: make goal reminder email transactional (#29134) 2021-10-29 14:30:30 -04:00
julianajlk
c98cdbd1f7 feat: Add Track Selection error handling (#28941)
REV-2355
2021-10-29 10:02:41 -04:00
AsadAzam
2b14c3157b Revert "Revert "feat: Add support for returning thread counts for all topics in a course [BD-38] [TNL-8724] [BB-4927] (#29062)" (#29087)" (#29152)
This reverts commit ed74db1daf.
2021-10-29 16:20:48 +05:00
Uzair Rasheed
1ac850f501 Merge pull request #29103 from edx/yob-coppa-compliance
make year of birth compliant with coppa
2021-10-28 18:18:54 +05:00
Rebecca Graber
9c0fd6296b chore: add oauth2_provider ownership (#29128) 2021-10-28 08:23:14 -04:00
Awais Qureshi
d061f64b27 test: Tests were failing due to different sorting order. (#29151) 2021-10-28 17:16:37 +05:00
uzairr
a7d89aae06 feat: make year of birth compliant with coppa
Adds a check to make the UX in compliant with Coppa suggestions.
After this change only users older than 13 years are able to
cascade between their limited and full profile.

Fixes: VAN-753
2021-10-28 16:55:27 +05:00
Waheed Ahmed
d5ab616ca8 feat: COPPA - remove primary/elementary option from education (#29147)
Remove `primary/elementary` option from education field if COPPA compliance
feature flag is enabled. Also, renamed the flag.

VAN-762
2021-10-28 14:12:53 +05:00
Kshitij Sobti
79cd0b1ef8 feat: Adds discussions settings for new discusions experience [BD-38] [TNL-8621] [BB-4854] (#29131)
* feat: Adds discussions settings for new discusions experience
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.

* fix: tests
2021-10-28 11:56:17 +05:00
connorhaugh
b01e773d2b feat: cache course index queries (#29107)
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
2021-10-27 13:53:10 -04:00
Matthew Piatetsky
ec7205e679 fix: update goal reminder email urls (#29133) 2021-10-27 13:49:28 -04:00
Dillon Dumesnil
f5b284ce46 Merge pull request #29105 from edx/ddumesnil/reset-completion-exam-aa-1063
fix: AA-1063: Reset Completion data for problems on exam reset
2021-10-27 09:58:48 -07:00
Awais Jibran
cfd78455df fix: update discussion API permissions (#29127) 2021-10-27 21:05:34 +05:00
David Ormsbee
5d0132305d Revert "feat: Adds discussions settings for new discusions experience (#28749)"
This reverts commit bb0c03123c.
2021-10-27 10:40:12 -04:00
Zachary Hancock
249f4e6fbf feat: batch verifications when creating in bulk (#29119)
batch verifications when creating in bulk
2021-10-27 10:12:02 -04:00
Dillon Dumesnil
05afe78552 fix: AA-1063: Reset Completion data for problems on exam reset
We encountered a bug where learners were sometimes not having their
completion information reset when their exam is reset. It was unclear
what was actually causing the completion to not be reset (it usually
is via a signal listener), but the effect was learners being unable to
reset their due dates in order to attempt the exam again since the exam
believed it was still complete.

This PR will likely be duplicating calls to the Completion API, but we
believe that is worthwhile to ensure successful completion state reset.
2021-10-27 09:42:52 -04:00
Andrew Shultz
09778cf9f9 Merge pull request #29118 from edx/ashultz0/regenerate-noidv-certs
provide a management command to regenerate unverified certs when honor code is turned on, per course
2021-10-27 09:14:18 -04:00
Kshitij Sobti
bb0c03123c feat: Adds discussions settings for new discusions experience (#28749)
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.
2021-10-27 15:41:27 +05:00
M. Zulqarnain
39b91154fb feat: New codemods on OpenedX 1 (#28776) 2021-10-27 13:07:36 +05:00
Andy Shultz
5ec9fca8d4 feat: provide command to regenerate certs for honor code courses
Only courses which actually have honor code on and only unverified
certs will be regenerated. All other certs wouldn't change so
don't cause trouble.

MST-855
2021-10-26 14:36:28 -04:00
Andy Shultz
1be408046b feat: explain certificate delay seconds and make it overrideable
this appears to be a throttling method on first glance but is instead
an async task handling hack, explain it

and then provide a way to override it for tight control of timing
while regenerating multiple certs
2021-10-26 14:32:33 -04:00
Matthew Piatetsky
f7050bfa3b feat: make use of more reasonable timezone when recording user activity (#29075) 2021-10-26 13:42:39 -04:00
João Victor Martins
418fd7f4b1 feat: Add optional cohort and course mode to bulk register
This adds two optional columns to the bulk register/enroll csv: cohort and
course mode. This enables setting the course mode and cohort for a user in the
same process as bulk enrolling/registering.
2021-10-26 19:14:21 +02:00
Braden MacDonald
6c85668099 feat: write split modulestore's course indexes to Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-26 10:06:52 -07:00
Simon Chen
817a4d5b1d fix: add logging message to debug the idv celery trigger logic (#29113)
We suspect the IDV code do not trigger name_affirmation update celery task correctly. Add the logging in code so we can trace the order of operation and figure out what is missing

Co-authored-by: Simon Chen <schen@edx-c02fw0guml85.lan>
2021-10-26 11:48:47 -04:00
Awais Jibran
f1957a7cf7 fix: lms discussion tab url should be accurate (#29110)
* fix: lms discussion tab url should be accurate

* refactor: change supports_lti feature

* refactor: rename variable

* refactor: rename feature variable.

* fix: update supports_lti method

* test: fix failing tests.

* test: fix quality test failure

Co-authored-by: SaadYousaf <saadyousaf@A006-00314.local>
2021-10-26 19:57:41 +05:00
Kshitij Sobti
d4cd3bd611 fix: enabling the notes tool using the course authoring will now add the notes tab (#29093)
If an existing course doesn't already have the notes tab, enabling notes will not make it show up. This change fixes this by adding the tab in case it isn't already in the course.
2021-10-26 14:57:19 +05:00
Max Sokolski
acd83d1c05 fix: use common default for LANGUAGE_COOKIE_NAME (#29096)
Changes:
- use default LANGUAGE_COOKIE_NAME value from common for cases when there is no
`LANGUAGE_COOKIE` and `LANGUAGE_COOKIE_NAME` in ENV_TOKENS
2021-10-25 14:22:00 -04:00
Andrew Shultz
f48ba1ae4c Merge pull request #29091 from edx/ashultz0/no-idv-cert
feat: make IDV optional for cert generation
2021-10-25 10:02:29 -04:00
Dillon Dumesnil
4917c8dc8b Merge pull request #29073 from edx/ddumesnil/update-edx-when-version-aa-1058
chore: AA-1058: Update edx-when version
2021-10-25 06:06:33 -07:00
Kshitij Sobti
140a1bf3d8 docs: ADR for new way to enable/disable teams (#28891)
Enabling or disabling team currently requires adding a team set/topic/group or removing them all. This ADR proposes a new mechanism.
2021-10-25 15:50:52 +05:00
Andy Shultz
7e3ef93a63 feat: make IDV optional for cert generation
IDV is on its way to retirement, so it's not going to be necessary for cert
generation forever.

Introduces a function to combine the honor code flag with IDV to tell
cert generation if it should care about a missing verification.

Various tests expanded to cover the retired case. The additional calls
in test_task_helper.py are caused by one call to fetch course
overrides which finds none, and that forces one check of the
background flag per student, 71 + 1 + 5 = 77.

MST-854
2021-10-22 14:46:07 -04:00
Dillon Dumesnil
78fbd28f41 chore: AA-1058: Update edx-when version
Updates query counts for the additional query to look up the Schedule
in edx-when
2021-10-22 09:36:04 -04:00
Awais Jibran
ed74db1daf Revert "feat: Add support for returning thread counts for all topics in a course [BD-38] [TNL-8724] [BB-4927] (#29062)" (#29087)
This reverts commit a2f04fb78b.
2021-10-22 17:21:44 +05:00
M. Zulqarnain
9833134fae Post Django 3.2 Cleanup (#29069)
* chore: Post Django 3.2 Cleanup
2021-10-22 13:55:16 +05:00
Zainab Amir
6019971dd4 feat: put year of birth behind feature flag (#29007) 2021-10-22 12:17:06 +05:00