Commit Graph

60300 Commits

Author SHA1 Message Date
Braden MacDonald
43bc683d80 feat: Add a data migration to copy all course index data into MySQL 2021-11-03 10:26:03 -04:00
Zachary Hancock
abf1f21668 fix: bulk verification creation failure for large input (#29197)
Fixes an issue where bulk create was only batching writes. With a sufficiently large input the users queryset would fail to evaluate due to the size of the filter list sent to the db.
2021-11-02 17:45:43 -04:00
Jansen Kantor
b6c92513c2 style: add blank line (#29199) 2021-11-02 16:05:17 -04:00
M. Zulqarnain
1003f262ad feat: Django codemods on LMS (1) (#28851) 2021-11-02 17:00:46 +05:00
Maxim Beder
eabd6e8019 feat: add anonymous settings to discussions api (#28981)
If course allows anonymous posts, and user is an author of a post, add
`anonymous` to posts' editable fields.
If course allows anonymous to peers posts, and user is an author of a
post, add `anonymous_to_peers` to posts' editable fields.

Course endpoint response to get request will now include course's
`allow_anonymous` and `allow_anonymous_to_peers` settings.

Added `anonymous` and `anonymous_to_peers` fields to the content
serializer, from which ThreadSerializer and CommentSerializer inherit.
Both fields have a default value of False, because there are cases where
course configuration doesn't allow them to be initialized (if a course
doesn't allow anonymous posts, the fields won't be included in the list
of initializable/editable fields).
2021-11-02 11:25:40 +05:00
Binod Pant
60da5db4b6 feat: degreed2 integrated channels (#29183)
* feat: degreed2 integrated channels

ENT-2789

* feat:  New integrated channel via edx-enterprise

* fix: pull in edx-enterprise 3.33.1

fixes db_overrides check failure by renaming field `key` to `client_id`
2021-11-01 21:18:59 -04:00
jawad khan
9c1fc4cbba feat: LEARNER-8493 Remove upgrade banner from mobile upgrade banner (#29160)
* feat: remove upgrade banner from mobile upgrade banner

After investigating the best way to keep a user logged in, and solve rendering issues, we decided to show the unit page in a full webview within the app. However, in order to comply with app store guidelines, we need to remove the upgrade banner from this page.

LEARNER-8493
2021-11-02 04:47:48 +05:00
David Ormsbee
3754bd219a fix: force reset of modulestore data for test cases
In #8bdf59b, the SplitModuleStore was given a Django ORM backed store
for active version data (i.e. "which version is currently published?").
This started to cause sporadic test failures depending on the test
ordering, such as this module:

  openedx/features/course_experience/tests/views/test_course_home.py

The root cause was that the database table holding these active versions
was not being properly cleared after tests, probably because of the odd
ordering we do MongoDB vs. Django ORM data initialization in the
ModuleStoreTestCase and SharedModuleStoreTestCase classes. This is an
overly broad hammer fix for this, because:

1. The obvious thing to add it into the ModuleStoreIsolationMixin didn't
   seem to work.
2. While overly broad, it's a small bit of code and should be safe.
3. It's more urgent to fix this flakiness in the build (affecting maybe
   1/4 test runs?) ASAP, rather than tracking this down.
2021-11-01 17:41:08 -04:00
Michael Roytman
b953ef044e Merge pull request #29155 from edx/mroytman/MST-1130-management-command-cleanup
feat: Add Management Command to Re-Emit SoftwareSecurePhotoVerification post_save Signal
2021-11-01 15:01:49 -04:00
Bianca Severino
1bb5685897 Merge pull request #29182 from edx/bseverino/hide-proctored-after-due
MST-980: Hide proctored exams after due by default
2021-11-01 14:39:23 -04:00
Michael Roytman
7ae8ed37a3 Merge pull request #29136 from edx/mroytman/MST-1130-remove-verified-name-waffle-flag
Remove Use of VERIFIED_NAME_FLAG Waffle Flag and is_verified_enabled Utility
2021-11-01 14:29:51 -04:00
kenclary
a9b0570d95 Merge pull request #29172 from edx/kclary/coursegraph
fix: move coursegraph celery tasks to cms (from lms), and on a non-default, non-high-priority queue.
2021-11-01 14:06:45 -04:00
michaelroytman
bb299c9521 feat: Remove Use of VERIFIED_NAME_FLAG Waffle Flag and is_verified_enabled Utility
The VERIFIED_NAME_FLAG, the VerifiedNameEnabledView, and the verified_name_enabled key removed from responses for both VerifiedNameView view and VerifiedNameHistoryView
were removed as part https://github.com/edx/edx-name-affirmation/pull/12. This was released in version 2.0.0 of the edx-name-affirmation PyPI package. Please see below for additional context for the removal, copied from the name-affirmation commit message.

The VERIFIED_NAME_FLAG was added as part https://github.com/edx/edx-name-affirmation/pull/12, [MST-801](https://openedx.atlassian.net/browse/MST-801) in order to control the release of the Verified Name project. It was used for a phased roll out by percentage of users.

The release reached a percentage of 50% before it was observed that, due to the way percentage roll out works in django-waffle, the code to create or update VerifiedName records was not working properly. The code was written such that any change to a SoftwareSecurePhotoVerification model instance sent a signal, which was received and handled by the Name Affirmation application. If the VERIFIED_NAME_FLAG was on for the requesting user, a Celery task was launched from the Name Affirmation application to perform the creation of or update to the appropriate VerifiedName model instances based on the verify_student application signal. However, we observed that when SoftwareSecurePhotoVerification records were moved into the "created" or "ready" status, a Celery task in Name Affirmation was created, but when SoftwareSecurePhotoVerification records were moved into the "submitted" status, the corresponding Celery task in Name Affirmation was not created. This caused VerifiedName records to stay in the "pending" state.

The django-waffle waffle flag used by the edx-toggle library implements percentage rollout by setting a cookie in a learner's browser session to assign them to the enabled or disabled group.
It turns out that the code that submits a SoftwareSecurePhotoVerification record, which moves it into the "submitted" state, happens as part of a Celery task in the verify_student application in the edx-platform. Therefore, we believe that because there is no request object in a Celery task, the edx-toggle code is defaulting to the case where there is no request object. In this case, the code checks whether the flag is enabled for everyone when determining whether the flag is enabled. Because of the percentage rollout (i.e. waffle flag not enabled for everyone), the Celery task in Name Affirmation is not created. This behavior was confirmed by logging added as part of https://github.com/edx/edx-name-affirmation/pull/62.

We have determined that we do not need the waffle flag, as we are comfortable that enabling the waffle flag for everyone will fix the issue and are comfortable releasing the feature to all users. For this reason, we are removing references to the flag.

[MST-1130](https://openedx.atlassian.net/browse/MST-1130)
2021-11-01 13:33:55 -04:00
michaelroytman
7ffdae90be feat: Add Management Command to Re-Emit SoftwareSecurePhotoVerification post_save Signal
The verify_student Django app contains a Signal receiver that receives the SoftwareSecurePhotoVerification post_save signal. It then emits an idv_update_signal to communicate that a change to IDV has occured. This Signal is received by the nameaffirmation app, which uses it to start a Celery task to create or update VerifiedName records based on the changes to the SoftwareSecurePhotoVerification model.

During the phased roll out of the Verified Name feature, due to the way percentage rollout is implemented by django-waffle and the way SoftwareSecurePhotoVerifications are updated, a mismatch of states occured where created VerifiedNames stayed in the "pending" state, even as the corresponding SoftwareSecurePhotoVerifications moved into "submitted", "approved", or "denied". Please see below for additional details.

This management commands takes as an argument a list of SoftwareSecurePhotoVerification IDs verification-ids, as well as a batch-size and sleep-time. In batches of batch-size, the command saves the SoftwareSecurePhotoVerification associated with the IDs argument. Each batch is separated by a pause of sleep_time in seconds.

By saving each SoftwareSecurePhotoVerification, the post_save signal is re-emitted, thereby re-emitting the idv_update_signal. Now that the aforementioned bug has been fixed, this will correctly trigger the Celery task and sync the SoftwareSecurePhotoVerification and VerifiedName objects.

Please find additional details about the bug below.

The release reached a percentage of 50% before it was observed that, due to the way percentage roll out works in django-waffle, the code to create or update VerifiedName records was not working properly. The code was written such that any change to a SoftwareSecurePhotoVerification model instance sent a signal, which was received and handled by the Name Affirmation application. If the VERIFIED_NAME_FLAG was on for the requesting user, a Celery task was launched from the Name Affirmation application to perform the creation of or update to the appropriate VerifiedName model instances based on the verify_student application signal. However, we observed that when SoftwareSecurePhotoVerification records were moved into the "created" or "ready" status, a Celery task in Name Affirmation was created, but when SoftwareSecurePhotoVerification records were moved into the "submitted" status, the corresponding Celery task in Name Affirmation was not created. This caused VerifiedName records to stay in the "pending" state.

The django-waffle waffle flag used by the edx-toggle library implements percentage rollout by setting a cookie in a learner's browser session to assign them to the enabled or disabled group.
It turns out that the code that submits a SoftwareSecurePhotoVerification record, which moves it into the "submitted" state, happens as part of a Celery task in the verify_student application in the edx-platform. Therefore, we believe that because there is no request object in a Celery task, the edx-toggle code is defaulting to the case where there is no request object. In this case, the code checks whether the flag is enabled for everyone when determining whether the flag is enabled. Because of the percentage rollout (i.e. waffle flag not enabled for everyone), the Celery task in Name Affirmation is not created. This behavior was confirmed by logging added as part of https://github.com/edx/edx-name-affirmation/pull/62.

[MST-1130](https://openedx.atlassian.net/browse/MST-1130)
2021-11-01 13:20:35 -04:00
Bianca Severino
6f4811d7e7 fix: hide proctored exams after due by default 2021-11-01 13:18:40 -04:00
Ken Clary
109fbb9f7c fix: move coursegraph celery tasks to cms (from lms), and on a non-default, non-high-priority queue.
This is so that the lms default celery queue does not get backed up
when coursegraph is hosed (which is likely when coursegraph has been
redeployed and needs to get the full set of courses).

TNL-8386
2021-11-01 13:13:42 -04:00
Ken Clary
c783684a09 fix: move coursegraph celery tasks to cms (from lms), and on a non-default, non-high-priority queue.
This is so that the lms default celery queue does not get backed up
when coursegraph is hosed (which is likely when coursegraph has been
redeployed and needs to get the full set of courses).

TNL-8386
2021-11-01 10:05:22 -04:00
alangsto
d70ebe6343 chore: update edx-proctoring version (#29179) 2021-11-01 09:34:48 -04:00
M. Zulqarnain
7aadf55387 feat: New codemods on OpenedX 4 (#28779) 2021-11-01 18:05:02 +05:00
Kshitij Sobti
ff61434893 feat: Add a new way to enable/disable teams [BD-38] [TNL-9175] [BB-5066] (#29099)
* feat: Add a new way to enable/disable teams

Adds a new mechanism for enabling/disabling the team feature in a course using an 'enabled' field to the teams config.
If this field is set to true, teams is enabled (team sets/groups) still need to be defined. If this is set to false then teams is disabled whether or not team sets are defined.

* fix: review feedback
2021-11-01 17:12:06 +05:00
Matthew Piatetsky
4583b639e6 fix: change svg to png and other email fixes (#29145) 2021-10-29 15:12:36 -04:00
Matthew Piatetsky
1a8d322e71 fix: make goal reminder email transactional (#29134) 2021-10-29 14:30:30 -04:00
Simon Chen
91f36d348d fix: remove ProctoredExamSoftwareSecureReview.video_url column on edx-proctoring (#29169)
Co-authored-by: Simon Chen <schen@edx-c02fw0guml85.lan>
2021-10-29 13:45:20 -04:00
Tim McCormack
18c273f150 feat: Add debug info to safe-sessions: Multiple cookies, response cookies (#29168)
Also:

- Normalize response of a helper function to always be a boolean
- Make unit test accepting of unrelated custom attr calls

ref: ARCHBOM-1939, ARCHBOM-1941
2021-10-29 17:11:23 +00:00
Manjinder Singh
0ba74be102 docs: improve docs (#29158) 2021-10-29 10:58:26 -04:00
Matthew Piatetsky
0c4f45a98b fix: don't introduce integrity errors when updating LastSeenCoursewareTimezone (#29162) 2021-10-29 10:39:11 -04:00
Simon Chen
841f4ff001 fix: remove ProctoredExamSoftwareSecureReview.video_url from proctoring library (#29167)
Co-authored-by: Simon Chen <schen@edx-c02fw0guml85.lan>
2021-10-29 10:18:53 -04:00
Awais Qureshi
3b12e11e48 temp: Adding logs for debugging purpose. (#29165)
* temp: Adding logs for debugging purpose.

BOM-2961

* temp: Adding logs for debugging purpose.

BOM-2961
2021-10-29 19:08:00 +05:00
julianajlk
c98cdbd1f7 feat: Add Track Selection error handling (#28941)
REV-2355
2021-10-29 10:02:41 -04:00
AsadAzam
2b14c3157b Revert "Revert "feat: Add support for returning thread counts for all topics in a course [BD-38] [TNL-8724] [BB-4927] (#29062)" (#29087)" (#29152)
This reverts commit ed74db1daf.
2021-10-29 16:20:48 +05:00
Simon Chen
6870898f68 fix: upgrade edx-proctoring library to 4.3.1 (#29157)
This would set the ProctoredExamSoftwareSecureReview.video_url field to be nullable

Co-authored-by: Simon Chen <schen@dhcp-10-31-74-156.dyn.MIT.EDU>
2021-10-28 16:43:42 -04:00
Simon Chen
22a7ab2705 fix: upgrade the edx-proctoring library version (#29154)
With this new version, the edx-proctoring library is running on Django3.2 instead of Django2.2

Co-authored-by: Simon Chen <schen@dhcp-10-31-74-156.dyn.mit.edu>
2021-10-28 11:54:33 -04:00
Robert Raposa
91ac846e71 fix: safe session code cleanup (#29139)
1. Minor code change to get user_id more consistently.
2. Remove outdated comments.
2021-10-28 10:06:50 -04:00
Robert Raposa
b82144bb83 feat!: remove misleading JWT monitoring (#29142)
BREAKING CHANGE: Removed unused and misleading custom attributes
  used for monitoring:
* `jwt_expires_in`: Removed because it is constant for all requests
  and just repeating the setting value. It was also misleading.
* `jwt_is_asymmetric`: Removed because multiple JWTs might be created,
  and only the last would be reported, which is misleading. This was
  meant to help with a future project to get everyone on to asymmetric
  JWTs, but can be added when needed, and probably should ONLY monitor
  deprecated (symmetric) usage that needs to be fixed. That can be left
  for another day.
2021-10-28 10:06:13 -04:00
Robert Raposa
b6a943c392 fix: errors with anonymous user (#29042)
There are several errors that appear in monitoring when calls
are made with an anonymous user. This resolves one (or more).
2021-10-28 10:05:48 -04:00
Rebecca Graber
af7fb754ab chore: upgrade event-tracking (#29141) 2021-10-28 09:55:09 -04:00
Uzair Rasheed
1ac850f501 Merge pull request #29103 from edx/yob-coppa-compliance
make year of birth compliant with coppa
2021-10-28 18:18:54 +05:00
Rebecca Graber
9c0fd6296b chore: add oauth2_provider ownership (#29128) 2021-10-28 08:23:14 -04:00
Awais Qureshi
d061f64b27 test: Tests were failing due to different sorting order. (#29151) 2021-10-28 17:16:37 +05:00
uzairr
a7d89aae06 feat: make year of birth compliant with coppa
Adds a check to make the UX in compliant with Coppa suggestions.
After this change only users older than 13 years are able to
cascade between their limited and full profile.

Fixes: VAN-753
2021-10-28 16:55:27 +05:00
M. Zulqarnain
834dabb399 feat: New codemods on OpenedX 2 (#28777) 2021-10-28 16:45:53 +05:00
Adeel Ehsan
2aa220094c feat: Remove el option from education levels (#29150)
If COPPA flag is set to true then remove el options from education levels

VAN-760
2021-10-28 15:50:47 +05:00
Awais Jibran
5f97bc60cf test: adds discussion API permissions test cases (#29135) 2021-10-28 14:54:32 +05:00
Waheed Ahmed
d5ab616ca8 feat: COPPA - remove primary/elementary option from education (#29147)
Remove `primary/elementary` option from education field if COPPA compliance
feature flag is enabled. Also, renamed the flag.

VAN-762
2021-10-28 14:12:53 +05:00
Kshitij Sobti
79cd0b1ef8 feat: Adds discussions settings for new discusions experience [BD-38] [TNL-8621] [BB-4854] (#29131)
* feat: Adds discussions settings for new discusions experience
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.

* fix: tests
2021-10-28 11:56:17 +05:00
kenclary
13818b9630 Merge pull request #29138 from edx/coursegraph-fix-neo4j-dump
fix: default for CertificatesDisplayBehaviors is enum string value, not enum.
2021-10-27 17:31:22 -04:00
edX Transifex Bot
c13cf4cca1 geoip2: update maxmind geolite country database 2021-10-28 02:03:37 +05:00
Jansen Kantor
2cce4f2c1e chore: bump ora version (#29120)
Bump ORA to 3.7.4. This version introduces custom window.confirm-esque prompts, because Chrome and Safari have begun to disallow window.confirm from within cross-origin iframes.
2021-10-27 16:36:21 -04:00
Ken Clary
bfd09236a2 fix: default for CertificatesDisplayBehaviors is enum string value, not enum.
This should correct a large number of errors encountered when trying
to dump courses to neo4j.

TNL-8386
2021-10-27 14:59:36 -04:00
connorhaugh
b01e773d2b feat: cache course index queries (#29107)
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
2021-10-27 13:53:10 -04:00