Commit Graph

1316 Commits

Author SHA1 Message Date
Kshitij Sobti
79cd0b1ef8 feat: Adds discussions settings for new discusions experience [BD-38] [TNL-8621] [BB-4854] (#29131)
* feat: Adds discussions settings for new discusions experience
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.

* fix: tests
2021-10-28 11:56:17 +05:00
connorhaugh
b01e773d2b feat: cache course index queries (#29107)
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
2021-10-27 13:53:10 -04:00
Dillon Dumesnil
05afe78552 fix: AA-1063: Reset Completion data for problems on exam reset
We encountered a bug where learners were sometimes not having their
completion information reset when their exam is reset. It was unclear
what was actually causing the completion to not be reset (it usually
is via a signal listener), but the effect was learners being unable to
reset their due dates in order to attempt the exam again since the exam
believed it was still complete.

This PR will likely be duplicating calls to the Completion API, but we
believe that is worthwhile to ensure successful completion state reset.
2021-10-27 09:42:52 -04:00
João Victor Martins
418fd7f4b1 feat: Add optional cohort and course mode to bulk register
This adds two optional columns to the bulk register/enroll csv: cohort and
course mode. This enables setting the course mode and cohort for a user in the
same process as bulk enrolling/registering.
2021-10-26 19:14:21 +02:00
Braden MacDonald
6c85668099 feat: write split modulestore's course indexes to Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-26 10:06:52 -07:00
Usama Sadiq
9ee8df0980 fix: Remove pylint constraint and fix warnings (#28646) 2021-10-20 23:00:13 +05:00
David Ormsbee
ae124bd554 Revert "feat: store split modulestore's course indexes in Django/MySQL"
This reverts commit 96e5ff8dce.
2021-10-07 15:07:42 -04:00
Braden MacDonald
96e5ff8dce feat: store split modulestore's course indexes in Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-07 10:59:47 -04:00
Julia Eskew
cc3747add6 feat: Optimize the querying of course block dates when querying dates for use in a
course outline, which don't need block dates below the subsection level of a course.
Pass the course's published version to all the appropriate places where edx-when's API
is called - to allow edx-when to more efficiently cache queried/processed results.

TNL-8061
2021-09-27 12:02:59 -04:00
Usama Sadiq
023f4144ab fix: fix new pylint warnings (#28725) 2021-09-23 17:54:15 +05:00
Awais Jibran
4196ac7730 refactor: remove duplicate decorator (#28763) 2021-09-16 16:06:29 +05:00
David Ormsbee
309cadc973 Revert "Use new version of edx-when." (#28684)
Reverting because of errors seen in proctoring. The error from splunk:

 File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/edx_proctoring/views.py", line 590, in get
    data['onboarding_release_date'] = effective_start.isoformat()
AttributeError: 'NoneType' object has no attribute 'isoformat'

The effective date is coming from learning_sequences, which ultimately
comes from edx-when.
2021-09-08 11:05:35 -04:00
Julia Eskew
a4ba37b882 feat: Optimize the querying of course block dates when querying dates for use in a
course outline, which don't need block dates below the subsection level of a course.
Pass the course's published version to all the appropriate places where edx-when's API
is called - to allow edx-when to more efficiently cache queried/processed results.

TNL-8061
2021-09-02 17:45:13 -04:00
mohtamba
25dfb9d4ca Remove waffle flag for bulk allowance
Remove the waffle flag for bulk allowance, so by default the bulk allowance feature is enabled and is the only option.
2021-08-17 07:42:39 -04:00
Christie Rice
cf3a6c16d6 fix: Stop showing course certificate buttons to beta testers (#28416)
Beta testers can’t earn course certificates, so they should not see a “Request Certificate” button or other info describing how they can earn a cert.

MICROBA-992
2021-08-10 09:06:14 -04:00
Justin Hynes
7f5cf2fabd refactor: update certificates app Python API functions
[MICROBA-1354]

* Update Python API function named `cert_generated_enabled` to `has_self_generated_certificates_enabled` to more accurately reflect the purpose of the function
2021-08-04 08:43:12 -04:00
Justin Hynes
d4a7a44909 refactor!: Remove generate_example_certificates and update Instructor Dashboard with updated messaging.
[MICROBA-1087]
[DEPR-155]

* Remove `generate_example_certificates` functionality
* Adjust Instructor dashboard slightly to prevent people from clicking the `Generate Example Certificates` button, remove form/code that called the `generate_example_certificates` endpoint.
2021-08-02 13:26:28 -04:00
Justin Hynes
0069bd724f fix: remove disabled v1 course certificate queue code
[MICROBA-1227]
[DEPR-155]

* remove queue.py (and associated unit tests)
* remove references to `queue.py` from functions in api.py (also update unit tests)
2021-07-22 11:03:49 -04:00
mohtamba
1c53194250 Add Waffle Flag for Bulk Allowances
Add bulk allowance flag to allow for new feature to be enabled.
2021-07-15 13:41:45 -04:00
Matthew Piatetsky
73bbe8272f Merge pull request #28048 from edx/AA-874c
[AA-874] fix student progress url on student admin dashboard
2021-07-08 12:40:38 -04:00
Manjinder Singh
9616a1c367 fix: adding code owner info to celery tasks (#28040)
Celery tasks needs decorator to correctly add ownership info:
https://github.com/edx/edx-django-utils/blob/master/edx_django_utils/monitoring/docs/how_tos/add_code_owner_custom_attribute_to_an_ida.rst#handling-celery-tasks
2021-06-28 09:15:31 -04:00
Matthew Piatetsky
36de3b8677 fix: fix student progress url on student admin dashboard 2021-06-28 09:15:14 -04:00
SaadYousaf
16972b4991 test: add tests to test override_legacy_discussion_settings flag with different roles 2021-06-28 11:54:17 +05:00
SaadYousaf
608f1994b7 refactor: move waffle flag calculation to utils file and other updates 2021-06-28 11:54:16 +05:00
SaadYousaf
90921e19e2 test: add tests for discussion settings and tab visibility with override_discussions_setting flag. 2021-06-28 11:54:16 +05:00
SaadYousaf
039a816e57 refactor: move pages_and_resources_mfe WaffleFlag to use in both LMS and Studio 2021-06-28 11:54:16 +05:00
SaadYousaf
09c8ef931e feat: Add CourseWaffleFlag for overriding course discussion settings for legacy experience. 2021-06-28 11:54:16 +05:00
Christie Rice
69580aa592 feat!: Read from the allowlist model, instead of from the whitelist model. Only write to the allowlist model. (#27974)
MICROBA-982
2021-06-25 09:47:32 -04:00
Dillon Dumesnil
61358b230e feat: AA-869: Add in logic to mark special exams as complete
This is technically a revert of a revert that also includes some new
logic. The original PR was https://github.com/edx/edx-platform/pull/27770
and the revert PR was https://github.com/edx/edx-platform/pull/27973.

The new logic is to mock a request and create a module with user state
taken into account. We also put this into a celery task to help avoid
any potential concerns with recursing through children.
2021-06-24 12:06:11 -07:00
Justin Hynes
205911653b refactor: Rename "whitelist" references to "allowlist" where possible in Instructor Dashboard/Certificates apps
[MICROBA-1179]
- Continue renaming/removal of code referring to the Certificate "white list".

The Certificates Django app `CertificateWhitelist` model is going away in an effort to make our codebase more inclusive. It is being replaced
with the `CertificateAllowlist` model. This PR continues to replace references to the Certificate "whitelist" with "allowlist" wherever
possible. There should be no change in functionality, nor are there any changes in appearance.
2021-06-22 08:18:15 -04:00
Dillon Dumesnil
68cce7dbbb Revert "feat: AA-773: Add in logic to mark special exams as complete" 2021-06-16 10:16:27 -07:00
Dillon Dumesnil
cc7be98f91 feat: AA-773: Add in logic to mark special exams as complete
We have had numerous support tickets/bugs related to special exams
not properly displaying completeness (see https://openedx.atlassian.net/browse/AA-773 for
details). Along with a corresponding edx-proctoring PR, this will now
submit completions for all completable children within a special
exam upon the exam being completed for the first time (as dictated by
the logic in the edx-proctoring PR).
2021-06-10 11:58:21 -07:00
Christie Rice
97f6d0fce8 test: Remove external tests that exercise the internal certificates code. Update tests in preparation for enabling V2 of course certificates globally. (#27881)
MICROBA-1082
2021-06-08 15:29:05 -04:00
Albert (AJ) St. Aubin
c37e88fdbf refactor: Move CertificateStatuses to data.py file per OEP-49
[MICROBA-678]

To allow access to CertificateStatuses constants in other Django apps we are moving the
CertificateStatuses enum to data.py per OEP-49.
2021-06-08 08:56:21 -04:00
Justin Hynes
6a0f57a266 refactor: update references to deprecated whitelist model
[MICROBA-1012]
- update python files to reference the allowlist instead of whitelist

As part of the Certificates v2 work, and in an effort to make the codebase more inclusive, we are moving away from using the term "whitelist" in favor of "allowlist". This PR is part of our renaming efforts across the Certificates Django app, and other Django apps that make use of Certificates functionality.
2021-06-03 14:09:24 -04:00
Justin Hynes
48f4f9f475 feat: add edx.certificate.revoked event
[MICROBA-1075]
- Adds a new `edx.certificate.revoked` event to the LMS.
- Refactor of our certificate revocation functions in the GeneratedCertificate model.

This new event will be emit when a GeneratedCertificate with the status of `downloadable` is revoked (through the `invalidate(), mark_notpassing(), or mark_unverified() functions). Event will have a `source` field that will allow us how our certificates are being revoked from learners and can be broken down in the following way:
*Invalidate*
- allowlist_removal
- certificate_generation
- certificate_regeneration
- certificate_service
- certificate_invalidation_list
- bulk_certificate_regeneration

*mark_notpassing*
- certificate_generation
- notpassing_signal

*unverified*
- certificate_generation
2021-05-25 13:44:01 -04:00
Justin Hynes
e0cfae745f Merge pull request #27646 from edx/jhynes/microba-1209_function-cleanup
refactor: consolidate (mostly) duplicated function
2021-05-17 12:59:42 -04:00
Justin Hynes
d4f3e11092 refactor: consolidate (mostly) duplicated functions
[MICROBA-1209]
Part of the modulestore cleanup/removal in the certificates app. The function `list_with_level_from_course_overview` was temporary. Cleaning this up. Modifying the original `list_with_level` to accept a course-id/course-key as it doesn't actually need the entire course object.
* Update `list_with_level` function to accept a course run id/course-key
* Remove `list_with_level_from_course_key` function (and update any functions using it to use `list_with_level` again)
2021-05-17 09:50:39 -04:00
Justin Hynes
313a295c95 refactor: update function calls outside of certificates app for updated generate_user_certificates and regenerate_user_certificates functions
[MICROBA-1238]
* remove unused `course` argument from `generate_user_certificates` function in the certificates app
* remove unused `course` argument from `regenerate_user_certificates` functioni n the certificates app
* remove `course` argument if passed in edx-platform apps outside of the certificates app
2021-05-17 09:39:36 -04:00
Giovanni Cimolin da Silva
9a334a383e feat: Add changes for waiting step details
Bump ORA version and fix method signature.
2021-05-13 09:27:53 -03:00
Kshitij Sobti
161e3560dd feat: Add REST endpoints for problem response report generation that can be driven by MFEs. (#27313) 2021-05-06 14:05:14 -04:00
Christie Rice
ce591fd3ea refactor: Rename whitelist to allowlist (#27504)
MICROBA-1021
2021-05-05 10:51:31 -04:00
Justin Hynes
6313c1ba67 Merge pull request #27501 from edx/jhynes/microba-1178_modulestore
refactor: reduce certificate django apps dependence on the modulestore (generation_handler)
2021-05-04 13:03:15 -04:00
Justin Hynes
5db4fd5bb2 refactor: reduce certificate django apps dependence on the modulestore (generation_handler)
[MICROBA-1178]
- remove modulestore usage in `generation_handler.py`
- add duplicate functions that utilize a CourseKey or CourseOverview to remove dependence on modulestore (this will be cleaned up (if possible) at a later part of this refactor)
- add python API function to `content`/`course_overview` app that will retrieve a single CourseOverview (rather than a serialized list of dicts of CourseOverview data)
2021-05-04 07:47:58 -04:00
Christie Rice
82bf0411ca test: Use allowlist factory (#27490)
MICROBA-1021
2021-05-03 12:53:36 -04:00
Usama Sadiq
228c54d222 refactor: pyupgrade second iteration 2021-05-03 14:55:47 +05:00
stvn
1b76999c2f refactor: Remove get_course_discussion_settings helper 2021-04-30 11:47:34 -07:00
Usama Sadiq
ab41a7daee refactor: Apply lint-amnesty on pylint violations 2021-04-20 21:43:59 +05:00
stvn
ffcfc05a61 refactor: Move BetaTesterFactory helper to common/ 2021-04-14 07:26:39 -07:00
stvn
22fb1ec124 refactor: Move InstructorFactory helper to common/ 2021-04-14 07:26:39 -07:00