Commit Graph

1336 Commits

Author SHA1 Message Date
Justin Hynes
31d3fcc01a feat: add models for scheduled instructor tasks
[MICROBA-1508]
- Adds the InstructorTaskSchedule model
- Adds the HistoricalInstructorTaskSchedule model
- Adds utility function used to create a InstructorTaskSchedule instance
- Adds a public `create_course_email` Python API function to the bulk_email app that can be imported and used in external apps
- Adds a new `test_api_helper.py` test file (with tests for the new `schedule_task` function) to the instructor_task app
- Adds a new `test_api.py` test file (with tests for the new `create_course_email` function to the bulk_email app
2022-03-25 14:34:37 -04:00
Thomas Tracy
700829bd4b feat: [Microba-1758] link new bulk email tool (#30099)
Creates a link to the new communication mfe's bulk email tool in the
instructor dashboard version. Staff can now use either experience. In
the future, we plan on turning off the old experience, like analytics.
2022-03-22 13:47:08 -04:00
Braden MacDonald
dcb7ef8821 feat: Read course indexes from MySQL, not MongoDB (#29184)
Description
This is a follow up to #29058 and #29413. This is the next step in moving part of the modulestore data (the course indexes / "active versions" table) from MongoDB to MySQL.

There are four steps planned in moving course index data to MySQL:

Step 1: create the tables in MySQL, start writing to MySQL + MongoDB  done
Step 2: migrate all remaining courses to MySQL  done
Step 3: switch reads from MongoDB to MySQL (this PR)
Step 4 (much later, once we know this is working well): stop writing to MongoDB altogether.
Supporting information
OpenCraft Jira ticket: MNG-2557

Status
 Tested with a large Open edX instance is in progress.

Testing instructions
Try making changes in Studio and verify that they work fine.

Deadline
None
2022-03-09 10:21:09 -05:00
Justin Hynes
bbb0bc77d9 revert: reverts 53041a2, causing email issues
[MICROBA-1666]

This reverts commit 53041a2d34 after course team started reporting issues of images in emails not respecting dimensions set with the email editor.

After a brief investigation we found unexpected attributes (like `width` and `height` of an image) being stripped from the HTML.
2022-03-02 14:03:55 -05:00
Justin Hynes
53041a2d34 feat: sanitize bulk course email message content before storing in database
[MICROBA-1666]

* Use bleach to sanitize user-provided content of bulk course emails before storing in the database.
* Add new `BULK_COURSE_EMAIL_ALLOWED_HTML_TAGS` setting to enable configuration of allowed HTML tags in bulk course emails.
2022-02-28 08:49:06 -05:00
Bianca Severino
af81b3a609 chore: replace integrity signature flag with django setting 2022-02-15 09:57:00 -05:00
Michael Terry
cb1bb7fa64 test: switch default test store to the split store
It's long past time that the default test modulestore was Split,
instead of Old Mongo. This commit switches the default store and
fixes some tests that now fail:
- Tests that didn't expect MFE to be enabled (because we don't
  enable MFE for Old Mongo) - opt out of MFE for those
- Tests that hardcoded old key string formats
- Lots of other random little differences

In many places, I didn't spend much time trying to figure out how to
properly fix the test, and instead just set the modulestore to Old
Mongo.

For those tests that I didn't spend time investigating, I've set
the modulestore to TEST_DATA_MONGO_AMNESTY_MODULESTORE - search for
that string to find further work.
2022-02-04 14:32:50 -05:00
Jansen Kantor
30dcdb487a fix: uncaught ItemNotFoundErrors (#29870) 2022-02-04 10:24:15 -05:00
Simon Chen
895cc10c1b fix: do not validate proctoring_provider if proctoring is disabled (#29803)
Co-authored-by: Simon Chen <schen@edx-c02fw0guml85.lan>
2022-01-21 12:45:43 -05:00
Julia Eskew
89b3efc982 fix: Email sent via ACE cannot use Braze as a mail-delivery backend unless
a valid LMS user ID is supplied. If Braze cannot be used, AWS SES is used
instead. However, sometimes the SES score can prevent email from being delivered.
Make a best effort in every case to obtain a user ID and send it along with
the email to be sent by ACE and Braze.

TNL-9417
2022-01-06 10:49:01 -05:00
Jawayria
309669f974 chore: Applied lint-amnesty on lms/djangoapps/instructor (#29399) 2021-12-21 13:37:04 +05:00
Matt Hughes
a92f372a3e fix: redirect behavior to learner-specific progress pages
JIRA:AU-375
2021-12-13 15:52:00 -05:00
Abdurrehman
6c6cf0ad0b feat: external plugin integration in instructor dashboard (#29376)
* feat: external plugin integration fixes
2021-12-07 15:54:00 +05:00
Thomas Tracy
038d68d53c feat: [MICROBA-1531] bulk email deny list logic (#29465)
* feat: [MICROBA-1531] bulk_email deny list logic
2021-12-02 13:11:55 -05:00
alangsto
a1b967a2fb feat: remove verification status from profile data report (#29428)
The verification status should be removed from the student profile data report if the integrity signature toggle has been enabled.
2021-11-24 11:41:20 -05:00
Kshitij Sobti
e8e8f4acbe feat!: Change the way tabs are ordered [BD-38] [TNL-9174] [BB-5076] (#29262)
* feat!: Change the way tabs are ordered
The change imposes a new ordering for tabs based on their new priority. When reordering tabs, this ordering will be maintained.

* fix: Apply suggestions from code review

Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>

* fix: review feedback

Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>
2021-11-22 11:17:30 +05:00
M. Zulqarnain
95dbcaa323 feat: Django codemods on LMS (4) (#28854) 2021-11-19 15:10:58 +05:00
M. Zulqarnain
b84b862cc0 feat: Django codemods on LMS (3) (#28853) 2021-11-16 18:27:21 +05:00
Awais Jibran
deba009e25 refactor: remove unused variables (#29304)
I have come across these unused variables and thought it's good to remove and clean up the code.
2021-11-12 00:28:35 +05:00
Ned Batchelder
d9dd10dc97 style: code cleanups from Steven Burch (#29292)
* chore: update deprecated import from collections

* chore: remove outdated imports from markdown library

as it hasn't been supported since 2.0.3 and we're on 3.x.
This was deprecated at least as early as 2012!

* docs: add docstring and remove lint-amnesty to markdown plugin

* chore: remove deprecated etree import

* style: remove unnecessary-comprehension for sets

* style: resolve a number of amnestied pylint complaints

Co-authored-by: stvn <stvn@mit.edu>
2021-11-10 07:11:57 -08:00
Kshitij Sobti
79cd0b1ef8 feat: Adds discussions settings for new discusions experience [BD-38] [TNL-8621] [BB-4854] (#29131)
* feat: Adds discussions settings for new discusions experience
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.

* fix: tests
2021-10-28 11:56:17 +05:00
connorhaugh
b01e773d2b feat: cache course index queries (#29107)
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
2021-10-27 13:53:10 -04:00
Dillon Dumesnil
05afe78552 fix: AA-1063: Reset Completion data for problems on exam reset
We encountered a bug where learners were sometimes not having their
completion information reset when their exam is reset. It was unclear
what was actually causing the completion to not be reset (it usually
is via a signal listener), but the effect was learners being unable to
reset their due dates in order to attempt the exam again since the exam
believed it was still complete.

This PR will likely be duplicating calls to the Completion API, but we
believe that is worthwhile to ensure successful completion state reset.
2021-10-27 09:42:52 -04:00
João Victor Martins
418fd7f4b1 feat: Add optional cohort and course mode to bulk register
This adds two optional columns to the bulk register/enroll csv: cohort and
course mode. This enables setting the course mode and cohort for a user in the
same process as bulk enrolling/registering.
2021-10-26 19:14:21 +02:00
Braden MacDonald
6c85668099 feat: write split modulestore's course indexes to Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-26 10:06:52 -07:00
Usama Sadiq
9ee8df0980 fix: Remove pylint constraint and fix warnings (#28646) 2021-10-20 23:00:13 +05:00
David Ormsbee
ae124bd554 Revert "feat: store split modulestore's course indexes in Django/MySQL"
This reverts commit 96e5ff8dce.
2021-10-07 15:07:42 -04:00
Braden MacDonald
96e5ff8dce feat: store split modulestore's course indexes in Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-07 10:59:47 -04:00
Julia Eskew
cc3747add6 feat: Optimize the querying of course block dates when querying dates for use in a
course outline, which don't need block dates below the subsection level of a course.
Pass the course's published version to all the appropriate places where edx-when's API
is called - to allow edx-when to more efficiently cache queried/processed results.

TNL-8061
2021-09-27 12:02:59 -04:00
Usama Sadiq
023f4144ab fix: fix new pylint warnings (#28725) 2021-09-23 17:54:15 +05:00
Awais Jibran
4196ac7730 refactor: remove duplicate decorator (#28763) 2021-09-16 16:06:29 +05:00
David Ormsbee
309cadc973 Revert "Use new version of edx-when." (#28684)
Reverting because of errors seen in proctoring. The error from splunk:

 File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/edx_proctoring/views.py", line 590, in get
    data['onboarding_release_date'] = effective_start.isoformat()
AttributeError: 'NoneType' object has no attribute 'isoformat'

The effective date is coming from learning_sequences, which ultimately
comes from edx-when.
2021-09-08 11:05:35 -04:00
Julia Eskew
a4ba37b882 feat: Optimize the querying of course block dates when querying dates for use in a
course outline, which don't need block dates below the subsection level of a course.
Pass the course's published version to all the appropriate places where edx-when's API
is called - to allow edx-when to more efficiently cache queried/processed results.

TNL-8061
2021-09-02 17:45:13 -04:00
mohtamba
25dfb9d4ca Remove waffle flag for bulk allowance
Remove the waffle flag for bulk allowance, so by default the bulk allowance feature is enabled and is the only option.
2021-08-17 07:42:39 -04:00
Christie Rice
cf3a6c16d6 fix: Stop showing course certificate buttons to beta testers (#28416)
Beta testers can’t earn course certificates, so they should not see a “Request Certificate” button or other info describing how they can earn a cert.

MICROBA-992
2021-08-10 09:06:14 -04:00
Justin Hynes
7f5cf2fabd refactor: update certificates app Python API functions
[MICROBA-1354]

* Update Python API function named `cert_generated_enabled` to `has_self_generated_certificates_enabled` to more accurately reflect the purpose of the function
2021-08-04 08:43:12 -04:00
Justin Hynes
d4a7a44909 refactor!: Remove generate_example_certificates and update Instructor Dashboard with updated messaging.
[MICROBA-1087]
[DEPR-155]

* Remove `generate_example_certificates` functionality
* Adjust Instructor dashboard slightly to prevent people from clicking the `Generate Example Certificates` button, remove form/code that called the `generate_example_certificates` endpoint.
2021-08-02 13:26:28 -04:00
Justin Hynes
0069bd724f fix: remove disabled v1 course certificate queue code
[MICROBA-1227]
[DEPR-155]

* remove queue.py (and associated unit tests)
* remove references to `queue.py` from functions in api.py (also update unit tests)
2021-07-22 11:03:49 -04:00
mohtamba
1c53194250 Add Waffle Flag for Bulk Allowances
Add bulk allowance flag to allow for new feature to be enabled.
2021-07-15 13:41:45 -04:00
Matthew Piatetsky
73bbe8272f Merge pull request #28048 from edx/AA-874c
[AA-874] fix student progress url on student admin dashboard
2021-07-08 12:40:38 -04:00
Manjinder Singh
9616a1c367 fix: adding code owner info to celery tasks (#28040)
Celery tasks needs decorator to correctly add ownership info:
https://github.com/edx/edx-django-utils/blob/master/edx_django_utils/monitoring/docs/how_tos/add_code_owner_custom_attribute_to_an_ida.rst#handling-celery-tasks
2021-06-28 09:15:31 -04:00
Matthew Piatetsky
36de3b8677 fix: fix student progress url on student admin dashboard 2021-06-28 09:15:14 -04:00
SaadYousaf
16972b4991 test: add tests to test override_legacy_discussion_settings flag with different roles 2021-06-28 11:54:17 +05:00
SaadYousaf
608f1994b7 refactor: move waffle flag calculation to utils file and other updates 2021-06-28 11:54:16 +05:00
SaadYousaf
90921e19e2 test: add tests for discussion settings and tab visibility with override_discussions_setting flag. 2021-06-28 11:54:16 +05:00
SaadYousaf
039a816e57 refactor: move pages_and_resources_mfe WaffleFlag to use in both LMS and Studio 2021-06-28 11:54:16 +05:00
SaadYousaf
09c8ef931e feat: Add CourseWaffleFlag for overriding course discussion settings for legacy experience. 2021-06-28 11:54:16 +05:00
Christie Rice
69580aa592 feat!: Read from the allowlist model, instead of from the whitelist model. Only write to the allowlist model. (#27974)
MICROBA-982
2021-06-25 09:47:32 -04:00
Dillon Dumesnil
61358b230e feat: AA-869: Add in logic to mark special exams as complete
This is technically a revert of a revert that also includes some new
logic. The original PR was https://github.com/edx/edx-platform/pull/27770
and the revert PR was https://github.com/edx/edx-platform/pull/27973.

The new logic is to mock a request and create a module with user state
taken into account. We also put this into a celery task to help avoid
any potential concerns with recursing through children.
2021-06-24 12:06:11 -07:00
Justin Hynes
205911653b refactor: Rename "whitelist" references to "allowlist" where possible in Instructor Dashboard/Certificates apps
[MICROBA-1179]
- Continue renaming/removal of code referring to the Certificate "white list".

The Certificates Django app `CertificateWhitelist` model is going away in an effort to make our codebase more inclusive. It is being replaced
with the `CertificateAllowlist` model. This PR continues to replace references to the Certificate "whitelist" with "allowlist" wherever
possible. There should be no change in functionality, nor are there any changes in appearance.
2021-06-22 08:18:15 -04:00