Commit Graph

19237 Commits

Author SHA1 Message Date
connorhaugh
2ebfe47077 fix: remove slashmark 2021-11-22 13:58:34 -05:00
connorhaugh
74bda16638 Revert "feat: Add a data migration to copy all course index data into MySQL (#29293)" (#29387)
This reverts commit b5299674d2.
2021-11-22 11:59:56 -05:00
Braden MacDonald
b5299674d2 feat: Add a data migration to copy all course index data into MySQL (#29293) 2021-11-22 10:34:25 -05:00
Kshitij Sobti
e8e8f4acbe feat!: Change the way tabs are ordered [BD-38] [TNL-9174] [BB-5076] (#29262)
* feat!: Change the way tabs are ordered
The change imposes a new ordering for tabs based on their new priority. When reordering tabs, this ordering will be maintained.

* fix: Apply suggestions from code review

Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>

* fix: review feedback

Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>
2021-11-22 11:17:30 +05:00
Adeel Ehsan
66654b2edb feat: don't allow account creation for secondary emails (#28880)
VAN-726
2021-11-19 17:06:24 +05:00
Aarif
39fe1a20e4 feat: move unit tests to github actions on self-hosted runners (#28873) 2021-11-18 19:09:31 +05:00
edX Transifex Bot
a60f9ff82d geoip2: update maxmind geolite country database 2021-11-18 02:13:52 +05:00
Justin Lapierre
f6f3a79a15 Revert "[BD-13] Deprecate ModuleSystem.render_template" 2021-11-16 15:16:08 -05:00
Piotr Surowiec
4bf829dcbd fix: do not remove comments from XML during course import (#28557)
This is a follow-up to edx#1087, which reverted this change.
According to the PR comments, parsing strings with XML comments inside them was
causing errors. This does not seem to be the case anymore - these strings are
just hidden when the block is rendered, but they are not breaking XBlocks.
This also handles (ignores) the comments that could be added directly to the
LibraryContentBlock in the XML export by users.
2021-11-16 12:42:08 -05:00
edX Transifex Bot
52a2da61f1 geoip2: update maxmind geolite country database 2021-11-11 02:18:37 +05:00
Ned Batchelder
d9dd10dc97 style: code cleanups from Steven Burch (#29292)
* chore: update deprecated import from collections

* chore: remove outdated imports from markdown library

as it hasn't been supported since 2.0.3 and we're on 3.x.
This was deprecated at least as early as 2012!

* docs: add docstring and remove lint-amnesty to markdown plugin

* chore: remove deprecated etree import

* style: remove unnecessary-comprehension for sets

* style: resolve a number of amnestied pylint complaints

Co-authored-by: stvn <stvn@mit.edu>
2021-11-10 07:11:57 -08:00
David Ormsbee
2d60224125 Merge pull request #28571 from open-craft/jill/bd-13-render_template
feat!: [BD-13] Deprecate ModuleSystem.render_template
2021-11-09 12:20:59 -05:00
David Ormsbee
0ec2a3b9b5 docs: ADR for removing MongoDB usage. 2021-11-09 10:55:28 -05:00
Jillian Vogel
26b43465a4 refactor: use MakoService.render_template to remove deprecation warnings
from test code.
2021-11-09 14:03:54 +10:30
Jillian Vogel
8d62d337f5 refactor: use MakoService.render_template to remove deprecation warnings
from block code.
2021-11-09 14:03:54 +10:30
Jillian Vogel
457f959356 refactor: deprecates ModuleSystem.render_template
in favor of the added MakoSystem render_template method.

Related changes:
* Adds the MakoService to the StudioEditModuleRuntime,
  PreviewModuleSystem, LmsModuleSystem, and XBlockRuntime
* MakoService constructor takes a `namespace_prefix` string, so that the
  CMS PreviewModuleSystem can render to LMS templates, without needing
  the special render_from_lms helper method.
* ModuleSystem.render_template becomes a read-only property, so the
  constructor calls and test module systems are updated accordingly.
* Adds tests for the MakoService and module system shims.
2021-11-09 14:03:50 +10:30
Andy Shultz
6884361bce feat: only show IDV panel until honor code replaces it
the integrity flag is a course waffle flag so until it's on for
everyone this will not be triggered - that's OK and expected

MST-1153
2021-11-08 14:52:13 -05:00
Braden MacDonald
480e8997ec fix: A test (test_save_one_and_confirm) was flaky 2021-11-08 10:46:33 -08:00
Simon Chen
bcf093c1f6 feat: disable idv on dashboard course listing (#29276)
There are messages on each course card on learner dashboard that displays action panels to prompt learners to do IDV. With honor code signature feature on, those panel message should be disabled

Co-authored-by: Simon Chen <schen@edx-c02fw0guml85.lan>
2021-11-08 13:36:40 -05:00
David Ormsbee
15eca3ba59 Merge pull request #29190 from open-craft/jill/BD-13-user-service-fix
[FAL-2457] Let ModuleSystem use UserService
2021-11-08 09:18:10 -05:00
Farhaan Bukhsh
cfe75e95b9 feat: Add discussion_enabled for Unit (#28864)
For each unit discussion_enabled flag is added
so that each unit can be made discussable when needed.

Signed-off-by: Farhaan Bukhsh <farhaan@opencraft.com>
2021-11-08 16:17:07 +05:00
edX Transifex Bot
20535e12eb geoip2: update maxmind geolite country database 2021-11-04 02:08:52 +05:00
SaadYousaf
abdc46793f fix: add cache to forum role lookup 2021-11-04 00:24:29 +05:00
connorhaugh
b8d49e77cc Revert "feat: Add a data migration to copy all course index data into MySQL"
This reverts commit 43bc683d80.
2021-11-03 13:53:21 -04:00
Braden MacDonald
43bc683d80 feat: Add a data migration to copy all course index data into MySQL 2021-11-03 10:26:03 -04:00
Jansen Kantor
b6c92513c2 style: add blank line (#29199) 2021-11-02 16:05:17 -04:00
Jillian Vogel
a14f7e5953 fix: do not modify the django user object when creating a XBlockDjangoUserService
This user object is shared between invocations, and so the
anonymous_user_id gets altered for different types of blocks.
2021-11-02 20:42:21 +10:30
Jillian Vogel
f8c96c1df7 chore: pylint 2021-11-02 20:04:33 +10:30
Jillian Vogel
b1b3c646ad refactor: fixes tests
* UserStubService now takes user, user_is_staff, and anonymous_user_id
* get_test_system() creates a UserStubService with an anonymous_user_id of 'student'
* Removes references to deprecated ModuleSystem attributes from test code
* Fixes and simplifies the ConditionalBlock tests, using get_module provided by TestModuleSystem instead of trying to mock out all the pieces.

(cherry picked from commit 927016a8df)
2021-11-02 18:25:52 +10:30
Jillian Vogel
bafc0cd91f refactor: removes calls to deprecated ModuleSystem attributes
Removes references to these deprecated attributes from the platform code:
* runtime.anonymous_student_id
* runtime.seed
* runtime.user_id
* runtime.user_is_staff

Related changes:

* Ensure that all platform XBlocks which use these attributes "need" the user service.
* ProblemBlock: Removes check for existence of runtime.seed attribute in preparation for removal of this attribute from ModuleSystem.
* edxnotes: Catches NoSuchServiceError just in case some XBlocks using notes don't have the user service.
* UserTagsService refactor: pass user and course_id on creation

(cherry picked from commit 753839276f)
2021-11-02 18:24:14 +10:30
Jillian Vogel
cf1064616c refactor: deprecate ModuleSystem user attributes in favor of user service
The following ModuleSystem attributes are deprecated by this change, and should be pulled directly from the user service instead:

* anonymous_student_id
* seed
* user_id
* user_is_staff

Related changes:

* Removes the `user` and `anonymous_student_id` parameters from the ModuleService constructor.
* Stores anonymous_user_id in XBlockDjangoUserService's opt_attr
* Pulls out constants used by DjangoXBlockUserService opt_attr so they can be used in the platform code.
* LmsModuleSystem uses the user service created in wrapper function for runtime.publish to avoid requiring the user
  service to be "needed" by all XBlocks.
* LmsModuleSystem no longer checks for instances of XModuleDescriptor when deciding what kind of anonymous_user_id to
  provide:  all XModules are XBlocks, so this check is unnecessary.
* XBlockRuntime returns a user service when requested
* Adds tests for deprecated ModuleSystem attributes and changes to XBlockDjangoUserService.

(cherry picked from commit c41e7fb93a)
2021-11-02 18:24:06 +10:30
David Ormsbee
3754bd219a fix: force reset of modulestore data for test cases
In #8bdf59b, the SplitModuleStore was given a Django ORM backed store
for active version data (i.e. "which version is currently published?").
This started to cause sporadic test failures depending on the test
ordering, such as this module:

  openedx/features/course_experience/tests/views/test_course_home.py

The root cause was that the database table holding these active versions
was not being properly cleared after tests, probably because of the odd
ordering we do MongoDB vs. Django ORM data initialization in the
ModuleStoreTestCase and SharedModuleStoreTestCase classes. This is an
overly broad hammer fix for this, because:

1. The obvious thing to add it into the ModuleStoreIsolationMixin didn't
   seem to work.
2. While overly broad, it's a small bit of code and should be safe.
3. It's more urgent to fix this flakiness in the build (affecting maybe
   1/4 test runs?) ASAP, rather than tracking this down.
2021-11-01 17:41:08 -04:00
Kshitij Sobti
ff61434893 feat: Add a new way to enable/disable teams [BD-38] [TNL-9175] [BB-5066] (#29099)
* feat: Add a new way to enable/disable teams

Adds a new mechanism for enabling/disabling the team feature in a course using an 'enabled' field to the teams config.
If this field is set to true, teams is enabled (team sets/groups) still need to be defined. If this is set to false then teams is disabled whether or not team sets are defined.

* fix: review feedback
2021-11-01 17:12:06 +05:00
Matthew Piatetsky
4583b639e6 fix: change svg to png and other email fixes (#29145) 2021-10-29 15:12:36 -04:00
Simon Chen
91f36d348d fix: remove ProctoredExamSoftwareSecureReview.video_url column on edx-proctoring (#29169)
Co-authored-by: Simon Chen <schen@edx-c02fw0guml85.lan>
2021-10-29 13:45:20 -04:00
Simon Chen
841f4ff001 fix: remove ProctoredExamSoftwareSecureReview.video_url from proctoring library (#29167)
Co-authored-by: Simon Chen <schen@edx-c02fw0guml85.lan>
2021-10-29 10:18:53 -04:00
julianajlk
c98cdbd1f7 feat: Add Track Selection error handling (#28941)
REV-2355
2021-10-29 10:02:41 -04:00
Awais Jibran
5f97bc60cf test: adds discussion API permissions test cases (#29135) 2021-10-28 14:54:32 +05:00
Kshitij Sobti
79cd0b1ef8 feat: Adds discussions settings for new discusions experience [BD-38] [TNL-8621] [BB-4854] (#29131)
* feat: Adds discussions settings for new discusions experience
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.

* fix: tests
2021-10-28 11:56:17 +05:00
kenclary
13818b9630 Merge pull request #29138 from edx/coursegraph-fix-neo4j-dump
fix: default for CertificatesDisplayBehaviors is enum string value, not enum.
2021-10-27 17:31:22 -04:00
edX Transifex Bot
c13cf4cca1 geoip2: update maxmind geolite country database 2021-10-28 02:03:37 +05:00
Ken Clary
bfd09236a2 fix: default for CertificatesDisplayBehaviors is enum string value, not enum.
This should correct a large number of errors encountered when trying
to dump courses to neo4j.

TNL-8386
2021-10-27 14:59:36 -04:00
connorhaugh
b01e773d2b feat: cache course index queries (#29107)
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
2021-10-27 13:53:10 -04:00
David Ormsbee
5d0132305d Revert "feat: Adds discussions settings for new discusions experience (#28749)"
This reverts commit bb0c03123c.
2021-10-27 10:40:12 -04:00
Kshitij Sobti
bb0c03123c feat: Adds discussions settings for new discusions experience (#28749)
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.
2021-10-27 15:41:27 +05:00
Braden MacDonald
6c85668099 feat: write split modulestore's course indexes to Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-26 10:06:52 -07:00
M. Zulqarnain
9833134fae Post Django 3.2 Cleanup (#29069)
* chore: Post Django 3.2 Cleanup
2021-10-22 13:55:16 +05:00
Andrew Shultz
f79e18b442 Merge pull request #29076 from edx/ashultz0/mst-853
feat: leave IDV out of proctoring requirements mail if honorcode
2021-10-21 11:02:38 -04:00
Usama Sadiq
95427251dc fix: fixed pylint warnings 2021-10-21 09:54:22 -04:00
Kyle McCormick
9bf266f717 fix: add missing __init__.py files 2021-10-21 09:54:22 -04:00