Commit Graph

9426 Commits

Author SHA1 Message Date
Jillian Vogel
ad5ad72273 [BD-13] Deprecate ModuleSystem.render_template (fixed) (#29354)
* refactor: deprecates ModuleSystem.render_template

in favor of the added MakoSystem render_template method.

Related changes:
* Adds the MakoService to the StudioEditModuleRuntime,
  PreviewModuleSystem, LmsModuleSystem, and XBlockRuntime
* MakoService constructor takes a `namespace_prefix` string, so that the
  CMS PreviewModuleSystem can render to LMS templates, without needing
  the special render_from_lms helper method.
* ModuleSystem.render_template becomes a read-only property, so the
  constructor calls and test module systems are updated accordingly.
* Adds tests for the MakoService and module system shims.

(cherry picked from commit 457f959356)

* refactor: use MakoService.render_template to remove deprecation warnings

from block code.

(cherry picked from commit 8d62d337f5)

* refactor: use MakoService.render_template to remove deprecation warnings

from test code.

(cherry picked from commit 26b43465a4)

* test: Adds a test to verify the bug introduced by the previous changes

The AuthoringMixin is automatically added to all XBlocks (see
settings.XBLOCK_MIXINS), and AuthoringMixin.visibility_view expects the
"mako" service.

This test verifies the bug by testing the PureXBlock, which does not
require the "mako" service, and so fails when the visibility_view is
rendered.

* fix: AuthoringMixin needs mako service

which fixes the visibility_view for XBlocks which don't explicitly
require the mako service.

Also removes the unneeded class property _services_requested from
AuthoringMixin and StudioEditableBlock. This property is better provided
by the XBlockMixin class.
2021-11-29 14:42:52 -05:00
Kshitij Sobti
e8e8f4acbe feat!: Change the way tabs are ordered [BD-38] [TNL-9174] [BB-5076] (#29262)
* feat!: Change the way tabs are ordered
The change imposes a new ordering for tabs based on their new priority. When reordering tabs, this ordering will be maintained.

* fix: Apply suggestions from code review

Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>

* fix: review feedback

Co-authored-by: Farhaan Bukhsh <farhaan@opencraft.com>
2021-11-22 11:17:30 +05:00
Justin Lapierre
f6f3a79a15 Revert "[BD-13] Deprecate ModuleSystem.render_template" 2021-11-16 15:16:08 -05:00
Piotr Surowiec
4bf829dcbd fix: do not remove comments from XML during course import (#28557)
This is a follow-up to edx#1087, which reverted this change.
According to the PR comments, parsing strings with XML comments inside them was
causing errors. This does not seem to be the case anymore - these strings are
just hidden when the block is rendered, but they are not breaking XBlocks.
This also handles (ignores) the comments that could be added directly to the
LibraryContentBlock in the XML export by users.
2021-11-16 12:42:08 -05:00
Ned Batchelder
d9dd10dc97 style: code cleanups from Steven Burch (#29292)
* chore: update deprecated import from collections

* chore: remove outdated imports from markdown library

as it hasn't been supported since 2.0.3 and we're on 3.x.
This was deprecated at least as early as 2012!

* docs: add docstring and remove lint-amnesty to markdown plugin

* chore: remove deprecated etree import

* style: remove unnecessary-comprehension for sets

* style: resolve a number of amnestied pylint complaints

Co-authored-by: stvn <stvn@mit.edu>
2021-11-10 07:11:57 -08:00
David Ormsbee
2d60224125 Merge pull request #28571 from open-craft/jill/bd-13-render_template
feat!: [BD-13] Deprecate ModuleSystem.render_template
2021-11-09 12:20:59 -05:00
David Ormsbee
0ec2a3b9b5 docs: ADR for removing MongoDB usage. 2021-11-09 10:55:28 -05:00
Jillian Vogel
26b43465a4 refactor: use MakoService.render_template to remove deprecation warnings
from test code.
2021-11-09 14:03:54 +10:30
Jillian Vogel
8d62d337f5 refactor: use MakoService.render_template to remove deprecation warnings
from block code.
2021-11-09 14:03:54 +10:30
Jillian Vogel
457f959356 refactor: deprecates ModuleSystem.render_template
in favor of the added MakoSystem render_template method.

Related changes:
* Adds the MakoService to the StudioEditModuleRuntime,
  PreviewModuleSystem, LmsModuleSystem, and XBlockRuntime
* MakoService constructor takes a `namespace_prefix` string, so that the
  CMS PreviewModuleSystem can render to LMS templates, without needing
  the special render_from_lms helper method.
* ModuleSystem.render_template becomes a read-only property, so the
  constructor calls and test module systems are updated accordingly.
* Adds tests for the MakoService and module system shims.
2021-11-09 14:03:50 +10:30
Braden MacDonald
480e8997ec fix: A test (test_save_one_and_confirm) was flaky 2021-11-08 10:46:33 -08:00
David Ormsbee
15eca3ba59 Merge pull request #29190 from open-craft/jill/BD-13-user-service-fix
[FAL-2457] Let ModuleSystem use UserService
2021-11-08 09:18:10 -05:00
Farhaan Bukhsh
cfe75e95b9 feat: Add discussion_enabled for Unit (#28864)
For each unit discussion_enabled flag is added
so that each unit can be made discussable when needed.

Signed-off-by: Farhaan Bukhsh <farhaan@opencraft.com>
2021-11-08 16:17:07 +05:00
Jansen Kantor
b6c92513c2 style: add blank line (#29199) 2021-11-02 16:05:17 -04:00
Jillian Vogel
f8c96c1df7 chore: pylint 2021-11-02 20:04:33 +10:30
Jillian Vogel
b1b3c646ad refactor: fixes tests
* UserStubService now takes user, user_is_staff, and anonymous_user_id
* get_test_system() creates a UserStubService with an anonymous_user_id of 'student'
* Removes references to deprecated ModuleSystem attributes from test code
* Fixes and simplifies the ConditionalBlock tests, using get_module provided by TestModuleSystem instead of trying to mock out all the pieces.

(cherry picked from commit 927016a8df)
2021-11-02 18:25:52 +10:30
Jillian Vogel
bafc0cd91f refactor: removes calls to deprecated ModuleSystem attributes
Removes references to these deprecated attributes from the platform code:
* runtime.anonymous_student_id
* runtime.seed
* runtime.user_id
* runtime.user_is_staff

Related changes:

* Ensure that all platform XBlocks which use these attributes "need" the user service.
* ProblemBlock: Removes check for existence of runtime.seed attribute in preparation for removal of this attribute from ModuleSystem.
* edxnotes: Catches NoSuchServiceError just in case some XBlocks using notes don't have the user service.
* UserTagsService refactor: pass user and course_id on creation

(cherry picked from commit 753839276f)
2021-11-02 18:24:14 +10:30
Jillian Vogel
cf1064616c refactor: deprecate ModuleSystem user attributes in favor of user service
The following ModuleSystem attributes are deprecated by this change, and should be pulled directly from the user service instead:

* anonymous_student_id
* seed
* user_id
* user_is_staff

Related changes:

* Removes the `user` and `anonymous_student_id` parameters from the ModuleService constructor.
* Stores anonymous_user_id in XBlockDjangoUserService's opt_attr
* Pulls out constants used by DjangoXBlockUserService opt_attr so they can be used in the platform code.
* LmsModuleSystem uses the user service created in wrapper function for runtime.publish to avoid requiring the user
  service to be "needed" by all XBlocks.
* LmsModuleSystem no longer checks for instances of XModuleDescriptor when deciding what kind of anonymous_user_id to
  provide:  all XModules are XBlocks, so this check is unnecessary.
* XBlockRuntime returns a user service when requested
* Adds tests for deprecated ModuleSystem attributes and changes to XBlockDjangoUserService.

(cherry picked from commit c41e7fb93a)
2021-11-02 18:24:06 +10:30
David Ormsbee
3754bd219a fix: force reset of modulestore data for test cases
In #8bdf59b, the SplitModuleStore was given a Django ORM backed store
for active version data (i.e. "which version is currently published?").
This started to cause sporadic test failures depending on the test
ordering, such as this module:

  openedx/features/course_experience/tests/views/test_course_home.py

The root cause was that the database table holding these active versions
was not being properly cleared after tests, probably because of the odd
ordering we do MongoDB vs. Django ORM data initialization in the
ModuleStoreTestCase and SharedModuleStoreTestCase classes. This is an
overly broad hammer fix for this, because:

1. The obvious thing to add it into the ModuleStoreIsolationMixin didn't
   seem to work.
2. While overly broad, it's a small bit of code and should be safe.
3. It's more urgent to fix this flakiness in the build (affecting maybe
   1/4 test runs?) ASAP, rather than tracking this down.
2021-11-01 17:41:08 -04:00
Kshitij Sobti
ff61434893 feat: Add a new way to enable/disable teams [BD-38] [TNL-9175] [BB-5066] (#29099)
* feat: Add a new way to enable/disable teams

Adds a new mechanism for enabling/disabling the team feature in a course using an 'enabled' field to the teams config.
If this field is set to true, teams is enabled (team sets/groups) still need to be defined. If this is set to false then teams is disabled whether or not team sets are defined.

* fix: review feedback
2021-11-01 17:12:06 +05:00
Awais Jibran
5f97bc60cf test: adds discussion API permissions test cases (#29135) 2021-10-28 14:54:32 +05:00
Kshitij Sobti
79cd0b1ef8 feat: Adds discussions settings for new discusions experience [BD-38] [TNL-8621] [BB-4854] (#29131)
* feat: Adds discussions settings for new discusions experience
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.

* fix: tests
2021-10-28 11:56:17 +05:00
Ken Clary
bfd09236a2 fix: default for CertificatesDisplayBehaviors is enum string value, not enum.
This should correct a large number of errors encountered when trying
to dump courses to neo4j.

TNL-8386
2021-10-27 14:59:36 -04:00
connorhaugh
b01e773d2b feat: cache course index queries (#29107)
* feat: store split modulestore's course indexes in Django/MySQL

Course outline generation is calling active_versions hundreds or even thousands of times on course_publish. In practice, the handling of a block requires a fetch of the course index. That requires a call to the active_versions table for each touching of a block. In larger courses, that can mean 1000s of calls to the db to retrieve them. A quick solution to this problem is to use a request cache, so we don't have to query the slow db every time and memoize the relevant data
[An example run can be seen here.](https://one.newrelic.com/launcher/nr1-core.explorer?platform[filters]=IihuYW1lIGxpa2UgJ3Byb2QtZWR4LWVkeGFwcCcgb3IgaWQgPSAncHJvZC1lZHgtZWR4YXBwJyBvciBkb21haW5JZCA9ICdwcm9kLWVkeC1lZHhhcHAnKSBhbmQgKG5hbWUgbGlrZSAnbG1zJyBvciBpZCA9ICdsbXMnIG9yIGRvbWFpbklkID0gJ2xtcycpIg==&platform[accountId]=88178&platform[timeRange][begin_time]=1626715880701&platform[timeRange][end_time]=1627320680701&pane=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb25zIiwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSIsInNlbGVjdGVkU2VyaWVzIjoiZjAzYjNmNzY5OTQ0MjlmOTFhYWQ4MDBkNTEwZTU5MDM5OWNjMzNhMSIsImRyaWxsZG93biI6eyJ0cmFuc2FjdGlvbk5hbWUiOiJPdGhlclRyYW5zYWN0aW9uL0NlbGVyeS9jbXMuZGphbmdvYXBwcy5jb250ZW50c3RvcmUudGFza3MudXBkYXRlX291dGxpbmVfZnJvbV9tb2R1bGVzdG9yZV90YXNrIn19&cards[0]=eyJuZXJkbGV0SWQiOiJhcG0tbmVyZGxldHMudHJhbnNhY3Rpb24tdHJhY2UiLCJ0cmFjZUlkIjoiY2I2OGNkMDktZWI5Yi0xMWViLWJkY2QtMDI0MmFjMTEwMDBlXzI1MjU0MV8zMTgzMTMiLCJlbnRpdHlHdWlkIjoiT0RneE56aDhRVkJOZkVGUVVFeEpRMEZVU1U5T2ZEWTVNVE0zTkRRNE5BIn0=&sidebars[0]=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5hY3Rpb25zIiwic2VsZWN0ZWROZXJkbGV0Ijp7Im5lcmRsZXRJZCI6ImFwbS1uZXJkbGV0cy50cmFuc2FjdGlvbnMifSwiZW50aXR5R3VpZCI6Ik9EZ3hOemg4UVZCTmZFRlFVRXhKUTBGVVNVOU9mRFk1TVRNM05EUTROQSJ9&state=d9946155-ea53-cb11-c1c0-cc873d6c7d39)

Useful information to include:
In theory, this should provide a minor performance boost to authors and learners, and be noticeable in the above function trace, once live on prod. Note, that in several places the cache must be invalidated, as to prevent a stale cache.


Co-authored-by: Braden MacDonald <braden@opencraft.com>
2021-10-27 13:53:10 -04:00
David Ormsbee
5d0132305d Revert "feat: Adds discussions settings for new discusions experience (#28749)"
This reverts commit bb0c03123c.
2021-10-27 10:40:12 -04:00
Kshitij Sobti
bb0c03123c feat: Adds discussions settings for new discusions experience (#28749)
This commit adds new discussions settings for the new discussions experience. These are stored in the course so they can be a part of course import/export flow.
These are also added to the discussions configuraiton API to allow MFEs to update the settings.
The discussions API is currently available via LMS, however that means it cannot save changes to the modulestore. This also adds the API to the studio config so it can now also be accessed from studio and be used to save course settings.
2021-10-27 15:41:27 +05:00
Braden MacDonald
6c85668099 feat: write split modulestore's course indexes to Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

This commit starts writing course indexes (active_versions) to both MySQL and Mongo, but continues to read from MongoDB only.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-26 10:06:52 -07:00
M. Zulqarnain
ba75bb6569 feat: django codemods changes for common folder (#28775) 2021-10-21 13:52:19 +05:00
Usama Sadiq
9ee8df0980 fix: Remove pylint constraint and fix warnings (#28646) 2021-10-20 23:00:13 +05:00
Usama Sadiq
ab4c14afe3 fix: fix pylint warnings (#29049) 2021-10-20 16:28:18 +05:00
Eric Herrera
fe6882819a fix: Address more PR comments
Address @felipemonotoya recommendations related to add versioning to safe_exec_remote functions.
2021-10-15 08:19:48 -05:00
Eric Herrera
fc7104cde5 fix: Address PR comments
Address fixes recommended by @timmc-edx.
2021-10-14 08:11:40 -05:00
Eric Herrera
d672d309e8 fix: Improve and fix codejail service integration
Address recommendations and fixes discussed in PR.
Use similar structure to notes app.
Fix errors detected in tests.
2021-10-14 08:11:40 -05:00
Eric Herrera
3a736eefa2 feat: Codejail as external service (optional)
Add codejail service settings (endpoint and feature flag).
Add conditional to allow running codejail using a REST API service when flag is enabled.
2021-10-14 08:11:40 -05:00
David Ormsbee
ae124bd554 Revert "feat: store split modulestore's course indexes in Django/MySQL"
This reverts commit 96e5ff8dce.
2021-10-07 15:07:42 -04:00
Braden MacDonald
96e5ff8dce feat: store split modulestore's course indexes in Django/MySQL
Split modulestore persists data in three MongoDB "collections": course_index (list of courses and the current version of each), structure (outline of the courses, and some XBlock fields), and definition (other XBlock fields). While "structure" and "definition" data can get very large, which is one of the reasons MongoDB was chosen for modulestore, the course index data is very small.

By moving course index data to MySQL / a django model, we get these advantages:
* Full history of changes to the course index data is now preserved
* Includes a django admin view to inspect the list of courses and libraries
* It's much easier to "reset" a corrupted course to a known working state, by using the simple-history revert tools from the django admin.
* The remaining MongoDB collections (structure and definition) are essentially just used as key-value stores of large JSON data structures. This paves the way for future changes that allow migrating courses one at a time from MongoDB to S3, and thus eliminating any use of MongoDB by split modulestore, simplifying the stack.
2021-10-07 10:59:47 -04:00
Aarif
7d2cc016ec fix: added type annotations to fix mypy failure in CI (#28933) 2021-10-05 17:04:11 +05:00
ha-D
d3bc4601ae feat: Add course-wide custom scripts
Imlements OEP-15 by adding two fields to the course settings:
- Course-wide Custom JS
- Course-wide Custom CSS
The resources defined in these fields will be rendered in all course pages.

Rebase b6cb629849..0578e1c4c6 onto b6cb629849:
- Add course-wide resources to API for MFE use
- Revert "Add course-wide resources to API for MFE use" reverts commit 53648dcf0afe3cd171c9dc2eb5e56b871b2bcfb2

Signed-off-by: Gabor Boros <gabor.brs@gmail.com>
2021-10-04 11:24:45 -07:00
Agrendalath
4b8421dfd8 fix: handle invalid source library version on course import
Importing a course with an invalid source library version (e.g. created on
another instance) should not make the import fail.
2021-09-30 10:19:56 -07:00
David Ormsbee
deaf80f346 revert: "[BD-13] Let ModuleSystem use UserService" (#28857)
This caused a regression where anonymous_id values were not as expected.
Reverting for now, with investigation to follow.
2021-09-27 15:42:35 -04:00
David Ormsbee
3827f4c90d Merge pull request #28440 from open-craft/jill/BD-13-user-service
[BD-13] Let ModuleSystem use UserService
2021-09-24 09:26:43 -04:00
Usama Sadiq
78799f25ee fix: Fixed new pylint warnings (#28726) 2021-09-23 17:54:27 +05:00
Usama Sadiq
023f4144ab fix: fix new pylint warnings (#28725) 2021-09-23 17:54:15 +05:00
Jillian Vogel
927016a8df refactor: fixes tests
* UserStubService now takes user, user_is_staff, and anonymous_user_id
* get_test_system() creates a UserStubService with an anonymous_user_id of 'student'
* Removes references to deprecated ModuleSystem attributes from test code
* Fixes and simplifies the ConditionalBlock tests, using get_module provided by TestModuleSystem instead of trying to mock out all the pieces.
2021-09-23 12:04:03 +09:30
Jillian Vogel
753839276f refactor: removes calls to deprecated ModuleSystem attributes
Removes references to these deprecated attributes from the platform code:
* runtime.anonymous_student_id
* runtime.seed
* runtime.user_id
* runtime.user_is_staff

Related changes:

* Ensure that all platform XBlocks which use these attributes "need" the user service.
* ProblemBlock: Removes check for existence of runtime.seed attribute in preparation for removal of this attribute from ModuleSystem.
* edxnotes: Catches NoSuchServiceError just in case some XBlocks using notes don't have the user service.
* UserTagsService refactor: pass user and course_id on creation
2021-09-23 11:46:43 +09:30
Jillian Vogel
c41e7fb93a refactor: deprecate ModuleSystem user attributes in favor of user service
The following ModuleSystem attributes are deprecated by this change, and should be pulled directly from the user service instead:

* anonymous_student_id
* seed
* user_id
* user_is_staff

Related changes:

* Removes the `user` and `anonymous_student_id` parameters from the ModuleService constructor.
* Stores anonymous_user_id in XBlockDjangoUserService's opt_attr
* Pulls out constants used by DjangoXBlockUserService opt_attr so they can be used in the platform code.
* LmsModuleSystem uses the user service created in wrapper function for runtime.publish to avoid requiring the user
  service to be "needed" by all XBlocks.
* LmsModuleSystem no longer checks for instances of XModuleDescriptor when deciding what kind of anonymous_user_id to
  provide:  all XModules are XBlocks, so this check is unnecessary.
* XBlockRuntime returns a user service when requested
* Adds tests for deprecated ModuleSystem attributes and changes to XBlockDjangoUserService.
2021-09-23 11:41:43 +09:30
Matjaz Gregoric
5487ff5906 Fix RTL issue with capa problems (#28562)
* fix: resolve styling cap RTL styling issue

Margins were incorrect in the RTL version of the stylesheet.

* fix: move style overrides from lms-course to capa

These styles were only applied to the LTR version of the stylesheet, but
we want them in the RTL version as well.
2021-09-21 10:44:35 -04:00
sarina
0c74d347f7 fix: abstract edX-specific strings 2021-09-19 14:01:20 -04:00
Michael Terry
cd7d13cd6d feat: notify MFE when a sequence is hidden-after-due
Currently, if a learner manually loads a sequence page that would
normally be skipped for them because it is hidden-after-due, the
sequence renders anyway.

This commit tells the frontend when it should not show a sequence
because it's hidden.

AA-1000
2021-09-09 14:43:00 -04:00
DawoudSheraz
e986d360c9 refactor: move course location util to common djangoapp instead of xmodule 2021-09-03 12:40:28 +05:00