Rationale: The instructor may create short labels that are longer than
3 characters, and they can be hard to work with in the mobile UI. Thus,
on mobile, it was decided to add short labels to the api response by
getting them from section breakdown, which ensures they are consistent
with the labels the user sees in the Grading section.
* feat: allow editing html block imported from upstream
The modified field is left untouched in future sync while storing the
upstream values in hidden fields to allow authors to revert to upstream
version at any point.
* fix: sync downstream_customized field for copy-pasted modified block
* test: add more tests
* fix: lint issues
* test: copy paste
* feat: skip sync if html data is modified
* feat: update upstream fields only when modified
* refactor: use version_synced field to skip sync
* feat: edit title inplace for library source components
* fixup! feat: edit title inplace for library source components
* fix: edit title button style
* fix: test case
* fix: lint issue
* refactor: don't show different icon for modified upstream blocks
* Revert "refactor: use version_synced field to skip sync"
This reverts commit 8b784fff2f49b43702c952e7f955bd4048e8cc69.
* feat: only skip sync for modified blocks if updated as part of container
* refactor: update sync behaviour when synced individually and as part of parent
* feat: include ready to sync children info in downstream link get api
* test: fix failing tests
* fix: lint issues
* feat: new tests and update api to allow overriding modified fields in sync
* test: api changes
* refactor: edit options should be visible for individual imports
* docs: update api docs
* chore: remove old comments
- Adds the `top_level_parent_usage_key` to the `EntityLinkBase`
- This field is used to save the top-level parent of a component or container when it is imported into a course. Example: A unit with components imported into a course. The unit is the top-level parent of the components.
- Updates the `DownstreamListView` to return the top-level parents instead of downstream child, if this parent exists.
- Each time containers with children were synchronized, a new downstream block was created for each child instead of updating the existing one. This occurred because the `upstream_key` was incorrectly validated as an `Opaquekey` against a list of key strings. This was fixed by converting the `upstream_key` to a string before the verification. (see 34cd5a4781 and 29647831dc)
- Which edX user roles will this change impact? "Course Author", "Developer".
Add ResourceTemplates to XBLOCK_MIXINS so it’s applied to all CMS XBlocks at
runtime instead of being directly inherited. This keeps the Studio-only feature
in edx-platform (where it belongs), while still making it available to built-in
and extracted XBlocks.
When we extract built-in blocks from the platform, they will not be able to
inherit ResourcesTemplates directly; they will get it from XBLOCK_MIXINS. So,
we also needed to update a few template-related tests to use the mixed block
class (or an instance of it) rather than the unmixed base class, because the
unmixed base classes will soon be extracted and thus lack ResourceTemplates.
Related to https://github.com/openedx/edx-platform/issues/34827
Run tests for both the built-in and extracted WordCloud block.
The tests are mostly compatible with both versions of the block,
except for a few places where the XBlock framework and the
built-in XModule system differ which we've had to handle using
conditionals.
This moves us closer to enabling the extracted WordCloud block
by default and eventually removing the built-in block.
Part of: https://github.com/openedx/edx-platform/issues/34840
In the effort to simplify settings in edx-platform, as discussed in ADR 22 -
Settings Simplification, this PR brings some of the production defaults defined
in `lms/envs/production.py` and `cms/envs/production.py` up to
`openedx/envs/common.py` or `lms/envs/common.py` and `cms/envs/common.py` as
appropriate.
Bringing these defaults up from the `production.py` settings modules caused
changes in the rendered settings of the `test.py` modules, and so I have
settings to the `test.py` modules to bring the rendered settings back in line
with what is has been. I have not deeply looked at which settings are needed
for tests to pass or not, but just the differences between the rendered
settings between `master` and this branch.
ADR 22: https://github.com/openedx/edx-platform/blob/master/docs/decisions/0022-settings-simplification.rst
Fixes https://github.com/openedx/edx-platform/issues/36892.
This PR removes the usage of the custom `mongodb_proxy` package from
edx-platform, replacing it with MongoDB's built-in retry functionality via
`retryReads=True`.
Changes:
* Removed all references to `mongodb_proxy.MongoProxy` from connection logic.
* Updated `connect_to_mongodb()` to always use `retryReads=True`.
* Uninstalled `openedx-mongodbproxy` from edx-platform requirements.
Fixes: https://github.com/openedx/edx-platform/issues/35210
fix below bson_obj document size error due to mongo aggregate function by calculating the count and using find instead of aggregate.
```
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: BSONObj size: 19117457 (0x123B591) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: null, full error: {'ok': 0.0, 'errmsg': 'BSONObj size: 19117457 (0x123B591) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: null', 'code': 10334, 'codeName': 'BSONObjectTooLarge', '$clusterTime': {'clusterTime': Timestamp(1750073176, 2), 'signature': {'hash': b'\xd7\xdd\xf4\x7f\xe4\xcea\xa1\xa7P\xba\xab\xcb\x12\x7f1A$(\xb4', 'keyId': 7471973240714297345}}, 'operationTime': Timestamp(1750073176, 2)}
```
This brings an important security improvement -- codejail won't default to
running in unsafe mode, which can happen if certain configuration errors
are present.
Properly configured installations shouldn't be affected. We just need to
adjust some unit tests to opt into unsafe mode.
Changes:
- Update `edx-codejail` dependency to [version 4.0.0](https://github.com/openedx/codejail/blob/master/CHANGELOG.rst#400---2025-06-13)
- Define a `use_unsafe_codejail` decorator that allows running a unit test (or entire TestCase class) in unsafe mode
- Use that decorator as needed, based on which tests started failing
The original issue was that when a sequence was locked due to prerequisites, the API returned an empty items array ([]). This prevented the frontend from knowing what units were inside the locked sequence, meaning it couldn't construct the URLs correctly for navigation so the next/previous buttons stop working.
Some of the calls to `safe_exec` were missing the `limit_overrides_context`
parameter. This normally conveys the course key to codejail so that we can
give some courses different resource limits, but it's also valuable for
diagnosing codejail issues in logs and other telemetry.
I wasn't able to test all of these paths manually, but the utility function
`get_course_id_from_capa_block` will swallow errors, so the situation
should be no worse if the `LoncapaResponse.capa_block` field has something
unexpected.
For darklaunch comparisons where the two sides have different Python
versions, we'll want a more comprehensive list of normalizers.
- Expand the default list to include patterns discovered during a Python
3.8 vs. 3.12 comparison.
- Append the setting value by default, rather than replacing (but still
allow replacing).
- Use default normalizers if custom ones can't be loaded.
- Add log message when loading normalizers fails.
- Validate the replacement pattern, not just the search pattern.
This is intended to make logs more or less a standalone source for
analyzing mismatches.
- Only log mismatches or exceptions
- Merge local and remote log messages into one so they can be correlated
more easily
- Different log messages for mismatch vs. unexpected exceptions
- Include course ID (as limit overrides context) in log message
`unsafely` should be a boolean, not None. I believe this is what was
causing a 400 in remote codejail-service calls (during darklaunch) when
the `calculate_problem_responses_csv.v2` instructor task ran:
https://github.com/edx/edx-arch-experiments/issues/1028
There are also places in the code where this member appears to be a scalar
property instead of a method, or possibly that's just buggy code that
doesn't run. I decided not to touch that in this PR (we're probably going
to DEPR the entire unsafe mechanism anyhow) but if this PR doesn't work,
that's the next place to look -- because higher up in this file there's
a `can_execute_unsafe_code=None` that we could try changing.
* feat: library unit sync
* feat: create component link only for component xblocks
* feat: container link model
* feat: update downstream api views
* feat: delete extra components in container on sync (not working)
* fix: duplicate definitions of LibraryXBlockMetadata
* test: add a new integration test suite for syncing
* feat: partially implement container+child syncing
* fix: blockserializer wasn't always serializing all HTML block fields
* feat: handle reorder, addition and deletion of components in sync
Updates children components of unit in course based on upstream unit,
deletes removed component, adds new ones and updates order as per
upstream.
* feat: return unit upstreamInfo and disallow edits to units in courses that are sourced from a library (#773)
* feat: Add upstream_info to unit
* feat: disallow edits to units in courses that are sourced from a library (#774)
---------
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
* docs: capitalization of XBlock
Co-authored-by: David Ormsbee <dave@axim.org>
* refactor: (minor) change python property name to reflect type better
* fix: lots of "Tried to inspect a missing...upstream link" warnings
when viewing a unit in Studio
* docs: mention potential REST API for future refactor
* fix: check if upstream actually exists before making unit read-only
* chore: fix camel-case var
* fix: test failure when mocked XBlock doesn't have UpstreamSyncMixin
---------
Co-authored-by: Braden MacDonald <braden@opencraft.com>
Co-authored-by: Chris Chávez <xnpiochv@gmail.com>
Co-authored-by: Jillian Vogel <jill@opencraft.com>
Co-authored-by: Rômulo Penido <romulo.penido@gmail.com>
Co-authored-by: Braden MacDonald <mail@bradenm.com>
Co-authored-by: David Ormsbee <dave@axim.org>
- Fix bug where we were overwriting `remote_emsg` with None, and add test
that would have caught it.
- Suppress differences due solely to the codejail sandbox directory name
differing (in stack traces), and add test for this. Configurable because
we'll need to add an additional search/replace pair for the sandbox venv
paths.
- Add a variety of custom attributes, replacing existing ones. The attrs
now have a prefixed naming scheme to simplify searching.
- Add slug to log output so we can more readily correlate traces and logs,
as well as logs across services.
- Fix typo in error message.
- Fix existing import sort order lint.
- Separate test for misconfiguration
- Add helper method for generic dark launch testing
- Test two darklaunch scenarios: Globals interference, and error that would
previously have caused the remote side not to run
- Rename mocks to have our usual `mock_` prefix
- Catch all exceptions, not just Exception, to better prevent errors from
interfering with mainline responses.
- Introduce a separate try block around the monitoring code so that bugs
there don't cause issues.
- Print exception information as well for both sides (but only if not a
SafeExecException, which is redundant with emsg).
Some formatting changes to log messages as well.
Example outputs:
For `1/0`:
```
2025-04-14 17:26:34,239 INFO 10232 [xmodule.capa.safe_exec.safe_exec] [user 3] [ip 172.18.0.1] safe_exec.py:240 - Remote execution in darklaunch mode produces globals={'expect': None, 'ans': '1/0'}, emsg=None, exception=None
2025-04-14 17:26:34,239 INFO 10232 [xmodule.capa.safe_exec.safe_exec] [user 3] [ip 172.18.0.1] safe_exec.py:245 - Local execution in darklaunch mode produces globals={'expect': None, 'ans': '1/0'}, emsg='ZeroDivisionError: division by zero', exception=None
```
For `raise BaseException("hi")`:
```
2025-04-14 17:26:13,359 INFO 10232 [xmodule.capa.safe_exec.safe_exec] [user 3] [ip 172.18.0.1] safe_exec.py:240 - Remote execution in darklaunch mode produces globals={'expect': None, 'ans': 'raise BaseException("hi")'}, emsg=None, exception=None
2025-04-14 17:26:13,359 INFO 10232 [xmodule.capa.safe_exec.safe_exec] [user 3] [ip 172.18.0.1] safe_exec.py:245 - Local execution in darklaunch mode produces globals={'expect': None, 'ans': 'raise BaseException("hi")'}, emsg='hi', exception=BaseException('hi')
```
With codejail-service down, and `out = 1 + 2`:
```
2025-04-14 17:30:28,597 INFO 12484 [xmodule.capa.safe_exec.safe_exec] [user 3] [ip 172.18.0.1] safe_exec.py:241 - Remote execution in darklaunch mode produces globals={'expect': None, 'ans': 'out = 1 + 2', 'out': 3, 'cfn_return': {'input_list': [{'ok': True, 'msg': 'Output:\n3', 'grade_decimal': 1}]}}, emsg=None, exception=CodejailServiceUnavailable('Codejail API Service is unavailable. Please try again in a few minutes.')
2025-04-14 17:30:28,597 INFO 12484 [xmodule.capa.safe_exec.safe_exec] [user 3] [ip 172.18.0.1] safe_exec.py:246 - Local execution in darklaunch mode produces globals={'expect': None, 'ans': 'out = 1 + 2', 'out': 3, 'cfn_return': {'input_list': [{'ok': True, 'msg': 'Output:\n3', 'grade_decimal': 1}]}}, emsg=None, exception=None
```
During dark launch of remote codejail, we want to ensure we always run both
local and remote execution -- otherwise we're missing data for the remote
side in an important situation.
This will help answer the question of whether the unexpected exception
happens on both sides, even though it may not look exactly the same due to
differences in how unexpected errors are handled.
An example input that provokes this in unsafe execution mode is
`raise BaseException("hi")`; in safe execution mode, printing to
`sys.__stdout__` should also produce an appropriate error.
fix: Restructuring to send course_id and score to edx submission
fix: Refactoring of sending information to sent_to_submission
fix: Elimination of unnecessary functions
fix: Added usage comment to ProblemBlock in XqueueInterface constructor
fix: update doc ADR
fix: setting for Quality Others test (ubuntu-24.04, 3.11, 20)
fix: Deprecation for django-trans-escape-filter-parse-error
test: Add @pytest.mark.django_db decorator to test functions
test: Fix for pylint being disabled
fix: updated changes for pylint disable
fix: update error from docs ADR-0005
update: xmodule/docs/decisions/0005-send-data-to-edx-submission.rst
Co-authored-by: Sarina Canelake <sarina@axim.org>
update: xmodule/docs/decisions/0005-send-data-to-edx-submission.rst
Co-authored-by: Sarina Canelake <sarina@axim.org>
fix: Adjusted correction
fix: update date for docs ADR
Revert "fix: update date for docs ADR"
This reverts commit 0b4229c51c4937f95cb407872645dd448df45418.
fix: replace call created_submission to create_external_grader_detail
fix: update test xqueue_submission
fix: add docstring in test_xqueue_submission
fix: update date doc ADR
fix: update version edx-submission 3.8.6
fix: add @pytest.mark.xfail
fix: add 20 chances in test_capa_block:
fix: increase retry attempts for seed generation in ProblemBlockTest
fix: change version to edx-submission lib
fix: new version edx-submission in testings
fix: replace parameter file to files
fix: update variable grader_file_name and points_possible
fix: Adjustment in the is_flag_active function to always take the last record edited in the waffle
fix: wrap large line of code
fix: update function is_flag_active
fix: code style adjustment
fix: changes for 60 retry
feat: use CourseWaffleFlag to determine xqueue callback path
fix: Code style adjustment
fix: remove deprecated xqueue callback route and simplify callback type logic
fix: Deleting a comment in the ADR document
fix: add log in self.block is None
fix: Code style adjustment in log
This PR adds contentstore.use_react_markdown_editor course waffle flag in cms.
This flag helps in enabling the markdown editor in the authoring mfe.
This PR also adds the markdown_edited field in the Problem Xblock to persist the
user's choice of switching to the markdown editor on the authoring MFE.
More details in the authoring MFE PR: openedx/frontend-app-authoring#1805
In the Annotatable XBlock, the <instructions> element was appearing
twice in the student view:
* Once in "annotatable-instructions" (where it should be).
* Again in "annotatable-content" (where annotations and other content are rendered).
The _render_content method processed and rendered the entire XML
data, including <instructions>, without removing it. The
_extract_instructions method, which is responsible for removing
<instructions>, was not called in _render_content, leading to
duplication.
This fix will:
* Prevents duplicate instructions in the student view.
* Maintains the expected behavior of showing instructions only in
"annotatable-instructions".
* No impact on existing annotation functionality.
