Video SJSON transcripts are supposed to be UTF-8 encoded, but SJSON
is an ad hoc thing we made up to make it easier to build the
transcripts viewer in the VideoBlock, and it's not well specified.
Prior to this commit, if you had an SJSON file with Latin-1 encoded
text outside the standard ASCII range (e.g. û), then we'd error out
while trying to export it.
This was blocking an effort to export some Old Mongo courses (TNL-8007).
A variety of updates were made to improve the toggle documentation:
* Added comments to help ensure that the waffle(), waffle_switches(),
waffle_flags() anti-pattern won't be contagious (copied).
* Some minor toggle_description updates.
* Removed empty toggle_target_removal_date annotations for
non-temporary toggles.
* Removed empty optional toggle_warnings annotations.
* Removed empty optional toggle_tickets annotations.
* Removed deprecated toggle_category, toggle_status,
and toggle_expiration_date annotations.
* Fixed some indents, use cases, and implementations.
ARCHBOM-1721
We're fixing an error that appears when you click the "edit access"
button on an XBlock in the library page.
Libraries in the "Add library" page shouldn't have the accessibility
config icon. Access settings for libraries are changed at the top, on
Settings > User Access. When clicking the access-button (the cog) on a
Library, an error appears. This is known to happen on koa.master and
open-release/juniper.3.
This happens due to the way XBlocks are rendered; as they get rendered
initially, the previews are unaware of whether they're part of a
library. We can confirm this by refreshing the page, this makes the
button disappear.
This fix removes the cogwheel when the XBlock is part of a library.
This doesn't affect the XBlocks imported to courses because those are
rendered on a Randomized Content XBlock, which does have its access
options.
Authored by: Daniel Francis <daniel.francis@opencraft.com>
tests are failings and complaining related objects doest not exists in
User table. Create object in test setup to fix it.
In another fix article id was giving integrity error.
Fixing task test.
tests are failings and complaining related objects doest not exists in User table. Create object in test setup to fix it.
In another fix article id was giving integrity error.
Fixing task test.
* Introduces the idea of content errors into the learning_sequences
public API, accessible using get_content_errors().
* Makes course outline generation much more resilient to unusual
structures (e.g. Section -> Unit with no Sequence in between),
with the understanding that anything that doesn't conform to the
standard structure will simply be skipped.
* Improves the Django Admin for learning_sequences to display
content errors and improve sequence data browsing within a course.
* Switches the main table viewed in the Django admin from
LearningContext to CourseContext, which is appropriate since only
course runs generate outlines.
This was done as part of TNL-8057, with the end goal of making
course outline generation resilient enough to switch over apps
to using the learning_sequences outline API. The types of course
structure errors that this PR addresses cause display issues even
in the current Outline Page experience, but would break the outline
generation for learning_sequences altogether.
The approach for error messages here is very generic, to keep
modulestore concepts from seeping into learning_sequences (which is
not aware of the modulestore/contentstore). We may need to address
this later, with a more normalized content error data model.
While the Django admin page is backwards compatible with the old
versions of the models, we should run the backfill_course_outlines
management command after deploying this change, to get the full
benefits.
In https://github.com/edx/edx-platform/pull/25955 `HiddenDescriptor`
(which was a subclass of `RawDescriptor` with a custom `student_view()`)
was converted to an XBlock. It is used as the `default_class` by the
`CachingDescriptorSystem` classes. However `RawDescriptor` is still
being used by `XMLModuleStore`. This has been replaced by
`HiddenDescriptor` as well.
The Studio UI prevents you from creating a Section or Subsection
with no title (display_name). But OLX import allows you to bypass
these checks and create Sections ("chapter" tag) and Subsections
("sequential" tag) without display_name information specified in
the XML. When this happens, Studio and the LMS fall back to using
the url_name (the last part of the UsageKey) as a title, using the
display_name_with_default method.
This usually works, because url_names are derived from the import
file name, and if you're hand-editing a course in XML, your file
names are probably more intelligible than Mongo object IDs. In any
case, this commit updates get_outline_from_modulestore to match the
behavior of Studio and the LMS with respect to this situation.
This is part of the course outlines backfill rollout. TNL-8056
* Adds the backfill_course_outlines management command to contentstore
* Adds a read-only Django admin interface to learning_sequences for the
support team and debugging.
* Adds two new functions to the learning_sequences public API:
key_supports_outlines and get_course_keys_with_outlines
The learning_sequences app isn't supposed to know about contentstore or
modulestore, as it's intended to be extracted out of edx-platform in the
long term. Therefore, the backfill_course_outlines command is in
contentstore, and not learning_sequences.
This work was tracked in TNL-7983, but it also fixes a bug where we were
trying to generate course outlines for libraries (TNL-7981).
All Open edX instances upgrading to Lilac should run the
backfill_course_outlines command as part of their upgrade process.
django-not-configured is an error raised by pylint (with
the pylint-django plugin) when it's not correctly configured.
We should not be applying lint amnesty for such a violation.
This commit removes several waffle toggles that have been enabled
on edx.org for years. It's time to remove the rollout gating for
these features and enable them by default.
This doesn't directly change any behavior. But it does create new
database objects by default now and allows for enabling other
schedule based features more easily.
Specifically, the following toggles were affected.
schedules.create_schedules_for_course
- Waffle flag removed as always-enabled
- We now always create a schedule when an enrollment is created
schedules.send_updates_for_course
- Waffle flag removed as always-enabled
- Course update emails are sent as long as the ScheduleConfig
allows it.
- This is not a change in default behavior, because ScheduleConfig
is off by default.
dynamic_pacing.studio_course_update
- Waffle switch removed as always-enabled
- Course teams can now always edit course updates directly in Studio
ScheduleConfig.create_schedules
ScheduleConfig.hold_back_ratio
- Model fields for rolling out the schedules feature
- Schedules are now always created
- This commit only removes references to these fields, they still
exist in the database. A future commit will remove them entirely
This commit also adds a new has_highlights field to CourseOverview.
This is used to cache whether a course has highlights, used to
decide which course update email behavior they get. Previously every
enrollment had to dig into the modulestore to determine that.
The learning_sequences app has its own model for Course Outlines.
Prior to this commit, these course outlines were only populated by
a management command in the learning_sequences app that queried
modulestore. This commit does a few things:
1. Move the update_course_outline command to live in contentstore
(i.e. Studio). This makes learning_sequences unaware of
modulestore, and makes it easier for us to extract it from
edx-platform (or to plug in different kinds of course outlines).
2. Add tests.
3. Add performance and debug logging to course outline creation.
4. Make course outline creation happen every time a course publish
happens.
This will allow us to start collecting data about how long building
course outlines takes, and get error reporting around any content
edge cases that break the course outline code.
* import task decorator from celery APP instance instead of celery package in CMS
* replaced task decorator with shared_task in cms and common
* Fixed import of shared_task
* Fixed import
On a platform that is configured to upload video transcripts to S3
(`DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"`),
uploads from the studio fail with a TypeError: "Unicode-objects must be
encoded before hashing"
A full stacktrace of the issue can be found here:
https://sentry.overhang.io/share/issue/2249b6f67d794c7e986cc288758f4ebe/
This error is triggered by md5 hashing in the botocore library, which
itself is used by the S3Boto3Storage storage class. This error does not
occur with filesystem-based uploads because it does not perform checksum
verification. The reason why this error would not occur on edx.org is
unknown. Similar issues were already fixed from edxval.
To address this issue, we encode the transcript file content prior to
sending it to s3.
