On a platform that is configured to upload video transcripts to S3
(`DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"`),
uploads from the studio fail with a TypeError: "Unicode-objects must be
encoded before hashing"
A full stacktrace of the issue can be found here:
https://sentry.overhang.io/share/issue/2249b6f67d794c7e986cc288758f4ebe/
This error is triggered by md5 hashing in the botocore library, which
itself is used by the S3Boto3Storage storage class. This error does not
occur with filesystem-based uploads because it does not perform checksum
verification. The reason why this error would not occur on edx.org is
unknown. Similar issues were already fixed from edxval.
To address this issue, we encode the transcript file content prior to
sending it to s3.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
Treat transcript content as unicode strings and convert them at any edge
where we encounter them. One decision made here was to not update
edx-val it treats transcripts as byte but is much closer to the actual
files so it makes more sense over there. But within the platform they
are generally passed around as serialized json and so it's much better
for them to be unicode.
Adds val trancripts in outcome of get_transcripts_info and rest of flow remains the same and also add fallback to edx-val rtanscripts for mobile accessible video endpoints.
Purpose of this, is to serve transcripts from edx-val if the requested transcripts are not found in the contentstore i.e. that's where all the transcripts are living to date.
- has_course_access renamed to has_course_author_access for clarity
- Changed doc string to clearly state that it determines whether or not
a user has write access to a course
This commit adds all of cms.
These keys are now objects with a limited interface, and the particular
internal representation is managed by the data storage layer (the
modulestore).
For the LMS, there should be no outward-facing changes to the system.
The keys are, for now, a change to internal representation only. For
Studio, the new serialized form of the keys is used in urls, to allow
for further migration in the future.
Co-Author: Andy Armstrong <andya@edx.org>
Co-Author: Christina Roberts <christina@edx.org>
Co-Author: David Baumgold <db@edx.org>
Co-Author: Diana Huang <dkh@edx.org>
Co-Author: Don Mitchell <dmitchell@edx.org>
Co-Author: Julia Hansbrough <julia@edx.org>
Co-Author: Nimisha Asthagiri <nasthagiri@edx.org>
Co-Author: Sarina Canelake <sarina@edx.org>
[LMS-2370]
Fix donwload subs for non youtube videos and non-en language - continue.
Add acceptance tests.
Add detetion of assets on request.
Updated docstring.
Add fixes and acceptance tests.
Fix acceptance tests.
Update docsrtings and cleanup code, resful for language_id.
Specify exception type in POST.
Fix url in upload module.
Improve exception handling.
Remove 'en' and catching in editable_metadata.
Move descriptor.get_context test to lms tests.
Add query parameter to translation dispatch.
Response to format parameter of translatin GET request.
Fix Acceprance test: Metadata Editor.
move handlers to proper scores.
Split video player into smaller files.
Add ugettext and fix typoes.
Add changelog.
Support for downloading non-ascii filenames.
Change event binding.
Add content-language to download requests.
Reractor POST handler to not update self.transcripts.
