The Video Player XBlock will sometimes make API calls to /couses/yt_video_metadata, a REST API endpoint that in turn loads video metadata from YouTube using the configured settings.YOUTUBE_API_KEY.
However, in the Blockstore-based XBlock runtime, we are running XBlocks in a secure sandbox, and the user's browser cannot pass session cookies when calling REST API endpoints. So currently, the video XBlock tries to request YouTube metadata from that API endpoint, but it fails if run within such a sandbox.
The existing API also doesn't work for anonymous users (users who are allowed to see video XBlocks but who have not logged in to an LMS user account).
This commit updates the Video XBlock so that it can use a handler to load the data from YouTube instead of a generic REST API. This works well in the new runtime, because it has code to support calling handlers within the sandbox, including by anonymous users.
I also fixed a bug where on a default devstack, the endpoint will try calling YouTube using PUT_YOUR_API_KEY_HERE as an API key, and get a "bad request" error from YouTube.
The code could be re-organized by moving things around, but I've left everything as-is for now to keep the diff as small as possible.
This fixes a bug that causes the video XBlock parsing to break in the new XBlock runtime if:
* the html5_sources field is set, and
* the download_video field is not set
The error is:
File "/edx/app/edxapp/edx-platform/openedx/core/djangoapps/xblock/rest_api/views.py", line 52, in render_block_view
block = load_block(usage_key, request.user)
File "/edx/app/edxapp/edx-platform/openedx/core/djangoapps/xblock/api.py", line 84, in load_block
return runtime.get_block(usage_key)
File "/edx/app/edxapp/edx-platform/openedx/core/djangoapps/xblock/runtime/blockstore_runtime.py", line 70, in get_block
block = block_class.parse_xml_new_runtime(xml_node, runtime=self, keys=keys)
File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/video_module/video_module.py", line 620, in parse_xml_new_runtime
setattr(video_block, key, cls.fields[key].from_json(val))
KeyError: 'source'
The reason for the error is that parse_video_xml() will sometimes return a 'source' attribute in its field_data return value, even though source is not a video field (anymore). This then causes an error when trying to look up cls.fields['source']
A workaround in the meantime is to add download_video="false" to the OLX.
This adds request caching to the following places:
* course expiration wrapper (displayed in Units)
* offer banner generation (displayed in Units)
* get_enrollment
* user_by_anonymous_id
* youtube_disabled_for_course
On a sample course with edx-val enabled, this reduced the queries
for a large sequence from 450 to 155.
Fix the issue that was preventing us from upgrading pytest. pytest does some manipulation of test packages that prevents `pkg_resources` from loading resources from them, but used to contain a workaround for the problem. That workaround was [removed](https://github.com/pytest-dev/pytest/issues/5392) in 4.6.0 as a performance enhancement when pytest switched from `pkg_resources` to `importlib-metadata` for its own entrypoint handling. This tripped up one of our test modules which defined classes that loaded templates from inside a test package. Moving these resources to the parent package fixes the problem.
More and more, `pkg_resources` is being abandoned in favor of `importlib-metadata` and `importlib_resources` as they have a simpler design with much better performance. However, `importlib_resources` doesn't support loading files from any directory which isn't itself a Python package (and doesn't allow direct use of paths including directories within the package). Jinja2 chose a [different approach](https://github.com/pallets/jinja/pull/1082) that we may want to emulate in our resource handling.
Also fixed usage of a removed `pytest.raises()` parameter and a bug in our configuration of the `common/lib` tests that became a problem after the upgrade.
Quality / encoding
Remove comments
remove debug
Changed service implementation to make unit testing easier
Corrected service
code style
unit test cleanup
Clean up imports
Implementation details:
* Anonymous users are assigned a unique ID (like
`anon42c08f9996194e2a9339`) which gets stored in the django session.
`block.scope_ids.user_id` and `block.runtime.anonymous_student_id`
will both return this value.
* User state for anonymous users is stored in the django cache and
automatically expires as the cache gets pruned. Because user state is
stored, anonymous users can use interactive blocks like capa problems.
* There is no mechanism for upgrading to a registered account and
keeping user state since the user state store for anonymous users
(EphemeralKeyValueStore) is completely different than the one for
registered users (DjangoKeyValueStore/"CSM"), and has no "list all
keys" functionality.
* "User State Summary" field values are shared among [recently active]
anonymous users but are not shared with registered users.
* Anonymous users can only access the `public_view` of XBlocks, not the
regular `student_view`.
This commit upgrades the version of pymongo from 2.x to 3.x, removing usages to deprecated functions usage and fixing tests where necessary.
This version of pymongo supports MongoDB 2.x all the way up to 4.2, and this ensures that the platform will be able to run on a supported MongoDB version in the next release.
When going between python 2 and python 3.5 we can get pickeld course
structires that are incompatible no matter what we do do to the bug
linked in the comment. In this case, handle the error and delete the
corrupt data from the cache.
Making this fairly generic because if we have any bad data in cache we
don't want it to blow up the whole process. Just delete the bad data
and try again.
It was an internal class that we can't easily compare to. So I'm opting
to just make the xml output be more consistent and ordered. I was
hoping to avoid this since the order shouldn't matter but the juggling
we have to do to validate the unorderd data is more complication than
it's worth.
Improves navigation within Studio for Learning Sequences, speeding up authors who want to see how a learner progresses through content without needing to jump over to the LMS.
This adds a dropdown section navigator to the breadcrumbs on the unit page and copies the sequence navigator from LMS to the studio unit page.
This is a follow up from MST-16, which was commited
in 3858036a4e.
Changes:
* Enrich course teams_configuration from a plain Dict
to a custom XBlock field that uses the new TeamsConfig
wrapper class.
* Remove teams_conf property from course, as the previous
change made it redundant.
* Update teams_enabled implementation.
* Remove teams_max_size field from course, which is
no longer semantically correct, as max team size
is now defined on a teamset level.
* Remove teams_topics in order to discourage use of raw
teams config dict.
* Add convenience properties teamsets and teamsets_by_id
to course.
* Allow periods and spaces in teamset IDs to avoid breaking
existing course teams.
Some parts of the code still use the old raw config data
(identifiable by searching "cleaned_data_old_format"),
which we expect to be slowly factored away as we build
new teams features. MST-40 has been created to remove any
remaining references if necessary.
MST-18
* fix: bokchoy test
* fix: remove pdb break
This is to deal with a special issue in the video descriptor xml output.
The value of the transcript attribute of the video tag is a serialized
json string. This can have comparison problems in python3 where the order
of the dictionary output is not gauranteed to be the same. So the strings
don't match equally. We convert the parsed json instead so that the
comparison can be correct.
See the commit that added round_away_from_zero for details.
IN the case of dashboard_data.py:
We don't first cast to decimal here because in python 2 this didn't do
anything. The python 2 rounding method would just cast things to float
before doing the rounding anyway so it just wastes cpu cycles. In
python 3 this causes some typing issues so just removing it.
Wrap CourseModule.teams_configuration in TeamsConfig
class, centralizing parsing, validation, error handling,
etc. Wrapped object is exposed on 'teams_conf' field.
Old code still uses 'teams_configuration' dict;
we should change this in the future (MST-18).
MST-16
