Also improve usefulness of some blockstore runtime logs for debugging
Context:
Sometimes when trying to load an XBlock's XML file from Amazon S3, AWS will return a 4xx or 5xx response along with error XML like:
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>foo/bar</Key>...</Error>
A bug in the get_bundle_file_data_with_cache method would cause this XML to be returned to the runtime anyways, as if it were the expected OLX. This would then (obviously) lead to strange parsing bugs, e.g. when trying to interpret <Code> as an <xblock-include>.
This fixes the bug and improves the logging, both to make this sort of issue easier to debug in the future and to return whatever detailed error code S3 provides (or Blockstore, if S3 is not being used).
* Revert "Ran make migration on third_party_auth (#23253)"
This reverts commit 49be65cc58.
* Removing provider.util import
* Removing further provider things
* Adding hash tests
* Removing from provider imports from openedx
* removed all uses of retire_dop_oauth2_models
* Removing provider library from lms, common, and cms
Created/copied function short_token(from django-oauth-provider) and create_hash256 to help with conversion
* WIP: add third-party-auth scope and usage
BOM-1264
* Fix tests now that we do permissions in a more standard way.
Rather than manually setting the permission class we previously
explicitly raised a PermissionDenied exception. The way DRF
permissoning logic works, if we use the WWW-Authenticate header in the
highest priority auth class, it will return a 401 instead of a 403.
* Added test to make sure having permissions gives access to user mapping api
* Test new filters logic.
Ensure that the filters we add to the application access model make it
into the JWT correctly.
* quality fix
* quality fix
* disable pylint warning
* quality fix
* fix indent prob
Co-authored-by: Feanil Patel <feanil@edx.org>
Co-authored-by: Manjinder Singh <49171515+jinder1s@users.noreply.github.com>
* Renamed OAuth2Authentication to BearerAuthentication
* Added back OAuth2Authentication name
-there are libraries such as edx-enterprise that still import OAuth2Authentication. The OAuth2Authentication class should be fully removed when everything is importing BearerAuthentication correctly
- Removing deprecated auth classes
- Replacing OAuth2AuthenticationAllowInactiveUserDeprecated with OAuth2AuthenticationAllowInactiveUser
- the old class was a child class of a deprecated class that we are removing and replacing.
Updating to the current highest number instead of making it the default
which is highest know by the language. This is beacuse when we update
between python versions if there is a new proctocol version, we don't
want that to cause cache thrashing as we try to load things from the
various caches. We can get into a situation where new machines pickle
things with a newer versions of the procotocl, put them in the cache and
then break older machines that don't know about the new protocol
version.
* Added new auth class
- Created new class called OAuth2AuthenticationAllowInactiveUser, which replaces old class with same name
- Renames previous OAuth2AuthenticationAllowInactiveUser to OAuth2AuthenticationAllowInactiveUserDeprecated
- Replaced all imports of OAuth2AuthenticationAllowInactiveUser to call deprecated class instead
- testing new class in discussion(added flag based on django setting)
* Added new authentication class(meets drf standards)
The new class replaces the deprecated oauth2authetnication class from rest_framework_auth library(repo django-rest-framework-oauth).
Majority of the code is combination of copy-pasta from old oauth2authentication class and Oauth2AuthenticationAllowInactiveUser class
* Added ability to switch to new authentication class in bookmarks app
* Changed error type reported by Outh class. It now outputs a json rather than a string.
Currently, we are working on removing the rest_framework_auth library from edx-platform. For this push, we need to remove the oauth2Authentication class. This PR creates a new class oauth2AuthenticationDeprecated that adds additional new relic metrics. The metrics would allow us to see how often this class is used and its success rate. The hope is that this information will help us with transitioning to a different authentication class.
This reverts commit cdb0619846,
which itself reverted 3ca006214e.
The original commit (3ca006) was reverted because it was suspected
that it was causing unexpectedly-increased memcached usage and
500s in the Gradebook API. It is not clear whether that is actually
the case. We are optimistically reinstating 3ca006 and will monitor
production to see if there is an adverse effect.
MST-105
The functions `CourseOverview.get_from_id_if_exists`
and `CourseOverview.get_from_ids if exists` are bug-
prone, in that they do not go to the modulestore
when a CourseOverview is out-of-date or uncached.
The docstrings spell this out, but callers seem
not to understand the implications.
So, in this commit, we:
* Remove both functions
* Add `CourseOverview.get_from_ids` for almost-as-fast
yet safer bulk-fetching of overviews
* Update call sites to use `course_exists` or `get_from_ids`
in place of the old functions.
* Add tests for @verify_course_exists.
This PR should specifically fix an issue where
the program_enrollments API was breaking when the CourseOverview
version was bumped.
CR-1708
Ignore `ResourceWarning` to clear noise in logs until we replace `EdxRestApiClient` with `OAuthAPIClient`. We have too many warnings like this
unclosed <ssl.SSLSocket fd=33, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('*.*.*', 47040), raddr=('*.*.*.*', 443)
We use pickling in conjunction with caches in these cases so we should
be able to fallback to the uncached behavior if the pickled data is
corrupt in some way.
eg. it's a python 2 pickle we don't know how to read.
This is a follow up from MST-16, which was commited
in 3858036a4e.
Changes:
* Enrich course teams_configuration from a plain Dict
to a custom XBlock field that uses the new TeamsConfig
wrapper class.
* Remove teams_conf property from course, as the previous
change made it redundant.
* Update teams_enabled implementation.
* Remove teams_max_size field from course, which is
no longer semantically correct, as max team size
is now defined on a teamset level.
* Remove teams_topics in order to discourage use of raw
teams config dict.
* Add convenience properties teamsets and teamsets_by_id
to course.
* Allow periods and spaces in teamset IDs to avoid breaking
existing course teams.
Some parts of the code still use the old raw config data
(identifiable by searching "cleaned_data_old_format"),
which we expect to be slowly factored away as we build
new teams features. MST-40 has been created to remove any
remaining references if necessary.
MST-18
* fix: bokchoy test
* fix: remove pdb break
This PR introduces some backend python + REST APIs for storing static
asset files along with an XBlock in a content library. It also updates
the new runtime to be able to load such static asset files.
Example use cases:
* Store an image file with an HTML block and then use the image inline
in the HTML block.
* Store a PDF file with an HTML block and provide a link in the HTML for
the learner to download the PDF.
* Store .srt files or even video .mp4 files that belong to a video
XBlock.
Within the bundle, these static asset files are stored in a "static/"
subfolder of the folder that contains the OLX file. Extending an
existing LMS/Studio convention, a static asset file such as "image.png"
is referenced within the OLX as "/static/image.png" and the URL will be
rewritten by the runtime.
Python 3 has no way to access this implementation of rounding in the
standard library. We implement it here so that we can continue to use
it for grades calculation to keep regrading consistent.
Currently we can't be confident that if we change the rounding behaviour
we won't impact students. We can't be sure that students that were
previously passing wouldn't suddenly no longer be passing. Given this,
it's lower risk to just implement the old rounding strategy here and use
it when we are rounding to calculate grades.
Wrap CourseModule.teams_configuration in TeamsConfig
class, centralizing parsing, validation, error handling,
etc. Wrapped object is exposed on 'teams_conf' field.
Old code still uses 'teams_configuration' dict;
we should change this in the future (MST-18).
MST-16
We need to do this because when I tride to go to the JSON serializer a
bunch of tests started failing because various parts of our code are
putting things into the session that are not JSON serializable.
We can't keep using the default pickle serializer because it defaluts to
using the highest available protocol and that will cause issues with the
python 2 to 3 upgrade since both will be running in production at the
same time. We need to use a version of the pickle protocol that both
can use interchangably.
We also need to make sure we read with latin1 encoding to make datetimes
work correctly between the two versions of python.
