This includes an optimization to the get_bundle_version_files_cached method, which is used very often when loading blockstore data; it was previously being cached only in a process-local cache (lru_cache). My hunch is that in production, with many appservers and LMS workers and frequent deployments and a large number of bundles, the process-local cache is not being hit very often.
I also increased the MAX_BLOCKSTORE_CACHE_DELAY from 60s to 300s; this reduces the frequency with which we check if either (A) an external system modified the blockstore bundle and/or (B) we have a cache invalidation bug somewhere. I am increasing it because that check is more expensive than I thought (calling blockstore API to ascertain latest version of a particular bundle), and I haven't seen any cache invalidation errors that this would help to work around. (Plus, increasing this will make such bugs more obvious.)
Also improve usefulness of some blockstore runtime logs for debugging
Context:
Sometimes when trying to load an XBlock's XML file from Amazon S3, AWS will return a 4xx or 5xx response along with error XML like:
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>foo/bar</Key>...</Error>
A bug in the get_bundle_file_data_with_cache method would cause this XML to be returned to the runtime anyways, as if it were the expected OLX. This would then (obviously) lead to strange parsing bugs, e.g. when trying to interpret <Code> as an <xblock-include>.
This fixes the bug and improves the logging, both to make this sort of issue easier to debug in the future and to return whatever detailed error code S3 provides (or Blockstore, if S3 is not being used).
We upgraded these before in https://github.com/edx/edx-platform/pull/23119 along with django-countries (which released a new version just before that), but had to revert in https://github.com/edx/edx-platform/pull/23237 because of a performance regression in the admin page for EnterpriseCustomer. We think django-countries was the culprit and have pinned it, so trying these two again by themselves.
Add integration settings to enable upload of videos from from studio.
Settings enable user to connect to s3 bucket using mfa and assume role
functionality.
PROD-1214
* Enable pagiation to show all a learner's teams on the "My Teams" tab
* Selectively hide/show pagination on the "My Teams" tab
* Increase "My Teams" page size to 5
This includes an optimization to the get_bundle_version_files_cached method, which is used very often when loading blockstore data; it was previously being cached only in a process-local cache (lru_cache). My hunch is that in production, with many appservers and LMS workers and frequent deployments and a large number of bundles, the process-local cache is not being hit very often.
I also increased the MAX_BLOCKSTORE_CACHE_DELAY from 60s to 300s; this reduces the frequency with which we check if either (A) an external system modified the blockstore bundle and/or (B) we have a cache invalidation bug somewhere. I am increasing it because that check is more expensive than I thought (calling blockstore API to ascertain latest version of a particular bundle), and I haven't seen any cache invalidation errors that this would help to work around. (Plus, increasing this will make such bugs more obvious.)
Also improve usefulness of some blockstore runtime logs for debugging
Context:
Sometimes when trying to load an XBlock's XML file from Amazon S3, AWS will return a 4xx or 5xx response along with error XML like:
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>foo/bar</Key>...</Error>
A bug in the get_bundle_file_data_with_cache method would cause this XML to be returned to the runtime anyways, as if it were the expected OLX. This would then (obviously) lead to strange parsing bugs, e.g. when trying to interpret <Code> as an <xblock-include>.
This fixes the bug and improves the logging, both to make this sort of issue easier to debug in the future and to return whatever detailed error code S3 provides (or Blockstore, if S3 is not being used).
