Deprecated edx-platform import paths
(for example, `student` instead of
`common.djangoapps.student`) currently raise
warnings when used. We want to fully remove
support for those paths.
As an easily reversible way to initially remove
support, we add a new setting to LMS and Studio
called `ERROR_ON_DEPRECATED_EDX_PLATFORM_IMPORTS`,
defaulting to False. We set it to True for devstack
and will set it to True in Stage and Production
soon. If critical errors occur, we can easily
flip the setting back to False.
* Generate common/djangoapps import shims for LMS
* Generate common/djangoapps import shims for Studio
* Stop appending project root to sys.path
* Stop appending common/djangoapps to sys.path
* Import from common.djangoapps.course_action_state instead of course_action_state
* Import from common.djangoapps.course_modes instead of course_modes
* Import from common.djangoapps.database_fixups instead of database_fixups
* Import from common.djangoapps.edxmako instead of edxmako
* Import from common.djangoapps.entitlements instead of entitlements
* Import from common.djangoapps.pipline_mako instead of pipeline_mako
* Import from common.djangoapps.static_replace instead of static_replace
* Import from common.djangoapps.student instead of student
* Import from common.djangoapps.terrain instead of terrain
* Import from common.djangoapps.third_party_auth instead of third_party_auth
* Import from common.djangoapps.track instead of track
* Import from common.djangoapps.util instead of util
* Import from common.djangoapps.xblock_django instead of xblock_django
* Add empty common/djangoapps/__init__.py to fix pytest collection
* Fix pylint formatting violations
* Exclude import_shims/ directory tree from linting
Some tests were still relying on deprecated
WaffleSwitchNamespace.override method, which was working before because
we were importing WaffleSwitchNamespace from waffle_utils.__init__. This
no longer works after we import WaffleSwitchNamespace from edx-toggles.
The old folder name is somewhat confusing, because the
folder contains shims to _compensate for the removal
of sys.path hacks_, but does not contain the sys.path
hacks themselves.
Furthermore, this import_shims/ system could also be
used for other import path changes, such as turning
the locally-installed packages in common/lib/
into regular, importable modules
(e.g. `from common.lib.xmodule import abc` instead of
`from xmodule import abc`). So, a name that is not
specific to the sys.path hacks may be better
in the medium-to-long term.
Along the same lines, we also rename SysPathHackWarning
to DeprecatedEdxPlatformImportWarning.
This change will prevent Library Content from being marked as
complete on view and the corresponding version bump to
edx-completion contains code that will start looking at the
children of the library content for completeness.
Fixing 56 GuessedAtParserWarnings, in commit edx#24098
Background: BeautifulSoup automatically picks the fastest parser available. By default, it picks the "lxml" parser.
Per the [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-a-parser) documentation:
> Beautiful Soup supports the HTML parser included in Python’s standard library, but it also supports a number of third-party Python parsers. One is the lxml parser. Depending on your setup, you might install lxml with one of these commands.
> Another alternative is the pure-Python html5lib parser, which parses HTML the way a web browser does.
Context: We changed two statements, one in lms and another in openedx. Both statements fire up BeautifulSoup. Now we explicitly ask for "lxml," following the recommendation on BeautifulSoup's documentation:
> If you can, I recommend you install and use lxml for speed. If you’re using a very old version of Python – earlier than 2.7.3 or 3.2.2 – it’s essential that you install lxml or html5lib. Python’s built-in HTML parser is just not very good in those old versions.
Before:
`soup = BeautifulSoup(content)`
After:
`soup = BeautifulSoup(markup=content, features="lxml")`
The warnings are gone, tests are passing in local.
1. Created a new celery queue with key `SOFTWARE_SECURE_VERIFICATION_ROUTING_KEY`.
2. Added a celery task with retry logic.
3. sorted imports with isort.
4. Changed deprecated `log.warn` => `log.warning`.
Python3 json can not work with bytes type input so used simplejson
instead of python json which works with both bytes and unicode
type input.
Previously we mistakenly updated StringIO to six.String io. which
caused the tests failure. Used io.BytesIO to fix the issues as
assci character bytes can not be converted to unicode.
