The requirements/edx/private.txt file is for dev's own private package
needs. There are two installation mechanisms in edx-platform, and
neither handled the file properly:
- `paver install_prereqs` had the wrong file name. The file was moved
almost three years ago, and paver wasn't kept up.
- `make requirements` used `private.*` which included private.in, which
pip-sync balks at.
Part of the notifier service deprecation (DEPR-106).
Also removed pdfminer from the package uninstall list, since we no longer install the package it conflicts with either.
enum34 is causing problems in python3.8 tests of edx-platform because it has incompatibility with recent versions of python, the error the error happens while importing the re module in these python versions.
This library is a backport of enum of python3.4 made for python<3.4, therefore is not needed anymore.
edx-val was updated to remove that dependency: https://github.com/edx/edx-val/pull/245
'npm install' command execution was changed from paver.easy.sh()
to subprocess.Popen() in commit 0dd13fad1b,
but exception handling was not updated accordingly. Due to this,
errors in 'npm install' execution were not detected.
Unpin several more outdated dependencies whose changelogs don't contain any significant backwards incompatible changes. Also add "moto" to the list of packages to uninstall from existing environments, since it requires a jsondiff version that clashes with the one we now use (triggering a harmless but distracting warning on dependency updates).
We can potentially stop using path.py/path altogether by switching to pathlib in the Python 3 standard library, but that merits a separate PR of its own.
Also, note that I'm not actually unpinning freezegun; different PRs restricted it in both constraints.txt and test.in, I'm just removing the latter redundant constraint.
Fix the issue that was preventing us from upgrading pytest. pytest does some manipulation of test packages that prevents `pkg_resources` from loading resources from them, but used to contain a workaround for the problem. That workaround was [removed](https://github.com/pytest-dev/pytest/issues/5392) in 4.6.0 as a performance enhancement when pytest switched from `pkg_resources` to `importlib-metadata` for its own entrypoint handling. This tripped up one of our test modules which defined classes that loaded templates from inside a test package. Moving these resources to the parent package fixes the problem.
More and more, `pkg_resources` is being abandoned in favor of `importlib-metadata` and `importlib_resources` as they have a simpler design with much better performance. However, `importlib_resources` doesn't support loading files from any directory which isn't itself a Python package (and doesn't allow direct use of paths including directories within the package). Jinja2 chose a [different approach](https://github.com/pallets/jinja/pull/1082) that we may want to emulate in our resource handling.
Also fixed usage of a removed `pytest.raises()` parameter and a bug in our configuration of the `common/lib` tests that became a problem after the upgrade.
to accelerate installation, people usually set npm registry to a faster mirror by running 'npm config set registry BLAH_BLAH'
and even if the customized config not available, by default it is 'NPM_REGISTRY = "https://registry.npmjs.org/"', so delete these lines always works.
The NO_PYTHON_UNINSTALL environment variable can now be used to disable the task that removes older versions of Python libraries. This task is not necessary for Docker images/containers since we always create a fresh virtualenv when we create the image.
These are the only two egg names that are wrong in this file. We have
to uninstall i18n-tools, because the distribution name changed. The
lti-consumer XBlock hasn't changed named, we just had the wrong name in
the requirements file.
Problem:
Intermittent "cb() never called!" errors on npm installs for platform.
This is likely due to a race condition on installing OS-specific bits that are built during time of install. When more than one compilation takes place at one time, this error can occur.
Workaround:
Detect the error and re-run the install. (When retrying, one or all of the bits have been compiled. So running it again just finishes the install.)
This change:
* gives us the flexibility of managing node from pull-request to pull-request. It'll be a huge lift to folks that want to test out node upgrades like @bjacobel or @andy-armstrong
* Sets us up for a port to Xenial, which will be accomplished in part by using nodeenv for managing the node environment. See a companion PR on the configuration repo for edxapp: edx/configuration#3444
This pull request does not:
* Attempt to solve any stability issues in builds related to node.
* Make node installs any faster
* Create any caching infrastructure to speed up node installs. They will be approximately as fast as they were before. Downloading the node bits build-to-build adds about 3 seconds, which is worth it for the added flexibility.
