The last time we tried this upgrade we encountered timeouts on the quality job, which it now appears were due to the worker running pylint common running out of memory and killing the Jenkins process. Switching to a different worker type with double the RAM (8 GB vs. 4 GB) seems to have fixed this; about 5.5 GB was used. Upstream is aware of the high memory usage on large projects, it's apparently due primarily to a cache of parsed modules: https://github.com/PyCQA/pylint/issues/1495 .
Even after disabling some of the new checks that have been added, the new version of pylint found about twice as much to complain about. Just bumping the threshold for now to unblock the Django upgrade, we can try automated utilities like pyupgrade to fix some of these later.
Remove duplicate constraints in requirements/constraints.txt uncovered by the recent change in pip-compile output format. I sorted the pinned dependencies by name to try and make it more obvious if this happens again. I also upgraded to pip-tools 4.5.1, which removes the line numbers from the enhanced pip-compile output added in 4.5.0, which should reduce future diff churn and merge conflicts but means that there's a large diff this one last time.
Also unpin edx-search again after the previous change to do that was apparently broken by a merge conflict.
- Reimplement `djcelery.common.respect_language` utility function so we
can drop the dependency.
- Loosen `celery` pinning to a range, which brings in a fix for a
possible crash bug
Also, pin `path` to 13.1.0 for now in order to continue supporting py35.
There is a planned edx-drf-extension update to 3.0.0 to remove the
toggle used for ENFORCE_JWT_SCOPES. Currently, edx-platform is dependent
on the this toggle. This constraint will keep other `make upgrade` PRs
working in the short window before the edx-paltform PR to remove the
same toggle can land.
BOM-1324
-Updated middleware setting to use use middleware insted of
middleware_classes
-github install of django-method-override fork to support
new style middleware in django1.11
Fix constraints to not have a lingering pin on edx-when. And
add constraint on version of mock that supports python 3.5.
And then a general make upgrade.
Unpin several more outdated dependencies whose changelogs don't contain any significant backwards incompatible changes. Also add "moto" to the list of packages to uninstall from existing environments, since it requires a jsondiff version that clashes with the one we now use (triggering a harmless but distracting warning on dependency updates).
We can potentially stop using path.py/path altogether by switching to pathlib in the Python 3 standard library, but that merits a separate PR of its own.
Also, note that I'm not actually unpinning freezegun; different PRs restricted it in both constraints.txt and test.in, I'm just removing the latter redundant constraint.
* Added pytest-json-report plugin
- modifying app-opts in setup.cfg
- adding hook to all conftest.py files in repo
- setting report to be saved to test_root/log/warnings.json
- Writing custom logic to save json report to avoid overwrite if pytest called twice
This was created to allow us to easily parse through test warnings in jenkins
Fix the issue that was preventing us from upgrading pytest. pytest does some manipulation of test packages that prevents `pkg_resources` from loading resources from them, but used to contain a workaround for the problem. That workaround was [removed](https://github.com/pytest-dev/pytest/issues/5392) in 4.6.0 as a performance enhancement when pytest switched from `pkg_resources` to `importlib-metadata` for its own entrypoint handling. This tripped up one of our test modules which defined classes that loaded templates from inside a test package. Moving these resources to the parent package fixes the problem.
More and more, `pkg_resources` is being abandoned in favor of `importlib-metadata` and `importlib_resources` as they have a simpler design with much better performance. However, `importlib_resources` doesn't support loading files from any directory which isn't itself a Python package (and doesn't allow direct use of paths including directories within the package). Jinja2 chose a [different approach](https://github.com/pallets/jinja/pull/1082) that we may want to emulate in our resource handling.
Also fixed usage of a removed `pytest.raises()` parameter and a bug in our configuration of the `common/lib` tests that became a problem after the upgrade.
- Upgrade edx-drf-extensions to 2.4.5
- Removed constraint to 2.4.0, because 2.4.2 introduces a workaround for
ARCH-1210 by putting the problematic code behind a django setting.
- Remove unused JWT_AUTH_REFRESH_COOKIE setting.
ARCH-418, ARCH-1269, ARCH-1044
fix broken toggle
