build: Clear out stale dependencies when installing requirements (#31080)

We've been seeing some cross-version caching issues in the
edx-platform-runner unit tests, which apparently run on a "dirty"
environment—the virtualenv does not get cleared out between runs,
which probably improves performance but allows installed dependencies
to "leak" between runs. This results in errors between master and older
open-releases but could also prevent us from noticing missing deps.

By using pip-sync in the new CI Make targets (as we already do for
the regular `make requirements` target) we can ensure that any stale
dependencies from runs by other branches (or older versions of the
code) are removed.

Calling `make local-requirements` at the end of each `*-requirements`
target rather than making it a prerequisite is necessary for using sync,
since otherwise the local reqs would be wiped out.

The `requirements` target is also deduplicated into the newer
`dev-requirements` (aliased to it, with both installing private deps now.)

Adding a prerequisite of `pre-requirements` allows us to simplify some
workflow calls slightly. This ends up being the bulk of the commit by
line count. The pip lockfile also wasn't being used in the Makefile, so
I added that to pre-requirements as well.

Also fix leading whitespace issue in Makefile.
This commit is contained in:
Tim McCormack
2022-10-03 20:26:40 +00:00
committed by GitHub
parent d2377d348a
commit 2edbdcf7f4
8 changed files with 14 additions and 19 deletions

View File

@@ -63,7 +63,6 @@ jobs:
- name: Install Required Python Dependencies
run: |
pip install -r requirements/pip.txt
make base-requirements
- uses: c-hive/gha-npm-cache@v1

View File

@@ -50,7 +50,6 @@ jobs:
- name: Install Python dependencies
run: |
pip install -r requirements/pip.txt
make dev-requirements
pip uninstall -y mysqlclient
pip install --no-binary mysqlclient mysqlclient

View File

@@ -60,7 +60,6 @@ jobs:
- name: Install Python dependencies
run: |
pip install -r requirements/pip.txt
make dev-requirements
pip uninstall -y mysqlclient
pip install --no-binary mysqlclient mysqlclient

View File

@@ -57,7 +57,6 @@ jobs:
- name: Install required Python dependencies
run: |
pip install -r requirements/pip.txt
make dev-requirements
pip uninstall -y mysqlclient
pip install --no-binary mysqlclient mysqlclient

View File

@@ -49,7 +49,6 @@ jobs:
- name: Install Required Python Dependencies
run: |
pip install -r requirements/pip.txt
make base-requirements
- name: Initiate Mongo DB Service

View File

@@ -61,7 +61,6 @@ jobs:
- name: install requirements
run: |
sudo pip install -r requirements/pip.txt
sudo make test-requirements
if [[ "${{ matrix.django-version }}" == "pinned" ]]; then
sudo pip install -r requirements/edx/django.txt

View File

@@ -17,7 +17,6 @@ jobs:
- uses: actions/checkout@v2
- name: install requirements
run: |
sudo pip install -r requirements/pip.txt
sudo make test-requirements
- name: verify unit tests count