2edbdcf7f4
We've been seeing some cross-version caching issues in the edx-platform-runner unit tests, which apparently run on a "dirty" environment—the virtualenv does not get cleared out between runs, which probably improves performance but allows installed dependencies to "leak" between runs. This results in errors between master and older open-releases but could also prevent us from noticing missing deps. By using pip-sync in the new CI Make targets (as we already do for the regular `make requirements` target) we can ensure that any stale dependencies from runs by other branches (or older versions of the code) are removed. Calling `make local-requirements` at the end of each `*-requirements` target rather than making it a prerequisite is necessary for using sync, since otherwise the local reqs would be wiped out. The `requirements` target is also deduplicated into the newer `dev-requirements` (aliased to it, with both installing private deps now.) Adding a prerequisite of `pre-requirements` allows us to simplify some workflow calls slightly. This ends up being the bulk of the commit by line count. The pip lockfile also wasn't being used in the Makefile, so I added that to pre-requirements as well. Also fix leading whitespace issue in Makefile.
116 lines
3.3 KiB
YAML
116 lines
3.3 KiB
YAML
name: unit-tests
|
|
|
|
on:
|
|
pull_request:
|
|
push:
|
|
branches:
|
|
- master
|
|
|
|
jobs:
|
|
run-tests:
|
|
name: python-${{ matrix.python-version }},django-${{ matrix.django-version }},${{ matrix.shard_name }}
|
|
if: github.repository == 'openedx/edx-platform' || github.repository == 'edx/edx-platform-private'
|
|
runs-on: [ edx-platform-runner ]
|
|
strategy:
|
|
matrix:
|
|
python-version:
|
|
- "3.8"
|
|
django-version:
|
|
- "pinned"
|
|
#- "4.0"
|
|
shard_name:
|
|
- "lms-1"
|
|
- "lms-2"
|
|
- "lms-3"
|
|
- "lms-4"
|
|
- "lms-5"
|
|
- "lms-6"
|
|
- "openedx-1"
|
|
- "openedx-2"
|
|
- "openedx-3"
|
|
- "openedx-4"
|
|
- "cms-1"
|
|
- "cms-2"
|
|
- "common-1"
|
|
- "common-2"
|
|
- "xmodule-1"
|
|
# We expect Django 4.0 to fail, so don't stop when it fails.
|
|
continue-on-error: ${{ matrix.django-version == '4.0' }}
|
|
|
|
steps:
|
|
- name: sync directory owner
|
|
run: sudo chown runner:runner -R .*
|
|
|
|
- name: checkout repo
|
|
uses: actions/checkout@v2
|
|
|
|
# This gives Mongo several chances to start. We started getting flakiness
|
|
# around 2022-02-15 wherein the start command would sometimes exit with:
|
|
#
|
|
# * Starting database mongodb
|
|
# ...fail!
|
|
#
|
|
# ...not having produced any logs or other output. We couldn't figure out
|
|
# what was causing Mongo to fail, so this is a (temporary?) hack to get
|
|
# PRs unblocked.
|
|
- name: start mongod server for tests
|
|
run: |
|
|
sudo mkdir -p /data/db
|
|
sudo chmod -R a+rw /data/db
|
|
mongod &
|
|
|
|
- name: install requirements
|
|
run: |
|
|
sudo make test-requirements
|
|
if [[ "${{ matrix.django-version }}" == "pinned" ]]; then
|
|
sudo pip install -r requirements/edx/django.txt
|
|
else
|
|
sudo pip install "django~=${{ matrix.django-version }}.0"
|
|
fi
|
|
|
|
- name: list installed package versions
|
|
run: |
|
|
sudo pip freeze
|
|
|
|
- name: Setup and run tests
|
|
uses: ./.github/actions/unit-tests
|
|
|
|
success:
|
|
name: Tests successful
|
|
# A collection step to give a simple name for required status checks.
|
|
# https://github.com/orgs/community/discussions/33579
|
|
needs: run-tests
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: "Success"
|
|
run: |
|
|
echo Tests successful
|
|
|
|
compile-warnings-report:
|
|
runs-on: [ edx-platform-runner ]
|
|
needs: [ run-tests ]
|
|
steps:
|
|
- name: sync directory owner
|
|
run: sudo chown runner:runner -R .*
|
|
- uses: actions/checkout@v2
|
|
- name: collect pytest warnings files
|
|
uses: actions/download-artifact@v2
|
|
with:
|
|
name: pytest-warnings-json
|
|
path: test_root/log
|
|
|
|
- name: display structure of downloaded files
|
|
run: ls -la test_root/log
|
|
|
|
- name: compile warnings report
|
|
run: |
|
|
python openedx/core/process_warnings.py --dir-path test_root/log --html-path reports/pytest_warnings/warning_report_all.html
|
|
|
|
- name: save warning report
|
|
if: success()
|
|
uses: actions/upload-artifact@v2
|
|
with:
|
|
name: pytest-warning-report-html
|
|
path: |
|
|
reports/pytest_warnings/warning_report_all.html
|