2edbdcf7f4
We've been seeing some cross-version caching issues in the edx-platform-runner unit tests, which apparently run on a "dirty" environment—the virtualenv does not get cleared out between runs, which probably improves performance but allows installed dependencies to "leak" between runs. This results in errors between master and older open-releases but could also prevent us from noticing missing deps. By using pip-sync in the new CI Make targets (as we already do for the regular `make requirements` target) we can ensure that any stale dependencies from runs by other branches (or older versions of the code) are removed. Calling `make local-requirements` at the end of each `*-requirements` target rather than making it a prerequisite is necessary for using sync, since otherwise the local reqs would be wiped out. The `requirements` target is also deduplicated into the newer `dev-requirements` (aliased to it, with both installing private deps now.) Adding a prerequisite of `pre-requirements` allows us to simplify some workflow calls slightly. This ends up being the bulk of the commit by line count. The pip lockfile also wasn't being used in the Makefile, so I added that to pre-requirements as well. Also fix leading whitespace issue in Makefile.
93 lines
2.8 KiB
YAML
93 lines
2.8 KiB
YAML
name: Migrations check on MySql 57
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
pull_request:
|
|
push:
|
|
branches:
|
|
- master
|
|
|
|
jobs:
|
|
check_migrations:
|
|
name: check migrations
|
|
runs-on: ${{ matrix.os }}
|
|
strategy:
|
|
matrix:
|
|
os: [ ubuntu-20.04 ]
|
|
python-version: [ 3.8 ]
|
|
|
|
steps:
|
|
- name: Checkout repo
|
|
uses: actions/checkout@v2
|
|
|
|
- name: Setup Python ${{ matrix.python-version }}
|
|
uses: actions/setup-python@v2
|
|
with:
|
|
python-version: ${{ matrix.python-version }}
|
|
|
|
- name: Install system Packages
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y libxmlsec1-dev
|
|
|
|
- name: Get pip cache dir
|
|
id: pip-cache-dir
|
|
run: |
|
|
echo "::set-output name=dir::$(pip cache dir)"
|
|
|
|
- name: Cache pip dependencies
|
|
id: cache-dependencies
|
|
uses: actions/cache@v2
|
|
with:
|
|
path: ${{ steps.pip-cache-dir.outputs.dir }}
|
|
key: ${{ runner.os }}-pip-${{ hashFiles('requirements/edx/development.txt') }}
|
|
restore-keys: ${{ runner.os }}-pip-
|
|
|
|
- name: Install MySQL 5.7
|
|
run: |
|
|
mkdir ${RUNNER_WORKSPACE}/mysql_packages && cd ${RUNNER_WORKSPACE}/mysql_packages
|
|
sudo apt-get remove --purge *mysql* -y
|
|
sudo rm -rvf /etc/init.d/mysql* /etc/mysql* /var/lib/mysql*
|
|
sudo apt-get install libaio1 libnuma1 libtinfo5 -y
|
|
wget https://cdn.mysql.com/archives/mysql-5.7/mysql-server_5.7.35-1ubuntu18.04_amd64.deb-bundle.tar
|
|
tar -xf mysql-server_5.7*.tar
|
|
sudo DEBIAN_FRONTEND=noninteractive dpkg --install mysql-common_5.7*.deb libmysqlclient*.deb mysql-server_5.7*.deb mysql-client_5.7*.deb mysql-community-server_5.7*.deb mysql-community-client_5.7*.deb
|
|
|
|
- name: Ubuntu and sql Versions
|
|
run: |
|
|
lsb_release -a
|
|
mysql -V
|
|
|
|
- name: Install Python dependencies
|
|
run: |
|
|
make dev-requirements
|
|
pip uninstall -y mysqlclient
|
|
pip install --no-binary mysqlclient mysqlclient
|
|
pip uninstall -y xmlsec
|
|
pip install --no-binary xmlsec xmlsec
|
|
|
|
- name: Initiate Services
|
|
run: |
|
|
sudo systemctl start mongod
|
|
sudo /etc/init.d/mysql start
|
|
|
|
- name: Reset mysql password
|
|
run: |
|
|
cat <<EOF | sudo mysql -u root
|
|
UPDATE mysql.user SET plugin = 'mysql_native_password' WHERE user = 'root';
|
|
UPDATE mysql.user SET authentication_string = null WHERE user = 'root';
|
|
FLUSH PRIVILEGES;
|
|
EOF
|
|
|
|
- name: Run Tests
|
|
env:
|
|
LMS_CFG: lms/envs/bok_choy.yml
|
|
run: |
|
|
echo "CREATE DATABASE IF NOT EXISTS edxtest;" | sudo mysql -u root
|
|
echo "CREATE DATABASE IF NOT EXISTS student_module_history_test;" | sudo mysql -u root
|
|
|
|
echo "Running the LMS migrations."
|
|
./manage.py lms --settings bok_choy migrate
|
|
echo "Running the CMS migrations."
|
|
./manage.py cms --settings bok_choy migrate
|