build: run ShellCheck
Adds a ShellCheck check to edx-platform PRs and master,
using the shared workflow & template from the .github repo.
This will become a "required" check once it passes for 2 straight weeks on master.
Brings all existing shell scripts into compliance with ShellCheck.
These scripts were written to replace some of the
Ansible provisioning steps in Devstack,
before tCRIL's focus turned to Tutor.
These scripts never ended up being used by Devstack, but
Devstack's maintainers did end up replacing
the Ansible provisioning steps with some direct
Bash commands in Devstack's provision-lms.sh:
23e16c1dda/provision-lms.sh (L51-L68)
Tutor has its own user & course provisioning scripts,
written in Sh using Tutor's jobs framework.
Thus, these scripts are no longer needed and do
not seem to be in use.
The file was introduced as part of the "decentralized devstack" spike.
We wanted a working Dockerfile in the edx-platform root,
and we wanted to speed it up by not relying on Paver.
edx-platform now has a working Dockerfile, and it does not use this script.
The paver-free-assets initiative is being continued here:
https://github.com/openedx/edx-platform/issues/31798
It will result in a new, prod-ready asset building script.
These files were used to run tests on
build.testeng.edx.org, a Jenkins instance.
All tests have since been moved over to
GitHub Actions. The build.testeng.edx.org
instance is decomissioned.
Removed files include:
- scripts/jenkins-*.sh
- scripts/Jenkinsfiles/*
- scripts/xdist/*
Note on xdist scripts: pytest-xdist in general is still
useful for local test runs in order to take advantage
of all available CPU cores, but the scripts here were
specifically for running tests on multiple remote machines,
particularly via Jenkins.
We have a need to lock the version of Django for production and tests, but
also to test on newer versions of Django so that we can get the repo ready
for long-term-support releases.
We've been doing that by extracting the `django==x.y.z` from the
pip-compiled files and moving it to a django.txt that is then co-installed
but can be overridden during tests. The problem is that this can result
in broken packages.
The approach here is to have `make test-requirements` continue to
ensure a consistent set of packages, and then install a different
Django on top of that in the CI script -- and call `pip check` to make
sure that combination isn't broken.
Adding Django 4.0 to the unit-tests.yml matrix will now correctly
result in this error and a failing job:
`django-splash 1.2.1 has requirement Django<4.0, but you have django 4.0.8.`
The other half of this is to change other CI runners to remove their
ability to control the Django version, since it's complicated to make
this work, and we probably only need it in unit-tests.yml. Convert them
to just use `make test-requirements`.
Also:
- Simplify handling of `pip --src` by setting `PIP_SRC` (rather than our
own `PIP_SRC_DIR`, which pip ignores because `--src-dir` isn't an option
that it knows). This is needed to allow `make test-requirements` to do
the pip calls. An alternative would be to set a pip-options env var for
the make target to use, but `PIP_SRC` already exists.
- Remove outdated modifications to common_constraints
- Add comment explaining why pylint tests need dev-requirements
As part of dissolving our sub-projects in edx-platform, we are moving this package under the xmodule directory.
We have fixed all the occurences of import of this package and also fixed all documents related references.
This might break your platform if you have any reference of `import capa` or `from capa import` in your codebase or in any Xblock.
Ref: https://openedx.atlassian.net/browse/BOM-2582
- Moving xmodule folder to root as we're dissolving sub-projects of common folder in edx-platform
- More info: https://openedx.atlassian.net/browse/BOM-2579
- -e common/lib/xmodule has been removed from the requirements as xmodule has itself become the part of edx-platform and not being installed through requirements
- The test files common/lib/xmodule/test_files/ have been removed as they are not being used anymore
The script for finding order dependency test failures
was dependent on inputs from Jenkins logs which we no
longer use.
This quick fix was to just use a list of tests with the
right format. A future iteration might process a new type
of output, like that from (pytest -v).
* Revert "Merge pull request #29909 from openedx/revert-29869-Carlos-Muniz/symmath-removal"
This reverts commit 8c0db8ddff, reversing
changes made to 1156c62014.
* fix: Remove misplaced `-e`
`-e` was wrongfully placed in front of `common/lib/sandbox-packages`,
which may have most likely been causing the edxAPP to break.
* fix: Change regex to apply to right dirs
`py38.txt` does not include `common/lib/xmodule` so it doesn't match the
regex. Therefore, it never got its lines in `common/lib/sandbox-packages`
fixed. If we change the regex to match any `common/lib/<packagename>` it
should work correctly.
scripts/post-pip-compile.sh uses common/lib/symmath/ as its sample
directory to clean up after using pip-tools. Since symmath has been
removed, xmodule, the largest of the directories in common/lib/ will
replace it.
In my case, xsslint was failing with a UnicodeDecodeError -- having this
try/except block helped me discover that it was reading files in my
virtualenv.
This commits prepares edx-platform's experimental Dockerfile
for optional use in devstack. Presently, the image built by this
Dockerfile isn't used anywhere.
Notable changes:
* Drop the openedx/edx-platform image name in favor of
openedx/lms and openedx/cms.
* Drop the newrelic stages and tags.
* Create openedx/lms-dev and openedx/cms-dev image
variants which use Django devserver, install dev
requirements, and specify devstack Django settings.
* Add config files at (lms,cms)/envs/devstack-experimental.yml,
extracted from the existing edxapp docker image.
* Adds three new scripts, each of which replaces an Ansible
or Paver-supported function with a pure bash + Django
management command implementation.
