* feat: Add a script to enhance JWKs in preparation for move from pyjwkest
This script accepts a signing JWK (presumably `JWT_PRIVATE_SIGNING_JWK`)
and ensures that it has all of the precomputed private numbers that are
required for top performance. This is necessary before moving away from
pyjwkest to PyJWT for signing JWTs. See issue
<https://github.com/openedx/edx-drf-extensions/issues/290>. (Alternatively,
one could remove the p, q, dp, dq, and qi params, but there is an unknown
performance cost to doing so as we are not currently caching these keys,
and the precompution happens on every load due to the way pyjwkest's API
works.)
* fixup! Upgrade devstack at the same time
There was a `requirements/pip.txt` with old versions, and a newer
`requirements/edx/pip.txt` managed via a `pip.in` file. The old one was
used in most places, but came out of sync with pip-tools.txt, which was
managed properly. Eventually this caused a `pip check` failure due to the
mismatch.
This should resolve at least part of https://github.com/edx/edx-arch-experiments/issues/267
This PR moves pip.in and pip-tools.in and their corresponding pin files
up to the `requirements/` dir, since they should be shared between the edx
and sandbox environments. This also has the effect of upgrading pip to
match the version in the file we've been uselessly upgrading.
Other improvements:
- Remove `-q` option from pip and pip-sync calls, as it was hiding some
debugging information that would have resolved this sooner.
- Depend on `pre-requirements` from `compile-requirements`, rather than
from `upgrade`. (The base target is the one that actually needs it.)
This also lets us remove the explicit `pip install pip-tools` line.
- Install the recompiled pip and pip-tools files right away, not after the
loop. When we upgrade pip-tools, we want to use the upgraded version,
not the previous version. This requires moving the pip-tools.txt
recompilation outside of the loop and into its own explicit line.
- Don't upgrade pip if we're not running `make upgrade` (respect the
compile options).
- Remove apparently-unneeded `--no-emit-trusted-host --no-emit-index-url`
options (we don't pass trusted-host or index-url options).
build: run ShellCheck
Adds a ShellCheck check to edx-platform PRs and master,
using the shared workflow & template from the .github repo.
This will become a "required" check once it passes for 2 straight weeks on master.
Brings all existing shell scripts into compliance with ShellCheck.
These scripts were written to replace some of the
Ansible provisioning steps in Devstack,
before tCRIL's focus turned to Tutor.
These scripts never ended up being used by Devstack, but
Devstack's maintainers did end up replacing
the Ansible provisioning steps with some direct
Bash commands in Devstack's provision-lms.sh:
23e16c1dda/provision-lms.sh (L51-L68)
Tutor has its own user & course provisioning scripts,
written in Sh using Tutor's jobs framework.
Thus, these scripts are no longer needed and do
not seem to be in use.
The file was introduced as part of the "decentralized devstack" spike.
We wanted a working Dockerfile in the edx-platform root,
and we wanted to speed it up by not relying on Paver.
edx-platform now has a working Dockerfile, and it does not use this script.
The paver-free-assets initiative is being continued here:
https://github.com/openedx/edx-platform/issues/31798
It will result in a new, prod-ready asset building script.
These files were used to run tests on
build.testeng.edx.org, a Jenkins instance.
All tests have since been moved over to
GitHub Actions. The build.testeng.edx.org
instance is decomissioned.
Removed files include:
- scripts/jenkins-*.sh
- scripts/Jenkinsfiles/*
- scripts/xdist/*
Note on xdist scripts: pytest-xdist in general is still
useful for local test runs in order to take advantage
of all available CPU cores, but the scripts here were
specifically for running tests on multiple remote machines,
particularly via Jenkins.
We have a need to lock the version of Django for production and tests, but
also to test on newer versions of Django so that we can get the repo ready
for long-term-support releases.
We've been doing that by extracting the `django==x.y.z` from the
pip-compiled files and moving it to a django.txt that is then co-installed
but can be overridden during tests. The problem is that this can result
in broken packages.
The approach here is to have `make test-requirements` continue to
ensure a consistent set of packages, and then install a different
Django on top of that in the CI script -- and call `pip check` to make
sure that combination isn't broken.
Adding Django 4.0 to the unit-tests.yml matrix will now correctly
result in this error and a failing job:
`django-splash 1.2.1 has requirement Django<4.0, but you have django 4.0.8.`
The other half of this is to change other CI runners to remove their
ability to control the Django version, since it's complicated to make
this work, and we probably only need it in unit-tests.yml. Convert them
to just use `make test-requirements`.
Also:
- Simplify handling of `pip --src` by setting `PIP_SRC` (rather than our
own `PIP_SRC_DIR`, which pip ignores because `--src-dir` isn't an option
that it knows). This is needed to allow `make test-requirements` to do
the pip calls. An alternative would be to set a pip-options env var for
the make target to use, but `PIP_SRC` already exists.
- Remove outdated modifications to common_constraints
- Add comment explaining why pylint tests need dev-requirements
As part of dissolving our sub-projects in edx-platform, we are moving this package under the xmodule directory.
We have fixed all the occurences of import of this package and also fixed all documents related references.
This might break your platform if you have any reference of `import capa` or `from capa import` in your codebase or in any Xblock.
Ref: https://openedx.atlassian.net/browse/BOM-2582
- Moving xmodule folder to root as we're dissolving sub-projects of common folder in edx-platform
- More info: https://openedx.atlassian.net/browse/BOM-2579
- -e common/lib/xmodule has been removed from the requirements as xmodule has itself become the part of edx-platform and not being installed through requirements
- The test files common/lib/xmodule/test_files/ have been removed as they are not being used anymore
The script for finding order dependency test failures
was dependent on inputs from Jenkins logs which we no
longer use.
This quick fix was to just use a list of tests with the
right format. A future iteration might process a new type
of output, like that from (pytest -v).
* Revert "Merge pull request #29909 from openedx/revert-29869-Carlos-Muniz/symmath-removal"
This reverts commit 8c0db8ddff, reversing
changes made to 1156c62014.
* fix: Remove misplaced `-e`
`-e` was wrongfully placed in front of `common/lib/sandbox-packages`,
which may have most likely been causing the edxAPP to break.
* fix: Change regex to apply to right dirs
`py38.txt` does not include `common/lib/xmodule` so it doesn't match the
regex. Therefore, it never got its lines in `common/lib/sandbox-packages`
fixed. If we change the regex to match any `common/lib/<packagename>` it
should work correctly.
scripts/post-pip-compile.sh uses common/lib/symmath/ as its sample
directory to clean up after using pip-tools. Since symmath has been
removed, xmodule, the largest of the directories in common/lib/ will
replace it.
