Adds a GH workflow that will run `make compile-requirements` and create
a PR for it and overhauls the requirements README with info on how to
use the new workflows.
README changes:
- Group the workflow/makefile info under a new heading
- Switch to imperative for headings
- Move "Upgrade all dependencies" to end of list (uncommon for people to
actually need this command, I suspect)
- Add "Add a dependency"
- Preamble about .in and .txt files and Mac vs. Linux, as well as about forks
Also, some style fixes to upgrade-one-python-dependency GH workflow:
- Fix indentation in inputs block
- Remove "-workflow" from job name (probably copied from another workflow)
- Wrap a long line
We have a few situations where requirements files can become
inconsistent or cause unnecessary review churn in later `make upgrade`
runs either due to manual editing, inconsistent environments, or
incorrectly specified git dependencies. This will produce a failing
check for any PR that does not produce a clean run of `make
compile-requirements` on Linux.
Addresses https://github.com/openedx/edx-platform/issues/31372
@feanil is overhauling the docs structure of edx-platform currently.
Recently, a change he made upped the docs build time from <15mins
to >40mins. Rather than revert the changes he is making, we're going
to take the docs build check out of CI for now. He plans to re-enable
the build check, but as a ReadTheDocs build, not a GitHub Action.
He's actively working on the docs, so he's not worried about any
regressions that may pop up while the check is disabled.
It's worth noting that the current docs check does not actually
publish its output anywhere.
Now that we're using standard service containers, we can use a matrix to
test both the current supported versions in one job. This should also
make it easier to test with future versions as we get ready to upgrade.
This will allow people to optionally describe the dependency upgrade
without having to rewrite the auto-generated commit or edit the PR body.
(The PR body isn't even guaranteed to make it into the eventual merged
commit.)
* feat: added enrollment API for notification config
* feat: added apps.py in notifications
* feat: added waffle flag for notification app
* feat: added proper docs for the API
This adds a Make target that should simplify the common task of
upgrading a single dependency. Sometimes people manually edit the pin
files, which we would like to avoid; hopefully this will make it
easier for them to do the right thing.
The GitHub workflow should also make it easier for people on Mac to
recompile requirements in a Linux environment, reducing the number of
times spurious dependency changes show up in the pin files (due to
OS-dependent requirements.)
Also, separate upgrade/downgrade instructions and simplify the latter.
(Min constraints are rare and we usually move beyond them quickly.)
Adds a tiny `openedx.core.djangoapps.staticfiles` app so that
static asset ignore patterns can be coded into configuration rather
than supplied on the command line or coded into pavelib.
Makes it easier to run static asset collection without Paver.
See ADR for details:
openedx/core/djangoapps/staticfiles/docs/decisions/0001-purpose-of-app.rst
Closes: https://github.com/openedx/edx-platform/issues/31658
build: run ShellCheck
Adds a ShellCheck check to edx-platform PRs and master,
using the shared workflow & template from the .github repo.
This will become a "required" check once it passes for 2 straight weeks on master.
Brings all existing shell scripts into compliance with ShellCheck.
The .github/workflows/add-depr-ticket-to-depr-board.yml workflow is missing or needs an update to stay in
sync with the current standard for this workflow as defined in the
`.github` repo of the `openedx` GitHub org.
The .github/workflows/add-remove-label-on-comment.yml workflow is missing or needs an update to stay in
sync with the current standard for this workflow as defined in the
`.github` repo of the `openedx` GitHub org.
The .github/workflows/self-assign-issue.yml workflow is missing or needs an update to stay in
sync with the current standard for this workflow as defined in the
`.github` repo of the `openedx` GitHub org.
- Remove pylint warning suppression from SHAKE-128 hexdigest calls; these
are no longer needed as of astroid 2.12.12. For background, see issue
<https://github.com/PyCQA/pylint/issues/4039>. Also, correct a comment
that referred to SHAKE-256 instead of SHAKE-128.
- Replace "released" with "beta" in several places where there was a copy
& paste error in dark_lang. Add mention of `beta_lang` to a docstring
where it was omitted.
- Remove comment regarding Mongo startup; the code it referred to has since
been removed.
- Fix typos.
We have a need to lock the version of Django for production and tests, but
also to test on newer versions of Django so that we can get the repo ready
for long-term-support releases.
We've been doing that by extracting the `django==x.y.z` from the
pip-compiled files and moving it to a django.txt that is then co-installed
but can be overridden during tests. The problem is that this can result
in broken packages.
The approach here is to have `make test-requirements` continue to
ensure a consistent set of packages, and then install a different
Django on top of that in the CI script -- and call `pip check` to make
sure that combination isn't broken.
Adding Django 4.0 to the unit-tests.yml matrix will now correctly
result in this error and a failing job:
`django-splash 1.2.1 has requirement Django<4.0, but you have django 4.0.8.`
The other half of this is to change other CI runners to remove their
ability to control the Django version, since it's complicated to make
this work, and we probably only need it in unit-tests.yml. Convert them
to just use `make test-requirements`.
Also:
- Simplify handling of `pip --src` by setting `PIP_SRC` (rather than our
own `PIP_SRC_DIR`, which pip ignores because `--src-dir` isn't an option
that it knows). This is needed to allow `make test-requirements` to do
the pip calls. An alternative would be to set a pip-options env var for
the make target to use, but `PIP_SRC` already exists.
- Remove outdated modifications to common_constraints
- Add comment explaining why pylint tests need dev-requirements
If a job "needs" earlier jobs, and one of the earlier jobs fails, then
the "needs" job will be marked as Skipped. A required check that is
skipped doesn't block merging.
The alls-green action has the correct logic to fail the aggregation job
if any of its required jobs fail.
We were seeing this error whenever unit tests
were run from a fork (only forks use the GH-hosted
workflow, which is why this was able to merge
into master):
You have an error in your yaml syntax on line 65
The comment hash (#) needs to be aligned
with the rest of the text block.
We've been seeing some cross-version caching issues in the
edx-platform-runner unit tests, which apparently run on a "dirty"
environment—the virtualenv does not get cleared out between runs,
which probably improves performance but allows installed dependencies
to "leak" between runs. This results in errors between master and older
open-releases but could also prevent us from noticing missing deps.
By using pip-sync in the new CI Make targets (as we already do for
the regular `make requirements` target) we can ensure that any stale
dependencies from runs by other branches (or older versions of the
code) are removed.
Calling `make local-requirements` at the end of each `*-requirements`
target rather than making it a prerequisite is necessary for using sync,
since otherwise the local reqs would be wiped out.
The `requirements` target is also deduplicated into the newer
`dev-requirements` (aliased to it, with both installing private deps now.)
Adding a prerequisite of `pre-requirements` allows us to simplify some
workflow calls slightly. This ends up being the bulk of the commit by
line count. The pip lockfile also wasn't being used in the Makefile, so
I added that to pre-requirements as well.
Also fix leading whitespace issue in Makefile.
