We haven't quite dropped Py3.8 support yet, so we can't use these workflows if
they're compiling with 3.11:
* upgrade-one-python-dependency (upgrading a single, targeted dep)
* compile-python-requirements (recompiling all deps without upgrading)
So, we "downgrade" those workflows to use 3.8, for now. We should revert this
commit when we drop 3.8 support.
Note: The upgrade-python-requirements.yml workflow is still using 3.8, so this
commit will update the other workflows to match that one.
We were seeing the following error:
```
/usr/bin/git add -- requirements scripts/**/requirements
fatal: pathspec 'scripts/**/requirements' did not match any files
```
Once we introduce wildcards, the whole path needs to be valid so adding
a trailing wildcard to catch all the relevant directories and files.
The `upgrade-one-python-dependency` workflow would fail if there were
changes in the scripts directory because they wouldn't get added to the
PR. Update the list of `add-paths` for the workflow so that it doesn't
fail like this in the future.
This reverts commit 52adce48d1 because we were getting this in the build pipeline:
```
WARNING: lxml 5.2.1 does not provide the extra 'html-clean'
ERROR: Exception:
Traceback (most recent call last):
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
status = self.run(options, args)
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
return func(self, options, args)
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 315, in run
requirement_set = resolver.resolve(
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
result = self._result = resolver.resolve(
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 472, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 366, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 221, in _attempt_to_pin_criterion
satisfied = all(
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 222, in <genexpr>
self._p.is_satisfied_by(requirement=r, candidate=candidate)
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 178, in is_satisfied_by
return requirement.is_satisfied_by(candidate)
File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/requirements.py", line 84, in is_satisfied_by
assert candidate.name == self.name, (
AssertionError: Internal issue: Candidate is not for this requirement lxml[html-clean,html-clean] vs lxml[html-clean]
WARNING: You are using pip version 21.2.1; however, version 24.0 is available.
You should consider upgrading via the '/edx/app/edxapp/venvs/edxapp/bin/python -m pip install --upgrade pip' command.
```
Adds a GH workflow that will run `make compile-requirements` and create
a PR for it and overhauls the requirements README with info on how to
use the new workflows.
README changes:
- Group the workflow/makefile info under a new heading
- Switch to imperative for headings
- Move "Upgrade all dependencies" to end of list (uncommon for people to
actually need this command, I suspect)
- Add "Add a dependency"
- Preamble about .in and .txt files and Mac vs. Linux, as well as about forks
Also, some style fixes to upgrade-one-python-dependency GH workflow:
- Fix indentation in inputs block
- Remove "-workflow" from job name (probably copied from another workflow)
- Wrap a long line
This will allow people to optionally describe the dependency upgrade
without having to rewrite the auto-generated commit or edit the PR body.
(The PR body isn't even guaranteed to make it into the eventual merged
commit.)
This adds a Make target that should simplify the common task of
upgrading a single dependency. Sometimes people manually edit the pin
files, which we would like to avoid; hopefully this will make it
easier for them to do the right thing.
The GitHub workflow should also make it easier for people on Mac to
recompile requirements in a Linux environment, reducing the number of
times spurious dependency changes show up in the pin files (due to
OS-dependent requirements.)
Also, separate upgrade/downgrade instructions and simplify the latter.
(Min constraints are rare and we usually move beyond them quickly.)
