Implements the `npm run watch` section of the assets ADR [1], plus some
modifications since I decided to switch from pywatchman to watchdog (see
ADR changes for justification). This will replace `paver watch_assets`
(edx-platform) and `openedx-assets watch-themes` (Tutor).
Specifically, this PR adds three experimental commands:
* `npm run watch-sass` : Watch for Sass changes with watchdog.
* `npm run watch-webpack` : Invoke Webpack-watch for JS changes.
* `npm run watch` : Invoke both `watch-sass` and `watch-webpack` simultaneously.
These commands are only intended to work in development mode. They have
been tested both on bare-metal edx-platform and through `tutor dev run`
on on Linux.
Before removing the "experimental" label, we need to:
* Test through Devstack on Linux.
* Test through Devstack and `tutor dev run` on macOS.
* Test on bare-metal macOS. Might not work, which is OK, but we should
document that.
* Document the commands in edx-platform's README.
* Confirm that this not only works through `tutor dev run`, but also as
a suitable replacement in the `watchthemes` service that Tutor runs
automatically as part of `tutor dev start`. Tweak if necessary.
References:
1. https://github.com/openedx/edx-platform/blob/master/docs/decisions/0017-reimplement-asset-processing.rst
Part of: https://github.com/openedx/edx-platform/issues/31612
This PR implements much of the static assets rework ADR [1], including:
* `npm run build[-dev]`, and its subcommands,
* `npm run webpack[-dev]` and
* `npm run compile-sass[-dev]`.
This is backwards-compatible. `paver update_assets` should not be affected.
The new command warns that it is "experimental" for a few reasons:
* `npm run build` will fail in the webpack phase unless you first
run `xmodule_assets`. This will be changed soon [2].
* We have tested the new build, but not quite so thoroughly that we'd
recommend it as the production default yet. Once the xmodule_assets
work lands, we'll share this on the forums so early adopters can try it
out.
* The commands lack some top-level documentation. Once they stabilize more,
we'll add a section to the README that explains how and when to use `npm run
build` and its subcommands and its env vars.
* `npm run watch` is not yet implemented.
References:
1. https://github.com/openedx/edx-platform/blob/master/docs/decisions/0017-reimplement-asset-processing.rst
2. https://github.com/openedx/edx-platform/pull/32685
Part of: https://github.com/openedx/edx-platform/issues/31604
* chore: typing + fixes for content_staging
* chore: typing + fixes for learning_sequences
* chore: typing + fixes for content_libraries
* chore: typing + fixes for new XBlock runtime
* feat: type hinting more code with mypy
Some doc buildig deps were in development.in explicitly while others
weren't. Update development.in to pull in doc.txt and then remove
direct additions of the doc requirements from development.in
As part of the deprecation process for edx-sphinx-theme a previous PR switched
to sphinx-book-theme, however edx-sphinx-theme still remained as a dependency in
one place, which is now removed in this PR.
There was a `requirements/pip.txt` with old versions, and a newer
`requirements/edx/pip.txt` managed via a `pip.in` file. The old one was
used in most places, but came out of sync with pip-tools.txt, which was
managed properly. Eventually this caused a `pip check` failure due to the
mismatch.
This should resolve at least part of https://github.com/edx/edx-arch-experiments/issues/267
This PR moves pip.in and pip-tools.in and their corresponding pin files
up to the `requirements/` dir, since they should be shared between the edx
and sandbox environments. This also has the effect of upgrading pip to
match the version in the file we've been uselessly upgrading.
Other improvements:
- Remove `-q` option from pip and pip-sync calls, as it was hiding some
debugging information that would have resolved this sooner.
- Depend on `pre-requirements` from `compile-requirements`, rather than
from `upgrade`. (The base target is the one that actually needs it.)
This also lets us remove the explicit `pip install pip-tools` line.
- Install the recompiled pip and pip-tools files right away, not after the
loop. When we upgrade pip-tools, we want to use the upgraded version,
not the previous version. This requires moving the pip-tools.txt
recompilation outside of the loop and into its own explicit line.
- Don't upgrade pip if we're not running `make upgrade` (respect the
compile options).
- Remove apparently-unneeded `--no-emit-trusted-host --no-emit-index-url`
options (we don't pass trusted-host or index-url options).
The edx-platform codebase already includes quite a few type annotations, but
they were not regularly checked. This may cause problems, when the annotations
themselves include errors (as we found out in some of the learning_sequences
annotations). So here, we add mypy as a dev requirement and introduce a make
command to run mypy regularly. Mypy runs on a very small portion of the total
edx-platform, as configured in mypy.ini. Our hope is that developers will add
more and more modules to this configuration file, until we can eventually run
mypy on the full code base.
See discussion: https://discuss.openedx.org/t/dev-notes-running-mypy-on-edx-platform/4860
We are currently running this plugin from a has directly off of master
to get support for a newer version of DJDT. However even on master
there is not yet support for python 3. We're running into this when
running runserver and when collecting static assets.
A quick survey of the development team at edX shows that people are no
longer actively using this debug panel.(14 responders all saying no to
the question "In the last year have you used the mongo portion of the
django-debug-toolbar in edx-platform?")
This makes sense since the module store and courseware related
developement is not very active right now. Given all this, I'm removing
this toolbar for now and if we need it again we can survey what's
available at that time to choose the best tool.
