This commits prepares edx-platform's experimental Dockerfile
for optional use in devstack. Presently, the image built by this
Dockerfile isn't used anywhere.
Notable changes:
* Drop the openedx/edx-platform image name in favor of
openedx/lms and openedx/cms.
* Drop the newrelic stages and tags.
* Create openedx/lms-dev and openedx/cms-dev image
variants which use Django devserver, install dev
requirements, and specify devstack Django settings.
* Add config files at (lms,cms)/envs/devstack-experimental.yml,
extracted from the existing edxapp docker image.
* Adds three new scripts, each of which replaces an Ansible
or Paver-supported function with a pure bash + Django
management command implementation.
These comments are meant for the Makefile maintainers to read, not the
user running make. We can prefix them with @ to keep make from echoing
them while running.
As we have upgraded pyjwt in edx-platform and also removed these pins from common constraint files in edx-l;int, so we don't need this anymore.
https://github.com/edx/edx-lint/pull/203
Pull Translations has been broken in edx-platform for a number of months
partially in part because of an import issue with local packages during
paver setup. This solves that problem by running make requirements as part
of the pull_translations make target.
Note: This should not be the default behavior. This is only a patch so the
translation job works again. The paver job being run (i18n_compilejs) does
the same install before it runs so we are doubling up there which is not
great.
PROD-2455
The edx-platform codebase already includes quite a few type annotations, but
they were not regularly checked. This may cause problems, when the annotations
themselves include errors (as we found out in some of the learning_sequences
annotations). So here, we add mypy as a dev requirement and introduce a make
command to run mypy regularly. Mypy runs on a very small portion of the total
edx-platform, as configured in mypy.ini. Our hope is that developers will add
more and more modules to this configuration file, until we can eventually run
mypy on the full code base.
See discussion: https://discuss.openedx.org/t/dev-notes-running-mypy-on-edx-platform/4860
We have decided not to pursue the decentralized devstack design right
now and so we want to cleanup files and task that were built as a part
of the process.
Related Ticket: ARCHBOM-1685
The requirements/edx/private.txt file is for dev's own private package
needs. There are two installation mechanisms in edx-platform, and
neither handled the file properly:
- `paver install_prereqs` had the wrong file name. The file was moved
almost three years ago, and paver wasn't kept up.
- `make requirements` used `private.*` which included private.in, which
pip-sync balks at.
I grew tired of modifying the Makefile to compile requirements without
upgrading them. Also, installing pip-tools should not be part of the
compile-requirements target, so a separate target was created.
When running "make pull_translations", this will print any
validation errors on the console, rather than stuffing them into
a .prob file.
This helps when looking at failing jenkins log files.
* Add and push Dockerfile; add decentralized devstack settings
Co-Authored-By: Diana Huang <dkh@edx.org>
Co-Authored-By: Kyle McCormick <kmccormick@edx.org>
* Remove Python requirements hack
Remove the attempted optimization to the installation of Python
package dependencies. The dependencies in edx-platform change
about three times per day, so this was of dubious value. And
because npm is run through nodeenv, which is a Python package,
the Python dependencies installation has to happen first.
* ARCHBOM-1439: Changing workdir to /edx/app/edxapp/edx-platform (#24835)
Context: The Dockerfile tries to stay in sych with legacy stuff.
In the ansible we configure the directory structure such that things
relating to the app but not in the codebase,
such as the env file wind up in /edx/app/edxapp/.
And the codebase winds up in /edx/app/edxapp/edx-platform.
I think due to accident, the dockerfile does
/edx/app/edx-platform/edx-platform instead of /edx/app/edxapp/edx-platform.
This commit tries to have Dockerfile more reflect what is currently happening in production
* Update ports for decentralized devstack ARCHBOM-1447 (#24841)
Switch from the LMS ports we've historically used for NGINX to those used for gunicorn, and fix the Studio ports to match the ones we've historically used for its gunicorn service. Also removed some leftover bits of the requirements hack.
Co-authored-by: Adam Blackwell <ablackwell@edx.org>
Co-authored-by: Diana Huang <dkh@edx.org>
Co-authored-by: jinder1s <msingh@edx.org>
Co-authored-by: Jeremy Bowman <jbowman@edx.org>
Co-authored-by: Manjinder Singh <49171515+jinder1s@users.noreply.github.com>
We introduce a new documentation target, where we use the featuretoggles
Sphinx extension from code-annotations to generate human-readable
documentation of feature toggles in edx-platform. The annotation report
is generated on-the-fly based on the standard feature toggle
configuration file in code-annotations.
In addition, we add new doc.in & doc.txt requirement files that will be
pip-installed by readthedocs to generate the documentation targets.
Cleaned up a few more things about the testing configuration now that we've deployed Django 2.2 to production:
* Go back to a single version of django-oauth-toolkit, tox doesn't need to vary it anymore
* Go back to PyPI release of django-method-override
* Remove no longer needed dependency on python-oauth2 (we already removed the package it was used by)
* Update prepare_xdist_nodes.sh with new Django requirements file options
* Revert "Use pip-sync to make sure that dep cache tarball can go safely stale"
This reverts commit d435f4cd3e.
* Revert "Extract worker setup into own shell script, as much as possible"
This reverts commit 0a079e757c.
This patch adds compilejs
management command to be executed
on pull translations enabling
availability of js/jsx translations
for lms/cms code.
PROD-960
Django-auth-toolkit update
Pinned the latest commit from github to get Django2.2 support in testing.
Pinned the zipp<2.0.0 to get Python<3.5 support too.
The API documentation decorators do not have to leak which solution we
use to generate the docs. Here, and as discussed in PR #21820, we rename
the `openapi` module to `apidocs`, and we make sure that this module
includes all the right functions to document API Views without referring
to Open API.
The xblock counting script had a requirements file that was not
previously being tracked and had an out of date version of requests.
Add it to the `upgrade` make target so that it stays up to date.
