Andal.LND/edx-platform
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor top-level docs directory

Browse Source
This commit is contained in:
Nimisha Asthagiri
2019-04-28 18:45:05 -04:00
parent 06bc6fbd82
commit 7f3727d84f
16 changed files with 193 additions and 252 deletions
Download Patch File Download Diff File Expand all files Collapse all files

88
docs/frontend/bootstrap.rst Normal file
View File

@@ -0,0 +1,88 @@
.. _ui_bootstrap:
######################
Working with Bootstrap
######################
This topic describes the Bootstrap framework, and how it should be used to
build user interfaces within edX applications. Note that Bootstrap adoption
started after the Ginkgo release, and so this document applies to the edX
master branch and to the forthcoming Hawthorn release.
If you are interested in the rationale for edX choosing Bootstrap, you can
read about the decision in `OEP-16: Adopting Bootstrap
<https://open-edx-proposals.readthedocs.io/en/latest/oep-0016-bp-adopt-bootstrap.html>`_.
.. highlight:: none
***************
Getting Started
***************
Bootstrap is an open source front end component library that is used by many of
the world's most popular applications. It allows for rapid assembly of front end
components using a responsive grid system, a robust component library and easy
to configure theming capabilities to ensure that new components are rendered
consistently. EdX is using `Bootstrap 4`_ which is a reimplemented version using
Sass and that is currently in beta release.
All edX applications should use the `edx-bootstrap`_ package that can be
installed via `npm`_. This package provides two themes (a default Open edX
theme, as well as an edX branded version), and eventually will provide custom
Open edX styles for common patterns.
.. _ui_bootstrap_custom_designs:
*************************
Developing Custom Designs
*************************
Bootstrap provides a large number of components and layouts out-of-the-box, but
there will always be a need to implement custom designs. There are a number of
considerations to take into account when implementing your designs using Sass.
The most important rule is to avoid hard-coding values such as colors and fonts.
Using hard-coded values means that Bootstrap themes will not be able to affect
your styles, and so your new elements will look out of place. Whenever possible
you should instead use the functions and variables provided by Bootstrap to
access theme colors or fonts. See `Bootstrap customization options`_ for more
details.
For example, here is an example of a hard-coded style::
.my-element {
font-family: "Open Sans";
color: #0000ff;
}
The recommended alternative is as follows::
.my-element {
font-family: $font-family-sans-serif;
color: theme-color("primary");
}
If you do find the need for a custom color or font that isn't provided by
the edX Bootstrap library, consider first whether it makes sense to contribute
it back so that other applications can use this value too. If you decide to
add a custom value, define it as a variable that can be overridden by a theme by
using the ``!default`` flag. This allows themes to provide a different value
for this variable if they choose. See the Sass documentation for `default flag`_
for more details.
For example::
$my-custom-color: #0000ff !default;
.my-element {
font-family: $font-family-sans-serif;
color: $my-custom-color;
}
.. _Bootstrap 4: https://getbootstrap.com/docs/4.0/getting-started/introduction/
.. _Bootstrap customization options: https://getbootstrap.com/docs/4.0/getting-started/options/
.. _default flag: http://sass-lang.com/documentation/file.SASS_REFERENCE.html#Variable_Defaults___default
.. _edx-bootstrap: https://www.npmjs.com/package/@edx/edx-bootstrap
.. _npm: https://www.npmjs.com/

110
docs/frontend/javascript.rst Normal file
View File

@@ -0,0 +1,110 @@
JavaScript in edx-platform
==========================
ES2015
------
All new JavaScript code in edx-platform should be written in ES2015.
ES2015 is not a framework or library -- rather, it is the latest and
greatest revision of the JavaScript language itself, natively supported
in all modern browsers and engines. Think of it as JavaScript's
equivalent to Python 3. ES2015 brings with it number of wonderful
syntactic features, such as classes, native imports, arrow functions,
and new data structures. To learn more about ES2015, check out `Luke
Hoban's comprehensive ES6 Features
repo <https://github.com/lukehoban/es6features>`__.
Although ES2015 is natively supported in modern browsers, older browsers
can't interpret it. Here at edX, we support the two latest versions of
every browser, plus IE11, so we need to do a little extra work to
support ES2015. This is where Webpack and Babel come in. Webpack is a
module bundler that transforms, minifies, and compiles frontend code
into pre-built "bundles" to include within pages. It works together with
Babel to transpile ES2015 code into ES5 code, which can safely be used
in all browsers.
Fortunately, you don't need to worry about the gritty details of Webpack
in order to write ES2015 code. You just need to make sure Webpack knows
where to find your files. It's also important to note that **Webpack is
not compatible with RequireJS**. Work is currently underway to move all
legacy RequireJS modules into Webpack, but until it is complete, you
will need to update legacy code yourself in order to use it with ES2015.
Adding a New ES2015 Module
~~~~~~~~~~~~~~~~~~~~~~~~~~
Don't mix ES2015 and ES5 modules within directories. If necessary,
create a new directory just for your new file. If you create a new
directory, run the following from edx-platform root to copy over an
appropriate eslint config:
::
cp cms/static/js/features_jsx/.eslintrc.js path/to/your/directory
Give your new file an UpperCamelCase filename, such as
``MyAwesomeModule.js``. If it is a React module, use the ``.jsx``
extension; otherwise, use the ``.js`` extension.
If you intend to include this module itself directly within a page, you
will need to tell Webpack about it. Add a line to the ``entry`` object
within ``webpack.common.config.js``.
::
'MyAwesomeModule': 'path/to/your/directory/MyAwesomeModule.js',
The new entry's key should be the name of your module (typically this is
the same as your filename), and its value should be the path to your
file relative to the edx-platform root.
Writing Your File
~~~~~~~~~~~~~~~~~
Structure your module using ``class``\ es. Typically, you'll want to
define and export one ``class`` per file. If you are going to be
including this module directly within a page and passing it through
Webpack, use a non-default export. ``MyAwesomeModule.js`` should look
something like this:
::
export class MyAwesomeModule {
// your awesome code here
}
Use two-space indentation. This is industry standard practice for
ES2015. If you need to pull in external dependencies, use ``import``
statements:
::
import moment from 'moment';
import 'jquery.cookie';
import { MyOtherModule } from './MyOtherModule';
Building Your File
~~~~~~~~~~~~~~~~~~
Devstack comes with two watcher containers specifically for building
assets. They compile frontend files very quickly, so you can see your
changes reflected in a browser almost immediately. You can run these
containers with:
::
make dev.up.watchers
and stop them with
::
make stop.watchers
If you make any changes to ``webpack.common.config.js`` while the
watchers are running, you will need to restart the watchers in order for
them to pick up your changes.
If your changes aren't being reflected in the browser, check the logs
with ``make logs`` to see if something went wrong. If you get stuck, ask
for help in the FedX hipchat room, or in #front-end on Slack.

205
docs/frontend/static_assets.rst Normal file
View File

@@ -0,0 +1,205 @@
#######################################
edx-platform Static Asset Pipeline Plan
#######################################
Static asset handling in edx-platform has evolved in a messy way over the years.
This has led to a lot of complexity and inconsistencies. This is a proposal for
how we can move forward to a simpler system and more modern toolchain. Note that
this is not a detailed guide for how to write React or Bootstrap code. This is
instead going to talk about conventions for how we arrange, extract, and compile
static assets.
Big Open Questions (TODO)
*************************
This document is a work in progress, as the design for some of this is still in
flux, particularly around extensibility.
* Pluggable third party apps and Webpack packaging.
* Keep the Django i18n mechanism?
* Stance on HTTP/2 and bundling granularity.
* Optimizing theme assets.
* Tests
Requirements
************
Any proposed solution must support:
* Externally developed and installed Django apps.
* Theming.
* XBlock assets.
* Existing tests.
* Fast builds.
* An incremental implementation path.
* Other kinds of pluggability???
Assumptions
***********
Some assumptions/opinions that this proposal is based on:
* We want to shift as much as possible to Webpack and the JavaScript stack of
technologies, leaving the Python layer as thin as possible.
* While we will try to make theming upgrades straightforward, we will be moving
around where files are located and where they're compiled out to.
* We will be pushing towards a world that is more Django app-centric than LMS
vs. Studio centric, to reduce duplication.
* At the same time, we want to consolidate assets far more efficiently than we
are doing today.
* Leaning towards more static front ends + API calls.
* However we still need to be compatible with Django's asset system for things
like third party apps (e.g. Django Rest Framework browsing assets, Swagger,
etc.)
* It should be possible to pre-build static assets and deploy them onto S3 or
similar.
Where We Are Today
******************
We have a static asset pipeline that is mostly driven by Django's built-in
staticfiles finders and the collectstatic process. We use the popular
``django-pipeline`` library, with UglifyJS as the JavaScript compressor (the
binary is installed via node into node_modules). We also use the less well known
``django-pipeline-forgiving`` extension to ``django-pipeline`` so we don't error
out when files are missing (added when we started dynamically scanning XBlocks
for assets).
The ``django-pipeline`` config is aware of CSS files for the purposes of
concatenation, but it does *not* know about the source Sass files.
Those are processed with paver tasks before ``django-pipeline`` ever sees them.
We also have the following custom extensions to Django's builtin ``STATICFILES``
mechanism:
``openedx.core.djangoapps.theming.finders.ThemeFilesFinder``
Custom finder that overrides any static asset with a version from the themes
directory (``COMPREHENSIVE_THEME_DIRS`` defined in ``lms.envs.json`` and
``cms.envs.json``).
``openedx.core.lib.xblock_pipeline.finder.XBlockPipelineFinder``
Custom finder that accesses and extracts assets from pip-installed XBlocks via
``pkg_resources``.
``openedx.core.storage.DevelopmentStorage/ProductionStorage``
Custom ``FileStorage`` classes that mostly exist for theme-awareness.
LMS and Studio/CMS Separation
-----------------------------
LMS and Studio have their own directories for source assets (``lms/static`` and
``cms/static``), and have symlinks to shared assets in ``common/static``. We
treat the static asset compilation and collection phase for LMS and Studio as
separate projects that happen to share a lot of pieces. They output to different
places (typically ``/edx/var/edxapp/staticfiles`` for LMS and
``/edx/var/edxapp/staticfiles/studio`` for Studio) and can be collected
separately. However in practice they're always run together because we deploy
them from the same commits and to the same servers.
Django vs. Webpack Conventions
******************************
The Django convention for having an app with bundled assets is to namespace them
locally with the app name so that they get their own directories when they are
gathered together into a common static directory by collectstatic. For example,
the edx-enterprise app has a ``static/enterprise`` folder, so its assets are
compiled to ``/edx/var/edxapp/staticfiles/enterprise`` by edx-platform and will
not conflict with assets from any other Django app.
Webpack conventions would have us create a single set of configuration files at
the root of edx-platform, which would specify all bundles in the project.
TODO: The big, "pluggable Webpack components" question.
Proposed Repo Structure
***********************
All assets that are in common spaces like ``common/static``, ``lms/static``,
and ``cms/static`` would be moved to be under the Django apps that they are a
part of and follow the Django naming convention (e.g.
``openedx/features/course_bookmarks/static/course_bookmarks``). An app's
``templates/{appname}`` directory will only be for server side templates, and
any client-side templates will be put in ``static/{appname}/templates``.
Proposed Compiled Structure
***************************
This is meant to be a sample of the different types of things we'd have, not a
full list:
::
# Webpack bundles/post-processed assets
/webpack/css
/fonts
/js
/vendor ?
# Django apps that are in the edx-platform repo
/course_bookmarks
/course_experience
# edX authored, installed via separate repo
/enterprise
# Entirely third party apps that we need to maintain compatiblity with.
/admin
/rest_framework
# Themes are part of the "theming" app
/theming/themes/open-edx
/red-theme
/edx.org
# XBlocks still collect their assets into a common space (/xmodule goes away)
# We consider this to be the XBlock Runtime's app, and it collects static
# assets from installed XBlocks.
/xblock
Django vs. Webpack Roles
************************
Rule of thumb: Django/Python still serves static assets, Webpack processes and
optimizes them.
Webpack would be responsible for all Sass compilation in edx-platform. It would
also be responsible for the optimization/minification of JavaScript assets, but
those optimized assets would only appear under the ``/webpack`` directory. Third
party assets that Webpack is not aware of may have hash suffixes applied to them
by the Django collectstatic layer, but will not otherwise be processed or
optimized in any way -- so no sass compilation, no uglifyjs minification, etc.
The django-pipeline dependency should be removed altogether.
Themes
------
Theme handling is muddled. The fact that themes can override server-side
templates means that Python has to be aware of them. At the same time, we want
to shift over Sass compilation as a whole to Webpack, meaning that at least some
knowledge about where they are and how to compile them has to exist there. Also,
there are JS assets in some themes that provide additional functionality, and it
would be a performance degradation if those assets were no longer optimized.
What I do NOT want to happen:
* Significant end user performance degradation.
* Having an *additional* system in the asset pipeline (e.g. keeping
django-pipeline around while having additional systems).
I think that means that conceptually, there exists a larger Static Asset system
that exists and that we think of both Webpack and Django being consumers of its
configuration. This is also very fuzzy at the moment.
Asset Groups
------------
There will be logical groupings of static assets. There should be uniformity and
no duplication within a group, but we would allow duplication between groups to
better facilitate independent deployment and isolation.
Example Groups:
* XBlock/XModule Assets
* LMS/Studio apps in edx-platform
* Third party app, such as edx-enterprise

163
docs/frontend/styling.rst Normal file
View File

@@ -0,0 +1,163 @@
#######################
Styling in edx-platform
#######################
Over time, our Sass styling has become a little convoluted, with three major
reworkings often leading to confusion when a developer needs to style in
multiple locations across edx-platform. The main endeavors were v1, v2 (or
pattern library) and Bootstrap. We are trying to move away from using v2, as the
pattern library is deprecated, but there may still be locations in the code that
reference those styles (please remove them as you see fit).
Our platform uses a system of Sass partials that combine to compile into a
single large css file to be rendered on the page. From the Sass docs:
You can create partial Sass files that contain little snippets of CSS
that you can include in other Sass files. This is a great way to
modularize your CSS and help keep things easier to maintain. A partial
is simply a Sass file named with a leading underscore. You might name it
something like _partial.scss. The underscore lets Sass know that the
file is only a partial file and that it should not be generated into a
CSS file. Sass partials are used with the ``@import`` directive.
This structure allows us to break up our styling into small pieces, making
readability and maintenance easier, while often at the expense of structural
complexity as a code base grows. Here is an example, directly from edx-platform,
for how the partials flow to a single scss file that compiles into CSS for the
page to use. This is a page that uses v1 styling.
.. figure:: v1_sass_pipeline.webp
:alt: Sass Compilation for v1 CSS
Sass Compilation for v1 CSS
Note that this only shows an example, there are far more partials that come
together to make the ``lms-main-v1.css`` final file.
As you can see, the ``lms-main-v1.scss`` file does not have a leading
underscore, telling the compilation to turn that scss file into an actual css
file to be rendered on the page. If you were to use the Chrome Inspector tool
and look at the styling, you will see that there is one unified ``lms-
main-v1.css`` file that contains all the element styles.
The ``lms-main-v1.scss`` file uses the ``@import`` statement to pull in the
``_build- base-v1.scss`` and ``build-lms-v1.scss`` files which in turn import
more partials down the road. It is critical to recognize that this tree works in
a depth-first, first-come-first-serve manner. That means that styles that are
imported early in the process cannot reference Sass variables imported later in
the process (ie: ``bootstrap/variables`` cannot reference variables from
``shared/header``).
This diagram describes the process for the **v1 styles**, and there is a similar
setup for ``lms-main-v2.scss`` as well as ``bootstrap/lms-main.scss``. Each
individual HTML page on the edx-platform specifies which of the three that page
wants to use for styling. Please note that as an organization, we are slowly
trying to 1) move everything over to the bootstrap/lms-main.scss file and 2)
deprecate and stop using any v2 files. ``lms-main-v1.scss`` can still be used,
but a migration over to ``bootstrap/lms-main.scss``, and a migration of any
relevant partials from the v1 structure, would be a valuable endeavor.
Please note that as you add partials, make sure that they are not already being
imported in another file (ie: you add a node to the tree above that already
exists on the tree).
What theming does, and how to do it
***********************************
By thinking about the styling as a tree, theming becomes a lot simpler. All
theming means is that you can override one of the partials above by matching
the exact path in the ``edx-platform/themes`` directory. So, for example, to
override the shared header file (located at ``lms/static/sass/shared/_header``),
you would simply go into the ``edx-platform/themes/[theme you want to
override]/lms/static/shared`` folder and add a _header.scss file. When django
compiles the assets, it will use this file as a replacement to the main
edx-platform implementation.
If you look at the actual code base, you will see that we have a standard of
using the partials directory for these overrides files. This keeps the specific
components isolated from the core styling to reduce errors when people override
files and forget to import other files that are needed elsewhere on the site.
This is a good practice that future development should adhere to.
Final Note: When dealing with front end changes, it is a good idea to also check
the edx-themes repo, that works exactly like our themes folder, but also
includes html templates that can further confuse things.
Bootstrap and edx-platform
**************************
In a month and a half endeavor in the Fall, Andy and I worked on integrating
Bootstrap into the platform for three main reasons.
1. **To unify our styling:** By specifying colors and variables that can be used
universally, reducing the '50 shades of grey' issue and inconsistencies in the
ways that we style components.
2. **To add a widely used component library:** Bootstrap has a robust community
of developers that contribute to their open source component library, allowing
for easier prototyping and create of front end experiences.
3. **To simplify theming:** Since we now have a single unified source of truth
for our variables, openedx instances and alternative themes can simply override
those files to customly style their sites.
Relating to the above styling conversation, we have pulled in the entire
bootstrap styling library into ``lms-main.scss``, but only partially pulled it
into the v1 and v2 implementations due to naming conflicts. For example, we
could not pull in the bootstrap modal, due to conflicts with our own modal
styling (both use the generic ``.modal`` class). We were, however, able **to
pull in the entire bootstrap javascript file**, since that file overrides jQuery
in ways that our current application does not (meaning no conflicts).
Therefore, any pages that use lms-main.scss can use any bootstrap component out
of the box. **To use bootstrap components in older, v1, v2 pages** we have to
manually specify the exact Sass partial from the ``edx-
platform/node_modules/bootstrap`` that we want. For example, to add an alert to
a v1 page, you would add to ``base/build`` the partial ``bootstrap/scss/alert``.
Steps outlined below.
1. Determine from the ``node_modules/bootstrap/scss`` directory which component
you want to import.
2. Add that import (i.e: ``bootstrap/scss/alert``) somewhere in the Sass tree,
most likely to ``base/build.scss``
3. NOTE: Run ``paver compile_sass`` to make sure it works, if not, there is
likely a bootstrap mixin that you are missing. If so, search for the mixin in
``node_modules/bootstrap/scss/mixins`` and import it (i.e:
``bootstrap/scss/mixins/alert``)
4. You are done. Add the component using HTML or Javascript. Use the Bootstrap
Component Docs for examples.
FAQ
***
I want to add a new scss file for a feature on a particular page, how do I add it?
##################################################################################
First, you want to check which root file the page uses, whether it is
lms-main-v1, lms-main-v2 or lms-main. Then, go to that file and trace down the
tree to find a good spot for the new scss file to live. So, for example, if you
are adding an LMS feature and the page uses lms-main-v1, you can trace down lms-
main-v1 > _build_lms_v1 and see that there are plenty of standalone imports that
look like features. Simply add your file to the lms/static/sass directory in a
similar manner and add it as an @import to that page.
Why isn't my bootstrap component styling like in the bootstrap docs?
####################################################################
In any pages that use the lms-main.css compiled file, the bootstrap component
will render. This case only arises when we are working with a legacy v1 or v2
page. As outlined earlier in this document, we cannot import the entire
bootstrap repository due to conflicts with old styling. For v1 and v2, we are
gradually pulling in styles for components, so this must mean that you are
building a component that has not yet been used in the LMS/Studio.
To add the styles, you first need to find them in the bootstrap package. To do
so, search at edx-platform/node_modules/bootstrap/scss for the file that you
need. Then add this to the v1 or v2 tree, most likely to the build/base.scss
file. Note that you may hit issues with mixins when you try to compile the SASS.
In this case, check the edx-platform/node_modules/bootstrap/scss/mixins folder
and import that into the lms or studio before trying to import the component
styling.