Andal.LND/edx-platform
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Switch to common structure and shared resources

Browse Source
This commit is contained in:
Julian Arni
2013-07-16 13:01:12 -04:00
parent e3fa27a3d8
commit bf60b24aa8
44 changed files with 185 additions and 585 deletions
Download Patch File Download Diff File Expand all files Collapse all files

112
docs/internal/code_standards.txt Normal file
View File

@@ -0,0 +1,112 @@
Scope
This document describes code quality standards for the i4x
system.
1. Coding Standards
Code falls into four categories:
* Deployed. Running on a live server.
* Production. Intended for deployment.
* Scaffolding. Intended to define interfaces for future work, and
minimal implementations to support further development.
* Prototype. Experimental new features.
1.1 Deployed
The standards for deployed code are identical to production. In
general, we tend to do either:
1) Perform a final verification QA cycle on changed parts of code
before deploying.
2) Use code on a staging or internal server for a week before
deploying.
1.2 Production
All production code must be peer-reviewed. The code must meet the
following standards:
1) Test Suite. Code must have reasonable, although not complete, test
coverage.
2) Consistent. Code must follow PEP8
3) Clean Abstractions.
4) Future-Compatible. Code must not be incompatible with the
long-term vision of either the codebase or of edX.
5) Properly Documented
6) Maintainable and deployable
7) Robust.
All code paths must be manually or automatically verified.
1.3 Scaffolding
All scaffolding code should be peer-reviewed. The code must meet the
following standards:
1) Testable. We do not require test coverage, but we do require the
code to be structured such that it is possible to build tests.
2) Consistent. Code must follow PEP8
3) Clean abstractions or obvious throw-away code. One of the goals
of scaffolding is to define proper abstractions.
4) Future-Compatible. Code must not be incompatible with the
long-term vision of either the codebase or of edX.
5) Somewhat documented
6) Unpluggable. There should be a setting to disable scaffolding code.
By default, and by policy, it should never be enabled on production
servers.
7) Purpose. The scaffolding must provide a clean reason for existence
(e.g. define a specific interface, etc.)
1.4 Prototype
Prototype code should live in a separate branch. It should strive
to follow PEP8, be readable, testable, and future-proof, but we have
no hard standards.
2. Process Standards
* Code should be integrated in small pull requests. Large commits
should be broken down into small commits for integration.
* Every piece of production and deployed code must be reviewed prior
to integration.
* Anyone on the edX team competent to review a piece of code may
review it (this may change as the team grows).
* Each contributor is responsible for finding a person to review their
code. If it is not clear to the contributor who is appropriate, each
project has an owner who is the default go-to.
2.1 Rapid pull
Unmerged code can lead to merge conflicts, and slow down
development. We have an experimental procedure for handling rapid
pulls and merges. To qualify:
* A piece of code must only have minor issues remaining (nothing which
we would be uncomfortable placing on a server).
* Either the requester or the puller takes ownership for guaranteeing
that those issues are resolved within a short timeframe.
* Both the requester and the puller must be comfortable with it.
* Both the requester and the owner must have a history of/ability to
resolve remaining issues quickly.
If code qualifies:
* It can be merged, and repaired in master.
* The pull message should specify '## pending fixes/OWNER' where ## is
the pull request number, and OWNER is the owner.
* All required fixes are documented in github in the (now closed) pull
request, and should be marked off there when applied (potentially,
directly to master).
* Once all fixes are applied, the final commit should specify
'## closed'.
3. Documentation Standards
* Whenever possible, documentation should live in code.
* When impossible, it should live in the github repo.
* Discussion should live on github, Basecamp or Pivotal, depending on
context.
* Notes for later fixes should in general be put into Pivotal as stories.
If they are left in the code, they should be prefixed by
# TODO (<name>)

120
docs/internal/development.md Normal file
View File

@@ -0,0 +1,120 @@
# Development Tasks
## Prerequisites
### Ruby
To install all of the libraries needed for our rake commands, run `bundle install`.
This will read the `Gemfile` and install all of the gems specified there.
### Python
Run the following::
pip install -r requirements.txt
### Binaries
Install the following:
* Mongodb (http://www.mongodb.org/)
### Databases
First start up the mongo daemon. E.g. to start it up in the background
using a config file:
mongod --config /usr/local/etc/mongod.conf &
Check out the course data directories that you want to work with into the
`GITHUB_REPO_ROOT` (by default, `../data`). Then run the following command:
rake resetdb
## Installing
To create your development environment, run the shell script in the root of
the repo:
scripts/create-dev-env.sh
## Starting development servers
Both the LMS and Studio can be started using the following shortcut tasks
rake lms # Start the LMS
rake cms # Start studio
rake lms[cms.dev] # Start LMS to run alongside Studio
rake lms[cms.dev_preview] # Start LMS to run alongside Studio in preview mode
Under the hood, this executes `django-admin.py runserver --pythonpath=$WORKING_DIRECTORY --settings=lms.envs.dev`,
which starts a local development server.
Both of these commands take arguments to start the servers in different environments
or with additional options:
# Start the LMS using the test configuration, on port 5000
rake lms[test,5000] # Executes django-admin.py runserver --pythonpath=$WORKING_DIRECTORY --setings=lms.envs.test 5000
*N.B.* You may have to escape the `[` characters, depending on your shell: `rake "lms[test,5000]"`
To get a full list of available rake tasks, use:
rake -T
### Troubleshooting
#### Reference Error: XModule is not defined (javascript)
This means that the javascript defining an xmodule hasn't loaded correctly. There are a number
of different things that could be causing this:
1. See `Error: watch EMFILE`
#### Error: watch EMFILE (coffee)
When running a development server, we also start a watcher process alongside to recompile coffeescript
and sass as changes are made. On Mac OSX systems, the coffee watcher process takes more file handles
than are allowed by default. This will result in `EMFILE` errors when coffeescript is running, and
will prevent javascript from compiling, leading to the error 'XModule is not defined'
To work around this issue, we use `Process::setrlimit` to set the number of allowed open files.
Coffee watches both directories and files, so you will need to set this fairly high (anecdotally,
8000 seems to do the trick on OSX 10.7.5, 10.8.3, and 10.8.4)
## Running Tests
See `testing.md` for instructions on running the test suite.
## Content development
If you change course content, while running the LMS in dev mode, it is unnecessary to restart to refresh the modulestore.
Instead, hit /migrate/modules to see a list of all modules loaded, and click on links (eg /migrate/reload/edx4edx) to reload a course.
### Gitreload-based workflow
github (or other equivalent git-based repository systems) used for
course content can be setup to trigger an automatic reload when changes are pushed. Here is how:
1. Each content directory in mitx_all/data should be a clone of a git repo
2. The user running the mitx gunicorn process should have its ssh key registered with the git repo
3. The list settings.ALLOWED_GITRELOAD_IPS should contain the IP address of the git repo originating the gitreload request.
By default, this list is ['207.97.227.253', '50.57.128.197', '108.171.174.178'] (the github IPs).
The list can be overridden in the startup file used, eg lms/envs/dev*.py
4. The git post-receive-hook should POST to /gitreload with a JSON payload. This payload should define at least
{ "repository" : { "name" : reload_dir }
where reload_dir is the directory name of the content to reload (ie mitx_all/data/reload_dir should exist)
The mitx server will then do "git reset --hard HEAD; git clean -f -d; git pull origin" in that directory. After the pull,
it will reload the modulestore for that course.
Note that the gitreload-based workflow is not meant for deployments on AWS (or elsewhere) which use collectstatic, since collectstatic is not run by a gitreload event.
Also, the gitreload feature needs MITX_FEATURES['ENABLE_LMS_MIGRATION'] = True in the django settings.

159
docs/internal/discussion.md Normal file
View File

@@ -0,0 +1,159 @@
# Running the discussion service
## Instruction for Mac
## Installing Mongodb
If you haven't done so already:
brew install mongodb
Make sure that you have mongodb running. You can simply open a new terminal tab and type:
mongod
## Installing elasticsearch
brew install elasticsearch
For debugging, it's often more convenient to have elasticsearch running in a terminal tab instead of in background. To do so, simply open a new terminal tab and then type:
elasticsearch -f
## Setting up the discussion service
First, make sure that you have access to the [github repository](https://github.com/rll/cs_comments_service). If this were not the case, send an email to dementrock@gmail.com.
First go into the mitx_all directory. Then type
git clone git@github.com:rll/cs_comments_service.git
cd cs_comments_service/
If you see a prompt asking "Do you wish to trust this .rvmrc file?", type "y"
Now if you see this error "Gemset 'cs_comments_service' does not exist," run the following command to create the gemset and then use the rvm environment manually:
rvm gemset create 'cs_comments_service'
rvm use 1.9.3@cs_comments_service
Now use the following command to install required packages:
bundle install
The following command creates database indexes:
bundle exec rake db:init
Now use the following command to generate seeds (basically some random comments in Latin):
bundle exec rake db:seed
It's done! Launch the app now:
ruby app.rb
## Running the delayed job worker
In the discussion service, notifications are handled asynchronously using a third party gem called delayed_job. If you want to test this functionality, run the following command in a separate tab:
bundle exec rake jobs:work
## Initialize roles and permissions
To fully test the discussion forum, you might want to act as a moderator or an administrator. Currently, moderators can manage everything in the forum, and administrator can manage everything plus assigning and revoking moderator status of other users.
First make sure that the database is up-to-date:
rake django-admin[syncdb]
rake django-admin[migrate]
For convenience, add the following environment variables to the terminal (assuming that you're using configuration set lms.envs.dev):
export DJANGO_SETTINGS_MODULE=lms.envs.dev
export PYTHONPATH=.
Now initialzie roles and permissions, providing a course id eg.:
django-admin.py seed_permissions_roles "MITx/6.002x/2012_Fall"
To assign yourself as a moderator, use the following command (assuming your username is "test", and the course id is "MITx/6.002x/2012_Fall"):
django-admin.py assign_role test Moderator "MITx/6.002x/2012_Fall"
To assign yourself as an administrator, use the following command
django-admin.py assign_role test Administrator "MITx/6.002x/2012_Fall"
## Some other useful commands
### generate seeds for a specific forum
The seed generating command above assumes that you have the following discussion tags somewhere in the course data:
<discussion for="Welcome Video" id="video_1" discussion_category="Video"/>
<discussion for="Lab 0: Using the Tools" id="lab_1" discussion_category="Lab"/>
<discussion for="Lab Circuit Sandbox" id="lab_2" discussion_category="Lab"/>
For example, you can insert them into overview section as following:
<chapter name="Overview">
<section format="Video" name="Welcome">
<vertical>
<video youtube="0.75:izygArpw-Qo,1.0:p2Q6BrNhdh8,1.25:1EeWXzPdhSA,1.50:rABDYkeK0x8"/>
<discussion for="Welcome Video" id="video_1" discussion_category="Video"/>
</vertical>
</section>
<section format="Lecture Sequence" name="System Usage Sequence">
<%include file="sections/introseq.xml"/>
</section>
<section format="Lab" name="Lab0: Using the tools">
<vertical>
<html> See the <a href="/section/labintro"> Lab Introduction </a> or <a href="/static/handouts/schematic_tutorial.pdf">Interactive Lab Usage Handout </a> for information on how to do the lab </html>
<problem name="Lab 0: Using the Tools" filename="Lab0" rerandomize="false"/>
<discussion for="Lab 0: Using the Tools" id="lab_1" discussion_category="Lab"/>
</vertical>
</section>
<section format="Lab" name="Circuit Sandbox">
<vertical>
<problem name="Circuit Sandbox" filename="Lab_sandbox" rerandomize="false"/>
<discussion for="Lab Circuit Sandbox" id="lab_2" discussion_category="Lab"/>
</vertical>
</section>
</chapter>
Currently, only the attribute "id" is actually used, which identifies discussion forum. In the code for the data generator, the corresponding lines are:
generate_comments_for("video_1")
generate_comments_for("lab_1")
generate_comments_for("lab_2")
We also have a command for generating comments within a forum with the specified id:
bundle exec rake db:generate_comments[type_the_discussion_id_here]
For instance, if you want to generate comments for a new discussion tab named "lab_3", then use the following command
bundle exec rake db:generate_comments[lab_3]
### Running tests for the service
bundle exec rspec
Warning: the development and test environments share the same elasticsearch index. After running tests, search may not work in the development environment. You simply need to reindex:
bundle exec rake db:reindex_search
### debugging the service
You can use the following command to launch a console within the service environment:
bundle exec rake console
### show user roles and permissions
Use the following command to see the roles and permissions of a user in a given course (assuming, again, that the username is "test"):
django-admin.py show_permissions moderator
You need to make sure that the environment variables are exported. Otherwise you would need to do
django-admin.py show_permissions moderator --settings=lms.envs.dev --pythonpath=.

145
docs/internal/mongo.md Normal file
View File

@@ -0,0 +1,145 @@
# Notes on using mongodb backed LMS and CMS
These are some random notes for developers, on how things are stored in mongodb, and how to debug mongodb data.
## Databases
Two mongodb databases are used:
- xmodule: stores module definitions and metadata (modulestore)
- xcontent: stores filesystem content, like PDF files
modulestore documents are stored with an _id which has fields like this:
{"_id": {"tag":"i4x","org":"HarvardX","course":"CS50x","category":"chapter","name":"Week_1","revision":null}}
## Document fields
### Problems
Here is an example showing the fields available in problem documents:
{
"_id" : {
"tag" : "i4x",
"org" : "MITx",
"course" : "6.00x",
"category" : "problem",
"name" : "ps03:ps03-Hangman_part_2_The_Game",
"revision" : null
},
"definition" : {
"data" : " ..."
},
"metadata" : {
"display_name" : "Hangman Part 2: The Game",
"attempts" : "30",
"title" : "Hangman, Part 2",
"data_dir" : "6.00x",
"type" : "lecture"
}
}
## Sample interaction with mongodb
1. "mongo"
2. "use xmodule"
3. "show collections" should give "modulestore" and "system.indexes"
4. 'db.modulestore.find( {"_id.org": "MITx"} )' will produce a list of all MITx course documents
5. 'db.modulestore.find( {"_id.org": "MITx", "_id.category": "problem"} )' will produce a list of all problems in MITx courses
Example query for finding all files with "image" in the filename:
- use xcontent
- db.fs.files.find({'filename': /image/ } )
- db.fs.files.find({'filename': /image/ } ).count()
## Debugging the mongodb contents
A convenient tool is http://phpmoadmin.com/ (needs php)
Under ubuntu, do:
- apt-get install php5-fpm php-pear
- pecl install mongo
- edit /etc/php5/fpm/php.ini to add "extension=mongo.so"
- /etc/init.d/php5-fpm restart
and also setup nginx to run php through fastcgi.
## Backing up mongodb
- mogodump (dumps all dbs)
- mongodump --collection modulestore --db xmodule (dumps just xmodule/modulestore)
- mongodump -d xmodule -q '{"_id.org": "MITx"}' (dumps just MITx documents in xmodule)
- mongodump -q '{"_id.org": "MITx"}' (dumps all MITx documents)
## Deleting course content
Use "remove" instead of "find":
- db.modulestore.remove( {"_id.course": "8.01greytak"})
## Finding useful information from the mongodb modulestore
- Organizations
> db.modulestore.distinct( "_id.org")
[ "HarvardX", "MITx", "edX", "edx" ]
- Courses
> db.modulestore.distinct( "_id.course")
[
"CS50x",
"PH207x",
"3.091x",
"6.002x",
"6.00x",
"8.01esg",
"8.01rq_MW",
"8.02teal",
"8.02x",
"edx4edx",
"toy",
"templates"
]
- Find a problem which has the word "quantum" in its definition
db.modulestore.findOne( {"definition.data":/quantum/})n
- Find Location for all problems with the word "quantum" in its definition
db.modulestore.find( {"definition.data":/quantum/}, {'_id':1})
- Number of problems in each course
db.runCommand({
mapreduce: "modulestore",
query: { '_id.category': 'problem' },
map: function(){ emit(this._id.course, {count:1}); },
reduce: function(key, values){
var result = {count:0};
values.forEach(function(value) {
result.count += value.count;
});
return result;
},
out: 'pbyc',
verbose: true
});
produces:
> db.pbyc.find()
{ "_id" : "3.091x", "value" : { "count" : 184 } }
{ "_id" : "6.002x", "value" : { "count" : 176 } }
{ "_id" : "6.00x", "value" : { "count" : 147 } }
{ "_id" : "8.01esg", "value" : { "count" : 184 } }
{ "_id" : "8.01rq_MW", "value" : { "count" : 73 } }
{ "_id" : "8.02teal", "value" : { "count" : 5 } }
{ "_id" : "8.02x", "value" : { "count" : 99 } }
{ "_id" : "PH207x", "value" : { "count" : 25 } }
{ "_id" : "edx4edx", "value" : { "count" : 50 } }
{ "_id" : "templates", "value" : { "count" : 11 } }

143
docs/internal/overview.md Normal file
View File

@@ -0,0 +1,143 @@
# Documentation for edX code (edx-platform repo)
This document explains the general structure of the edX platform, and defines some of the acronyms and terms you'll see flying around in the code.
## Assumptions:
You should be familiar with the following. If you're not, go read some docs...
- python
- django
- javascript
- html, xml -- xpath, xslt
- css
- git
- mako templates -- we use these instead of django templates, because they support embedding real python.
## Other relevant terms
- CAPA -- lon-capa.org -- content management system that has defined a standard for online learning and assessment materials. Many of our materials follow this standard.
- TODO: add more details / link to relevant docs. lon-capa.org is not immediately intuitive.
- lcp = loncapa problem
## Parts of the system
- LMS -- Learning Management System. The student-facing parts of the system. Handles student accounts, displaying videos, tutorials, exercies, problems, etc.
- CMS -- Course Management System. The instructor-facing parts of the system. Allows instructors to see and modify their course, add lectures, problems, reorder things, etc.
- Forums -- this is a ruby on rails service that runs on Heroku. Contributed by berkeley folks. The LMS has a wrapper lib that talks to it.
- Data. In the data/ dir. There is currently a single `course.xml` file that describes an entire course. Speaking of which...
- Courses. A course is broken up into Chapters ("week 1", "week 2", etc). A chapter is broken up into Sections ("Lecture 1", "Simple Circuits Exercises", "HW1", etc). A section can contain modules: Problems, Html, Videos, Verticals, or Sequences.
- Problems: specified in problem files. May have python scripts embedded to both generate random parameters and check answers. Also allows specifying things like tolerance or precision in answers
- Html: any html - often description, or links to outside resources
- Videos: links to youtube or elsewhere
- Verticals: a nesting tag: collect several videos, problems, html modules and display them vertically.
- Sequences: a sequence of modules, displayed with a horizontal navigation bar, displaying one component at a time.
- see `data/course.xml` for more examples
## High Level Entities in the code
### Common libraries
- xmodule: generic learning modules. *x* can be sequence, video, template, html,
vertical, capa, etc. These are the things that one puts inside sections
in the course structure.
- XModuleDescriptor: This defines the problem and all data and UI needed to edit
that problem. It is unaware of any student data, but can be used to retrieve
an XModule, which is aware of that student state.
- XModule: The XModule is a problem instance that is particular to a student. It knows
how to render itself to html to display the problem, how to score itself,
and how to handle ajax calls from the front end.
- Both XModule and XModuleDescriptor take system context parameters. These are named
ModuleSystem and DescriptorSystem respectively. These help isolate the XModules
from any interactions with external resources that they require.
For instance, the DescriptorSystem has a function to load an XModuleDescriptor
from a Location object, and the ModuleSystem knows how to render things,
track events, and complain about 404s
- `course.xml` format. We use python setuptools to connect supported tags with the descriptors that handle them. See `common/lib/xmodule/setup.py`. There are checking and validation tools in `common/validate`.
- the xml import+export functionality is in `xml_module.py:XmlDescriptor`, which is a mixin class that's used by the actual descriptor classes.
- There is a distinction between descriptor _definitions_ that stay the same for any use of that descriptor (e.g. here is what a particular problem is), and _metadata_ describing how that descriptor is used (e.g. whether to allow checking of answers, due date, etc). When reading in `from_xml`, the code pulls out the metadata attributes into a separate structure, and puts it back on export.
- in `common/lib/xmodule`
- capa modules -- defines `LoncapaProblem` and many related things.
- in `common/lib/capa`
### LMS
The LMS is a django site, with root in `lms/`. It runs in many different environments--the settings files are in `lms/envs`.
- We use the Django Auth system, including the is_staff and is_superuser flags. User profiles and related code lives in `lms/djangoapps/student/`. There is support for groups of students (e.g. 'want emails about future courses', 'have unenrolled', etc) in `lms/djangoapps/student/models.py`.
- `StudentModule` -- keeps track of where a particular student is in a module (problem, video, html)--what's their grade, have they started, are they done, etc. [This is only partly implemented so far.]
- `lms/djangoapps/courseware/models.py`
- Core rendering path:
- `lms/urls.py` points to `courseware.views.index`, which gets module info from the course xml file, pulls list of `StudentModule` objects for this user (to avoid multiple db hits).
- Calls `render_accordion` to render the "accordion"--the display of the course structure.
- To render the current module, calls `module_render.py:render_x_module()`, which gets the `StudentModule` instance, and passes the `StudentModule` state and other system context to the module constructor the get an instance of the appropriate module class for this user.
- calls the module's `.get_html()` method. If the module has nested submodules, render_x_module() will be called again for each.
- ajax calls go to `module_render.py:modx_dispatch()`, which passes it to the module's `handle_ajax()` function, and then updates the grade and state if they changed.
- [This diagram](https://github.com/MITx/mitx/wiki/MITx-Architecture) visually shows how the clients communicate with problems + modules.
- See `lms/urls.py` for the wirings of urls to views.
- Tracking: there is support for basic tracking of client-side events in `lms/djangoapps/track`.
### CMS
The CMS is a django site, with root in `cms`. It can run in a number of different
environments, defined in `cms/envs`.
- Core rendering path: Still TBD
### Static file processing
- CSS -- we use a superset of CSS called SASS. It supports nice things like includes and variables, and compiles to CSS. The compiler is called `sass`.
- javascript -- we use coffeescript, which compiles to js, and is much nicer to work with. Look for `*.coffee` files. We use _jasmine_ for testing js.
- _mako_ -- we use this for templates, and have wrapper called mitxmako that makes mako look like the django templating calls.
We use a fork of django-pipeline to make sure that the js and css always reflect the latest `*.coffee` and `*.sass` files (We're hoping to get our changes merged in the official version soon). This works differently in development and production. Test uses the production settings.
In production, the django `collectstatic` command recompiles everything and puts all the generated static files in a static/ dir. A starting point in the code is `django-pipeline/pipeline/packager.py:pack`.
In development, we don't use collectstatic, instead accessing the files in place. The auto-compilation is run via `common/djangoapps/pipeline_mako/templates/static_content.html`. Details: templates include `<%namespace name='static' file='static_content.html'/>`, then something like `<%static:css group='application'/>` to call the functions in `common/djangoapps/pipeline_mako/__init__.py`, which call the `django-pipeline` compilers.
### Other modules
- Wiki -- in `lms/djangoapps/simplewiki`. Has some markdown extentions for embedding circuits, videos, etc.
## Testing
See `testing.md`.
## TODO:
- describe our production environment
- describe the front-end architecture, tools, etc. Starting point: `lms/static`
---
Note: this file uses markdown. To convert to html, run:
markdown2 overview.md > overview.html

47
docs/internal/remote_gradebook.md Normal file
View File

@@ -0,0 +1,47 @@
Grades can be pushed to a remote gradebook, and course enrollment membership can be pulled from a remote gradebook. This file documents how to setup such a remote gradebook, and what the API should be for writing new remote gradebook "xservers".
1. Definitions
An "xserver" is a web-based server that is part of the MITx eco system. There are a number of "xserver" programs, including one which does python code grading, an xqueue server, and graders for other coding languages.
"Stellar" is the MIT on-campus gradebook system.
2. Setup
The remote gradebook xserver should be specified in the lms.envs configuration using
MITX_FEATURES[REMOTE_GRADEBOOK_URL]
Each course, in addition, should define the name of the gradebook being used. A class "section" may also be specified. This goes in the policy.json file, eg:
"remote_gradebook": {
"name" : "STELLAR:/project/mitxdemosite",
"section" : "r01"
},
3. The API for the remote gradebook xserver is an almost RESTful service model, which only employs POSTs, to the xserver url, with form data for the fields:
- submit: get-assignments, get-membership, post-grades, or get-sections
- gradebook: name of gradebook
- user: username of staff person initiating the request (for logging)
- section: (optional) name of section
The return body content should be a JSON string, of the format {'msg': message, 'data': data}. The message is displayed in the instructor dashboard.
The data is a list of dicts (associative arrays). Each dict should be key:value.
## For submit=post-grades:
A file is also posted, with the field name "datafile". This file is CSV format, with two columns, one being "External email" and the other being the name of the assignment (that column contains the grades for the assignment).
## For submit=get-assignments
data keys = "AssignmentName"
## For submit=get-membership
data keys = "email", "name", "section"
## For submit=get-sections
data keys = "SectionName"

BIN
docs/internal/test_pyramid.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

232
docs/internal/testing.md Normal file
View File

@@ -0,0 +1,232 @@
# Testing
## Overview
We maintain three kinds of tests: unit tests, integration tests,
and acceptance tests.
### Unit Tests
* Each test case should be concise: setup, execute, check, and teardown.
If you find yourself writing tests with many steps, consider refactoring
the unit under tests into smaller units, and then testing those individually.
* As a rule of thumb, your unit tests should cover every code branch.
* Mock or patch external dependencies.
We use [voidspace mock](http://www.voidspace.org.uk/python/mock/).
* We unit test Python code (using [unittest](http://docs.python.org/2/library/unittest.html)) and
Javascript (using [Jasmine](http://pivotal.github.io/jasmine/))
### Integration Tests
* Test several units at the same time.
Note that you can still mock or patch dependencies
that are not under test! For example, you might test that
`LoncapaProblem`, `NumericalResponse`, and `CorrectMap` in the
`capa` package work together, while still mocking out template rendering.
* Use integration tests to ensure that units are hooked up correctly.
You do not need to test every possible input--that's what unit
tests are for. Instead, focus on testing the "happy path"
to verify that the components work together correctly.
* Many of our tests use the [Django test client](https://docs.djangoproject.com/en/dev/topics/testing/overview/) to simulate
HTTP requests to the server.
### UI Acceptance Tests
* Use these to test that major program features are working correctly.
* We use [lettuce](http://lettuce.it/) to write BDD-style tests. Most of
these tests simulate user interactions through the browser using
[splinter](http://splinter.cobrateam.info/).
Overall, you want to write the tests that **maximize coverage**
while **minimizing maintenance**.
In practice, this usually means investing heavily
in unit tests, which tend to be the most robust to changes in the code base.
![Test Pyramid](test_pyramid.png)
The pyramid above shows the relative number of unit tests, integration tests,
and acceptance tests. Most of our tests are unit tests or integration tests.
## Test Locations
* Python unit and integration tests: Located in
subpackages called `tests`.
For example, the tests for the `capa` package are located in
`common/lib/capa/capa/tests`.
* Javascript unit tests: Located in `spec` folders. For example,
`common/lib/xmodule/xmodule/js/spec` and `{cms,lms}/static/coffee/spec`
For consistency, you should use the same directory structure for implementation
and test. For example, the test for `src/views/module.coffee`
should be written in `spec/views/module_spec.coffee`.
* UI acceptance tests:
- Set up and helper methods: `common/djangoapps/terrain`
- Tests: located in `features` subpackage within a Django app.
For example: `lms/djangoapps/courseware/features`
## Factories
Many tests delegate set-up to a "factory" class. For example,
there are factories for creating courses, problems, and users.
This encapsulates set-up logic from tests.
Factories are often implemented using [FactoryBoy](https://readthedocs.org/projects/factoryboy/)
In general, factories should be located close to the code they use.
For example, the factory for creating problem XML definitions
is located in `common/lib/capa/capa/tests/response_xml_factory.py`
because the `capa` package handles problem XML.
# Running Tests
Before running tests, ensure that you have all the dependencies. You can install dependencies using:
rake install_prereqs
## Running Python Unit tests
We use [nose](https://nose.readthedocs.org/en/latest/) through
the [django-nose plugin](https://pypi.python.org/pypi/django-nose)
to run the test suite.
You can run tests using `rake` commands. For example,
rake test
runs all the tests. It also runs `collectstatic`, which prepares the static files used by the site (for example, compiling Coffeescript to Javascript).
You can also run the tests without `collectstatic`, which tends to be faster:
rake fasttest_lms
or
rake fasttest_cms
xmodule can be tested independently, with this:
rake test_common/lib/xmodule
other module level tests include
* `rake test_common/lib/capa`
* `rake test_common/lib/calc`
To run a single django test class:
rake test_lms[courseware.tests.tests:testViewAuth]
To run a single django test:
rake test_lms[courseware.tests.tests:TestViewAuth.test_dark_launch]
To run a single nose test file:
nosetests common/lib/xmodule/xmodule/tests/test_stringify.py
To run a single nose test:
nosetests common/lib/xmodule/xmodule/tests/test_stringify.py:test_stringify
Very handy: if you uncomment the `pdb=1` line in `setup.cfg`, it will drop you into pdb on error. This lets you go up and down the stack and see what the values of the variables are. Check out [the pdb documentation](http://docs.python.org/library/pdb.html)
### Running Javascript Unit Tests
To run all of the javascript unit tests, use
rake jasmine
If the `phantomjs` binary is on the path, or the `PHANTOMJS_PATH` environment variable is
set to point to it, then the tests will be run headless. Otherwise, they will be run in
your default browser
export PATH=/path/to/phantomjs:$PATH
rake jasmine # Runs headless
or
PHANTOMJS_PATH=/path/to/phantomjs rake jasmine # Runs headless
or
rake jasmine # Runs in browser
You can also force a run using phantomjs or the browser using the commands
rake jasmine:browser # Runs in browser
rake jasmine:phantomjs # Runs headless
You can run tests for a specific subsystems as well
rake jasmine:lms # Runs all lms javascript unit tests using the default method
rake jasmine:cms:browser # Runs all cms javascript unit tests in the browser
Use `rake -T` to get a list of all available subsystems
**Troubleshooting**: If you get an error message while running the `rake` task,
try running `bundle install` to install the required ruby gems.
### Running Acceptance Tests
We use [Lettuce](http://lettuce.it/) for acceptance testing.
Most of our tests use [Splinter](http://splinter.cobrateam.info/)
to simulate UI browser interactions. Splinter, in turn,
uses [Selenium](http://docs.seleniumhq.org/) to control the Chrome browser.
**Prerequisite**: You must have [ChromeDriver](https://code.google.com/p/selenium/wiki/ChromeDriver)
installed to run the tests in Chrome. The tests are confirmed to run
with Chrome (not Chromium) version 26.0.0.1410.63 with ChromeDriver
version r195636.
To run all the acceptance tests:
rake test_acceptance_lms
rake test_acceptance_cms
To test only a specific feature:
rake test_acceptance_lms[lms/djangoapps/courseware/features/problems.feature]
To start the debugger on failure, add the `--pdb` option:
rake test_acceptance_lms["lms/djangoapps/courseware/features/problems.feature --pdb"]
To run tests faster by not collecting static files, you can use
`rake fasttest_acceptance_lms` and `rake fasttest_acceptance_cms`.
**Note**: The acceptance tests can *not* currently run in parallel.
## Viewing Test Coverage
We currently collect test coverage information for Python unit/integration tests.
To view test coverage:
1. Run the test suite:
rake test
2. Generate reports:
rake coverage
3. Reports are located in the `reports` folder. The command
generates HTML and XML (Cobertura format) reports.
## Testing using queue servers
When testing problems that use a queue server on AWS (e.g. sandbox-xqueue.edx.org), you'll need to run your server on your public IP, like so.
`django-admin.py runserver --settings=lms.envs.dev --pythonpath=. 0.0.0.0:8000`
When you connect to the LMS, you need to use the public ip. Use `ifconfig` to figure out the number, and connect e.g. to `http://18.3.4.5:8000/`