From 731d8a4e6858487c1aef68221d81a45e552ff150 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vedran=20Kara=C4=8Di=C4=87?= Date: Mon, 2 Nov 2015 13:14:45 +0100 Subject: [PATCH] Changed command description texts to be sentences --- docs/en_us/internal/testing.rst | 206 +++++++++++++++++++++----------- 1 file changed, 139 insertions(+), 67 deletions(-) diff --git a/docs/en_us/internal/testing.rst b/docs/en_us/internal/testing.rst index 7cbb1194a3..e47ad6c2b1 100644 --- a/docs/en_us/internal/testing.rst +++ b/docs/en_us/internal/testing.rst @@ -112,7 +112,7 @@ example, the factory for creating problem XML definitions is located in Running Tests ============= -You can run all of the unit-level tests using the command +You can run all of the unit-level tests using this command. :: @@ -122,7 +122,7 @@ This includes python, javascript, and documentation tests. It does not, however, run any acceptance tests. Note - -`paver` is a scripting tool. To get information about various options, you can run the following command - +`paver` is a scripting tool. To get information about various options, you can run the this command. :: paver -h Running Python Unit tests @@ -132,44 +132,50 @@ We use `nose `__ through the `django-nose plugin `__ to run the test suite. -You can run all the python tests using ``paver`` commands. For example, +For example, this command runs all the python test scripts. :: paver test_python -runs all the tests. It also runs ``collectstatic``, which prepares the -static files used by the site (for example, compiling Coffeescript to -Javascript). +It also runs ``collectstatic``, which prepares the +static files used by the site (for example, compiling CoffeeScript to +JavaScript). -You can re-run all failed python tests by running: (see note at end of -section) +You can re-run all failed python tests by running this command (see note at end of +section). :: paver test_python --failed -To test lms or cms python, use:: +To test lms python tests use this command. + +:: paver test_system -s lms -or +To test cms python tests use this command. :: paver test_system -s cms -You can also run these tests without ``collectstatic``, which is faster:: +To run these tests without ``collectstatic``, which is faster, append the following argument. + +:: paver test_system -s lms --fasttest -or +To run cms python tests without ``collectstatic`` use this command. :: paver test_system -s cms --fasttest -To run a single django test class:: +To run a single django test class use this command. + +:: paver test_system -t lms/djangoapps/courseware/tests/tests.py:ActivateLoginTest @@ -177,16 +183,21 @@ When developing tests, it is often helpful to be able to really just run one single test without the overhead of PIP installs, UX builds, etc. In this case, it is helpful to look at the output of paver, and run just the specific command (optionally, stripping away coverage metrics). At -the time of this writing, the command is:: +the time of this writing, the command is the following. + +:: python ./manage.py lms test --verbosity=1 lms/djangoapps/courseware/tests/test_courses.py --traceback --settings=test -To run a single django test:: + +To run a single test format the command like this. + +:: paver test_system -t lms/djangoapps/courseware/tests/tests.py:ActivateLoginTest.test_activate_login To re-run all failing django tests from lms or cms, use the -``--failed``,\ ``-f`` flag (see note at end of section) +``--failed``,\ ``-f`` flag (see note at end of section). :: @@ -197,46 +208,58 @@ There is also a ``--fail_fast``, ``-x`` option that will stop nosetests after the first failure. common/lib tests are tested with the ``test_lib`` task, which also -accepts the ``--failed`` and ``--fail_fast`` options. For example:: - - paver test_lib -l common/lib/calc - -or +accepts the ``--failed`` and ``--fail_fast`` options. :: + paver test_lib -l common/lib/calc paver test_lib -l common/lib/xmodule --failed -To run a single nose test file:: +For example, this command runs a single nose test file. + +:: nosetests common/lib/xmodule/xmodule/tests/test_stringify.py -To run a single nose test:: +This command runs a single nose test within a specified file. + +:: nosetests common/lib/xmodule/xmodule/tests/test_stringify.py:test_stringify -To run a single test and get stdout, with proper env config:: + +This is an example of how to run a single test and get stdout, with proper env config. + +:: python manage.py cms --settings test test contentstore.tests.test_import_nostatic -s -To run a single test and get stdout and get coverage:: +These are examples of how to run a single test and get stdout and get coverage. + +:: python -m coverage run --rcfile=./common/lib/xmodule/.coveragerc which ./manage.py cms --settings test test --traceback --logging-clear-handlers --liveserver=localhost:8000-9000 contentstore.tests.test_import_nostatic -s # cms example python -m coverage run --rcfile=./lms/.coveragerc which ./manage.py lms --settings test test --traceback --logging-clear-handlers --liveserver=localhost:8000-9000 courseware.tests.test_module_render -s # lms example -generate coverage report:: +Use this command to generate coverage report. + +:: coverage report --rcfile=./common/lib/xmodule/.coveragerc -or to get html report:: +Use this command to generate an HTML report. + +:: coverage html --rcfile=./common/lib/xmodule/.coveragerc -then browse reports/common/lib/xmodule/cover/index.html +The report is then saved in reports/common/lib/xmodule/cover/index.html To run tests for stub servers, for example for `YouTube stub server `__, -you can do one of:: +you can run one of these commands. + +:: paver test_system -s cms -t common/djangoapps/terrain/stubs/tests/test_youtube_stub.py python -m coverage run --rcfile=cms/.coveragerc `which ./manage.py` cms --settings test test --traceback common/djangoapps/terrain/stubs/tests/test_youtube_stub.py @@ -271,7 +294,9 @@ tests:: paver test_js To run a specific set of JavaScript tests and print the results to the -console:: +console, run these commands. + +:: paver test_js_run -s lms paver test_js_run -s lms-coffee @@ -281,7 +306,9 @@ console:: paver test_js_run -s common paver test_js_run -s common-requirejs -To run JavaScript tests in a browser: +To run JavaScript tests in a browser, run these commands. + +:: paver test_js_dev -s lms paver test_js_dev -s lms-coffee @@ -331,42 +358,58 @@ supported development enviornment for the edX Platform. * mySQL -To run all the bok choy acceptance tests:: +To run all the bok choy acceptance tests run this command. + +:: paver test_bokchoy Once the database has been set up and the static files collected, you can use the 'fast' option to skip those tasks. This option can also be -used with any of the test specs below:: +used with any of the test specs below. + +:: paver test_bokchoy --fasttest -To run a single test, specify the name of the test file. For example:: +For example to run a single test, specify the name of the test file. + +:: paver test_bokchoy -t lms/test_lms.py Notice the test file location is relative to -common/test/acceptance/tests. For example:: +common/test/acceptance/tests. This is another example. + +:: paver test_bokchoy -t studio/test_studio_bad_data.py -To run a single test faster by not repeating setup tasks:: +To run a single test faster by not repeating setup tasks us the ``--fasttest`` option. + +:: paver test_bokchoy -t studio/test_studio_bad_data.py --fasttest -To test only a certain feature, specify the file and the testcase class:: +To test only a certain feature, specify the file and the testcase class. + +:: paver test_bokchoy -t studio/test_studio_bad_data.py:BadComponentTest To execute only a certain test case, specify the file name, class, and -test case method:: +test case method. + +:: paver test_bokchoy -t lms/test_lms.py:RegistrationTest.test_register During acceptance test execution, log files and also screenshots of failed tests are captured in test\_root/log. -To put a debugging breakpoint in a test use:: +Use this command to put a debugging breakpoint in a test. + +:: from nose.tools import set_trace; set_trace() @@ -374,8 +417,10 @@ By default, all bokchoy tests are run with the 'split' ModuleStore. To override the modulestore that is used, use the default\_store option. The currently supported stores are: 'split' (xmodule.modulestore.split\_mongo.split\_draft.DraftVersioningModuleStore) -and 'draft' (xmodule.modulestore.mongo.DraftMongoModuleStore). For -example:: +and 'draft' (xmodule.modulestore.mongo.DraftMongoModuleStore). This is an example +for the 'draft' store. + +:: paver test_bokchoy --default_store='draft' @@ -407,12 +452,16 @@ These prerequisites are all automatically installed and available in `Devstack * mySQL -To run all the bok choy accessibility tests:: +To run all the bok choy accessibility tests use this command. + +:: paver test_bokchoy --extra_args="-a 'a11y'" To run specific tests, use the ``-t`` flag to specify a nose-style test spec -relative to the ``common/test/acceptance/tests`` directory:: +relative to the ``common/test/acceptance/tests`` directory. This is an example for it. + +:: paver test_bokchoy --extra_args="-a 'a11y'" -t test_lms_dashboard.py:LmsDashboardA11yTest.test_dashboard_course_listings_a11y @@ -431,26 +480,34 @@ installed to run the tests in Chrome. The tests are confirmed to run with Chrome (not Chromium) version 34.0.1847.116 with ChromeDriver version 2.6.232917. -To run all the acceptance tests:: +To run all the acceptance tests, run this command. + +:: paver test_acceptance -To run only for lms or cms:: +To run only for lms or cms, run one of these commands. + +:: paver test_acceptance -s lms paver test_acceptance -s cms -To test only a specific feature:: +For example, this command tests only a specific feature. + +:: paver test_acceptance -s lms --extra_args="lms/djangoapps/courseware/features/problems.feature" -To test only a specific scenario +A command like this tests only a specific scenario. :: paver test_acceptance -s lms --extra_args="lms/djangoapps/courseware/features/problems.feature -s 3" -To start the debugger on failure, pass the ``--pdb`` option to the paver command:: +To start the debugger on failure, pass the ``--pdb`` option to the paver command like this. + +:: paver test_acceptance -s lms --pdb --extra_args="lms/djangoapps/courseware/features/problems.feature" @@ -506,24 +563,21 @@ according to the template string ``{scenario_number}__{step_number}__{step_function_name}__{"1_before"|"2_after"}``. If you don't want to have screenshots be captured for all steps, but -rather want fine grained control, you can use the decorator +rather want fine grained control, you can use this decorator before any Python function in ``feature_name.py`` file. :: @capture_screenshot_before_after -before any Python function in ``feature_name.py`` file. The decorator -will capture two screenshots - one before the decorated function runs, -and one after. Also, the function +The decorator will capture two screenshots: one before the decorated function runs, +and one after. Also, this function is available, and can be inserted at any point in code to capture a +screenshot specifically in that place. :: from lettuce import world; world.capture_screenshot("image_name") -is available, and can be inserted at any point in code to capture a -screenshot specifically in that place. In both cases the captured -screenshots will go to the same folder as when using the step method - -``{TEST_ROOT}/log/auto_screenshot``. +In both cases the captured screenshots will go to the same folder as when using the step method: ``{TEST_ROOT}/log/auto_screenshot``. A totally different approach to visually seeing acceptance tests run in Vagrant is to redirect Vagrant X11 session to your local machine. Please @@ -538,11 +592,15 @@ unit/integration tests. To view test coverage: -1. Run the test suite:: +1. Run the test suite with this command. + +:: paver test -2. Generate reports:: +2. Generate reports with this command. + +:: paver coverage @@ -550,21 +608,27 @@ To view test coverage: HTML and XML (Cobertura format) reports. Python Code Style Quality ------------------- +------------------------- -To view Python code style quality (including pep8 and pylint violations):: +To view Python code style quality (including pep8 and pylint violations) run this command. + +:: paver run_quality More specific options are below. -- Running a particular quality report:: +- These commands run a particular quality report. + +:: paver run_pep8 paver run_pylint -- Running a report, and setting it to fail if it exceeds a given number - of violations:: +- This command runs a report, and sets it to fail if it exceeds a given number + of violations. + +:: paver run_pep8 --limit=800 @@ -574,12 +638,16 @@ More specific options are below. that, the command can be set to fail if a certain diff threshold is not met. For example, to cause the process to fail if quality expectations are less than 100% when compared to master (or in other - words, if style quality is worse than what's already on master):: + words, if style quality is worse than what is already on master). + +:: paver run_quality --percentage=100 - Note that 'fixme' violations are not counted with run\_quality. To - see all 'TODO' lines, use:: + see all 'TODO' lines, use this command. + +:: paver find_fixme --system=lms @@ -590,11 +658,15 @@ More specific options are below. JavaScript Code Style Quality ------------------ -To view JavaScript code style quality:: +To view JavaScript code style quality run this command. + +:: paver run_jshint -- This command also comes with a ``--limit`` switch, for example:: +- This command also comes with a ``--limit`` switch, this is an example of that switch. + +:: paver run_jshint --limit=700