diff --git a/.gitignore b/.gitignore index b1a36e5f2e..4fd90cfe03 100644 --- a/.gitignore +++ b/.gitignore @@ -45,3 +45,4 @@ node_modules autodeploy.properties .ws_migrations_complete .vagrant/ +logs diff --git a/.tx/config b/.tx/config index 540c4732af..9288418924 100644 --- a/.tx/config +++ b/.tx/config @@ -1,25 +1,25 @@ [main] host = https://www.transifex.com -[edx-studio.django-partial] +[edx-platform.django-partial] file_filter = conf/locale//LC_MESSAGES/django-partial.po source_file = conf/locale/en/LC_MESSAGES/django-partial.po source_lang = en type = PO -[edx-studio.djangojs] +[edx-platform.djangojs] file_filter = conf/locale//LC_MESSAGES/djangojs.po source_file = conf/locale/en/LC_MESSAGES/djangojs.po source_lang = en type = PO -[edx-studio.mako] +[edx-platform.mako] file_filter = conf/locale//LC_MESSAGES/mako.po source_file = conf/locale/en/LC_MESSAGES/mako.po source_lang = en type = PO -[edx-studio.messages] +[edx-platform.messages] file_filter = conf/locale//LC_MESSAGES/messages.po source_file = conf/locale/en/LC_MESSAGES/messages.po source_lang = en diff --git a/AUTHORS b/AUTHORS index 70af9f318d..c700eab277 100644 --- a/AUTHORS +++ b/AUTHORS @@ -81,3 +81,6 @@ Felix Sun Adam Palay Ian Hoover Mukul Goyal +Robert Marks +Yarko Tymciurak + diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 4d117a9c73..68308980ad 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -5,10 +5,23 @@ These are notable changes in edx-platform. This is a rolling list of changes, in roughly chronological order, most recent first. Add your entries at or near the top. Include a label indicating the component affected. +LMS: Added user preferences (arbitrary user/key/value tuples, for which +which user/key is unique) and a REST API for reading users and +preferences. Access to the REST API is restricted by use of the +X-Edx-Api-Key HTTP header (which must match settings.EDX_API_KEY; if +the setting is not present, the API is disabled). + +LMS: Added endpoints for AJAX requests to enable/disable notifications +(which are not yet implemented) and a one-click unsubscribe page. + +Common: Add a manage.py that knows about edx-platform specific settings and projects + Common: Added *experimental* support for jsinput type. Common: Added setting to specify Celery Broker vhost +Common: Utilize new XBlock bulk save API in LMS and CMS. + Studio: Add table for tracking course creator permissions (not yet used). Update rake django-admin[syncdb] and rake django-admin[migrate] so they run for both LMS and CMS. @@ -21,6 +34,8 @@ Studio: Added support for uploading and managing PDF textbooks Common: Student information is now passed to the tracking log via POST instead of GET. +Blades: Added functionality and tests for new capa input type: choicetextresponse. + Common: Add tests for documentation generation to test suite Blades: User answer now preserved (and changeable) after clicking "show answer" in choice problems @@ -43,6 +58,13 @@ history of background tasks for a given problem and student. Blades: Small UX fix on capa multiple-choice problems. Make labels only as wide as the text to reduce accidental choice selections. +Studio: +- use xblock field defaults to initialize all new instances' fields and +only use templates as override samples. +- create new instances via in memory create_xmodule and related methods rather +than cloning a db record. +- have an explicit method for making a draft copy as distinct from making a new module. + Studio: Remove XML from the video component editor. All settings are moved to be edited as metadata. diff --git a/README.md b/README.md index e533459c8b..4ab07b3550 100644 --- a/README.md +++ b/README.md @@ -12,30 +12,35 @@ installation process. 1. Make sure you have plenty of available disk space, >5GB 2. Install Git: http://git-scm.com/downloads -3. Install VirtualBox: https://www.virtualbox.org/wiki/Download_Old_Builds_4_2 - (you need version 4.2.12, as later/earlier versions might not work well with - Vagrant) +3. Install VirtualBox: https://www.virtualbox.org/wiki/Downloads + See http://docs.vagrantup.com/v2/providers/index.html for a list of supported + Providers. You should use VirtualBox >= 4.2.12. + (Windows: later/earlier VirtualBox versions than 4.2.12 have been reported to not work well with + Vagrant. If this is still a problem, you can + install 4.2.12 from https://www.virtualbox.org/wiki/Download_Old_Builds_4_2). 4. Install Vagrant: http://www.vagrantup.com/ (Vagrant 1.2.2 or later) 5. Open a terminal 6. Download the project: `git clone git://github.com/edx/edx-platform.git` 7. Enter the project directory: `cd edx-platform/` -8. (Windows only) Run the commands to +8. (Windows only) Run the commands to [deal with line endings and symlinks under Windows](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#dealing-with-line-endings-and-symlinks-under-windows) -9. Start: `vagrant up` +9. Create the development environment and start it: `vagrant up` -The last step might require your host machine's administrator password to setup NFS. +The initial `vagrant up` will download a Linux image, then boot and ask for your +host machine's administrator password to setup file sharing between your computer and the VM. +Once file sharing is established, `edx-platform/scripts/create-dev-env.sh` will +install dependencies and configure the VM. +This will take a while; go grab a coffee. -Afterwards, it will download an image, install all the dependencies and configure -the VM. It will take a while, go grab a coffee. +When complete, you should see a _"Success!"_ message. +If not, refer to the +[troubleshooting section](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#troubleshooting). -Once completed, hopefully you should see a "Success!" message indicating that the -installation went fine. (If not, refer to the -[troubleshooting section](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#troubleshooting).) +Your development environment is initialized only on the first bring-up. +Subsequently `vagrant up` commands will boot your virtual machine normally. -Note: by default, the VM will get the IP `192.168.20.40`. If you need to use a -different IP, you can edit the file `Vagrantfile`. If you have already started the -VM with `vagrant up`, see "Stopping and restarting the VM" below to take the change -into account. +Note: by default, the VM will get the IP `192.168.20.40`. +You can change this in your `Vagrantfile` (the startup message will reflect your VM's actual IP). Accessing the VM ---------------- @@ -46,15 +51,24 @@ Once the installation is finished, to log into the virtual machine: $ vagrant ssh ``` -Note: This won't work from Windows, install install PuTTY from -http://www.chiark.greenend.org.uk/%7Esgtatham/putty/download.html instead. Then -connect to 127.0.0.1, port 2222, using vagrant/vagrant as a user/password. +Note: This won't work from Windows. Instead, install PuTTY from +http://www.chiark.greenend.org.uk/%7Esgtatham/putty/download.html. Then +connect to 192.168.20.40, port 2222, using vagrant/vagrant as a user/password. + Using edX --------- -Once inside the VM, you can start Studio and LMS with the following commands -(from the `/opt/edx/edx-platform` folder): +When you login to your VM, you are in +`/opt/edx/edx-platform` by default, which is shared from your host workspace. +Your host computer contains the edx-project development code and repository. +Your VM runs edx-platform code mounted from your host, so +you can develop by editing on your host. + +After logging into your VM with `vagrant ssh`, +start the _Studio_ and +_Learning management system (LMS)_ +servers (run these from `/opt/edx/edx-platform`): Learning management system (LMS): @@ -62,46 +76,85 @@ Learning management system (LMS): $ rake lms[cms.dev,0.0.0.0:8000] ``` -Studio: +Studio (CMS): ``` $ rake cms[dev,0.0.0.0:8001] ``` -Once started, open the following URLs in your browser: +The servers will come up to these URLs: -* Learning management system (LMS): http://192.168.20.40:8000/ -* Studio (CMS): http://192.168.20.40:8001/ +- LMS: http://192.168.20.40:8000/ +- CMS: http://192.168.20.40:8001/ -You can develop by editing the files directly in the `edx-platform/` directory you -downloaded before, you don't need to connect to the VM to edit them (the VM uses -those files to run edX, mirroring the folder in `/opt/edx/edx-platform`). +Your VM's port 8000 is forwarded to host port 9000 +so you can also access the LMS with [http://localhost:9000/](). +Similarly, VM port 8001 is forwarded to host port 9001. +These are set in your `Vagrantfile`. -You may also want to create a super-user with: - -``` -$ rake django-admin["createsuperuser"] -``` - -Also note that if you register a new user through the web interface, -the activiation email will be posted to your VM's terminal window (search for -lines similar to): +Note that when you register a new user through the web interface, +by default the activiation email will be appear on your VM's terminal. +Search for lines similar to: ``` Subject: Your account for edX Studio From: registration@edx.org ``` -and find the activation URL for the account you've created. +and find the activation URL. See the [Frequently Asked Questions](https://github.com/edx/edx-platform/wiki/Frequently-Asked-Questions) for more usage tips. +Django admin & debug toolbar +----------------------------- + +You can enable admin logins and the debug_toolbar by editing +`lms/envs/common.py`: + +- enable ADMIN login page by setting: + - ``` + 'ENABLE_DJANGO_ADMIN_SITE': True +``` + + +- enable debug toolbar by uncommenting: + - ``` + # 'debug_toolbar.middleware.DebugToolbarMiddleware', +``` + +These are also defined in `lms/envs/dev.py`, +and usually active on localhost. + +To get at your VM's 127.0.0.1, explicitly forward one of VM's available localhost ports to your computer. +Instead of `vagrant ssh`, login with: + +``` +$ ssh -L 6080:127.0.0.1:8080 vagrant@192.168.20.40 +``` + +The password is _vagrant_. + +From your VM, start the LMS as a localhost instance: + +``` +$ rake lms[cms.dev,127.0.0.1:8080] +``` + +You should see the debug toolbar now on [http:/localhost:6080/](). +You should now also see a login on [http://localhost:6080/admin/]() +You will need a privileged user for the admin login. +You can create a CMS/LMS super-user with: +``` +$ ./manage.py lms createsuperuser +``` + + Stopping & starting ------------------- -To stop the VM (from your `edx-platform/` directory): +To stop the VM (from your `edx-platform/` directory): ``` $ vagrant halt ``` @@ -112,16 +165,27 @@ To restart: $ vagrant up ``` -or, to start without attempting to update the dependencies: +To suspend and resume tasks in progress on your VM: +``` +$ vagrant suspend +$ # and later... +$ vagrant resume +``` +Your development environment is normally created once, on first `vagrant up`. +You can continue to fetch changes in edx-platform +as you work with your VM. +To re-create your VM and create a fresh development environment: ``` -$ vagrant up --no-provision +$ vagrant destroy +$ vagrant up # will make a new VM ``` + Troubleshooting --------------- -If anything doesn't work as expected, see the +If anything doesn't work as expected, see the [troubleshooting section](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#troubleshooting). Installation - Advanced @@ -229,24 +293,12 @@ or any other process management tool. Configuring Your Project ------------------------ -We use [`rake`](http://rake.rubyforge.org/) to execute common tasks in our -project. The `rake` tasks are defined in the `rakefile`, or you can run `rake -T` -to view a summary. - Before you run your project, you need to create a sqlite database, create -tables in that database, run database migrations, and populate templates for -CMS templates. Fortunately, `rake` will do all of this for you! Just run: +tables in that database, and run database migrations. Fortunately, `django` +will do all of this for you - $ rake django-admin[syncdb] - $ rake django-admin[migrate] - $ rake cms:update_templates - -If you are running these commands using the [`zsh`](http://www.zsh.org/) shell, -zsh will assume that you are doing -[shell globbing](https://en.wikipedia.org/wiki/Glob_%28programming%29), search for -a file in your directory named `django-adminsyncdb` or `django-adminmigrate`, -and fail. To fix this, just surround the argument with quotation marks, so that -you're running `rake "django-admin[syncdb]"`. + $ ./manage.py lms syncdb --migrate + $ ./manage.py cms syncdb --migrate Run Your Project ---------------- @@ -254,6 +306,10 @@ edX has two components: Studio, the course authoring system; and the LMS (learning management system) used by students. These two systems communicate through the MongoDB database, which stores course information. +We use [`rake`](http://rake.rubyforge.org/) to execute common tasks in our +project. The `rake` tasks are defined in the `rakefile`, or you can run `rake -T` +to view a summary. + To run Studio, run: $ rake cms diff --git a/cms/djangoapps/contentstore/course_info_model.py b/cms/djangoapps/contentstore/course_info_model.py index ada3873992..7e1e6470ff 100644 --- a/cms/djangoapps/contentstore/course_info_model.py +++ b/cms/djangoapps/contentstore/course_info_model.py @@ -20,8 +20,8 @@ def get_course_updates(location): try: course_updates = modulestore('direct').get_item(location) except ItemNotFoundError: - template = Location(['i4x', 'edx', "templates", 'course_info', "Empty"]) - course_updates = modulestore('direct').clone_item(template, Location(location)) + modulestore('direct').create_and_save_xmodule(location) + course_updates = modulestore('direct').get_item(location) # current db rep: {"_id" : locationjson, "definition" : { "data" : "
    [

  1. date

    content
    2. ]
"} "metadata" : ignored} location_base = course_updates.location.url() diff --git a/cms/djangoapps/contentstore/features/checklists.feature b/cms/djangoapps/contentstore/features/checklists.feature index 3767144c99..10db23c4fa 100644 --- a/cms/djangoapps/contentstore/features/checklists.feature +++ b/cms/djangoapps/contentstore/features/checklists.feature @@ -8,7 +8,7 @@ Feature: Course checklists Scenario: A course author can mark tasks as complete Given I have opened Checklists Then I can check and uncheck tasks in a checklist - And They are correctly selected after I reload the page + And They are correctly selected after reloading the page Scenario: A task can link to a location within Studio Given I have opened Checklists diff --git a/cms/djangoapps/contentstore/features/checklists.py b/cms/djangoapps/contentstore/features/checklists.py index 13d3ca99b7..e8dcd755a3 100644 --- a/cms/djangoapps/contentstore/features/checklists.py +++ b/cms/djangoapps/contentstore/features/checklists.py @@ -45,7 +45,7 @@ def i_can_check_and_uncheck_tasks(step): verifyChecklist2Status(2, 7, 29) -@step('They are correctly selected after I reload the page$') +@step('They are correctly selected after reloading the page$') def tasks_correctly_selected_after_reload(step): reload_the_page(step) verifyChecklist2Status(2, 7, 29) diff --git a/cms/djangoapps/contentstore/features/common.py b/cms/djangoapps/contentstore/features/common.py index cb24af47e0..d357c8ae96 100644 --- a/cms/djangoapps/contentstore/features/common.py +++ b/cms/djangoapps/contentstore/features/common.py @@ -208,8 +208,9 @@ def set_date_and_time(date_css, desired_date, time_css, desired_time): def i_created_a_video_component(step): world.create_component_instance( step, '.large-video-icon', - 'i4x://edx/templates/video/default', - '.xmodule_VideoModule' + 'video', + '.xmodule_VideoModule', + has_multiple_templates=False ) @@ -238,6 +239,17 @@ def save_button_disabled(step): assert world.css_has_class(button_css, disabled) +@step('I confirm the prompt') +def confirm_the_prompt(step): + prompt_css = 'a.button.action-primary' + world.css_click(prompt_css) + + +@step(u'I am shown a (.*)$') +def i_am_shown_a_notification(step, notification_type): + assert world.is_css_present('.wrapper-%s' % notification_type) + + def type_in_codemirror(index, text): world.css_click(".CodeMirror", index=index) g = world.css_find("div.CodeMirror.CodeMirror-focused > div > textarea") diff --git a/cms/djangoapps/contentstore/features/component.feature b/cms/djangoapps/contentstore/features/component.feature index 2291712f2d..a30ce96ae6 100644 --- a/cms/djangoapps/contentstore/features/component.feature +++ b/cms/djangoapps/contentstore/features/component.feature @@ -67,3 +67,21 @@ Feature: Component Adding When I will confirm all alerts And I delete all components Then I see no components + + Scenario: I see a prompt on delete + Given I have opened a new course in studio + And I am editing a new unit + And I add the following components: + | Component | + | Discussion | + And I delete a component + Then I am shown a prompt + + Scenario: I see a notification on save + Given I have opened a new course in studio + And I am editing a new unit + And I add the following components: + | Component | + | Discussion | + And I edit and save a component + Then I am shown a notification diff --git a/cms/djangoapps/contentstore/features/component.py b/cms/djangoapps/contentstore/features/component.py index 217ad84591..15727dd992 100644 --- a/cms/djangoapps/contentstore/features/component.py +++ b/cms/djangoapps/contentstore/features/component.py @@ -41,6 +41,17 @@ def see_no_components(steps): assert world.is_css_not_present('li.component') +@step(u'I delete a component') +def delete_one_component(step): + world.css_click('a.delete-button') + + +@step(u'I edit and save a component') +def edit_and_save_component(step): + world.css_click('.edit-button') + world.css_click('.save-button') + + def step_selector_list(data_type, path, index=1): selector_list = ['a[data-type="{}"]'.format(data_type)] if index != 1: diff --git a/cms/djangoapps/contentstore/features/component_settings_editor_helpers.py b/cms/djangoapps/contentstore/features/component_settings_editor_helpers.py index 43164f62be..2b206e4466 100644 --- a/cms/djangoapps/contentstore/features/component_settings_editor_helpers.py +++ b/cms/djangoapps/contentstore/features/component_settings_editor_helpers.py @@ -7,10 +7,16 @@ from terrain.steps import reload_the_page @world.absorb -def create_component_instance(step, component_button_css, instance_id, expected_css): - click_new_component_button(step, component_button_css) - click_component_from_menu(instance_id, expected_css) +def create_component_instance(step, component_button_css, category, + expected_css, boilerplate=None, + has_multiple_templates=True): + click_new_component_button(step, component_button_css) + + if has_multiple_templates: + click_component_from_menu(category, boilerplate, expected_css) + + assert_equal(1, len(world.css_find(expected_css))) @world.absorb def click_new_component_button(step, component_button_css): @@ -19,7 +25,7 @@ def click_new_component_button(step, component_button_css): @world.absorb -def click_component_from_menu(instance_id, expected_css): +def click_component_from_menu(category, boilerplate, expected_css): """ Creates a component from `instance_id`. For components with more than one template, clicks on `elem_css` to create the new @@ -27,12 +33,13 @@ def click_component_from_menu(instance_id, expected_css): as the user clicks the appropriate button, so we assert that the expected component is present. """ - elem_css = "a[data-location='%s']" % instance_id + if boilerplate: + elem_css = "a[data-category='{}'][data-boilerplate='{}']".format(category, boilerplate) + else: + elem_css = "a[data-category='{}']:not([data-boilerplate])".format(category) elements = world.css_find(elem_css) - assert(len(elements) == 1) - if elements[0]['id'] == instance_id: # If this is a component with multiple templates - world.css_click(elem_css) - assert_equal(1, len(world.css_find(expected_css))) + assert_equal(len(elements), 1) + world.css_click(elem_css) @world.absorb diff --git a/cms/djangoapps/contentstore/features/studio-overview-togglesection.feature b/cms/djangoapps/contentstore/features/course-overview.feature similarity index 86% rename from cms/djangoapps/contentstore/features/studio-overview-togglesection.feature rename to cms/djangoapps/contentstore/features/course-overview.feature index e746f3629a..b3041b9b18 100644 --- a/cms/djangoapps/contentstore/features/studio-overview-togglesection.feature +++ b/cms/djangoapps/contentstore/features/course-overview.feature @@ -1,7 +1,7 @@ -Feature: Overview Toggle Section - In order to quickly view the details of a course's section or to scan the inventory of sections +Feature: Course Overview + In order to quickly view the details of a course's section and set release dates and grading As a course author - I want to toggle the visibility of each section's subsection details in the overview listing + I want to use the course overview page Scenario: The default layout for the overview page is to show sections in expanded view Given I have a course with multiple sections @@ -57,3 +57,9 @@ Feature: Overview Toggle Section And I click the "Expand All Sections" link Then I see the "Collapse All Sections" link And all sections are expanded + + Scenario: Notification is shown on grading status changes + Given I have a course with 1 section + When I navigate to the course overview page + And I change an assignment's grading status + Then I am shown a notification diff --git a/cms/djangoapps/contentstore/features/studio-overview-togglesection.py b/cms/djangoapps/contentstore/features/course-overview.py similarity index 93% rename from cms/djangoapps/contentstore/features/studio-overview-togglesection.py rename to cms/djangoapps/contentstore/features/course-overview.py index 9ab17fbdac..10fa6453b2 100644 --- a/cms/djangoapps/contentstore/features/studio-overview-togglesection.py +++ b/cms/djangoapps/contentstore/features/course-overview.py @@ -22,7 +22,7 @@ def have_a_course_with_1_section(step): section = world.ItemFactory.create(parent_location=course.location) subsection1 = world.ItemFactory.create( parent_location=section.location, - template='i4x://edx/templates/sequential/Empty', + category='sequential', display_name='Subsection One',) @@ -33,18 +33,18 @@ def have_a_course_with_two_sections(step): section = world.ItemFactory.create(parent_location=course.location) subsection1 = world.ItemFactory.create( parent_location=section.location, - template='i4x://edx/templates/sequential/Empty', + category='sequential', display_name='Subsection One',) section2 = world.ItemFactory.create( parent_location=course.location, display_name='Section Two',) subsection2 = world.ItemFactory.create( parent_location=section2.location, - template='i4x://edx/templates/sequential/Empty', + category='sequential', display_name='Subsection Alpha',) subsection3 = world.ItemFactory.create( parent_location=section2.location, - template='i4x://edx/templates/sequential/Empty', + category='sequential', display_name='Subsection Beta',) @@ -118,3 +118,9 @@ def all_sections_are_collapsed(step): subsections = world.css_find(subsection_locator) for index in range(len(subsections)): assert_false(world.css_visible(subsection_locator, index=index)) + + +@step(u"I change an assignment's grading status") +def change_grading_status(step): + world.css_find('a.menu-toggle').click() + world.css_find('.menu li').first.click() diff --git a/cms/djangoapps/contentstore/features/discussion-editor.py b/cms/djangoapps/contentstore/features/discussion-editor.py index ae3da3c458..13927a7d89 100644 --- a/cms/djangoapps/contentstore/features/discussion-editor.py +++ b/cms/djangoapps/contentstore/features/discussion-editor.py @@ -8,8 +8,9 @@ from lettuce import world, step def i_created_discussion_tag(step): world.create_component_instance( step, '.large-discussion-icon', - 'i4x://edx/templates/discussion/Discussion_Tag', - '.xmodule_DiscussionModule' + 'discussion', + '.xmodule_DiscussionModule', + has_multiple_templates=False ) @@ -17,14 +18,14 @@ def i_created_discussion_tag(step): def i_see_only_the_settings_and_values(step): world.verify_all_setting_entries( [ - ['Category', "Week 1", True], - ['Display Name', "Discussion Tag", True], - ['Subcategory', "Topic-Level Student-Visible Label", True] + ['Category', "Week 1", False], + ['Display Name', "Discussion Tag", False], + ['Subcategory', "Topic-Level Student-Visible Label", False] ]) @step('creating a discussion takes a single click') def discussion_takes_a_single_click(step): assert(not world.is_css_present('.xmodule_DiscussionModule')) - world.css_click("a[data-location='i4x://edx/templates/discussion/Discussion_Tag']") + world.css_click("a[data-category='discussion']") assert(world.is_css_present('.xmodule_DiscussionModule')) diff --git a/cms/djangoapps/contentstore/features/html-editor.py b/cms/djangoapps/contentstore/features/html-editor.py index 054c0ea642..b03388c89a 100644 --- a/cms/djangoapps/contentstore/features/html-editor.py +++ b/cms/djangoapps/contentstore/features/html-editor.py @@ -7,11 +7,11 @@ from lettuce import world, step @step('I have created a Blank HTML Page$') def i_created_blank_html_page(step): world.create_component_instance( - step, '.large-html-icon', 'i4x://edx/templates/html/Blank_HTML_Page', + step, '.large-html-icon', 'html', '.xmodule_HtmlModule' ) @step('I see only the HTML display name setting$') def i_see_only_the_html_display_name(step): - world.verify_all_setting_entries([['Display Name', "Blank HTML Page", True]]) + world.verify_all_setting_entries([['Display Name', "Blank HTML Page", False]]) diff --git a/cms/djangoapps/contentstore/features/problem-editor.py b/cms/djangoapps/contentstore/features/problem-editor.py index 5d12b23d90..565a35f802 100644 --- a/cms/djangoapps/contentstore/features/problem-editor.py +++ b/cms/djangoapps/contentstore/features/problem-editor.py @@ -18,8 +18,9 @@ def i_created_blank_common_problem(step): world.create_component_instance( step, '.large-problem-icon', - 'i4x://edx/templates/problem/Blank_Common_Problem', - '.xmodule_CapaModule' + 'problem', + '.xmodule_CapaModule', + 'blank_common.yaml' ) @@ -35,8 +36,8 @@ def i_see_five_settings_with_values(step): [DISPLAY_NAME, "Blank Common Problem", True], [MAXIMUM_ATTEMPTS, "", False], [PROBLEM_WEIGHT, "", False], - [RANDOMIZATION, "Never", True], - [SHOW_ANSWER, "Finished", True] + [RANDOMIZATION, "Never", False], + [SHOW_ANSWER, "Finished", False] ]) @@ -94,7 +95,7 @@ def my_change_to_randomization_is_persisted(step): def i_can_revert_to_default_for_randomization(step): world.revert_setting_entry(RANDOMIZATION) world.save_component_and_reopen(step) - world.verify_setting_entry(world.get_setting_entry(RANDOMIZATION), RANDOMIZATION, "Always", False) + world.verify_setting_entry(world.get_setting_entry(RANDOMIZATION), RANDOMIZATION, "Never", False) @step('I can set the weight to "(.*)"?') @@ -156,7 +157,7 @@ def create_latex_problem(step): world.click_new_component_button(step, '.large-problem-icon') # Go to advanced tab. world.css_click('#ui-id-2') - world.click_component_from_menu("i4x://edx/templates/problem/Problem_Written_in_LaTeX", '.xmodule_CapaModule') + world.click_component_from_menu("problem", "latex_problem.yaml", '.xmodule_CapaModule') @step('I edit and compile the High Level Source') @@ -169,7 +170,8 @@ def edit_latex_source(step): @step('my change to the High Level Source is persisted') def high_level_source_persisted(step): def verify_text(driver): - return world.css_text('.problem') == 'hi' + css_sel = '.problem div>span' + return world.css_text(css_sel) == 'hi' world.wait_for(verify_text) @@ -203,7 +205,7 @@ def verify_modified_display_name_with_special_chars(): def verify_unset_display_name(): - world.verify_setting_entry(world.get_setting_entry(DISPLAY_NAME), DISPLAY_NAME, '', False) + world.verify_setting_entry(world.get_setting_entry(DISPLAY_NAME), DISPLAY_NAME, 'Blank Advanced Problem', False) def set_weight(weight): diff --git a/cms/djangoapps/contentstore/features/section.feature b/cms/djangoapps/contentstore/features/section.feature index 84a9bb991d..a08b490c6d 100644 --- a/cms/djangoapps/contentstore/features/section.feature +++ b/cms/djangoapps/contentstore/features/section.feature @@ -33,4 +33,5 @@ Feature: Create Section And I have added a new section When I will confirm all alerts And I press the "section" delete icon + And I confirm the prompt Then the section does not exist diff --git a/cms/djangoapps/contentstore/features/subsection.feature b/cms/djangoapps/contentstore/features/subsection.feature index a11467e3f9..9f5793dbe7 100644 --- a/cms/djangoapps/contentstore/features/subsection.feature +++ b/cms/djangoapps/contentstore/features/subsection.feature @@ -38,4 +38,5 @@ Feature: Create Subsection And I see my subsection on the Courseware page When I will confirm all alerts And I press the "subsection" delete icon + And I confirm the prompt Then the subsection does not exist diff --git a/cms/djangoapps/contentstore/features/video-editor.py b/cms/djangoapps/contentstore/features/video-editor.py index a6865fdd6d..e0f76b30ad 100644 --- a/cms/djangoapps/contentstore/features/video-editor.py +++ b/cms/djangoapps/contentstore/features/video-editor.py @@ -7,7 +7,7 @@ from lettuce import world, step @step('I see the correct settings and default values$') def i_see_the_correct_settings_and_values(step): world.verify_all_setting_entries([['Default Speed', 'OEoXaMPEzfM', False], - ['Display Name', 'default', True], + ['Display Name', 'Video Title', False], ['Download Track', '', False], ['Download Video', '', False], ['Show Captions', 'True', False], diff --git a/cms/djangoapps/contentstore/features/video.py b/cms/djangoapps/contentstore/features/video.py index cb59193f17..a6a362befc 100644 --- a/cms/djangoapps/contentstore/features/video.py +++ b/cms/djangoapps/contentstore/features/video.py @@ -14,7 +14,7 @@ def does_not_autoplay(_step): @step('creating a video takes a single click') def video_takes_a_single_click(_step): assert(not world.is_css_present('.xmodule_VideoModule')) - world.css_click("a[data-location='i4x://edx/templates/video/default']") + world.css_click("a[data-category='video']") assert(world.is_css_present('.xmodule_VideoModule')) diff --git a/cms/djangoapps/contentstore/management/commands/dump_course_structure.py b/cms/djangoapps/contentstore/management/commands/dump_course_structure.py new file mode 100644 index 0000000000..d9b7c55cbd --- /dev/null +++ b/cms/djangoapps/contentstore/management/commands/dump_course_structure.py @@ -0,0 +1,55 @@ +from django.core.management.base import BaseCommand, CommandError +from xmodule.course_module import CourseDescriptor +from xmodule.modulestore.django import modulestore +from json import dumps +from xmodule.modulestore.inheritance import own_metadata +from django.conf import settings + +filter_list = ['xml_attributes', 'checklists'] + + +class Command(BaseCommand): + help = '''Write out to stdout a structural and metadata information about a course in a flat dictionary serialized + in a JSON format. This can be used for analytics.''' + + def handle(self, *args, **options): + if len(args) < 2 or len(args) > 3: + raise CommandError("dump_course_structure requires two or more arguments: ||") + + course_id = args[0] + outfile = args[1] + + # use a user-specified database name, if present + # this is useful for doing dumps from databases restored from prod backups + if len(args) == 3: + settings.MODULESTORE['direct']['OPTIONS']['db'] = args[2] + + loc = CourseDescriptor.id_to_location(course_id) + + store = modulestore() + + course = None + try: + course = store.get_item(loc, depth=4) + except: + print 'Could not find course at {0}'.format(course_id) + return + + info = {} + + def dump_into_dict(module, info): + filtered_metadata = dict((key, value) for key, value in own_metadata(module).iteritems() + if key not in filter_list) + info[module.location.url()] = { + 'category': module.location.category, + 'children': module.children if hasattr(module, 'children') else [], + 'metadata': filtered_metadata + } + + for child in module.get_children(): + dump_into_dict(child, info) + + dump_into_dict(course, info) + + with open(outfile, 'w') as f: + f.write(dumps(info)) diff --git a/cms/djangoapps/contentstore/management/commands/update_templates.py b/cms/djangoapps/contentstore/management/commands/update_templates.py deleted file mode 100644 index 36348314b9..0000000000 --- a/cms/djangoapps/contentstore/management/commands/update_templates.py +++ /dev/null @@ -1,10 +0,0 @@ -from xmodule.templates import update_templates -from xmodule.modulestore.django import modulestore -from django.core.management.base import BaseCommand - - -class Command(BaseCommand): - help = 'Imports and updates the Studio component templates from the code pack and put in the DB' - - def handle(self, *args, **options): - update_templates(modulestore('direct')) diff --git a/cms/djangoapps/contentstore/module_info_model.py b/cms/djangoapps/contentstore/module_info_model.py index 726d4bb0ce..bce4b0326c 100644 --- a/cms/djangoapps/contentstore/module_info_model.py +++ b/cms/djangoapps/contentstore/module_info_model.py @@ -3,13 +3,13 @@ from xmodule.modulestore.exceptions import ItemNotFoundError from xmodule.modulestore import Location -def get_module_info(store, location, parent_location=None, rewrite_static_links=False): +def get_module_info(store, location, rewrite_static_links=False): try: module = store.get_item(location) except ItemNotFoundError: # create a new one - template_location = Location(['i4x', 'edx', 'templates', location.category, 'Empty']) - module = store.clone_item(template_location, location) + store.create_and_save_xmodule(location) + module = store.get_item(location) data = module.data if rewrite_static_links: @@ -29,7 +29,8 @@ def get_module_info(store, location, parent_location=None, rewrite_static_links= 'id': module.location.url(), 'data': data, # TODO (cpennington): This really shouldn't have to do this much reaching in to get the metadata - 'metadata': module._model_data._kvs._metadata + # what's the intent here? all metadata incl inherited & namespaced? + 'metadata': module.xblock_kvs._metadata } @@ -37,14 +38,11 @@ def set_module_info(store, location, post_data): module = None try: module = store.get_item(location) - except: - pass - - if module is None: - # new module at this location - # presume that we have an 'Empty' template - template_location = Location(['i4x', 'edx', 'templates', location.category, 'Empty']) - module = store.clone_item(template_location, location) + except ItemNotFoundError: + # new module at this location: almost always used for the course about pages; thus, no parent. (there + # are quite a handful of about page types available for a course and only the overview is pre-created) + store.create_and_save_xmodule(location) + module = store.get_item(location) if post_data.get('data') is not None: data = post_data['data'] @@ -79,4 +77,4 @@ def set_module_info(store, location, post_data): # commit to datastore # TODO (cpennington): This really shouldn't have to do this much reaching in to get the metadata - store.update_metadata(location, module._model_data._kvs._metadata) + store.update_metadata(location, module.xblock_kvs._metadata) diff --git a/cms/djangoapps/contentstore/tests/test_checklists.py b/cms/djangoapps/contentstore/tests/test_checklists.py index 99ffb8678d..02999f6567 100644 --- a/cms/djangoapps/contentstore/tests/test_checklists.py +++ b/cms/djangoapps/contentstore/tests/test_checklists.py @@ -46,6 +46,8 @@ class ChecklistTestCase(CourseTestCase): # Now delete the checklists from the course and verify they get repopulated (for courses # created before checklists were introduced). self.course.checklists = None + # Save the changed `checklists` to the underlying KeyValueStore before updating the modulestore + self.course.save() modulestore = get_modulestore(self.course.location) modulestore.update_metadata(self.course.location, own_metadata(self.course)) self.assertEqual(self.get_persisted_checklists(), None) diff --git a/cms/djangoapps/contentstore/tests/test_contentstore.py b/cms/djangoapps/contentstore/tests/test_contentstore.py index be122fa1a4..a51110163d 100644 --- a/cms/djangoapps/contentstore/tests/test_contentstore.py +++ b/cms/djangoapps/contentstore/tests/test_contentstore.py @@ -24,12 +24,11 @@ from auth.authz import add_user_to_creator_group from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase from xmodule.modulestore.tests.factories import CourseFactory, ItemFactory -from xmodule.modulestore import Location +from xmodule.modulestore import Location, mongo from xmodule.modulestore.store_utilities import clone_course from xmodule.modulestore.store_utilities import delete_course from xmodule.modulestore.django import modulestore from xmodule.contentstore.django import contentstore, _CONTENTSTORE -from xmodule.templates import update_templates from xmodule.modulestore.xml_exporter import export_to_xml from xmodule.modulestore.xml_importer import import_from_xml, perform_xlint from xmodule.modulestore.inheritance import own_metadata @@ -88,6 +87,8 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): self.user.is_active = True # Staff has access to view all courses self.user.is_staff = True + + # Save the data that we've just changed to the db. self.user.save() self.client = Client() @@ -118,6 +119,10 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): course.advanced_modules = component_types + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + course.save() + store.update_metadata(course.location, own_metadata(course)) # just pick one vertical @@ -135,7 +140,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): self.check_components_on_page(ADVANCED_COMPONENT_TYPES, ['Video Alpha', 'Word cloud', 'Annotation', - 'Open Ended Response', + 'Open Response Assessment', 'Peer Grading Interface']) def test_advanced_components_require_two_clicks(self): @@ -183,7 +188,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): html_module = draft_store.get_item(['i4x', 'edX', 'simple', 'html', 'test_html', None]) - draft_store.clone_item(html_module.location, html_module.location) + draft_store.convert_to_draft(html_module.location) # now query get_items() to get this location with revision=None, this should just # return back a single item (not 2) @@ -215,7 +220,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): self.assertEqual(html_module.lms.graceperiod, course.lms.graceperiod) self.assertNotIn('graceperiod', own_metadata(html_module)) - draft_store.clone_item(html_module.location, html_module.location) + draft_store.convert_to_draft(html_module.location) # refetch to check metadata html_module = draft_store.get_item(['i4x', 'edX', 'simple', 'html', 'test_html', None]) @@ -233,13 +238,16 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): self.assertNotIn('graceperiod', own_metadata(html_module)) # put back in draft and change metadata and see if it's now marked as 'own_metadata' - draft_store.clone_item(html_module.location, html_module.location) + draft_store.convert_to_draft(html_module.location) html_module = draft_store.get_item(['i4x', 'edX', 'simple', 'html', 'test_html', None]) new_graceperiod = timedelta(hours=1) self.assertNotIn('graceperiod', own_metadata(html_module)) html_module.lms.graceperiod = new_graceperiod + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + html_module.save() self.assertIn('graceperiod', own_metadata(html_module)) self.assertEqual(html_module.lms.graceperiod, new_graceperiod) @@ -255,7 +263,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): draft_store.publish(html_module.location, 0) # and re-read and verify 'own-metadata' - draft_store.clone_item(html_module.location, html_module.location) + draft_store.convert_to_draft(html_module.location) html_module = draft_store.get_item(['i4x', 'edX', 'simple', 'html', 'test_html', None]) self.assertIn('graceperiod', own_metadata(html_module)) @@ -278,7 +286,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): ) # put into draft - modulestore('draft').clone_item(problem.location, problem.location) + modulestore('draft').convert_to_draft(problem.location) # make sure we can query that item and verify that it is a draft draft_problem = modulestore('draft').get_item( @@ -304,17 +312,36 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): self.assertGreater(len(course.textbooks), 0) + def test_default_tabs_on_create_course(self): + module_store = modulestore('direct') + CourseFactory.create(org='edX', course='999', display_name='Robot Super Course') + course_location = Location(['i4x', 'edX', '999', 'course', 'Robot_Super_Course', None]) + + course = module_store.get_item(course_location) + + expected_tabs = [] + expected_tabs.append({u'type': u'courseware'}) + expected_tabs.append({u'type': u'course_info', u'name': u'Course Info'}) + expected_tabs.append({u'type': u'textbooks'}) + expected_tabs.append({u'type': u'discussion', u'name': u'Discussion'}) + expected_tabs.append({u'type': u'wiki', u'name': u'Wiki'}) + expected_tabs.append({u'type': u'progress', u'name': u'Progress'}) + + self.assertEqual(course.tabs, expected_tabs) + def test_static_tab_reordering(self): module_store = modulestore('direct') CourseFactory.create(org='edX', course='999', display_name='Robot Super Course') course_location = Location(['i4x', 'edX', '999', 'course', 'Robot_Super_Course', None]) - ItemFactory.create(parent_location=course_location, - template="i4x://edx/templates/static_tab/Empty", - display_name="Static_1") - ItemFactory.create(parent_location=course_location, - template="i4x://edx/templates/static_tab/Empty", - display_name="Static_2") + ItemFactory.create( + parent_location=course_location, + category="static_tab", + display_name="Static_1") + ItemFactory.create( + parent_location=course_location, + category="static_tab", + display_name="Static_2") course = module_store.get_item(Location(['i4x', 'edX', '999', 'course', 'Robot_Super_Course', None])) @@ -371,7 +398,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): course_location = Location(['i4x', 'edX', '999', 'course', 'Robot_Super_Course', None]) chapterloc = ItemFactory.create(parent_location=course_location, display_name="Chapter").location - ItemFactory.create(parent_location=chapterloc, template='i4x://edx/templates/sequential/Empty', display_name="Sequential") + ItemFactory.create(parent_location=chapterloc, category='sequential', display_name="Sequential") sequential = direct_store.get_item(Location(['i4x', 'edX', '999', 'sequential', 'Sequential', None])) chapter = direct_store.get_item(Location(['i4x', 'edX', '999', 'chapter', 'Chapter', None])) @@ -574,7 +601,6 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): def test_clone_course(self): course_data = { - 'template': 'i4x://edx/templates/course/Empty', 'org': 'MITx', 'number': '999', 'display_name': 'Robot Super Course', @@ -614,10 +640,10 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): CourseFactory.create(org='MITx', course='999', display_name='Robot Super Course') location = Location('i4x://MITx/999/chapter/neuvo') - self.assertRaises(InvalidVersionError, draft_store.clone_item, 'i4x://edx/templates/chapter/Empty', - location) - direct_store.clone_item('i4x://edx/templates/chapter/Empty', location) - self.assertRaises(InvalidVersionError, draft_store.clone_item, location, location) + # Ensure draft mongo store does not allow us to create chapters either directly or via convert to draft + self.assertRaises(InvalidVersionError, draft_store.create_and_save_xmodule, location) + direct_store.create_and_save_xmodule(location) + self.assertRaises(InvalidVersionError, draft_store.convert_to_draft, location) self.assertRaises(InvalidVersionError, draft_store.update_item, location, 'chapter data') @@ -652,9 +678,9 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): vertical = module_store.get_item(Location(['i4x', 'edX', 'toy', 'vertical', 'vertical_test', None]), depth=1) - draft_store.clone_item(vertical.location, vertical.location) + draft_store.convert_to_draft(vertical.location) for child in vertical.get_children(): - draft_store.clone_item(child.location, child.location) + draft_store.convert_to_draft(child.location) # delete the course delete_course(module_store, content_store, location, commit=True) @@ -687,26 +713,35 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): import_from_xml(module_store, 'common/test/data/', ['toy']) location = CourseDescriptor.id_to_location('edX/toy/2012_Fall') - # get a vertical (and components in it) to put into 'draft' - vertical = module_store.get_item(Location(['i4x', 'edX', 'toy', - 'vertical', 'vertical_test', None]), depth=1) - - draft_store.clone_item(vertical.location, vertical.location) - + # get a vertical (and components in it) to copy into an orphan sub dag + vertical = module_store.get_item( + Location(['i4x', 'edX', 'toy', 'vertical', 'vertical_test', None]), + depth=1 + ) # We had a bug where orphaned draft nodes caused export to fail. This is here to cover that case. - draft_store.clone_item(vertical.location, Location(['i4x', 'edX', 'toy', - 'vertical', 'no_references', 'draft'])) + vertical.location = mongo.draft.as_draft(vertical.location.replace(name='no_references')) + draft_store.save_xmodule(vertical) + orphan_vertical = draft_store.get_item(vertical.location) + self.assertEqual(orphan_vertical.location.name, 'no_references') + # get the original vertical (and components in it) to put into 'draft' + vertical = module_store.get_item( + Location(['i4x', 'edX', 'toy', 'vertical', 'vertical_test', None]), + depth=1) + self.assertEqual(len(orphan_vertical.children), len(vertical.children)) + draft_store.convert_to_draft(vertical.location) for child in vertical.get_children(): - draft_store.clone_item(child.location, child.location) + draft_store.convert_to_draft(child.location) root_dir = path(mkdtemp_clean()) - # now create a private vertical - private_vertical = draft_store.clone_item(vertical.location, - Location(['i4x', 'edX', 'toy', 'vertical', 'a_private_vertical', None])) + # now create a new/different private (draft only) vertical + vertical.location = mongo.draft.as_draft(Location(['i4x', 'edX', 'toy', 'vertical', 'a_private_vertical', None])) + draft_store.save_xmodule(vertical) + private_vertical = draft_store.get_item(vertical.location) + vertical = None # blank out b/c i destructively manipulated its location 2 lines above - # add private to list of children + # add the new private to list of children sequential = module_store.get_item(Location(['i4x', 'edX', 'toy', 'sequential', 'vertical_sequential', None])) private_location_no_draft = private_vertical.location.replace(revision=None) @@ -792,6 +827,34 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): shutil.rmtree(root_dir) + def test_export_course_with_metadata_only_video(self): + module_store = modulestore('direct') + draft_store = modulestore('draft') + content_store = contentstore() + + import_from_xml(module_store, 'common/test/data/', ['toy']) + location = CourseDescriptor.id_to_location('edX/toy/2012_Fall') + + # create a new video module and add it as a child to a vertical + # this re-creates a bug whereby since the video template doesn't have + # anything in 'data' field, the export was blowing up + verticals = module_store.get_items(['i4x', 'edX', 'toy', 'vertical', None, None]) + + self.assertGreater(len(verticals), 0) + + parent = verticals[0] + + ItemFactory.create(parent_location=parent.location, category="video", display_name="untitled") + + root_dir = path(mkdtemp_clean()) + + print 'Exporting to tempdir = {0}'.format(root_dir) + + # export out to a tempdir + export_to_xml(module_store, content_store, location, root_dir, 'test_export', draft_modulestore=draft_store) + + shutil.rmtree(root_dir) + def test_course_handouts_rewrites(self): module_store = modulestore('direct') @@ -846,6 +909,9 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): # add a bool piece of unknown metadata so we can verify we don't throw an exception metadata['new_metadata'] = True + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + course.save() module_store.update_metadata(location, metadata) print 'Exporting to tempdir = {0}'.format(root_dir) @@ -885,7 +951,6 @@ class ContentStoreTest(ModuleStoreTestCase): self.client.login(username=uname, password=password) self.course_data = { - 'template': 'i4x://edx/templates/course/Empty', 'org': 'MITx', 'number': '999', 'display_name': 'Robot Super Course', @@ -1029,17 +1094,17 @@ class ContentStoreTest(ModuleStoreTestCase): html=True ) - def test_clone_item(self): + def test_create_item(self): """Test cloning an item. E.g. creating a new section""" CourseFactory.create(org='MITx', course='999', display_name='Robot Super Course') section_data = { 'parent_location': 'i4x://MITx/999/course/Robot_Super_Course', - 'template': 'i4x://edx/templates/chapter/Empty', + 'category': 'chapter', 'display_name': 'Section One', } - resp = self.client.post(reverse('clone_item'), section_data) + resp = self.client.post(reverse('create_item'), section_data) self.assertEqual(resp.status_code, 200) data = parse_json(resp) @@ -1054,14 +1119,14 @@ class ContentStoreTest(ModuleStoreTestCase): problem_data = { 'parent_location': 'i4x://MITx/999/course/Robot_Super_Course', - 'template': 'i4x://edx/templates/problem/Blank_Common_Problem' + 'category': 'problem' } - resp = self.client.post(reverse('clone_item'), problem_data) + resp = self.client.post(reverse('create_item'), problem_data) self.assertEqual(resp.status_code, 200) payload = parse_json(resp) - problem_loc = payload['id'] + problem_loc = Location(payload['id']) problem = get_modulestore(problem_loc).get_item(problem_loc) # should be a CapaDescriptor self.assertIsInstance(problem, CapaDescriptor, "New problem is not a CapaDescriptor") @@ -1194,10 +1259,9 @@ class ContentStoreTest(ModuleStoreTestCase): CourseFactory.create(org='edX', course='999', display_name='Robot Super Course') new_component_location = Location('i4x', 'edX', '999', 'discussion', 'new_component') - source_template_location = Location('i4x', 'edx', 'templates', 'discussion', 'Discussion_Tag') # crate a new module and add it as a child to a vertical - module_store.clone_item(source_template_location, new_component_location) + module_store.create_and_save_xmodule(new_component_location) new_discussion_item = module_store.get_item(new_component_location) @@ -1218,10 +1282,9 @@ class ContentStoreTest(ModuleStoreTestCase): module_store.modulestore_update_signal.connect(_signal_hander) new_component_location = Location('i4x', 'edX', '999', 'html', 'new_component') - source_template_location = Location('i4x', 'edx', 'templates', 'html', 'Blank_HTML_Page') # crate a new module - module_store.clone_item(source_template_location, new_component_location) + module_store.create_and_save_xmodule(new_component_location) finally: module_store.modulestore_update_signal = None @@ -1239,14 +1302,14 @@ class ContentStoreTest(ModuleStoreTestCase): # let's assert on the metadata_inheritance on an existing vertical for vertical in verticals: self.assertEqual(course.lms.xqa_key, vertical.lms.xqa_key) + self.assertEqual(course.start, vertical.lms.start) self.assertGreater(len(verticals), 0) new_component_location = Location('i4x', 'edX', 'toy', 'html', 'new_component') - source_template_location = Location('i4x', 'edx', 'templates', 'html', 'Blank_HTML_Page') # crate a new module and add it as a child to a vertical - module_store.clone_item(source_template_location, new_component_location) + module_store.create_and_save_xmodule(new_component_location) parent = verticals[0] module_store.update_children(parent.location, parent.children + [new_component_location.url()]) @@ -1256,6 +1319,8 @@ class ContentStoreTest(ModuleStoreTestCase): # check for grace period definition which should be defined at the course level self.assertEqual(parent.lms.graceperiod, new_module.lms.graceperiod) + self.assertEqual(parent.lms.start, new_module.lms.start) + self.assertEqual(course.start, new_module.lms.start) self.assertEqual(course.lms.xqa_key, new_module.lms.xqa_key) @@ -1263,6 +1328,7 @@ class ContentStoreTest(ModuleStoreTestCase): # now let's define an override at the leaf node level # new_module.lms.graceperiod = timedelta(1) + new_module.save() module_store.update_metadata(new_module.location, own_metadata(new_module)) # flush the cache and refetch @@ -1271,29 +1337,25 @@ class ContentStoreTest(ModuleStoreTestCase): self.assertEqual(timedelta(1), new_module.lms.graceperiod) + def test_default_metadata_inheritance(self): + course = CourseFactory.create() + vertical = ItemFactory.create(parent_location=course.location) + course.children.append(vertical) + # in memory + self.assertIsNotNone(course.start) + self.assertEqual(course.start, vertical.lms.start) + self.assertEqual(course.textbooks, []) + self.assertIn('GRADER', course.grading_policy) + self.assertIn('GRADE_CUTOFFS', course.grading_policy) + self.assertGreaterEqual(len(course.checklists), 4) -class TemplateTestCase(ModuleStoreTestCase): - - def test_template_cleanup(self): + # by fetching module_store = modulestore('direct') - - # insert a bogus template in the store - bogus_template_location = Location('i4x', 'edx', 'templates', 'html', 'bogus') - source_template_location = Location('i4x', 'edx', 'templates', 'html', 'Blank_HTML_Page') - - module_store.clone_item(source_template_location, bogus_template_location) - - verify_create = module_store.get_item(bogus_template_location) - self.assertIsNotNone(verify_create) - - # now run cleanup - update_templates(modulestore('direct')) - - # now try to find dangling template, it should not be in DB any longer - asserted = False - try: - verify_create = module_store.get_item(bogus_template_location) - except ItemNotFoundError: - asserted = True - - self.assertTrue(asserted) + fetched_course = module_store.get_item(course.location) + fetched_item = module_store.get_item(vertical.location) + self.assertIsNotNone(fetched_course.start) + self.assertEqual(course.start, fetched_course.start) + self.assertEqual(fetched_course.start, fetched_item.lms.start) + self.assertEqual(course.textbooks, fetched_course.textbooks) + # is this test too strict? i.e., it requires the dicts to be == + self.assertEqual(course.checklists, fetched_course.checklists) diff --git a/cms/djangoapps/contentstore/tests/test_course_settings.py b/cms/djangoapps/contentstore/tests/test_course_settings.py index 21d7d69d41..0862eb462d 100644 --- a/cms/djangoapps/contentstore/tests/test_course_settings.py +++ b/cms/djangoapps/contentstore/tests/test_course_settings.py @@ -19,6 +19,7 @@ from xmodule.modulestore.tests.factories import CourseFactory from models.settings.course_metadata import CourseMetadata from xmodule.modulestore.xml_importer import import_from_xml +from xmodule.modulestore.django import modulestore from xmodule.fields import Date from .utils import CourseTestCase @@ -36,7 +37,6 @@ class CourseDetailsTestCase(CourseTestCase): self.assertIsNone(details.enrollment_start, "enrollment_start date somehow initialized " + str(details.enrollment_start)) self.assertIsNone(details.enrollment_end, "enrollment_end date somehow initialized " + str(details.enrollment_end)) self.assertIsNone(details.syllabus, "syllabus somehow initialized" + str(details.syllabus)) - self.assertEqual(details.overview, "", "overview somehow initialized" + details.overview) self.assertIsNone(details.intro_video, "intro_video somehow initialized" + str(details.intro_video)) self.assertIsNone(details.effort, "effort somehow initialized" + str(details.effort)) @@ -49,7 +49,6 @@ class CourseDetailsTestCase(CourseTestCase): self.assertIsNone(jsondetails['enrollment_start'], "enrollment_start date somehow initialized ") self.assertIsNone(jsondetails['enrollment_end'], "enrollment_end date somehow initialized ") self.assertIsNone(jsondetails['syllabus'], "syllabus somehow initialized") - self.assertEqual(jsondetails['overview'], "", "overview somehow initialized") self.assertIsNone(jsondetails['intro_video'], "intro_video somehow initialized") self.assertIsNone(jsondetails['effort'], "effort somehow initialized") @@ -291,6 +290,71 @@ class CourseGradingTest(CourseTestCase): altered_grader = CourseGradingModel.update_grader_from_json(test_grader.course_location, test_grader.graders[1]) self.assertDictEqual(test_grader.graders[1], altered_grader, "drop_count[1] + 2") + def test_update_cutoffs_from_json(self): + test_grader = CourseGradingModel.fetch(self.course.location) + CourseGradingModel.update_cutoffs_from_json(test_grader.course_location, test_grader.grade_cutoffs) + # Unlike other tests, need to actually perform a db fetch for this test since update_cutoffs_from_json + # simply returns the cutoffs you send into it, rather than returning the db contents. + altered_grader = CourseGradingModel.fetch(self.course.location) + self.assertDictEqual(test_grader.grade_cutoffs, altered_grader.grade_cutoffs, "Noop update") + + test_grader.grade_cutoffs['D'] = 0.3 + CourseGradingModel.update_cutoffs_from_json(test_grader.course_location, test_grader.grade_cutoffs) + altered_grader = CourseGradingModel.fetch(self.course.location) + self.assertDictEqual(test_grader.grade_cutoffs, altered_grader.grade_cutoffs, "cutoff add D") + + test_grader.grade_cutoffs['Pass'] = 0.75 + CourseGradingModel.update_cutoffs_from_json(test_grader.course_location, test_grader.grade_cutoffs) + altered_grader = CourseGradingModel.fetch(self.course.location) + self.assertDictEqual(test_grader.grade_cutoffs, altered_grader.grade_cutoffs, "cutoff change 'Pass'") + + def test_delete_grace_period(self): + test_grader = CourseGradingModel.fetch(self.course.location) + CourseGradingModel.update_grace_period_from_json(test_grader.course_location, test_grader.grace_period) + # update_grace_period_from_json doesn't return anything, so query the db for its contents. + altered_grader = CourseGradingModel.fetch(self.course.location) + self.assertEqual(test_grader.grace_period, altered_grader.grace_period, "Noop update") + + test_grader.grace_period = {'hours': 15, 'minutes': 5, 'seconds': 30} + CourseGradingModel.update_grace_period_from_json(test_grader.course_location, test_grader.grace_period) + altered_grader = CourseGradingModel.fetch(self.course.location) + self.assertDictEqual(test_grader.grace_period, altered_grader.grace_period, "Adding in a grace period") + + test_grader.grace_period = {'hours': 1, 'minutes': 10, 'seconds': 0} + # Now delete the grace period + CourseGradingModel.delete_grace_period(test_grader.course_location) + # update_grace_period_from_json doesn't return anything, so query the db for its contents. + altered_grader = CourseGradingModel.fetch(self.course.location) + # Once deleted, the grace period should simply be None + self.assertEqual(None, altered_grader.grace_period, "Delete grace period") + + def test_update_section_grader_type(self): + # Get the descriptor and the section_grader_type and assert they are the default values + descriptor = get_modulestore(self.course.location).get_item(self.course.location) + section_grader_type = CourseGradingModel.get_section_grader_type(self.course.location) + + self.assertEqual('Not Graded', section_grader_type['graderType']) + self.assertEqual(None, descriptor.lms.format) + self.assertEqual(False, descriptor.lms.graded) + + # Change the default grader type to Homework, which should also mark the section as graded + CourseGradingModel.update_section_grader_type(self.course.location, {'graderType': 'Homework'}) + descriptor = get_modulestore(self.course.location).get_item(self.course.location) + section_grader_type = CourseGradingModel.get_section_grader_type(self.course.location) + + self.assertEqual('Homework', section_grader_type['graderType']) + self.assertEqual('Homework', descriptor.lms.format) + self.assertEqual(True, descriptor.lms.graded) + + # Change the grader type back to Not Graded, which should also unmark the section as graded + CourseGradingModel.update_section_grader_type(self.course.location, {'graderType': 'Not Graded'}) + descriptor = get_modulestore(self.course.location).get_item(self.course.location) + section_grader_type = CourseGradingModel.get_section_grader_type(self.course.location) + + self.assertEqual('Not Graded', section_grader_type['graderType']) + self.assertEqual(None, descriptor.lms.format) + self.assertEqual(False, descriptor.lms.graded) + class CourseMetadataEditingTest(CourseTestCase): """ @@ -352,7 +416,7 @@ class CourseMetadataEditingTest(CourseTestCase): self.assertEqual(test_model['display_name'], 'Robot Super Course', "not expected value") self.assertIn('rerandomize', test_model, 'Missing rerandomize metadata field') # check for deletion effectiveness - self.assertEqual('closed', test_model['showanswer'], 'showanswer field still in') + self.assertEqual('finished', test_model['showanswer'], 'showanswer field still in') self.assertEqual(None, test_model['xqa_key'], 'xqa_key field still in') diff --git a/cms/djangoapps/contentstore/tests/test_course_updates.py b/cms/djangoapps/contentstore/tests/test_course_updates.py index 4f92806871..30114496c8 100644 --- a/cms/djangoapps/contentstore/tests/test_course_updates.py +++ b/cms/djangoapps/contentstore/tests/test_course_updates.py @@ -36,8 +36,11 @@ class CourseUpdateTest(CourseTestCase): 'provided_id': payload['id']}) content += '
div

p

' payload['content'] = content + # POST requests were coming in w/ these header values causing an error; so, repro error here resp = self.client.post(first_update_url, json.dumps(payload), - "application/json") + "application/json", + HTTP_X_HTTP_METHOD_OVERRIDE="PUT", + REQUEST_METHOD="POST") self.assertHTMLEqual(content, json.loads(resp.content)['content'], "iframe w/ div") diff --git a/cms/djangoapps/contentstore/tests/test_crud.py b/cms/djangoapps/contentstore/tests/test_crud.py new file mode 100644 index 0000000000..84643f7787 --- /dev/null +++ b/cms/djangoapps/contentstore/tests/test_crud.py @@ -0,0 +1,186 @@ +''' +Created on May 7, 2013 + +@author: dmitchell +''' +import unittest +from xmodule import templates +from xmodule.modulestore.tests import persistent_factories +from xmodule.course_module import CourseDescriptor +from xmodule.modulestore.django import modulestore +from xmodule.seq_module import SequenceDescriptor +from xmodule.x_module import XModuleDescriptor +from xmodule.capa_module import CapaDescriptor +from xmodule.modulestore.locator import CourseLocator, BlockUsageLocator +from xmodule.modulestore.exceptions import ItemNotFoundError +from xmodule.html_module import HtmlDescriptor + + +class TemplateTests(unittest.TestCase): + """ + Test finding and using the templates (boilerplates) for xblocks. + """ + + def test_get_templates(self): + found = templates.all_templates() + self.assertIsNotNone(found.get('course')) + self.assertIsNotNone(found.get('about')) + self.assertIsNotNone(found.get('html')) + self.assertIsNotNone(found.get('problem')) + self.assertEqual(len(found.get('course')), 0) + self.assertEqual(len(found.get('about')), 1) + self.assertGreaterEqual(len(found.get('html')), 2) + self.assertGreaterEqual(len(found.get('problem')), 10) + dropdown = None + for template in found['problem']: + self.assertIn('metadata', template) + self.assertIn('display_name', template['metadata']) + if template['metadata']['display_name'] == 'Dropdown': + dropdown = template + break + self.assertIsNotNone(dropdown) + self.assertIn('markdown', dropdown['metadata']) + self.assertIn('data', dropdown) + self.assertRegexpMatches(dropdown['metadata']['markdown'], r'^Dropdown.*') + self.assertRegexpMatches(dropdown['data'], r'\s*

Dropdown.*') + + def test_get_some_templates(self): + self.assertEqual(len(SequenceDescriptor.templates()), 0) + self.assertGreater(len(HtmlDescriptor.templates()), 0) + self.assertIsNone(SequenceDescriptor.get_template('doesntexist.yaml')) + self.assertIsNone(HtmlDescriptor.get_template('doesntexist.yaml')) + self.assertIsNotNone(HtmlDescriptor.get_template('announcement.yaml')) + + def test_factories(self): + test_course = persistent_factories.PersistentCourseFactory.create(org='testx', prettyid='tempcourse', + display_name='fun test course', user_id='testbot') + self.assertIsInstance(test_course, CourseDescriptor) + self.assertEqual(test_course.display_name, 'fun test course') + index_info = modulestore('split').get_course_index_info(test_course.location) + self.assertEqual(index_info['org'], 'testx') + self.assertEqual(index_info['prettyid'], 'tempcourse') + + test_chapter = persistent_factories.ItemFactory.create(display_name='chapter 1', + parent_location=test_course.location) + self.assertIsInstance(test_chapter, SequenceDescriptor) + # refetch parent which should now point to child + test_course = modulestore('split').get_course(test_chapter.location) + self.assertIn(test_chapter.location.usage_id, test_course.children) + + def test_temporary_xblocks(self): + """ + Test using load_from_json to create non persisted xblocks + """ + test_course = persistent_factories.PersistentCourseFactory.create(org='testx', prettyid='tempcourse', + display_name='fun test course', user_id='testbot') + + test_chapter = XModuleDescriptor.load_from_json({'category': 'chapter', + 'metadata': {'display_name': 'chapter n'}}, + test_course.system, parent_xblock=test_course) + self.assertIsInstance(test_chapter, SequenceDescriptor) + self.assertEqual(test_chapter.display_name, 'chapter n') + self.assertIn(test_chapter, test_course.get_children()) + + # test w/ a definition (e.g., a problem) + test_def_content = 'boo' + test_problem = XModuleDescriptor.load_from_json({'category': 'problem', + 'definition': {'data': test_def_content}}, + test_course.system, parent_xblock=test_chapter) + self.assertIsInstance(test_problem, CapaDescriptor) + self.assertEqual(test_problem.data, test_def_content) + self.assertIn(test_problem, test_chapter.get_children()) + test_problem.display_name = 'test problem' + self.assertEqual(test_problem.display_name, 'test problem') + + def test_persist_dag(self): + """ + try saving temporary xblocks + """ + test_course = persistent_factories.PersistentCourseFactory.create(org='testx', prettyid='tempcourse', + display_name='fun test course', user_id='testbot') + test_chapter = XModuleDescriptor.load_from_json({'category': 'chapter', + 'metadata': {'display_name': 'chapter n'}}, + test_course.system, parent_xblock=test_course) + test_def_content = 'boo' + test_problem = XModuleDescriptor.load_from_json({'category': 'problem', + 'definition': {'data': test_def_content}}, + test_course.system, parent_xblock=test_chapter) + # better to pass in persisted parent over the subdag so + # subdag gets the parent pointer (otherwise 2 ops, persist dag, update parent children, + # persist parent + persisted_course = modulestore('split').persist_xblock_dag(test_course, 'testbot') + self.assertEqual(len(persisted_course.children), 1) + persisted_chapter = persisted_course.get_children()[0] + self.assertEqual(persisted_chapter.category, 'chapter') + self.assertEqual(persisted_chapter.display_name, 'chapter n') + self.assertEqual(len(persisted_chapter.children), 1) + persisted_problem = persisted_chapter.get_children()[0] + self.assertEqual(persisted_problem.category, 'problem') + self.assertEqual(persisted_problem.data, test_def_content) + + def test_delete_course(self): + test_course = persistent_factories.PersistentCourseFactory.create( + org='testx', + prettyid='edu.harvard.history.doomed', + display_name='doomed test course', + user_id='testbot') + persistent_factories.ItemFactory.create(display_name='chapter 1', + parent_location=test_course.location) + + id_locator = CourseLocator(course_id=test_course.location.course_id, revision='draft') + guid_locator = CourseLocator(version_guid=test_course.location.version_guid) + # verify it can be retireved by id + self.assertIsInstance(modulestore('split').get_course(id_locator), CourseDescriptor) + # and by guid + self.assertIsInstance(modulestore('split').get_course(guid_locator), CourseDescriptor) + modulestore('split').delete_course(id_locator.course_id) + # test can no longer retrieve by id + self.assertRaises(ItemNotFoundError, modulestore('split').get_course, id_locator) + # but can by guid + self.assertIsInstance(modulestore('split').get_course(guid_locator), CourseDescriptor) + + def test_block_generations(self): + """ + Test get_block_generations + """ + test_course = persistent_factories.PersistentCourseFactory.create( + org='testx', + prettyid='edu.harvard.history.hist101', + display_name='history test course', + user_id='testbot') + chapter = persistent_factories.ItemFactory.create(display_name='chapter 1', + parent_location=test_course.location, user_id='testbot') + sub = persistent_factories.ItemFactory.create(display_name='subsection 1', + parent_location=chapter.location, user_id='testbot', category='vertical') + first_problem = persistent_factories.ItemFactory.create(display_name='problem 1', + parent_location=sub.location, user_id='testbot', category='problem', data="") + first_problem.max_attempts = 3 + updated_problem = modulestore('split').update_item(first_problem, 'testbot') + updated_loc = modulestore('split').delete_item(updated_problem.location, 'testbot') + + second_problem = persistent_factories.ItemFactory.create(display_name='problem 2', + parent_location=BlockUsageLocator(updated_loc, usage_id=sub.location.usage_id), + user_id='testbot', category='problem', data="") + + # course root only updated 2x + version_history = modulestore('split').get_block_generations(test_course.location) + self.assertEqual(version_history.locator.version_guid, test_course.location.version_guid) + self.assertEqual(len(version_history.children), 1) + self.assertEqual(version_history.children[0].children, []) + self.assertEqual(version_history.children[0].locator.version_guid, chapter.location.version_guid) + + # sub changed on add, add problem, delete problem, add problem in strict linear seq + version_history = modulestore('split').get_block_generations(sub.location) + self.assertEqual(len(version_history.children), 1) + self.assertEqual(len(version_history.children[0].children), 1) + self.assertEqual(len(version_history.children[0].children[0].children), 1) + self.assertEqual(len(version_history.children[0].children[0].children[0].children), 0) + + # first and second problem may show as same usage_id; so, need to ensure their histories are right + version_history = modulestore('split').get_block_generations(updated_problem.location) + self.assertEqual(version_history.locator.version_guid, first_problem.location.version_guid) + self.assertEqual(len(version_history.children), 1) # updated max_attempts + self.assertEqual(len(version_history.children[0].children), 0) + + version_history = modulestore('split').get_block_generations(second_problem.location) + self.assertNotEqual(version_history.locator.version_guid, first_problem.location.version_guid) diff --git a/cms/djangoapps/contentstore/tests/test_i18n.py b/cms/djangoapps/contentstore/tests/test_i18n.py index a292b7316e..88df19ec2d 100644 --- a/cms/djangoapps/contentstore/tests/test_i18n.py +++ b/cms/djangoapps/contentstore/tests/test_i18n.py @@ -35,7 +35,6 @@ class InternationalizationTest(ModuleStoreTestCase): self.user.save() self.course_data = { - 'template': 'i4x://edx/templates/course/Empty', 'org': 'MITx', 'number': '999', 'display_name': 'Robot Super Course', diff --git a/cms/djangoapps/contentstore/tests/test_item.py b/cms/djangoapps/contentstore/tests/test_item.py index 4e6c951d9b..578b82b3cf 100644 --- a/cms/djangoapps/contentstore/tests/test_item.py +++ b/cms/djangoapps/contentstore/tests/test_item.py @@ -1,6 +1,11 @@ from contentstore.tests.test_course_settings import CourseTestCase from xmodule.modulestore.tests.factories import CourseFactory from django.core.urlresolvers import reverse +from xmodule.capa_module import CapaDescriptor +import json +from xmodule.modulestore.django import modulestore +import datetime +from pytz import UTC class DeleteItem(CourseTestCase): @@ -11,14 +16,228 @@ class DeleteItem(CourseTestCase): def testDeleteStaticPage(self): # Add static tab - data = { + data = json.dumps({ 'parent_location': 'i4x://mitX/333/course/Dummy_Course', - 'template': 'i4x://edx/templates/static_tab/Empty' - } + 'category': 'static_tab' + }) - resp = self.client.post(reverse('clone_item'), data) + resp = self.client.post(reverse('create_item'), data, + content_type="application/json") self.assertEqual(resp.status_code, 200) # Now delete it. There was a bug that the delete was failing (static tabs do not exist in draft modulestore). resp = self.client.post(reverse('delete_item'), resp.content, "application/json") self.assertEqual(resp.status_code, 200) + + +class TestCreateItem(CourseTestCase): + """ + Test the create_item handler thoroughly + """ + def response_id(self, response): + """ + Get the id from the response payload + :param response: + """ + parsed = json.loads(response.content) + return parsed['id'] + + def test_create_nicely(self): + """ + Try the straightforward use cases + """ + # create a chapter + display_name = 'Nicely created' + resp = self.client.post( + reverse('create_item'), + json.dumps({ + 'parent_location': self.course.location.url(), + 'display_name': display_name, + 'category': 'chapter' + }), + content_type="application/json" + ) + self.assertEqual(resp.status_code, 200) + + # get the new item and check its category and display_name + chap_location = self.response_id(resp) + new_obj = modulestore().get_item(chap_location) + self.assertEqual(new_obj.category, 'chapter') + self.assertEqual(new_obj.display_name, display_name) + self.assertEqual(new_obj.location.org, self.course.location.org) + self.assertEqual(new_obj.location.course, self.course.location.course) + + # get the course and ensure it now points to this one + course = modulestore().get_item(self.course.location) + self.assertIn(chap_location, course.children) + + # use default display name + resp = self.client.post( + reverse('create_item'), + json.dumps({ + 'parent_location': chap_location, + 'category': 'vertical' + }), + content_type="application/json" + ) + self.assertEqual(resp.status_code, 200) + + vert_location = self.response_id(resp) + + # create problem w/ boilerplate + template_id = 'multiplechoice.yaml' + resp = self.client.post( + reverse('create_item'), + json.dumps({ + 'parent_location': vert_location, + 'category': 'problem', + 'boilerplate': template_id + }), + content_type="application/json" + ) + self.assertEqual(resp.status_code, 200) + prob_location = self.response_id(resp) + problem = modulestore('draft').get_item(prob_location) + # ensure it's draft + self.assertTrue(problem.is_draft) + # check against the template + template = CapaDescriptor.get_template(template_id) + self.assertEqual(problem.data, template['data']) + self.assertEqual(problem.display_name, template['metadata']['display_name']) + self.assertEqual(problem.markdown, template['metadata']['markdown']) + + def test_create_item_negative(self): + """ + Negative tests for create_item + """ + # non-existent boilerplate: creates a default + resp = self.client.post( + reverse('create_item'), + json.dumps( + {'parent_location': self.course.location.url(), + 'category': 'problem', + 'boilerplate': 'nosuchboilerplate.yaml' + }), + content_type="application/json" + ) + self.assertEqual(resp.status_code, 200) + +class TestEditItem(CourseTestCase): + """ + Test contentstore.views.item.save_item + """ + def response_id(self, response): + """ + Get the id from the response payload + :param response: + """ + parsed = json.loads(response.content) + return parsed['id'] + + def setUp(self): + """ Creates the test course structure and a couple problems to 'edit'. """ + super(TestEditItem, self).setUp() + # create a chapter + display_name = 'chapter created' + resp = self.client.post( + reverse('create_item'), + json.dumps( + {'parent_location': self.course.location.url(), + 'display_name': display_name, + 'category': 'chapter' + }), + content_type="application/json" + ) + chap_location = self.response_id(resp) + resp = self.client.post( + reverse('create_item'), + json.dumps( + {'parent_location': chap_location, + 'category': 'sequential' + }), + content_type="application/json" + ) + self.seq_location = self.response_id(resp) + # create problem w/ boilerplate + template_id = 'multiplechoice.yaml' + resp = self.client.post( + reverse('create_item'), + json.dumps({'parent_location': self.seq_location, + 'category': 'problem', + 'boilerplate': template_id + }), + content_type="application/json" + ) + self.problems = [self.response_id(resp)] + + def test_delete_field(self): + """ + Sending null in for a field 'deletes' it + """ + self.client.post( + reverse('save_item'), + json.dumps({ + 'id': self.problems[0], + 'metadata': {'rerandomize': 'onreset'} + }), + content_type="application/json" + ) + problem = modulestore('draft').get_item(self.problems[0]) + self.assertEqual(problem.rerandomize, 'onreset') + self.client.post( + reverse('save_item'), + json.dumps({ + 'id': self.problems[0], + 'metadata': {'rerandomize': None} + }), + content_type="application/json" + ) + problem = modulestore('draft').get_item(self.problems[0]) + self.assertEqual(problem.rerandomize, 'never') + + + def test_null_field(self): + """ + Sending null in for a field 'deletes' it + """ + problem = modulestore('draft').get_item(self.problems[0]) + self.assertIsNotNone(problem.markdown) + self.client.post( + reverse('save_item'), + json.dumps({ + 'id': self.problems[0], + 'nullout': ['markdown'] + }), + content_type="application/json" + ) + problem = modulestore('draft').get_item(self.problems[0]) + self.assertIsNone(problem.markdown) + + def test_date_fields(self): + """ + Test setting due & start dates on sequential + """ + sequential = modulestore().get_item(self.seq_location) + self.assertIsNone(sequential.lms.due) + self.client.post( + reverse('save_item'), + json.dumps({ + 'id': self.seq_location, + 'metadata': {'due': '2010-11-22T04:00Z'} + }), + content_type="application/json" + ) + sequential = modulestore().get_item(self.seq_location) + self.assertEqual(sequential.lms.due, datetime.datetime(2010, 11, 22, 4, 0, tzinfo=UTC)) + self.client.post( + reverse('save_item'), + json.dumps({ + 'id': self.seq_location, + 'metadata': {'start': '2010-09-12T14:00Z'} + }), + content_type="application/json" + ) + sequential = modulestore().get_item(self.seq_location) + self.assertEqual(sequential.lms.due, datetime.datetime(2010, 11, 22, 4, 0, tzinfo=UTC)) + self.assertEqual(sequential.lms.start, datetime.datetime(2010, 9, 12, 14, 0, tzinfo=UTC)) + diff --git a/cms/djangoapps/contentstore/tests/test_textbooks.py b/cms/djangoapps/contentstore/tests/test_textbooks.py index 02c64e9413..a21a1b1023 100644 --- a/cms/djangoapps/contentstore/tests/test_textbooks.py +++ b/cms/djangoapps/contentstore/tests/test_textbooks.py @@ -62,6 +62,9 @@ class TextbookIndexTestCase(CourseTestCase): } ] self.course.pdf_textbooks = content + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + self.course.save() store = get_modulestore(self.course.location) store.update_metadata(self.course.location, own_metadata(self.course)) @@ -220,6 +223,9 @@ class TextbookByIdTestCase(CourseTestCase): 'tid': 2, }) self.course.pdf_textbooks = [self.textbook1, self.textbook2] + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + self.course.save() self.store = get_modulestore(self.course.location) self.store.update_metadata(self.course.location, own_metadata(self.course)) self.url_nonexist = reverse('textbook_by_id', kwargs={ diff --git a/cms/djangoapps/contentstore/tests/utils.py b/cms/djangoapps/contentstore/tests/utils.py index bc9e9e8bae..a3f211a703 100644 --- a/cms/djangoapps/contentstore/tests/utils.py +++ b/cms/djangoapps/contentstore/tests/utils.py @@ -54,7 +54,6 @@ class CourseTestCase(ModuleStoreTestCase): self.client.login(username=uname, password=password) self.course = CourseFactory.create( - template='i4x://edx/templates/course/Empty', org='MITx', number='999', display_name='Robot Super Course', diff --git a/cms/djangoapps/contentstore/utils.py b/cms/djangoapps/contentstore/utils.py index 5fa0d949b0..4973bddaca 100644 --- a/cms/djangoapps/contentstore/utils.py +++ b/cms/djangoapps/contentstore/utils.py @@ -19,14 +19,14 @@ NOTES_PANEL = {"name": _("My Notes"), "type": "notes"} EXTRA_TAB_PANELS = dict([(p['type'], p) for p in [OPEN_ENDED_PANEL, NOTES_PANEL]]) -def get_modulestore(location): +def get_modulestore(category_or_location): """ Returns the correct modulestore to use for modifying the specified location """ - if not isinstance(location, Location): - location = Location(location) + if isinstance(category_or_location, Location): + category_or_location = category_or_location.category - if location.category in DIRECT_ONLY_CATEGORIES: + if category_or_location in DIRECT_ONLY_CATEGORIES: return modulestore('direct') else: return modulestore() diff --git a/cms/djangoapps/contentstore/views/assets.py b/cms/djangoapps/contentstore/views/assets.py index d0b202da19..0bb9551ac9 100644 --- a/cms/djangoapps/contentstore/views/assets.py +++ b/cms/djangoapps/contentstore/views/assets.py @@ -13,7 +13,7 @@ from django_future.csrf import ensure_csrf_cookie from django.core.urlresolvers import reverse from django.core.servers.basehttp import FileWrapper from django.core.files.temp import NamedTemporaryFile -from django.views.decorators.http import require_POST +from django.views.decorators.http import require_POST, require_http_methods from mitxmako.shortcuts import render_to_response from cache_toolbox.core import del_cached_content @@ -249,6 +249,7 @@ def remove_asset(request, org, course, name): @ensure_csrf_cookie +@require_http_methods(("GET", "POST", "PUT")) @login_required def import_course(request, org, course, name): """ @@ -256,7 +257,7 @@ def import_course(request, org, course, name): """ location = get_location_and_verify_access(request, org, course, name) - if request.method == 'POST': + if request.method in ('POST', 'PUT'): filename = request.FILES['course-data'].name if not filename.endswith('.tar.gz'): diff --git a/cms/djangoapps/contentstore/views/checklist.py b/cms/djangoapps/contentstore/views/checklist.py index fa0a7b7b62..bcf4a1a5b9 100644 --- a/cms/djangoapps/contentstore/views/checklist.py +++ b/cms/djangoapps/contentstore/views/checklist.py @@ -7,11 +7,11 @@ from django.views.decorators.http import require_http_methods from django_future.csrf import ensure_csrf_cookie from mitxmako.shortcuts import render_to_response -from xmodule.modulestore import Location from xmodule.modulestore.inheritance import own_metadata from ..utils import get_modulestore, get_url_reverse from .access import get_location_and_verify_access +from xmodule.course_module import CourseDescriptor __all__ = ['get_checklists', 'update_checklist'] @@ -28,17 +28,16 @@ def get_checklists(request, org, course, name): modulestore = get_modulestore(location) course_module = modulestore.get_item(location) - new_course_template = Location('i4x', 'edx', 'templates', 'course', 'Empty') - template_module = modulestore.get_item(new_course_template) # If course was created before checklists were introduced, copy them over from the template. copied = False if not course_module.checklists: - course_module.checklists = template_module.checklists + course_module.checklists = CourseDescriptor.checklists.default copied = True checklists, modified = expand_checklist_action_urls(course_module) if copied or modified: + course_module.save() modulestore.update_metadata(location, own_metadata(course_module)) return render_to_response('checklists.html', { @@ -71,6 +70,7 @@ def update_checklist(request, org, course, name, checklist_index=None): # seeming noop which triggers kvs to record that the metadata is not default course_module.checklists = course_module.checklists checklists, _ = expand_checklist_action_urls(course_module) + course_module.save() modulestore.update_metadata(location, own_metadata(course_module)) return JsonResponse(checklists[index]) else: @@ -81,6 +81,7 @@ def update_checklist(request, org, course, name, checklist_index=None): # In the JavaScript view initialize method, we do a fetch to get all the checklists. checklists, modified = expand_checklist_action_urls(course_module) if modified: + course_module.save() modulestore.update_metadata(location, own_metadata(course_module)) return JsonResponse(checklists) diff --git a/cms/djangoapps/contentstore/views/component.py b/cms/djangoapps/contentstore/views/component.py index 30958d5866..1be6ac2822 100644 --- a/cms/djangoapps/contentstore/views/component.py +++ b/cms/djangoapps/contentstore/views/component.py @@ -26,6 +26,8 @@ from models.settings.course_grading import CourseGradingModel from .requests import _xmodule_recurse from .access import has_access +from xmodule.x_module import XModuleDescriptor +from xblock.plugin import PluginMissingError __all__ = ['OPEN_ENDED_COMPONENT_TYPES', 'ADVANCED_COMPONENT_POLICY_KEY', @@ -101,7 +103,7 @@ def edit_subsection(request, location): return render_to_response('edit_subsection.html', {'subsection': item, 'context_course': course, - 'create_new_unit_template': Location('i4x', 'edx', 'templates', 'vertical', 'Empty'), + 'new_unit_category': 'vertical', 'lms_link': lms_link, 'preview_link': preview_link, 'course_graders': json.dumps(CourseGradingModel.fetch(course.location).graders), @@ -134,10 +136,26 @@ def edit_unit(request, location): item = modulestore().get_item(location, depth=1) except ItemNotFoundError: return HttpResponseBadRequest() - lms_link = get_lms_link_for_item(item.location, course_id=course.location.course_id) component_templates = defaultdict(list) + for category in COMPONENT_TYPES: + component_class = XModuleDescriptor.load_class(category) + # add the default template + component_templates[category].append(( + component_class.display_name.default or 'Blank', + category, + False, # No defaults have markdown (hardcoded current default) + None # no boilerplate for overrides + )) + # add boilerplates + for template in component_class.templates(): + component_templates[category].append(( + template['metadata'].get('display_name'), + category, + template['metadata'].get('markdown') is not None, + template.get('template_id') + )) # Check if there are any advanced modules specified in the course policy. These modules # should be specified as a list of strings, where the strings are the names of the modules @@ -145,29 +163,29 @@ def edit_unit(request, location): course_advanced_keys = course.advanced_modules # Set component types according to course policy file - component_types = list(COMPONENT_TYPES) if isinstance(course_advanced_keys, list): - course_advanced_keys = [c for c in course_advanced_keys if c in ADVANCED_COMPONENT_TYPES] - if len(course_advanced_keys) > 0: - component_types.append(ADVANCED_COMPONENT_CATEGORY) + for category in course_advanced_keys: + if category in ADVANCED_COMPONENT_TYPES: + # Do I need to allow for boilerplates or just defaults on the class? i.e., can an advanced + # have more than one entry in the menu? one for default and others for prefilled boilerplates? + try: + component_class = XModuleDescriptor.load_class(category) + + component_templates['advanced'].append(( + component_class.display_name.default or category, + category, + False, + None # don't override default data + )) + except PluginMissingError: + # dhm: I got this once but it can happen any time the course author configures + # an advanced component which does not exist on the server. This code here merely + # prevents any authors from trying to instantiate the non-existent component type + # by not showing it in the menu + pass else: log.error("Improper format for course advanced keys! {0}".format(course_advanced_keys)) - templates = modulestore().get_items(Location('i4x', 'edx', 'templates')) - for template in templates: - category = template.location.category - - if category in course_advanced_keys: - category = ADVANCED_COMPONENT_CATEGORY - - if category in component_types: - # This is a hack to create categories for different xmodules - component_templates[category].append(( - template.display_name_with_default, - template.location.url(), - hasattr(template, 'markdown') and template.markdown is not None - )) - components = [ component.location.url() for component @@ -219,7 +237,7 @@ def edit_unit(request, location): 'subsection': containing_subsection, 'release_date': get_default_time_display(containing_subsection.lms.start) if containing_subsection.lms.start is not None else None, 'section': containing_section, - 'create_new_unit_template': Location('i4x', 'edx', 'templates', 'vertical', 'Empty'), + 'new_unit_category': 'vertical', 'unit_state': unit_state, 'published_date': get_default_time_display(item.cms.published_date) if item.cms.published_date is not None else None }) @@ -227,6 +245,7 @@ def edit_unit(request, location): @expect_json @login_required +@require_http_methods(("GET", "POST", "PUT")) @ensure_csrf_cookie def assignment_type_update(request, org, course, category, name): ''' @@ -238,7 +257,7 @@ def assignment_type_update(request, org, course, category, name): if request.method == 'GET': return JsonResponse(CourseGradingModel.get_section_grader_type(location)) - elif request.method == 'POST': # post or put, doesn't matter. + elif request.method in ('POST', 'PUT'): # post or put, doesn't matter. return JsonResponse(CourseGradingModel.update_section_grader_type(location, request.POST)) @@ -253,7 +272,7 @@ def create_draft(request): # This clones the existing item location to a draft location (the draft is implicit, # because modulestore is a Draft modulestore) - modulestore().clone_item(location, location) + modulestore().convert_to_draft(location) return HttpResponse() diff --git a/cms/djangoapps/contentstore/views/course.py b/cms/djangoapps/contentstore/views/course.py index f8de053d95..02eb4c65b8 100644 --- a/cms/djangoapps/contentstore/views/course.py +++ b/cms/djangoapps/contentstore/views/course.py @@ -1,10 +1,9 @@ """ Views related to operations on course objects """ -#pylint: disable=W0402 import json import random -import string +import string # pylint: disable=W0402 from django.contrib.auth.decorators import login_required from django_future.csrf import ensure_csrf_cookie @@ -43,8 +42,8 @@ from .component import ( ADVANCED_COMPONENT_POLICY_KEY) from django_comment_common.utils import seed_permissions_roles -import datetime -from django.utils.timezone import UTC + +from xmodule.html_module import AboutDescriptor __all__ = ['course_index', 'create_new_course', 'course_info', 'course_info_updates', 'get_course_settings', 'course_config_graders_page', @@ -82,10 +81,11 @@ def course_index(request, org, course, name): 'sections': sections, 'course_graders': json.dumps(CourseGradingModel.fetch(course.location).graders), 'parent_location': course.location, - 'new_section_template': Location('i4x', 'edx', 'templates', 'chapter', 'Empty'), - 'new_subsection_template': Location('i4x', 'edx', 'templates', 'sequential', 'Empty'), # for now they are the same, but the could be different at some point... + 'new_section_category': 'chapter', + 'new_subsection_category': 'sequential', 'upload_asset_callback_url': upload_asset_callback_url, - 'create_new_unit_template': Location('i4x', 'edx', 'templates', 'vertical', 'Empty') + 'new_unit_category': 'vertical', + 'category': 'vertical' }) @@ -98,12 +98,6 @@ def create_new_course(request): if not is_user_in_creator_group(request.user): raise PermissionDenied() - # This logic is repeated in xmodule/modulestore/tests/factories.py - # so if you change anything here, you need to also change it there. - # TODO: write a test that creates two courses, one with the factory and - # the other with this method, then compare them to make sure they are - # equivalent. - template = Location(request.POST['template']) org = request.POST.get('org') number = request.POST.get('number') display_name = request.POST.get('display_name') @@ -121,29 +115,31 @@ def create_new_course(request): existing_course = modulestore('direct').get_item(dest_location) except ItemNotFoundError: pass - if existing_course is not None: return JsonResponse({'ErrMsg': 'There is already a course defined with this name.'}) course_search_location = ['i4x', dest_location.org, dest_location.course, 'course', None] courses = modulestore().get_items(course_search_location) - if len(courses) > 0: return JsonResponse({'ErrMsg': 'There is already a course defined with the same organization and course number.'}) - new_course = modulestore('direct').clone_item(template, dest_location) + # instantiate the CourseDescriptor and then persist it + # note: no system to pass + if display_name is None: + metadata = {} + else: + metadata = {'display_name': display_name} + modulestore('direct').create_and_save_xmodule(dest_location, metadata=metadata) + new_course = modulestore('direct').get_item(dest_location) - # clone a default 'about' module as well - - about_template_location = Location(['i4x', 'edx', 'templates', 'about', 'overview']) - dest_about_location = dest_location._replace(category='about', name='overview') - modulestore('direct').clone_item(about_template_location, dest_about_location) - - if display_name is not None: - new_course.display_name = display_name - - # set a default start date to now - new_course.start = datetime.datetime.now(UTC()) + # clone a default 'about' overview module as well + dest_about_location = dest_location.replace(category='about', name='overview') + overview_template = AboutDescriptor.get_template('overview.yaml') + modulestore('direct').create_and_save_xmodule( + dest_about_location, + system=new_course.system, + definition_data=overview_template.get('data') + ) initialize_course_tabs(new_course) @@ -179,6 +175,7 @@ def course_info(request, org, course, name, provided_id=None): @expect_json +@require_http_methods(("GET", "POST", "PUT", "DELETE")) @login_required @ensure_csrf_cookie def course_info_updates(request, org, course, provided_id=None): @@ -209,7 +206,7 @@ def course_info_updates(request, org, course, provided_id=None): except: return HttpResponseBadRequest("Failed to delete", content_type="text/plain") - elif request.method == 'POST': + elif request.method in ('POST', 'PUT'): # can be either and sometimes django is rewriting one to the other try: return JsonResponse(update_course_updates(location, request.POST, provided_id)) except: @@ -303,7 +300,7 @@ def course_settings_updates(request, org, course, name, section): if request.method == 'GET': # Cannot just do a get w/o knowing the course name :-( return JsonResponse(manager.fetch(Location(['i4x', org, course, 'course', name])), encoder=CourseSettingsEncoder) - elif request.method == 'POST': # post or put, doesn't matter. + elif request.method in ('POST', 'PUT'): # post or put, doesn't matter. return JsonResponse(manager.update_from_json(request.POST), encoder=CourseSettingsEncoder) @@ -482,7 +479,7 @@ def textbook_index(request, org, course, name): if request.is_ajax(): if request.method == 'GET': return JsonResponse(course_module.pdf_textbooks) - elif request.method == 'POST': + elif request.method in ('POST', 'PUT'): # can be either and sometimes django is rewriting one to the other try: textbooks = validate_textbooks_json(request.body) except TextbookValidationError as err: @@ -498,6 +495,9 @@ def textbook_index(request, org, course, name): if not any(tab['type'] == 'pdf_textbooks' for tab in course_module.tabs): course_module.tabs.append({"type": "pdf_textbooks"}) course_module.pdf_textbooks = textbooks + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + course_module.save() store.update_metadata(course_module.location, own_metadata(course_module)) return JsonResponse(course_module.pdf_textbooks) else: @@ -544,6 +544,9 @@ def create_textbook(request, org, course, name): tabs = course_module.tabs tabs.append({"type": "pdf_textbooks"}) course_module.tabs = tabs + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + course_module.save() store.update_metadata(course_module.location, own_metadata(course_module)) resp = JsonResponse(textbook, status=201) resp["Location"] = reverse("textbook_by_id", kwargs={ @@ -577,7 +580,7 @@ def textbook_by_id(request, org, course, name, tid): if not textbook: return JsonResponse(status=404) return JsonResponse(textbook) - elif request.method in ('POST', 'PUT'): + elif request.method in ('POST', 'PUT'): # can be either and sometimes django is rewriting one to the other try: new_textbook = validate_textbook_json(request.body) except TextbookValidationError as err: @@ -587,10 +590,13 @@ def textbook_by_id(request, org, course, name, tid): i = course_module.pdf_textbooks.index(textbook) new_textbooks = course_module.pdf_textbooks[0:i] new_textbooks.append(new_textbook) - new_textbooks.extend(course_module.pdf_textbooks[i+1:]) + new_textbooks.extend(course_module.pdf_textbooks[i + 1:]) course_module.pdf_textbooks = new_textbooks else: course_module.pdf_textbooks.append(new_textbook) + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + course_module.save() store.update_metadata(course_module.location, own_metadata(course_module)) return JsonResponse(new_textbook, status=201) elif request.method == 'DELETE': @@ -598,7 +604,8 @@ def textbook_by_id(request, org, course, name, tid): return JsonResponse(status=404) i = course_module.pdf_textbooks.index(textbook) new_textbooks = course_module.pdf_textbooks[0:i] - new_textbooks.extend(course_module.pdf_textbooks[i+1:]) + new_textbooks.extend(course_module.pdf_textbooks[i + 1:]) course_module.pdf_textbooks = new_textbooks + course_module.save() store.update_metadata(course_module.location, own_metadata(course_module)) return JsonResponse() diff --git a/cms/djangoapps/contentstore/views/item.py b/cms/djangoapps/contentstore/views/item.py index abc5f48564..efebded9b9 100644 --- a/cms/djangoapps/contentstore/views/item.py +++ b/cms/djangoapps/contentstore/views/item.py @@ -13,16 +13,26 @@ from util.json_request import expect_json from ..utils import get_modulestore from .access import has_access from .requests import _xmodule_recurse +from xmodule.x_module import XModuleDescriptor -__all__ = ['save_item', 'clone_item', 'delete_item'] +__all__ = ['save_item', 'create_item', 'delete_item'] # cdodge: these are categories which should not be parented, they are detached from the hierarchy DETACHED_CATEGORIES = ['about', 'static_tab', 'course_info'] - @login_required @expect_json def save_item(request): + """ + Will carry a json payload with these possible fields + :id (required): the id + :data (optional): the new value for the data + :metadata (optional): new values for the metadata fields. + Any whose values are None will be deleted not set to None! Absent ones will be left alone + :nullout (optional): which metadata fields to set to None + """ + # The nullout is a bit of a temporary copout until we can make module_edit.coffee and the metadata editors a + # little smarter and able to pass something more akin to {unset: [field, field]} item_location = request.POST['id'] # check permissions for this user within this course @@ -42,59 +52,98 @@ def save_item(request): children = request.POST['children'] store.update_children(item_location, children) - # cdodge: also commit any metadata which might have been passed along in the - # POST from the client, if it is there - # NOTE, that the postback is not the complete metadata, as there's system metadata which is - # not presented to the end-user for editing. So let's fetch the original and - # 'apply' the submitted metadata, so we don't end up deleting system metadata - if request.POST.get('metadata') is not None: - posted_metadata = request.POST['metadata'] - # fetch original + # cdodge: also commit any metadata which might have been passed along + if request.POST.get('nullout') is not None or request.POST.get('metadata') is not None: + # the postback is not the complete metadata, as there's system metadata which is + # not presented to the end-user for editing. So let's fetch the original and + # 'apply' the submitted metadata, so we don't end up deleting system metadata existing_item = modulestore().get_item(item_location) + for metadata_key in request.POST.get('nullout', []): + # [dhm] see comment on _get_xblock_field + _get_xblock_field(existing_item, metadata_key).write_to(existing_item, None) # update existing metadata with submitted metadata (which can be partial) - # IMPORTANT NOTE: if the client passed pack 'null' (None) for a piece of metadata that means 'remove it' - for metadata_key, value in posted_metadata.items(): + # IMPORTANT NOTE: if the client passed 'null' (None) for a piece of metadata that means 'remove it'. If + # the intent is to make it None, use the nullout field + for metadata_key, value in request.POST.get('metadata', {}).items(): + # [dhm] see comment on _get_xblock_field + field = _get_xblock_field(existing_item, metadata_key) - if posted_metadata[metadata_key] is None: - # remove both from passed in collection as well as the collection read in from the modulestore - if metadata_key in existing_item._model_data: - del existing_item._model_data[metadata_key] - del posted_metadata[metadata_key] + if value is None: + field.delete_from(existing_item) else: - existing_item._model_data[metadata_key] = value - + value = field.from_json(value) + field.write_to(existing_item, value) + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + existing_item.save() # commit to datastore - # TODO (cpennington): This really shouldn't have to do this much reaching in to get the metadata store.update_metadata(item_location, own_metadata(existing_item)) return HttpResponse() +# [DHM] A hack until we implement a permanent soln. Proposed perm solution is to make namespace fields also top level +# fields in xblocks rather than requiring dereference through namespace but we'll need to consider whether there are +# plausible use cases for distinct fields w/ same name in different namespaces on the same blocks. +# The idea is that consumers of the xblock, and particularly the web client, shouldn't know about our internal +# representation (namespaces as means of decorating all modules). +# Given top-level access, the calls can simply be setattr(existing_item, field, value) ... +# Really, this method should be elsewhere (e.g., xblock). We also need methods for has_value (v is_default)... +def _get_xblock_field(xblock, field_name): + """ + A temporary function to get the xblock field either from the xblock or one of its namespaces by name. + :param xblock: + :param field_name: + """ + def find_field(fields): + for field in fields: + if field.name == field_name: + return field + + found = find_field(xblock.fields) + if found: + return found + for namespace in xblock.namespaces: + found = find_field(getattr(xblock, namespace).fields) + if found: + return found + + @login_required @expect_json -def clone_item(request): +def create_item(request): parent_location = Location(request.POST['parent_location']) - template = Location(request.POST['template']) + category = request.POST['category'] display_name = request.POST.get('display_name') if not has_access(request.user, parent_location): raise PermissionDenied() - parent = get_modulestore(template).get_item(parent_location) - dest_location = parent_location._replace(category=template.category, name=uuid4().hex) + parent = get_modulestore(category).get_item(parent_location) + dest_location = parent_location.replace(category=category, name=uuid4().hex) - new_item = get_modulestore(template).clone_item(template, dest_location) + # get the metadata, display_name, and definition from the request + metadata = {} + data = None + template_id = request.POST.get('boilerplate') + if template_id is not None: + clz = XModuleDescriptor.load_class(category) + if clz is not None: + template = clz.get_template(template_id) + if template is not None: + metadata = template.get('metadata', {}) + data = template.get('data') - # replace the display name with an optional parameter passed in from the caller if display_name is not None: - new_item.display_name = display_name + metadata['display_name'] = display_name - get_modulestore(template).update_metadata(new_item.location.url(), own_metadata(new_item)) + get_modulestore(category).create_and_save_xmodule(dest_location, definition_data=data, + metadata=metadata, system=parent.system) - if new_item.location.category not in DETACHED_CATEGORIES: - get_modulestore(parent.location).update_children(parent_location, parent.children + [new_item.location.url()]) + if category not in DETACHED_CATEGORIES: + get_modulestore(parent.location).update_children(parent_location, parent.children + [dest_location.url()]) return HttpResponse(json.dumps({'id': dest_location.url()})) diff --git a/cms/djangoapps/contentstore/views/preview.py b/cms/djangoapps/contentstore/views/preview.py index ba393e72f4..35af3e9ac3 100644 --- a/cms/djangoapps/contentstore/views/preview.py +++ b/cms/djangoapps/contentstore/views/preview.py @@ -7,7 +7,7 @@ from django.core.urlresolvers import reverse from django.contrib.auth.decorators import login_required from mitxmako.shortcuts import render_to_response -from xmodule_modifiers import replace_static_urls, wrap_xmodule +from xmodule_modifiers import replace_static_urls, wrap_xmodule, save_module # pylint: disable=F0401 from xmodule.error_module import ErrorDescriptor from xmodule.errortracker import exc_info_to_str from xmodule.exceptions import NotFoundError, ProcessingError @@ -47,6 +47,8 @@ def preview_dispatch(request, preview_id, location, dispatch=None): # Let the module handle the AJAX try: ajax_return = instance.handle_ajax(dispatch, request.POST) + # Save any module data that has changed to the underlying KeyValueStore + instance.save() except NotFoundError: log.exception("Module indicating to user that request doesn't exist") @@ -166,6 +168,11 @@ def load_preview_module(request, preview_id, descriptor): course_namespace=Location([module.location.tag, module.location.org, module.location.course, None, None]) ) + module.get_html = save_module( + module.get_html, + module + ) + return module diff --git a/cms/djangoapps/contentstore/views/tabs.py b/cms/djangoapps/contentstore/views/tabs.py index 154f9fb55d..d55932e33d 100644 --- a/cms/djangoapps/contentstore/views/tabs.py +++ b/cms/djangoapps/contentstore/views/tabs.py @@ -76,6 +76,9 @@ def reorder_static_tabs(request): # OK, re-assemble the static tabs in the new order course.tabs = reordered_tabs + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + course.save() modulestore('direct').update_metadata(course.location, own_metadata(course)) return HttpResponse() diff --git a/cms/djangoapps/contentstore/views/user.py b/cms/djangoapps/contentstore/views/user.py index 948ed614d2..020e4b5cb9 100644 --- a/cms/djangoapps/contentstore/views/user.py +++ b/cms/djangoapps/contentstore/views/user.py @@ -27,6 +27,7 @@ def index(request): # filter out courses that we don't have access too def course_filter(course): return (has_access(request.user, course.location) + # TODO remove this condition when templates purged from db and course.location.course != 'templates' and course.location.org != '' and course.location.course != '' @@ -34,7 +35,6 @@ def index(request): courses = filter(course_filter, courses) return render_to_response('index.html', { - 'new_course_template': Location('i4x', 'edx', 'templates', 'course', 'Empty'), 'courses': [(course.display_name, get_url_reverse('CourseOutline', course), get_lms_link_for_item(course.location, course_id=course.location.course_id)) @@ -83,6 +83,9 @@ def add_user(request, location): } return JsonResponse(msg, 400) + # remove leading/trailing whitespace if necessary + email = email.strip() + # check that logged in user has admin permissions to this course if not has_access(request.user, location, role=INSTRUCTOR_ROLE_NAME): raise PermissionDenied() diff --git a/cms/djangoapps/models/settings/course_details.py b/cms/djangoapps/models/settings/course_details.py index 8ce8c2db34..7c3b883283 100644 --- a/cms/djangoapps/models/settings/course_details.py +++ b/cms/djangoapps/models/settings/course_details.py @@ -122,6 +122,10 @@ class CourseDetails(object): descriptor.enrollment_end = converted if dirty: + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() + get_modulestore(course_location).update_metadata(course_location, own_metadata(descriptor)) # NOTE: below auto writes to the db w/o verifying that any of the fields actually changed diff --git a/cms/djangoapps/models/settings/course_grading.py b/cms/djangoapps/models/settings/course_grading.py index 4ea9f2f5db..0746fc7a90 100644 --- a/cms/djangoapps/models/settings/course_grading.py +++ b/cms/djangoapps/models/settings/course_grading.py @@ -7,9 +7,12 @@ class CourseGradingModel(object): """ Basically a DAO and Model combo for CRUD operations pertaining to grading policy. """ + # Within this class, allow access to protected members of client classes. + # This comes up when accessing kvs data and caches during kvs saves and modulestore writes. + # pylint: disable=W0212 def __init__(self, course_descriptor): self.course_location = course_descriptor.location - self.graders = [CourseGradingModel.jsonize_grader(i, grader) for i, grader in enumerate(course_descriptor.raw_grader)] # weights transformed to ints [0..100] + self.graders = [CourseGradingModel.jsonize_grader(i, grader) for i, grader in enumerate(course_descriptor.raw_grader)] # weights transformed to ints [0..100] self.grade_cutoffs = course_descriptor.grade_cutoffs self.grace_period = CourseGradingModel.convert_set_grace_period(course_descriptor) @@ -81,15 +84,18 @@ class CourseGradingModel(object): Decode the json into CourseGradingModel and save any changes. Returns the modified model. Probably not the usual path for updates as it's too coarse grained. """ - course_location = jsondict['course_location'] + course_location = Location(jsondict['course_location']) descriptor = get_modulestore(course_location).get_item(course_location) - graders_parsed = [CourseGradingModel.parse_grader(jsonele) for jsonele in jsondict['graders']] descriptor.raw_grader = graders_parsed descriptor.grade_cutoffs = jsondict['grade_cutoffs'] - get_modulestore(course_location).update_item(course_location, descriptor._model_data._kvs._data) + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() + get_modulestore(course_location).update_item(course_location, descriptor.xblock_kvs._data) + CourseGradingModel.update_grace_period_from_json(course_location, jsondict['grace_period']) return CourseGradingModel.fetch(course_location) @@ -116,6 +122,9 @@ class CourseGradingModel(object): else: descriptor.raw_grader.append(grader) + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() get_modulestore(course_location).update_item(course_location, descriptor._model_data._kvs._data) return CourseGradingModel.jsonize_grader(index, descriptor.raw_grader[index]) @@ -131,6 +140,10 @@ class CourseGradingModel(object): descriptor = get_modulestore(course_location).get_item(course_location) descriptor.grade_cutoffs = cutoffs + + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() get_modulestore(course_location).update_item(course_location, descriptor._model_data._kvs._data) return cutoffs @@ -156,6 +169,10 @@ class CourseGradingModel(object): descriptor = get_modulestore(course_location).get_item(course_location) descriptor.lms.graceperiod = grace_timedelta + + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() get_modulestore(course_location).update_metadata(course_location, descriptor._model_data._kvs._metadata) @staticmethod @@ -172,23 +189,12 @@ class CourseGradingModel(object): del descriptor.raw_grader[index] # force propagation to definition descriptor.raw_grader = descriptor.raw_grader + + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() get_modulestore(course_location).update_item(course_location, descriptor._model_data._kvs._data) - # NOTE cannot delete cutoffs. May be useful to reset - @staticmethod - def delete_cutoffs(course_location, cutoffs): - """ - Resets the cutoffs to the defaults - """ - if not isinstance(course_location, Location): - course_location = Location(course_location) - - descriptor = get_modulestore(course_location).get_item(course_location) - descriptor.grade_cutoffs = descriptor.defaut_grading_policy['GRADE_CUTOFFS'] - get_modulestore(course_location).update_item(course_location, descriptor._model_data._kvs._data) - - return descriptor.grade_cutoffs - @staticmethod def delete_grace_period(course_location): """ @@ -199,6 +205,10 @@ class CourseGradingModel(object): descriptor = get_modulestore(course_location).get_item(course_location) del descriptor.lms.graceperiod + + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() get_modulestore(course_location).update_metadata(course_location, descriptor._model_data._kvs._metadata) @staticmethod @@ -209,7 +219,7 @@ class CourseGradingModel(object): descriptor = get_modulestore(location).get_item(location) return {"graderType": descriptor.lms.format if descriptor.lms.format is not None else 'Not Graded', "location": location, - "id": 99 # just an arbitrary value to + "id": 99 # just an arbitrary value to } @staticmethod @@ -225,6 +235,9 @@ class CourseGradingModel(object): del descriptor.lms.format del descriptor.lms.graded + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() get_modulestore(location).update_metadata(location, descriptor._model_data._kvs._metadata) @staticmethod @@ -232,7 +245,7 @@ class CourseGradingModel(object): # 5 hours 59 minutes 59 seconds => converted to iso format rawgrace = descriptor.lms.graceperiod if rawgrace: - hours_from_days = rawgrace.days*24 + hours_from_days = rawgrace.days * 24 seconds = rawgrace.seconds hours_from_seconds = int(seconds / 3600) hours = hours_from_days + hours_from_seconds diff --git a/cms/djangoapps/models/settings/course_metadata.py b/cms/djangoapps/models/settings/course_metadata.py index 5fb07fe806..8d9a292867 100644 --- a/cms/djangoapps/models/settings/course_metadata.py +++ b/cms/djangoapps/models/settings/course_metadata.py @@ -76,6 +76,9 @@ class CourseMetadata(object): setattr(descriptor.lms, key, value) if dirty: + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() get_modulestore(course_location).update_metadata(course_location, own_metadata(descriptor)) @@ -97,6 +100,10 @@ class CourseMetadata(object): elif hasattr(descriptor.lms, key): delattr(descriptor.lms, key) + # Save the data that we've just changed to the underlying + # MongoKeyValueStore before we update the mongo datastore. + descriptor.save() + get_modulestore(course_location).update_metadata(course_location, own_metadata(descriptor)) diff --git a/cms/envs/aws.py b/cms/envs/aws.py index 20217deaff..339425fee5 100644 --- a/cms/envs/aws.py +++ b/cms/envs/aws.py @@ -92,6 +92,7 @@ LOG_DIR = ENV_TOKENS['LOG_DIR'] CACHES = ENV_TOKENS['CACHES'] SESSION_COOKIE_DOMAIN = ENV_TOKENS.get('SESSION_COOKIE_DOMAIN') +SESSION_ENGINE = ENV_TOKENS.get('SESSION_ENGINE', SESSION_ENGINE) # allow for environments to specify what cookie name our login subsystem should use # this is to fix a bug regarding simultaneous logins between edx.org and edge.edx.org which can @@ -122,6 +123,10 @@ LOGGING = get_logger_config(LOG_DIR, debug=False, service_variant=SERVICE_VARIANT) +#theming start: +PLATFORM_NAME = ENV_TOKENS.get('PLATFORM_NAME', 'edX') + + ################ SECURE AUTH ITEMS ############################### # Secret things: passwords, access keys, etc. with open(ENV_ROOT / CONFIG_PREFIX + "auth.json") as auth_file: diff --git a/cms/envs/dev.py b/cms/envs/dev.py index 4f8174ac2b..0b0a62f05d 100644 --- a/cms/envs/dev.py +++ b/cms/envs/dev.py @@ -33,6 +33,10 @@ MODULESTORE = { 'direct': { 'ENGINE': 'xmodule.modulestore.mongo.MongoModuleStore', 'OPTIONS': modulestore_options + }, + 'split': { + 'ENGINE': 'xmodule.modulestore.split_mongo.SplitMongoModuleStore', + 'OPTIONS': modulestore_options } } diff --git a/cms/envs/test.py b/cms/envs/test.py index 431a2c4184..efc7c5a7ef 100644 --- a/cms/envs/test.py +++ b/cms/envs/test.py @@ -63,6 +63,10 @@ MODULESTORE = { 'draft': { 'ENGINE': 'xmodule.modulestore.draft.DraftModuleStore', 'OPTIONS': MODULESTORE_OPTIONS + }, + 'split': { + 'ENGINE': 'xmodule.modulestore.split_mongo.SplitMongoModuleStore', + 'OPTIONS': MODULESTORE_OPTIONS } } diff --git a/cms/manage.py b/cms/manage.py deleted file mode 100755 index 723fa59da1..0000000000 --- a/cms/manage.py +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env python -from django.core.management import execute_manager -import imp -try: - imp.find_module('settings') # Assumed to be in the same directory. -except ImportError: - import sys - sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. " - "It appears you've customized things.\nYou'll have to run django-admin.py, " - "passing it your settings module.\n" % __file__) - sys.exit(1) - -import settings - -if __name__ == "__main__": - execute_manager(settings) diff --git a/cms/static/coffee/spec/views/overview_spec.coffee b/cms/static/coffee/spec/views/overview_spec.coffee index ba9b5d8918..d900e4bfb1 100644 --- a/cms/static/coffee/spec/views/overview_spec.coffee +++ b/cms/static/coffee/spec/views/overview_spec.coffee @@ -40,17 +40,30 @@ describe "Course Overview", -> """#" + appendSetFixtures """ +

+ +
+ """#" + spyOn(window, 'saveSetSectionScheduleDate').andCallThrough() # Have to do this here, as it normally gets bound in document.ready() $('a.save-button').click(saveSetSectionScheduleDate) + $('a.delete-section-button').click(deleteSection) + @notificationSpy = spyOn(CMS.Views.Notification.Mini.prototype, 'show').andCallThrough() window.analytics = jasmine.createSpyObj('analytics', ['track']) window.course_location_analytics = jasmine.createSpy() - sinon.useFakeXMLHttpRequest() + @xhr = sinon.useFakeXMLHttpRequest() + requests = @requests = [] + @xhr.onCreate = (req) -> requests.push(req) afterEach -> delete window.analytics delete window.course_location_analytics + @notificationSpy.reset() it "should save model when save is clicked", -> $('a.edit-button').click() @@ -61,3 +74,21 @@ describe "Course Overview", -> $('a.edit-button').click() $('a.save-button').click() expect(@notificationSpy).toHaveBeenCalled() + + it "should delete model when delete is clicked", -> + deleteSpy = spyOn(window, '_deleteItem').andCallThrough() + $('a.delete-section-button').click() + $('a.action-primary').click() + expect(deleteSpy).toHaveBeenCalled() + expect(@requests[0].url).toEqual('/delete_item') + + it "should not delete model when cancel is clicked", -> + deleteSpy = spyOn(window, '_deleteItem').andCallThrough() + $('a.delete-section-button').click() + $('a.action-secondary').click() + expect(@requests.length).toEqual(0) + + it "should show a confirmation on delete", -> + $('a.delete-section-button').click() + $('a.action-primary').click() + expect(@notificationSpy).toHaveBeenCalled() diff --git a/cms/static/coffee/src/views/module_edit.coffee b/cms/static/coffee/src/views/module_edit.coffee index 5154591d6f..c45feecd41 100644 --- a/cms/static/coffee/src/views/module_edit.coffee +++ b/cms/static/coffee/src/views/module_edit.coffee @@ -56,14 +56,15 @@ class CMS.Views.ModuleEdit extends Backbone.View changedMetadata: -> return _.extend(@metadataEditor.getModifiedMetadataValues(), @customMetadata()) - cloneTemplate: (parent, template) -> - $.post("/clone_item", { - parent_location: parent - template: template - }, (data) => - @model.set(id: data.id) - @$el.data('id', data.id) - @render() + createItem: (parent, payload) -> + payload.parent_location = parent + $.post( + "/create_item" + payload + (data) => + @model.set(id: data.id) + @$el.data('id', data.id) + @render() ) render: -> @@ -83,11 +84,15 @@ class CMS.Views.ModuleEdit extends Backbone.View data.metadata = _.extend(data.metadata || {}, @changedMetadata()) @hideModal() + saving = new CMS.Views.Notification.Mini + title: gettext('Saving') + '…' + saving.show() @model.save(data).done( => # # showToastMessage("Your changes have been saved.", null, 3) @module = null @render() @$el.removeClass('editing') + saving.hide() ).fail( -> showToastMessage(gettext("There was an error saving your changes. Please try again."), null, 3) ) diff --git a/cms/static/coffee/src/views/tabs.coffee b/cms/static/coffee/src/views/tabs.coffee index 1034fc988e..58f52f27a3 100644 --- a/cms/static/coffee/src/views/tabs.coffee +++ b/cms/static/coffee/src/views/tabs.coffee @@ -55,9 +55,9 @@ class CMS.Views.TabsEdit extends Backbone.View editor.$el.removeClass('new') , 500) - editor.cloneTemplate( + editor.createItem( @model.get('id'), - 'i4x://edx/templates/static_tab/Empty' + {category: 'static_tab'} ) analytics.track "Added Static Page", diff --git a/cms/static/coffee/src/views/unit.coffee b/cms/static/coffee/src/views/unit.coffee index 058bcf0ce1..14b64b7b9e 100644 --- a/cms/static/coffee/src/views/unit.coffee +++ b/cms/static/coffee/src/views/unit.coffee @@ -67,8 +67,8 @@ class CMS.Views.UnitEdit extends Backbone.View type = $(event.currentTarget).data('type') @$newComponentTypePicker.slideUp(250) @$(".new-component-#{type}").slideDown(250) - $('html, body').animate({ - scrollTop: @$(".new-component-#{type}").offset().top + $('html, body').animate({ + scrollTop: @$(".new-component-#{type}").offset().top }, 500) closeNewComponent: (event) => @@ -89,9 +89,9 @@ class CMS.Views.UnitEdit extends Backbone.View @$newComponentItem.before(editor.$el) - editor.cloneTemplate( + editor.createItem( @$el.data('id'), - $(event.currentTarget).data('location') + $(event.currentTarget).data() ) analytics.track "Added a Component", @@ -115,27 +115,43 @@ class CMS.Views.UnitEdit extends Backbone.View @model.save() deleteComponent: (event) => - if not confirm 'Are you sure you want to delete this component? This action cannot be undone.' - return - $component = $(event.currentTarget).parents('.component') - $.post('/delete_item', { - id: $component.data('id') - }, => - analytics.track "Deleted a Component", - course: course_location_analytics - unit_id: unit_location_analytics - id: $component.data('id') + msg = new CMS.Views.Prompt.Warning( + title: gettext('Delete this component?'), + message: gettext('Deleting this component is permanent and cannot be undone.'), + actions: + primary: + text: gettext('Yes, delete this component'), + click: (view) => + view.hide() + deleting = new CMS.Views.Notification.Mini + title: gettext('Deleting') + '…', + deleting.show() + $component = $(event.currentTarget).parents('.component') + $.post('/delete_item', { + id: $component.data('id') + }, => + deleting.hide() + analytics.track "Deleted a Component", + course: course_location_analytics + unit_id: unit_location_analytics + id: $component.data('id') - $component.remove() - # b/c we don't vigilantly keep children up to date - # get rid of it before it hurts someone - # sorry for the js, i couldn't figure out the coffee equivalent - `_this.model.save({children: _this.components()}, - {success: function(model) { - model.unset('children'); - }} - );` + $component.remove() + # b/c we don't vigilantly keep children up to date + # get rid of it before it hurts someone + # sorry for the js, i couldn't figure out the coffee equivalent + `_this.model.save({children: _this.components()}, + {success: function(model) { + model.unset('children'); + }} + );` + ) + secondary: + text: gettext('Cancel'), + click: (view) -> + view.hide() ) + msg.show() deleteDraft: (event) -> @wait(true) @@ -236,7 +252,7 @@ class CMS.Views.UnitEdit.NameEdit extends Backbone.View class CMS.Views.UnitEdit.LocationState extends Backbone.View initialize: => @model.on('change:state', @render) - + render: => @$el.toggleClass("#{@model.previous('state')}-item #{@model.get('state')}-item") diff --git a/cms/static/js/base.js b/cms/static/js/base.js index b53d116808..3a30e0bc81 100644 --- a/cms/static/js/base.js +++ b/cms/static/js/base.js @@ -253,17 +253,13 @@ function syncReleaseDate(e) { } function getEdxTimeFromDateTimeVals(date_val, time_val) { - var edxTimeStr = null; - if (date_val != '') { if (time_val == '') time_val = '00:00'; - // Note, we are using date.js utility which has better parsing abilities than the built in JS date parsing - var date = Date.parse(date_val + " " + time_val); - edxTimeStr = date.toString('yyyy-MM-ddTHH:mm'); + return new Date(date_val + " " + time_val + "Z"); } - return edxTimeStr; + else return null; } function getEdxTimeFromDateTimeInputs(date_id, time_id) { @@ -338,7 +334,7 @@ function createNewUnit(e) { e.preventDefault(); var parent = $(this).data('parent'); - var template = $(this).data('template'); + var category = $(this).data('category'); analytics.track('Created a Unit', { 'course': course_location_analytics, @@ -346,9 +342,9 @@ function createNewUnit(e) { }); - $.post('/clone_item', { + $.post('/create_item', { 'parent_location': parent, - 'template': template, + 'category': category, 'display_name': 'New Unit' }, @@ -360,39 +356,61 @@ function createNewUnit(e) { function deleteUnit(e) { e.preventDefault(); - _deleteItem($(this).parents('li.leaf')); + _deleteItem($(this).parents('li.leaf'), 'Unit'); } function deleteSubsection(e) { e.preventDefault(); - _deleteItem($(this).parents('li.branch')); + _deleteItem($(this).parents('li.branch'), 'Subsection'); } function deleteSection(e) { e.preventDefault(); - _deleteItem($(this).parents('section.branch')); + _deleteItem($(this).parents('section.branch'), 'Section'); } -function _deleteItem($el) { - if (!confirm(gettext('Are you sure you wish to delete this item. It cannot be reversed!'))) return; +function _deleteItem($el, type) { + var confirm = new CMS.Views.Prompt.Warning({ + title: gettext('Delete this ' + type + '?'), + message: gettext('Deleting this ' + type + ' is permanent and cannot be undone.'), + actions: { + primary: { + text: gettext('Yes, delete this ' + type), + click: function(view) { + view.hide(); - var id = $el.data('id'); + var id = $el.data('id'); - analytics.track('Deleted an Item', { - 'course': course_location_analytics, - 'id': id - }); - - - $.post('/delete_item', { - 'id': id, - 'delete_children': true, - 'delete_all_versions': true - }, - - function(data) { - $el.remove(); + analytics.track('Deleted an Item', { + 'course': course_location_analytics, + 'id': id + }); + + var deleting = new CMS.Views.Notification.Mini({ + title: gettext('Deleting') + '…' + }); + deleting.show(); + + $.post('/delete_item', + {'id': id, + 'delete_children': true, + 'delete_all_versions': true}, + function(data) { + $el.remove(); + deleting.hide(); + } + ); + } + }, + secondary: { + text: gettext('Cancel'), + click: function(view) { + view.hide(); + } + } + } }); + confirm.show(); } function markAsLoaded() { @@ -551,7 +569,7 @@ function saveNewSection(e) { var $saveButton = $(this).find('.new-section-name-save'); var parent = $saveButton.data('parent'); - var template = $saveButton.data('template'); + var category = $saveButton.data('category'); var display_name = $(this).find('.new-section-name').val(); analytics.track('Created a Section', { @@ -559,9 +577,9 @@ function saveNewSection(e) { 'display_name': display_name }); - $.post('/clone_item', { + $.post('/create_item', { 'parent_location': parent, - 'template': template, + 'category': category, 'display_name': display_name, }, @@ -595,7 +613,6 @@ function saveNewCourse(e) { e.preventDefault(); var $newCourse = $(this).closest('.new-course'); - var template = $(this).find('.new-course-save').data('template'); var org = $newCourse.find('.new-course-org').val(); var number = $newCourse.find('.new-course-number').val(); var display_name = $newCourse.find('.new-course-name').val(); @@ -612,7 +629,6 @@ function saveNewCourse(e) { }); $.post('/create_new_course', { - 'template': template, 'org': org, 'number': number, 'display_name': display_name @@ -646,7 +662,7 @@ function addNewSubsection(e) { var parent = $(this).parents("section.branch").data("id"); $saveButton.data('parent', parent); - $saveButton.data('template', $(this).data('template')); + $saveButton.data('category', $(this).data('category')); $newSubsection.find('.new-subsection-form').bind('submit', saveNewSubsection); $cancelButton.bind('click', cancelNewSubsection); @@ -659,7 +675,7 @@ function saveNewSubsection(e) { e.preventDefault(); var parent = $(this).find('.new-subsection-name-save').data('parent'); - var template = $(this).find('.new-subsection-name-save').data('template'); + var category = $(this).find('.new-subsection-name-save').data('category'); var display_name = $(this).find('.new-subsection-name-input').val(); analytics.track('Created a Subsection', { @@ -668,9 +684,9 @@ function saveNewSubsection(e) { }); - $.post('/clone_item', { + $.post('/create_item', { 'parent_location': parent, - 'template': template, + 'category': category, 'display_name': display_name }, @@ -734,7 +750,7 @@ function saveSetSectionScheduleDate(e) { var $thisSection = $('.courseware-section[data-id="' + id + '"]'); var html = _.template( '' + - '' + gettext("Will Release: ") + '' + + '' + gettext("Will Release:") + ' ' + gettext("<%= date %> at <%= time %> UTC") + '' + '' + diff --git a/cms/static/js/views/assets.js b/cms/static/js/views/assets.js index 224ec928fb..282aeab69c 100644 --- a/cms/static/js/views/assets.js +++ b/cms/static/js/views/assets.js @@ -9,7 +9,7 @@ function removeAsset(e){ e.preventDefault(); var that = this; - var msg = new CMS.Views.Prompt.Confirmation({ + var msg = new CMS.Views.Prompt.Warning({ title: gettext("Delete File Confirmation"), message: gettext("Are you sure you wish to delete this item. It cannot be reversed!\n\nAlso any content that links/refers to this item will no longer work (e.g. broken images and/or links)"), actions: { diff --git a/cms/static/js/views/grader-select-view.js b/cms/static/js/views/grader-select-view.js index b22e763710..a16f5fa1b7 100644 --- a/cms/static/js/views/grader-select-view.js +++ b/cms/static/js/views/grader-select-view.js @@ -81,9 +81,18 @@ CMS.Views.OverviewAssignmentGrader = Backbone.View.extend({ this.removeMenu(e); + var saving = new CMS.Views.Notification.Mini({ + title: gettext('Saving') + '…' + }); + saving.show(); + // TODO I'm not happy with this string fetch via the html for what should be an id. I'd rather use the id attr // of the CourseGradingPolicy model or null for Not Graded (NOTE, change template's if check for is-selected accordingly) - this.assignmentGrade.save('graderType', $(e.target).text()); + this.assignmentGrade.save( + 'graderType', + $(e.target).text(), + {success: function () { saving.hide(); }} + ); this.render(); } diff --git a/cms/static/js/views/settings/main_settings_view.js b/cms/static/js/views/settings/main_settings_view.js index 5550c550c0..0cbf573ba9 100644 --- a/cms/static/js/views/settings/main_settings_view.js +++ b/cms/static/js/views/settings/main_settings_view.js @@ -107,6 +107,7 @@ CMS.Views.Settings.Details = CMS.Views.ValidatingView.extend({ // to pick up when the date is typed directly in the field. datefield.change(setfield); timefield.on('changeTime', setfield); + timefield.on('input', setfield); datefield.datepicker('setDate', this.model.get(fieldName)); // timepicker doesn't let us set null, so check that we have a time diff --git a/cms/static/js/views/textbook.js b/cms/static/js/views/textbook.js index 74eaae8601..f666143c0a 100644 --- a/cms/static/js/views/textbook.js +++ b/cms/static/js/views/textbook.js @@ -241,7 +241,7 @@ CMS.Views.EditChapter = Backbone.View.extend({ asset_path: this.$("input.chapter-asset-path").val() }); var msg = new CMS.Models.FileUpload({ - title: _.template(gettext("Upload a new asset to “<%= name %>”"), + title: _.template(gettext("Upload a new PDF to “<%= name %>”"), {name: section.escape('name')}), message: "Files must be in PDF format." }); diff --git a/cms/static/sass/views/_index.scss b/cms/static/sass/views/_index.scss index ddb2ecce89..d571949cba 100644 --- a/cms/static/sass/views/_index.scss +++ b/cms/static/sass/views/_index.scss @@ -71,8 +71,13 @@ body.index { color: $white; } + .wrapper-text-welcome, .logo { + display: inline-block; + } + .logo { font-weight: 600; + margin-left: ($baseline/2); } .tagline { diff --git a/cms/static/sass/views/_unit.scss b/cms/static/sass/views/_unit.scss index 45e9823c2e..06685ad96b 100644 --- a/cms/static/sass/views/_unit.scss +++ b/cms/static/sass/views/_unit.scss @@ -747,6 +747,7 @@ body.unit { // Unit Page Sidebar .unit-settings { + .window-contents { padding: $baseline/2 $baseline; } @@ -854,6 +855,24 @@ body.unit { } .unit-location { + + // unit id + .wrapper-unit-id { + + .unit-id { + + .label { + @extend .t-title7; + margin-bottom: ($baseline/4); + color: $gray-d1; + } + + .value { + margin-bottom: 0; + } + } + } + .url { box-shadow: none; width: 100%; diff --git a/cms/templates/edit-static-page.html b/cms/templates/edit-static-page.html deleted file mode 100644 index f1b2374b46..0000000000 --- a/cms/templates/edit-static-page.html +++ /dev/null @@ -1,41 +0,0 @@ -<%inherit file="base.html" /> -<%! from django.core.urlresolvers import reverse %> -<%block name="title">Editing Static Page -<%block name="bodyclass">is-signedin course pages edit-static-page - -<%block name="content"> -
-
-
-
-
- - -
-
- - -
-
-
-
-
-

Page Settings

-
-
- - -
-
- Save - Preview -
-
-
-
-
-
- \ No newline at end of file diff --git a/cms/templates/editable_preview.html b/cms/templates/editable_preview.html deleted file mode 100644 index 731fd9b1c8..0000000000 --- a/cms/templates/editable_preview.html +++ /dev/null @@ -1,13 +0,0 @@ -
-${content} -
- Edit - Delete -
- -
-
Edit Video Component
- - SaveCancel -
-
diff --git a/cms/templates/howitworks.html b/cms/templates/howitworks.html index a791f5d1fa..e3a92aa345 100644 --- a/cms/templates/howitworks.html +++ b/cms/templates/howitworks.html @@ -11,7 +11,7 @@
## "edX Studio" should not be translated -

${_('Welcome to')}

+

${_('Welcome to')}

${_("Studio helps manage your courses online, so you can focus on teaching them")}

diff --git a/cms/templates/index.html b/cms/templates/index.html index 57921641ce..f0baef4f09 100644 --- a/cms/templates/index.html +++ b/cms/templates/index.html @@ -25,8 +25,8 @@
- - + +
diff --git a/cms/templates/js/edit-chapter.underscore b/cms/templates/js/edit-chapter.underscore index 0f9a91e3e0..34f42daa64 100644 --- a/cms/templates/js/edit-chapter.underscore +++ b/cms/templates/js/edit-chapter.underscore @@ -9,6 +9,6 @@ " value="<%= asset_path %>" type="text"> <%= gettext("upload a PDF file or provide the path to a Studio asset file") %> - + <%= gettext("delete chapter") %> diff --git a/cms/templates/overview.html b/cms/templates/overview.html index ab7788c64a..3795e9d09b 100644 --- a/cms/templates/overview.html +++ b/cms/templates/overview.html @@ -66,7 +66,8 @@

- +

@@ -83,8 +84,9 @@ Click here to set the section name
- - + +
@@ -181,7 +183,7 @@
- + ${_("New Subsection")}
diff --git a/cms/templates/unit.html b/cms/templates/unit.html index ce4f239ab1..fdff92e389 100644 --- a/cms/templates/unit.html +++ b/cms/templates/unit.html @@ -59,8 +59,8 @@ % if type == 'advanced' or len(templates) > 1: % else: - % for __, location, __ in templates: - + % for __, category, __, __ in templates: + % endfor % endif @@ -74,49 +74,60 @@ % if len(templates) > 1 or type == 'advanced':
% if type == "problem": -
- - % endif -
-
- % endif - ${_("Cancel")} -
- % endif + % endif +
+
    + % for name, category, has_markdown, boilerplate_name in sorted(templates): + % if has_markdown or type != "problem": + % if boilerplate_name is None: +
  • + + ${name} + +
    • + + % else: +
  • + + ${name} + +
    • + % endif + % endif + + %endfor +
+
+ % if type == "problem": +
+
    + % for name, category, has_markdown, boilerplate_name in sorted(templates): + % if not has_markdown: +
  • + + ${name} + +
    • + % endif + % endfor +
+
+
+ % endif + Cancel +
+ % endif % endfor @@ -160,10 +171,15 @@

${_("Unit Location")}

-
+
+

+ ${_("Unit Identifier:")} + +

+
  1. - ${section.display_name_with_default} + ${section.display_name_with_default}
    1. diff --git a/cms/templates/widgets/units.html b/cms/templates/widgets/units.html index 5ac05e79eb..62c1fb62d7 100644 --- a/cms/templates/widgets/units.html +++ b/cms/templates/widgets/units.html @@ -34,7 +34,7 @@ This def will enumerate through a passed in subsection and list all of the units
      2. % endfor
    2. - + New Unit
      3. diff --git a/cms/urls.py b/cms/urls.py index 6db07dc2ed..711742d89f 100644 --- a/cms/urls.py +++ b/cms/urls.py @@ -17,7 +17,7 @@ urlpatterns = ('', # nopep8 url(r'^preview_component/(?P.*?)$', 'contentstore.views.preview_component', name='preview_component'), url(r'^save_item$', 'contentstore.views.save_item', name='save_item'), url(r'^delete_item$', 'contentstore.views.delete_item', name='delete_item'), - url(r'^clone_item$', 'contentstore.views.clone_item', name='clone_item'), + url(r'^create_item$', 'contentstore.views.create_item', name='create_item'), url(r'^create_draft$', 'contentstore.views.create_draft', name='create_draft'), url(r'^publish_draft$', 'contentstore.views.publish_draft', name='publish_draft'), url(r'^unpublish_unit$', 'contentstore.views.unpublish_unit', name='unpublish_unit'), diff --git a/common/djangoapps/external_auth/models.py b/common/djangoapps/external_auth/models.py index 6c2f38d8b3..b40e427434 100644 --- a/common/djangoapps/external_auth/models.py +++ b/common/djangoapps/external_auth/models.py @@ -4,9 +4,9 @@ WE'RE USING MIGRATIONS! If you make changes to this model, be sure to create an appropriate migration file and check it in at the same time as your model changes. To do that, -1. Go to the mitx dir -2. django-admin.py schemamigration student --auto --settings=lms.envs.dev --pythonpath=. description_of_your_change -3. Add the migration file created in mitx/common/djangoapps/external_auth/migrations/ +1. Go to the edx-platform dir +2. ./manage.py lms schemamigration student --auto description_of_your_change +3. Add the migration file created in edx-platform/common/djangoapps/external_auth/migrations/ """ from django.db import models diff --git a/common/djangoapps/heartbeat/views.py b/common/djangoapps/heartbeat/views.py index d7c3a32192..0cee7116b4 100644 --- a/common/djangoapps/heartbeat/views.py +++ b/common/djangoapps/heartbeat/views.py @@ -1,16 +1,18 @@ import json from datetime import datetime +from pytz import UTC from django.http import HttpResponse from xmodule.modulestore.django import modulestore from dogapi import dog_stats_api + @dog_stats_api.timed('edxapp.heartbeat') def heartbeat(request): """ Simple view that a loadbalancer can check to verify that the app is up """ output = { - 'date': datetime.now().isoformat(), + 'date': datetime.now(UTC).isoformat(), 'courses': [course.location.url() for course in modulestore().get_courses()], } return HttpResponse(json.dumps(output, indent=4)) diff --git a/common/djangoapps/static_replace/__init__.py b/common/djangoapps/static_replace/__init__.py index b73a658c5f..9e50d73b26 100644 --- a/common/djangoapps/static_replace/__init__.py +++ b/common/djangoapps/static_replace/__init__.py @@ -43,6 +43,35 @@ def try_staticfiles_lookup(path): return url +def replace_jump_to_id_urls(text, course_id, jump_to_id_base_url): + """ + This will replace a link to another piece of courseware to a 'jump_to' + URL that will redirect to the right place in the courseware + + NOTE: This is similar to replace_course_urls in terms of functionality + but it is intended to be used when we only have a 'id' that the + course author provides. This is much more helpful when using + Studio authored courses since they don't need to know the path. This + is also durable with respect to item moves. + + text: The content over which to perform the subtitutions + course_id: The course_id in which this rewrite happens + jump_to_id_base_url: + A app-tier (e.g. LMS) absolute path to the base of the handler that will perform the + redirect. e.g. /courses////jump_to_id. NOTE the will be appended to + the end of this URL at re-write time + + output: after the link rewriting rules are applied + """ + + def replace_jump_to_id_url(match): + quote = match.group('quote') + rest = match.group('rest') + return "".join([quote, jump_to_id_base_url + rest, quote]) + + return re.sub(_url_replace_regex('/jump_to_id/'), replace_jump_to_id_url, text) + + def replace_course_urls(text, course_id): """ Replace /course/$stuff urls with /courses/$course_id/$stuff urls @@ -53,7 +82,6 @@ def replace_course_urls(text, course_id): returns: text with the links replaced """ - def replace_course_url(match): quote = match.group('quote') rest = match.group('rest') diff --git a/common/djangoapps/student/management/commands/pearson_transfer.py b/common/djangoapps/student/management/commands/pearson_transfer.py index 75716c7443..39e1e7468e 100644 --- a/common/djangoapps/student/management/commands/pearson_transfer.py +++ b/common/djangoapps/student/management/commands/pearson_transfer.py @@ -20,7 +20,7 @@ class Command(BaseCommand): files and then uploads over SFTP to Pearson and stuffs the entry in an S3 bucket for archive purposes. - Usage: django-admin.py pearson-transfer --mode [import|export|both] + Usage: ./manage.py pearson-transfer --mode [import|export|both] """ option_list = BaseCommand.option_list + ( diff --git a/common/djangoapps/student/models.py b/common/djangoapps/student/models.py index af93c34317..97d7e8b028 100644 --- a/common/djangoapps/student/models.py +++ b/common/djangoapps/student/models.py @@ -6,9 +6,9 @@ Migration Notes If you make changes to this model, be sure to create an appropriate migration file and check it in at the same time as your model changes. To do that, -1. Go to the mitx dir -2. django-admin.py schemamigration student --auto --settings=lms.envs.dev --pythonpath=. description_of_your_change -3. Add the migration file created in mitx/common/djangoapps/student/migrations/ +1. Go to the edx-platform dir +2. ./manage.py lms schemamigration student --auto description_of_your_change +3. Add the migration file created in edx-platform/common/djangoapps/student/migrations/ """ from datetime import datetime import hashlib @@ -69,30 +69,33 @@ class UserProfile(models.Model): location = models.CharField(blank=True, max_length=255, db_index=True) # Optional demographic data we started capturing from Fall 2012 - this_year = datetime.now().year + this_year = datetime.now(UTC).year VALID_YEARS = range(this_year, this_year - 120, -1) year_of_birth = models.IntegerField(blank=True, null=True, db_index=True) GENDER_CHOICES = (('m', 'Male'), ('f', 'Female'), ('o', 'Other')) - gender = models.CharField(blank=True, null=True, max_length=6, db_index=True, - choices=GENDER_CHOICES) + gender = models.CharField( + blank=True, null=True, max_length=6, db_index=True, choices=GENDER_CHOICES + ) # [03/21/2013] removed these, but leaving comment since there'll still be # p_se and p_oth in the existing data in db. # ('p_se', 'Doctorate in science or engineering'), # ('p_oth', 'Doctorate in another field'), - LEVEL_OF_EDUCATION_CHOICES = (('p', 'Doctorate'), - ('m', "Master's or professional degree"), - ('b', "Bachelor's degree"), - ('a', "Associate's degree"), - ('hs', "Secondary/high school"), - ('jhs', "Junior secondary/junior high/middle school"), - ('el', "Elementary/primary school"), - ('none', "None"), - ('other', "Other")) + LEVEL_OF_EDUCATION_CHOICES = ( + ('p', 'Doctorate'), + ('m', "Master's or professional degree"), + ('b', "Bachelor's degree"), + ('a', "Associate's degree"), + ('hs', "Secondary/high school"), + ('jhs', "Junior secondary/junior high/middle school"), + ('el', "Elementary/primary school"), + ('none', "None"), + ('other', "Other") + ) level_of_education = models.CharField( - blank=True, null=True, max_length=6, db_index=True, - choices=LEVEL_OF_EDUCATION_CHOICES - ) + blank=True, null=True, max_length=6, db_index=True, + choices=LEVEL_OF_EDUCATION_CHOICES + ) mailing_address = models.TextField(blank=True, null=True) goals = models.TextField(blank=True, null=True) allow_certificate = models.BooleanField(default=1) @@ -307,18 +310,18 @@ class TestCenterUserForm(ModelForm): ACCOMMODATION_REJECTED_CODE = 'NONE' ACCOMMODATION_CODES = ( - (ACCOMMODATION_REJECTED_CODE, 'No Accommodation Granted'), - ('EQPMNT', 'Equipment'), - ('ET12ET', 'Extra Time - 1/2 Exam Time'), - ('ET30MN', 'Extra Time - 30 Minutes'), - ('ETDBTM', 'Extra Time - Double Time'), - ('SEPRMM', 'Separate Room'), - ('SRREAD', 'Separate Room and Reader'), - ('SRRERC', 'Separate Room and Reader/Recorder'), - ('SRRECR', 'Separate Room and Recorder'), - ('SRSEAN', 'Separate Room and Service Animal'), - ('SRSGNR', 'Separate Room and Sign Language Interpreter'), - ) + (ACCOMMODATION_REJECTED_CODE, 'No Accommodation Granted'), + ('EQPMNT', 'Equipment'), + ('ET12ET', 'Extra Time - 1/2 Exam Time'), + ('ET30MN', 'Extra Time - 30 Minutes'), + ('ETDBTM', 'Extra Time - Double Time'), + ('SEPRMM', 'Separate Room'), + ('SRREAD', 'Separate Room and Reader'), + ('SRRERC', 'Separate Room and Reader/Recorder'), + ('SRRECR', 'Separate Room and Recorder'), + ('SRSEAN', 'Separate Room and Service Animal'), + ('SRSGNR', 'Separate Room and Sign Language Interpreter'), +) ACCOMMODATION_CODE_DICT = {code: name for (code, name) in ACCOMMODATION_CODES} @@ -572,7 +575,6 @@ class TestCenterRegistrationForm(ModelForm): return code - def get_testcenter_registration(user, course_id, exam_series_code): try: tcu = TestCenterUser.objects.get(user=user) diff --git a/common/djangoapps/student/views.py b/common/djangoapps/student/views.py index 398a3f6efc..553643bde7 100644 --- a/common/djangoapps/student/views.py +++ b/common/djangoapps/student/views.py @@ -111,9 +111,9 @@ def get_date_for_press(publish_date): # strip off extra months, and just use the first: date = re.sub(multimonth_pattern, ", ", publish_date) if re.search(day_pattern, date): - date = datetime.datetime.strptime(date, "%B %d, %Y") + date = datetime.datetime.strptime(date, "%B %d, %Y").replace(tzinfo=UTC) else: - date = datetime.datetime.strptime(date, "%B, %Y") + date = datetime.datetime.strptime(date, "%B, %Y").replace(tzinfo=UTC) return date @@ -1100,7 +1100,7 @@ def confirm_email_change(request, key): meta = up.get_meta() if 'old_emails' not in meta: meta['old_emails'] = [] - meta['old_emails'].append([user.email, datetime.datetime.now().isoformat()]) + meta['old_emails'].append([user.email, datetime.datetime.now(UTC).isoformat()]) up.set_meta(meta) up.save() # Send it to the old email... @@ -1198,7 +1198,7 @@ def accept_name_change_by_id(id): meta = up.get_meta() if 'old_names' not in meta: meta['old_names'] = [] - meta['old_names'].append([up.name, pnc.rationale, datetime.datetime.now().isoformat()]) + meta['old_names'].append([up.name, pnc.rationale, datetime.datetime.now(UTC).isoformat()]) up.set_meta(meta) up.name = pnc.new_name diff --git a/common/djangoapps/terrain/course_helpers.py b/common/djangoapps/terrain/course_helpers.py index 27bf95099d..c62b1a1e79 100644 --- a/common/djangoapps/terrain/course_helpers.py +++ b/common/djangoapps/terrain/course_helpers.py @@ -12,7 +12,6 @@ from django.contrib.sessions.middleware import SessionMiddleware from student.models import CourseEnrollment from xmodule.modulestore.django import modulestore from xmodule.contentstore.django import contentstore -from xmodule.templates import update_templates from urllib import quote_plus @@ -84,5 +83,4 @@ def clear_courses(): # from the bash shell to drop it: # $ mongo test_xmodule --eval "db.dropDatabase()" modulestore().collection.drop() - update_templates(modulestore('direct')) contentstore().fs_files.drop() diff --git a/common/djangoapps/terrain/steps.py b/common/djangoapps/terrain/steps.py index bd0b448e11..cfa8de15b7 100644 --- a/common/djangoapps/terrain/steps.py +++ b/common/djangoapps/terrain/steps.py @@ -129,6 +129,13 @@ def should_have_link_with_id_and_text(step, link_id, text): assert_equals(link.text, text) +@step(r'should see a link to "([^"]*)" with the text "([^"]*)"$') +def should_have_link_with_path_and_text(step, path, text): + link = world.browser.find_link_by_text(text) + assert len(link) > 0 + assert_equals(link.first["href"], django_url(path)) + + @step(r'should( not)? see "(.*)" (?:somewhere|anywhere) (?:in|on) (?:the|this) page') def should_see_in_the_page(step, doesnt_appear, text): if doesnt_appear: diff --git a/common/djangoapps/tests.py b/common/djangoapps/tests.py index 8e78ee7f37..4a61a106d4 100644 --- a/common/djangoapps/tests.py +++ b/common/djangoapps/tests.py @@ -23,15 +23,15 @@ class TestXmoduleModfiers(ModuleStoreTestCase): number='313', display_name='histogram test') section = ItemFactory.create( parent_location=course.location, display_name='chapter hist', - template='i4x://edx/templates/chapter/Empty') + category='chapter') problem = ItemFactory.create( parent_location=section.location, display_name='problem hist 1', - template='i4x://edx/templates/problem/Blank_Common_Problem') + category='problem') problem.has_score = False # don't trip trying to retrieve db data late_problem = ItemFactory.create( parent_location=section.location, display_name='problem hist 2', - template='i4x://edx/templates/problem/Blank_Common_Problem') + category='problem') late_problem.lms.start = datetime.datetime.now(UTC) + datetime.timedelta(days=32) late_problem.has_score = False diff --git a/common/djangoapps/xmodule_modifiers.py b/common/djangoapps/xmodule_modifiers.py index 7a74e75591..dd40b5139d 100644 --- a/common/djangoapps/xmodule_modifiers.py +++ b/common/djangoapps/xmodule_modifiers.py @@ -42,6 +42,28 @@ def wrap_xmodule(get_html, module, template, context=None): return _get_html +def replace_jump_to_id_urls(get_html, course_id, jump_to_id_base_url): + """ + This will replace a link between courseware in the format + /jump_to/ with a URL for a page that will correctly redirect + This is similar to replace_course_urls, but much more flexible and + durable for Studio authored courses. See more comments in static_replace.replace_jump_to_urls + + course_id: The course_id in which this rewrite happens + jump_to_id_base_url: + A app-tier (e.g. LMS) absolute path to the base of the handler that will perform the + redirect. e.g. /courses////jump_to_id. NOTE the will be appended to + the end of this URL at re-write time + + output: a wrapped get_html() function pointer, which, when called, will apply the + rewrite rules + """ + @wraps(get_html) + def _get_html(): + return static_replace.replace_jump_to_id_urls(get_html(), course_id, jump_to_id_base_url) + return _get_html + + def replace_course_urls(get_html, course_id): """ Updates the supplied module with a new get_html function that wraps @@ -89,6 +111,21 @@ def grade_histogram(module_id): return grades +def save_module(get_html, module): + """ + Updates the given get_html function for the given module to save the fields + after rendering. + """ + @wraps(get_html) + def _get_html(): + """Cache the rendered output, save, then return the output.""" + rendered_html = get_html() + module.save() + return rendered_html + + return _get_html + + def add_histogram(get_html, module, user): """ Updates the supplied module with a new get_html function that wraps @@ -120,7 +157,7 @@ def add_histogram(get_html, module, user): # doesn't like symlinks) filepath = filename data_dir = osfs.root_path.rsplit('/')[-1] - giturl = getattr(module.lms, 'giturl', '') or 'https://github.com/MITx' + giturl = module.lms.giturl or 'https://github.com/MITx' edit_link = "%s/%s/tree/master/%s" % (giturl, data_dir, filepath) else: edit_link = False diff --git a/common/lib/capa/capa/capa_problem.py b/common/lib/capa/capa/capa_problem.py index 2c813f49d5..c2bdeadc21 100644 --- a/common/lib/capa/capa/capa_problem.py +++ b/common/lib/capa/capa/capa_problem.py @@ -32,6 +32,8 @@ import capa.xqueue_interface as xqueue_interface import capa.responsetypes as responsetypes from capa.safe_exec import safe_exec +from pytz import UTC + # dict of tagname, Response Class -- this should come from auto-registering response_tag_dict = dict([(x.response_tag, x) for x in responsetypes.__all__]) @@ -42,13 +44,22 @@ solution_tags = ['solution'] response_properties = ["codeparam", "responseparam", "answer", "openendedparam"] # special problem tags which should be turned into innocuous HTML -html_transforms = {'problem': {'tag': 'div'}, - 'text': {'tag': 'span'}, - 'math': {'tag': 'span'}, - } +html_transforms = { + 'problem': {'tag': 'div'}, + 'text': {'tag': 'span'}, + 'math': {'tag': 'span'}, +} # These should be removed from HTML output, including all subelements -html_problem_semantics = ["codeparam", "responseparam", "answer", "script", "hintgroup", "openendedparam", "openendedrubric"] +html_problem_semantics = [ + "codeparam", + "responseparam", + "answer", + "script", + "hintgroup", + "openendedparam", + "openendedrubric" +] log = logging.getLogger(__name__) @@ -242,11 +253,15 @@ class LoncapaProblem(object): return None # Get a list of timestamps of all queueing requests, then convert it to a DateTime object - queuetime_strs = [self.correct_map.get_queuetime_str(answer_id) - for answer_id in self.correct_map - if self.correct_map.is_queued(answer_id)] - queuetimes = [datetime.strptime(qt_str, xqueue_interface.dateformat) - for qt_str in queuetime_strs] + queuetime_strs = [ + self.correct_map.get_queuetime_str(answer_id) + for answer_id in self.correct_map + if self.correct_map.is_queued(answer_id) + ] + queuetimes = [ + datetime.strptime(qt_str, xqueue_interface.dateformat).replace(tzinfo=UTC) + for qt_str in queuetime_strs + ] return max(queuetimes) @@ -404,10 +419,16 @@ class LoncapaProblem(object): # open using ModuleSystem OSFS filestore ifp = self.system.filestore.open(filename) except Exception as err: - log.warning('Error %s in problem xml include: %s' % ( - err, etree.tostring(inc, pretty_print=True))) - log.warning('Cannot find file %s in %s' % ( - filename, self.system.filestore)) + log.warning( + 'Error %s in problem xml include: %s' % ( + err, etree.tostring(inc, pretty_print=True) + ) + ) + log.warning( + 'Cannot find file %s in %s' % ( + filename, self.system.filestore + ) + ) # if debugging, don't fail - just log error # TODO (vshnayder): need real error handling, display to users if not self.system.get('DEBUG'): @@ -418,8 +439,11 @@ class LoncapaProblem(object): # read in and convert to XML incxml = etree.XML(ifp.read()) except Exception as err: - log.warning('Error %s in problem xml include: %s' % ( - err, etree.tostring(inc, pretty_print=True))) + log.warning( + 'Error %s in problem xml include: %s' % ( + err, etree.tostring(inc, pretty_print=True) + ) + ) log.warning('Cannot parse XML in %s' % (filename)) # if debugging, don't fail - just log error # TODO (vshnayder): same as above @@ -579,8 +603,9 @@ class LoncapaProblem(object): # let each Response render itself if problemtree in self.responders: overall_msg = self.correct_map.get_overall_message() - return self.responders[problemtree].render_html(self._extract_html, - response_msg=overall_msg) + return self.responders[problemtree].render_html( + self._extract_html, response_msg=overall_msg + ) # let each custom renderer render itself: if problemtree.tag in customrender.registry.registered_tags(): @@ -628,9 +653,10 @@ class LoncapaProblem(object): answer_id = 1 input_tags = inputtypes.registry.registered_tags() - inputfields = tree.xpath("|".join(['//' + response.tag + '[@id=$id]//' + x - for x in (input_tags + solution_tags)]), - id=response_id_str) + inputfields = tree.xpath( + "|".join(['//' + response.tag + '[@id=$id]//' + x for x in (input_tags + solution_tags)]), + id=response_id_str + ) # assign one answer_id for each input type or solution type for entry in inputfields: diff --git a/common/lib/capa/capa/correctmap.py b/common/lib/capa/capa/correctmap.py index 950cd199fc..e50be92152 100644 --- a/common/lib/capa/capa/correctmap.py +++ b/common/lib/capa/capa/correctmap.py @@ -37,23 +37,27 @@ class CorrectMap(object): return self.cmap.__iter__() # See the documentation for 'set_dict' for the use of kwargs - def set(self, - answer_id=None, - correctness=None, - npoints=None, - msg='', - hint='', - hintmode=None, - queuestate=None, **kwargs): + def set( + self, + answer_id=None, + correctness=None, + npoints=None, + msg='', + hint='', + hintmode=None, + queuestate=None, + **kwargs + ): if answer_id is not None: - self.cmap[str(answer_id)] = {'correctness': correctness, - 'npoints': npoints, - 'msg': msg, - 'hint': hint, - 'hintmode': hintmode, - 'queuestate': queuestate, - } + self.cmap[str(answer_id)] = { + 'correctness': correctness, + 'npoints': npoints, + 'msg': msg, + 'hint': hint, + 'hintmode': hintmode, + 'queuestate': queuestate, + } def __repr__(self): return repr(self.cmap) diff --git a/common/lib/capa/capa/inputtypes.py b/common/lib/capa/capa/inputtypes.py index 9bb72ad4e1..29800a211b 100644 --- a/common/lib/capa/capa/inputtypes.py +++ b/common/lib/capa/capa/inputtypes.py @@ -460,10 +460,10 @@ class JSInput(InputTypeBase): DO NOT USE! HAS NOT BEEN TESTED BEYOND 700X PROBLEMS, AND MAY CHANGE IN BACKWARDS-INCOMPATIBLE WAYS. Inputtype for general javascript inputs. Intended to be used with - customresponse. + customresponse. Loads in a sandboxed iframe to help prevent css and js conflicts between - frame and top-level window. - + frame and top-level window. + iframe sandbox whitelist: - allow-scripts - allow-popups @@ -474,9 +474,9 @@ class JSInput(InputTypeBase): window elements. Example: - See the documentation in the /doc/public folder for more information. @@ -500,7 +500,7 @@ class JSInput(InputTypeBase): Attribute('width', "400"), # iframe width Attribute('height', "300")] # iframe height - + def _extra_context(self): context = { @@ -510,11 +510,12 @@ class JSInput(InputTypeBase): return context - + registry.register(JSInput) #----------------------------------------------------------------------------- + class TextLine(InputTypeBase): """ A text line input. Can do math preview if "math"="1" is specified. @@ -1368,3 +1369,209 @@ class AnnotationInput(InputTypeBase): return extra_context registry.register(AnnotationInput) + + +class ChoiceTextGroup(InputTypeBase): + """ + Groups of radiobutton/checkboxes with text inputs. + + Examples: + RadioButton problem + + + A person rolls a standard die 100 times and records the results. + On the first roll they received a "1". Given this information + select the correct choice and fill in numbers to make it accurate. + + + + The lowest number rolled was: + and the highest number rolled was: + . + The lowest number rolled was + and there is not enough information to determine the highest number rolled. + + There is not enough information to determine the lowest + number rolled, and the highest number rolled was: + . + + + + + + CheckboxProblem: + + + A person randomly selects 100 times, with replacement, from the list of numbers \(\sqrt{2}\) , 2, 3, 4 ,5 ,6 + and records the results. The first number they pick is \(\sqrt{2}\) Given this information + select the correct choices and fill in numbers to make them accurate. + + + + + The lowest number selected was + + + The highest number selected was . + + There is not enough information given to determine the highest number + which was selected. + + There is not enough information given to determine the lowest number + selected. + + + + + + In the preceding examples the is used to generate a textinput html element + in the problem's display. Since it is inside of an incorrect choice, no answer given + for it will be correct, and thus specifying an answer for it is not needed. + """ + template = "choicetext.html" + tags = ['radiotextgroup', 'checkboxtextgroup'] + + def setup(self): + """ + Performs setup for the initial rendering of the problem. + `self.html_input_type` determines whether this problem is displayed + with radiobuttons or checkboxes + + If the initial value of `self.value` is '' change it to {} so that + the template has an empty dictionary to work with. + + sets the value of self.choices to be equal to the return value of + `self.extract_choices` + """ + self.text_input_values = {} + if self.tag == 'radiotextgroup': + self.html_input_type = "radio" + elif self.tag == 'checkboxtextgroup': + self.html_input_type = "checkbox" + else: + raise Exception("ChoiceGroup: unexpected tag {0}".format(self.tag)) + + if self.value == '': + # Make `value` an empty dictionary, if it currently has an empty + # value. This is necessary because the template expects a + # dictionary. + self.value = {} + self.choices = self.extract_choices(self.xml) + + @classmethod + def get_attributes(cls): + """ + Returns a list of `Attribute` for this problem type + """ + return [ + Attribute("show_correctness", "always"), + Attribute("submitted_message", "Answer received.") + ] + + def _extra_context(self): + """ + Returns a dictionary of extra content necessary for rendering this InputType. + + `input_type` is either 'radio' or 'checkbox' indicating whether the choices for + this problem will have radiobuttons or checkboxes. + """ + return { + 'input_type': self.html_input_type, + 'choices': self.choices + } + + @staticmethod + def extract_choices(element): + """ + Extracts choices from the xml for this problem type. + If we have xml that is as follows(choice names will have been assigned + by now) + + + The number + + Is the mean of the list. + + " + + "tag; got {0} instead".format(choice.tag) + ) + + components = [] + choice_text = '' + if choice.text is not None: + choice_text += choice.text + # Initialize our dict for the next content + adder = { + 'type': 'text', + 'contents': choice_text, + 'tail_text': '', + 'value': '' + } + components.append(adder) + + for elt in choice: + # for elements in the choice e.g. + adder = { + 'type': 'text', + 'contents': '', + 'tail_text': '', + 'value': '' + } + tag_type = elt.tag + # If the current `elt` is a set the + # `adder`type to 'numtolerance_input', and 'contents' to + # the `elt`'s name. + # Treat decoy_inputs and numtolerance_inputs the same in order + # to prevent students from reading the Html and figuring out + # which inputs are valid + if tag_type in ('numtolerance_input', 'decoy_input'): + # We set this to textinput, so that we get a textinput html + # element. + adder['type'] = 'textinput' + adder['contents'] = elt.get('name') + else: + adder['contents'] = elt.text + + # Add any tail text("is the mean" in the example) + adder['tail_text'] = elt.tail if elt.tail else '' + components.append(adder) + + # Add the tuple for the current choice to the list of choices + choices.append((choice.get("name"), components)) + return choices + +registry.register(ChoiceTextGroup) diff --git a/common/lib/capa/capa/responsetypes.py b/common/lib/capa/capa/responsetypes.py index 3762c21976..03c82ea218 100644 --- a/common/lib/capa/capa/responsetypes.py +++ b/common/lib/capa/capa/responsetypes.py @@ -33,6 +33,7 @@ from shapely.geometry import Point, MultiPoint from calc import evaluator, UndefinedVariable from . import correctmap from datetime import datetime +from pytz import UTC from .util import * from lxml import etree from lxml.html.soupparser import fromstring as fromstring_bs # uses Beautiful Soup!!! FIXME? @@ -1365,9 +1366,11 @@ class CodeResponse(LoncapaResponse): # Note that submission can be a file submission = student_answers[self.answer_id] except Exception as err: - log.error('Error in CodeResponse %s: cannot get student answer for %s;' - ' student_answers=%s' % - (err, self.answer_id, convert_files_to_filenames(student_answers))) + log.error( + 'Error in CodeResponse %s: cannot get student answer for %s;' + ' student_answers=%s' % + (err, self.answer_id, convert_files_to_filenames(student_answers)) + ) raise Exception(err) # We do not support xqueue within Studio. @@ -1381,19 +1384,20 @@ class CodeResponse(LoncapaResponse): #------------------------------------------------------------ qinterface = self.system.xqueue['interface'] - qtime = datetime.strftime(datetime.now(), xqueue_interface.dateformat) + qtime = datetime.strftime(datetime.now(UTC), xqueue_interface.dateformat) anonymous_student_id = self.system.anonymous_student_id # Generate header - queuekey = xqueue_interface.make_hashkey(str(self.system.seed) + qtime + - anonymous_student_id + - self.answer_id) + queuekey = xqueue_interface.make_hashkey( + str(self.system.seed) + qtime + anonymous_student_id + self.answer_id + ) callback_url = self.system.xqueue['construct_callback']() xheader = xqueue_interface.make_xheader( lms_callback_url=callback_url, lms_key=queuekey, - queue_name=self.queue_name) + queue_name=self.queue_name + ) # Generate body if is_list_of_files(submission): @@ -1406,9 +1410,10 @@ class CodeResponse(LoncapaResponse): # Metadata related to the student submission revealed to the external # grader - student_info = {'anonymous_student_id': anonymous_student_id, - 'submission_time': qtime, - } + student_info = { + 'anonymous_student_id': anonymous_student_id, + 'submission_time': qtime, + } contents.update({'student_info': json.dumps(student_info)}) # Submit request. When successful, 'msg' is the prior length of the @@ -2097,6 +2102,333 @@ class AnnotationResponse(LoncapaResponse): return option_ids[0] return None + +class ChoiceTextResponse(LoncapaResponse): + """ + Allows for multiple choice responses with text inputs + Desired semantics match those of NumericalResponse and + ChoiceResponse. + """ + + response_tag = 'choicetextresponse' + max_inputfields = 1 + allowed_inputfields = ['choicetextgroup', + 'checkboxtextgroup', + 'radiotextgroup' + ] + + def setup_response(self): + """ + Sets up three dictionaries for use later: + `correct_choices`: These are the correct binary choices(radio/checkbox) + `correct_inputs`: These are the numerical/string answers for required + inputs. + `answer_values`: This is a dict, keyed by the name of the binary choice + which contains the correct answers for the text inputs separated by + commas e.g. "1, 0.5" + + `correct_choices` and `correct_inputs` are used for grading the problem + and `answer_values` is used for displaying correct answers. + + """ + context = self.context + self.correct_choices = {} + self.assign_choice_names() + self.correct_inputs = {} + self.answer_values = {self.answer_id: []} + correct_xml = self.xml.xpath('//*[@id=$id]//choice[@correct="true"]', + id=self.xml.get('id')) + for node in correct_xml: + # For each correct choice, set the `parent_name` to the + # current choice's name + parent_name = node.get('name') + # Add the name of the correct binary choice to the + # correct choices list as a key. The value is not important. + self.correct_choices[parent_name] = {'answer': ''} + # Add the name of the parent to the list of correct answers + self.answer_values[self.answer_id].append(parent_name) + answer_list = [] + # Loop over elements inside of the correct choices + for child in node: + answer = child.get('answer', None) + if not answer: + # If the question creator does not specify an answer for a + # inside of a correct choice, raise an error + raise LoncapaProblemError( + "Answer not provided for numtolerance_input" + ) + # Contextualize the answer to allow script generated answers. + answer = contextualize_text(answer, context) + input_name = child.get('name') + # Contextualize the tolerance to value. + tolerance = contextualize_text( + child.get('tolerance', '0'), + context + ) + # Add the answer and tolerance information for the current + # numtolerance_input to `correct_inputs` + self.correct_inputs[input_name] = { + 'answer': answer, + 'tolerance': tolerance + } + # Add the correct answer for this input to the list for show + answer_list.append(answer) + # Turn the list of numtolerance_input answers into a comma separated string. + self.answer_values[parent_name] = ', '.join(answer_list) + # Turn correct choices into a set. Allows faster grading. + self.correct_choices = set(self.correct_choices.keys()) + + def assign_choice_names(self): + """ + Initialize name attributes in and tags + for this response. + + Example: + Assuming for simplicity that `self.answer_id` = '1_2_1' + + Before the function is called `self.xml` = + + + The number + + Is the mean of the list. + + + False demonstration choice + + + + After this is called the choices and numtolerance_inputs will have a name + attribute initialized and self.xml will be: + + + + The number + + Is the mean of the list. + + + # "bc" is appended at the end to indicate that this is a + # binary choice as opposed to a numtolerance_input, this convention + # is used when grading the problem + choice.set( + "name", + self.answer_id + "_choiceinput_" + str(index) + "bc" + ) + # Set Name attributes for elements + # Look for all inside this choice. + numtolerance_inputs = choice.findall('numtolerance_input') + # Look for all inside this choice + decoys = choice.findall('decoy_input') + # would only be used in choices which do not contain + # + inputs = numtolerance_inputs if numtolerance_inputs else decoys + # Give each input inside of the choice a name combining + # The ordinality of the choice, and the ordinality of the input + # within that choice e.g. 1_2_1_choiceinput_0_numtolerance_input_1 + for ind, child in enumerate(inputs): + child.set( + "name", + self.answer_id + "_choiceinput_" + str(index) + + "_numtolerance_input_" + str(ind) + ) + + def get_score(self, student_answers): + """ + Returns a `CorrectMap` showing whether `student_answers` are correct. + + `student_answers` contains keys for binary inputs(radiobutton, + checkbox) and numerical inputs. Keys ending with 'bc' are binary + choice inputs otherwise they are text fields. + + This method first separates the two + types of answers and then grades them in separate methods. + + The student is only correct if they have both the binary inputs and + numerical inputs correct. + """ + answer_dict = student_answers.get(self.answer_id, "") + binary_choices, numtolerance_inputs = self._split_answers_dict(answer_dict) + # Check the binary choices first. + choices_correct = self._check_student_choices(binary_choices) + inputs_correct = self._check_student_inputs(numtolerance_inputs) + # Only return correct if the student got both the binary + # and numtolerance_inputs are correct + correct = choices_correct and inputs_correct + + return CorrectMap( + self.answer_id, + 'correct' if correct else 'incorrect' + ) + + def get_answers(self): + """ + Returns a dictionary containing the names of binary choices as keys + and a string of answers to any numtolerance_inputs which they may have + e.g {choice_1bc : "answer1, answer2", choice_2bc : ""} + """ + return self.answer_values + + def _split_answers_dict(self, a_dict): + """ + Returns two dicts: + `binary_choices` : dictionary {input_name: input_value} for + the binary choices which the student selected. + and + `numtolerance_choices` : a dictionary {input_name: input_value} + for the numtolerance_inputs inside of choices which were selected + + Determines if an input is inside of a binary input by looking at + the beginning of it's name. + + For example. If a binary_choice was named '1_2_1_choiceinput_0bc' + All of the numtolerance_inputs in it would have an idea that begins + with '1_2_1_choice_input_0_numtolerance_input' + + Splits the name of the numtolerance_input at the occurence of + '_numtolerance_input_' and appends 'bc' to the end to get the name + of the choice it is contained in. + + Example: + `a_dict` = { + '1_2_1_choiceinput_0bc': '1_2_1_choiceinput_0bc', + '1_2_1_choiceinput_0_numtolerance_input_0': '1', + '1_2_1_choiceinput_0_numtolerance_input_1': '2' + '1_2_1_choiceinput_1_numtolerance_input_0': '3' + } + + In this case, the binary choice is '1_2_1_choiceinput_0bc', and + the numtolerance_inputs associated with it are + '1_2_1_choiceinput_0_numtolerance_input_0', and + '1_2_1_choiceinput_0_numtolerance_input_1'. + + so the two return dictionaries would be + `binary_choices` = {'1_2_1_choiceinput_0bc': '1_2_1_choiceinput_0bc'} + and + `numtolerance_choices` ={ + '1_2_1_choiceinput_0_numtolerance_input_0': '1', + '1_2_1_choiceinput_0_numtolerance_input_1': '2' + } + + The entry '1_2_1_choiceinput_1_numtolerance_input_0': '3' is discarded + because it was not inside of a selected binary choice, and no validation + should be performed on numtolerance_inputs inside of non-selected choices. + """ + + # Initialize the two dictionaries that are returned + numtolerance_choices = {} + binary_choices = {} + + # `selected_choices` is a list of binary choices which were "checked/selected" + # when the student submitted the problem. + # Keys in a_dict ending with 'bc' refer to binary choices. + selected_choices = [key for key in a_dict if key.endswith("bc")] + for key in selected_choices: + binary_choices[key] = a_dict[key] + + # Convert the name of a numtolerance_input into the name of the binary + # choice that it is contained within, and append it to the list if + # the numtolerance_input's parent binary_choice is contained in + # `selected_choices`. + selected_numtolerance_inputs = [ + key for key in a_dict if key.partition("_numtolerance_input_")[0] + "bc" + in selected_choices + ] + + for key in selected_numtolerance_inputs: + numtolerance_choices[key] = a_dict[key] + + return (binary_choices, numtolerance_choices) + + def _check_student_choices(self, choices): + """ + Compares student submitted checkbox/radiobutton answers against + the correct answers. Returns True or False. + + True if all of the correct choices are selected and no incorrect + choices are selected. + """ + student_choices = set(choices) + required_selected = len(self.correct_choices - student_choices) == 0 + no_extra_selected = len(student_choices - self.correct_choices) == 0 + correct = required_selected and no_extra_selected + return correct + + def _check_student_inputs(self, numtolerance_inputs): + """ + Compares student submitted numerical answers against the correct + answers and tolerances. + + `numtolerance_inputs` is a dictionary {answer_name : answer_value} + + Performs numerical validation by means of calling + `compare_with_tolerance()` on all of `numtolerance_inputs` + + Performs a call to `compare_with_tolerance` even on values for + decoy_inputs. This is used to validate their numericality and + raise an error if the student entered a non numerical expression. + + Returns True if and only if all student inputs are correct. + """ + inputs_correct = True + for answer_name, answer_value in numtolerance_inputs.iteritems(): + # If `self.corrrect_inputs` does not contain an entry for + # `answer_name`, this means that answer_name is a decoy + # input's value, and validation of its numericality is the + # only thing of interest from the later call to + # `compare_with_tolerance`. + params = self.correct_inputs.get(answer_name, {'answer': 0}) + + correct_ans = params['answer'] + # Set the tolerance to '0' if it was not specified in the xml + tolerance = params.get('tolerance', '0') + # Make sure that the staff answer is a valid number + try: + correct_ans = complex(correct_ans) + except ValueError: + log.debug( + "Content error--answer" + + "'{0}' is not a valid complex number".format(correct_ans) + ) + raise StudentInputError( + "The Staff answer could not be interpreted as a number." + ) + # Compare the student answer to the staff answer/ or to 0 + # if all that is important is verifying numericality + try: + partial_correct = compare_with_tolerance( + evaluator(dict(), dict(), answer_value), + correct_ans, + tolerance + ) + except: + # Use the traceback-preserving version of re-raising with a + # different type + _, _, trace = sys.exc_info() + + raise StudentInputError( + "Could not interpret '{0}' as a number{1}".format( + cgi.escape(answer_value), + trace + ) + ) + # Ignore the results of the comparisons which were just for + # Numerical Validation. + if answer_name in self.correct_inputs and not partial_correct: + # If any input is not correct, set the return value to False + inputs_correct = False + return inputs_correct + #----------------------------------------------------------------------------- # TEMPORARY: List of all response subclasses @@ -2116,4 +2448,5 @@ __all__ = [CodeResponse, MultipleChoiceResponse, TrueFalseResponse, JavascriptResponse, - AnnotationResponse] + AnnotationResponse, + ChoiceTextResponse] diff --git a/common/lib/capa/capa/templates/choicetext.html b/common/lib/capa/capa/templates/choicetext.html new file mode 100644 index 0000000000..5f587e214a --- /dev/null +++ b/common/lib/capa/capa/templates/choicetext.html @@ -0,0 +1,76 @@ +<% element_checked = False %> +% for choice_id, _ in choices: + <%choice_id = choice_id %> + %if choice_id in value: + <% element_checked = True %> + %endif +%endfor +
      +
      +
      +
      + % if input_type == 'checkbox' or not element_checked: + % if status == 'unsubmitted': + + % elif status == 'correct': + + % elif status == 'incorrect': + + % elif status == 'incomplete': + + % endif + % endif +
      + +
      + % for choice_id, choice_description in choices: + <%choice_id= choice_id %> +
      + % if correctness: + class="choicetextgroup_${correctness}" + % endif + % endif + > + + + % for content_node in choice_description: + % if content_node['type'] == 'text': + + ${content_node['contents']} + + % else: + <% my_id = content_node.get('contents','') %> + <% my_val = value.get(my_id,'') %> + + %endif + + ${content_node['tail_text']} + + + % endfor +

      +
      + + % endfor + +
      + + % if show_correctness == "never" and (value or status not in ['unsubmitted']): +
      ${submitted_message}
      + %endif + +
      diff --git a/common/lib/capa/capa/tests/response_xml_factory.py b/common/lib/capa/capa/tests/response_xml_factory.py index 35c12800ae..4c015d6699 100644 --- a/common/lib/capa/capa/tests/response_xml_factory.py +++ b/common/lib/capa/capa/tests/response_xml_factory.py @@ -779,3 +779,109 @@ class SymbolicResponseXMLFactory(ResponseXMLFactory): def create_input_element(self, **kwargs): return ResponseXMLFactory.textline_input_xml(**kwargs) + + +class ChoiceTextResponseXMLFactory(ResponseXMLFactory): + """ Factory for producing xml """ + + def create_response_element(self, **kwargs): + """ Create a element """ + return etree.Element("choicetextresponse") + + def create_input_element(self, **kwargs): + """ Create a element. + choices can be specified in the following format: + [("true", [{"answer": "5", "tolerance": 0}]), + ("false", [{"answer": "5", "tolerance": 0}]) + ] + + This indicates that the first checkbox/radio is correct and it + contains a numtolerance_input with an answer of 5 and a tolerance of 0 + + It also indicates that the second has a second incorrect radiobutton + or checkbox with a numtolerance_input. + """ + choices = kwargs.get('choices', [("true", {})]) + choice_inputs = [] + # Ensure that the first element of choices is an ordered + # collection. It will start as a list, a tuple, or not a Container. + if type(choices[0]) not in [list, tuple]: + choices = [choices] + + for choice in choices: + correctness, answers = choice + numtolerance_inputs = [] + # If the current `choice` contains any("answer": number) + # elements, turn those into numtolerance_inputs + if answers: + # `answers` will be a list or tuple of answers or a single + # answer, representing the answers for numtolerance_inputs + # inside of this specific choice. + + # Make sure that `answers` is an ordered collection for + # convenience. + if type(answers) not in [list, tuple]: + answers = [answers] + + numtolerance_inputs = [ + self._create_numtolerance_input_element(answer) + for answer in answers + ] + + choice_inputs.append( + self._create_choice_element( + correctness=correctness, + inputs=numtolerance_inputs + ) + ) + # Default type is 'radiotextgroup' + input_type = kwargs.get('type', 'radiotextgroup') + input_element = etree.Element(input_type) + + for ind, choice in enumerate(choice_inputs): + # Give each choice text equal to it's position(0,1,2...) + choice.text = "choice_{0}".format(ind) + input_element.append(choice) + + return input_element + + def _create_choice_element(self, **kwargs): + """ + Creates a choice element for a choictextproblem. + Defaults to a correct choice with no numtolerance_input + """ + text = kwargs.get('text', '') + correct = kwargs.get('correctness', "true") + inputs = kwargs.get('inputs', []) + choice_element = etree.Element("choice") + choice_element.set("correct", correct) + choice_element.text = text + for inp in inputs: + # Add all of the inputs as children of this choice + choice_element.append(inp) + + return choice_element + + def _create_numtolerance_input_element(self, params): + """ + Creates a or element with + optionally specified tolerance and answer. + """ + answer = params['answer'] if 'answer' in params else None + # If there is not an answer specified, Then create a + # otherwise create a and set its tolerance + # and answer attributes. + if answer: + text_input = etree.Element("numtolerance_input") + text_input.set('answer', answer) + # If tolerance was specified, was specified use it, otherwise + # Set the tolerance to "0" + text_input.set( + 'tolerance', + params['tolerance'] if 'tolerance' in params else "0" + ) + + else: + text_input = etree.Element("decoy_input") + + return text_input diff --git a/common/lib/capa/capa/tests/test_input_templates.py b/common/lib/capa/capa/tests/test_input_templates.py index 00a9b3f6c2..dcab279614 100644 --- a/common/lib/capa/capa/tests/test_input_templates.py +++ b/common/lib/capa/capa/tests/test_input_templates.py @@ -714,3 +714,170 @@ class DragAndDropTemplateTest(TemplateTestCase): # escaping the HTML. We should be able to traverse the XML tree. xpath = "//div[@class='drag_and_drop_problem_json']/p/b" self.assert_has_text(xml, xpath, 'HTML') + + +class ChoiceTextGroupTemplateTest(TemplateTestCase): + """Test mako template for `` input""" + + TEMPLATE_NAME = 'choicetext.html' + VALUE_DICT = {'1_choiceinput_0bc': '1_choiceinput_0bc', '1_choiceinput_0_textinput_0': '0', + '1_choiceinput_1_textinput_0': '0'} + EMPTY_DICT = {'1_choiceinput_0_textinput_0': '', + '1_choiceinput_1_textinput_0': ''} + BOTH_CHOICE_CHECKBOX = {'1_choiceinput_0bc': 'choiceinput_0', + '1_choiceinput_1bc': 'choiceinput_1', + '1_choiceinput_0_textinput_0': '0', + '1_choiceinput_1_textinput_0': '0'} + WRONG_CHOICE_CHECKBOX = {'1_choiceinput_1bc': 'choiceinput_1', + '1_choiceinput_0_textinput_0': '0', + '1_choiceinput_1_textinput_0': '0'} + + def setUp(self): + choices = [('1_choiceinput_0bc', + [{'tail_text': '', 'type': 'text', 'value': '', 'contents': ''}, + {'tail_text': '', 'type': 'textinput', 'value': '', 'contents': 'choiceinput_0_textinput_0'}]), + ('1_choiceinput_1bc', [{'tail_text': '', 'type': 'text', 'value': '', 'contents': ''}, + {'tail_text': '', 'type': 'textinput', 'value': '', 'contents': 'choiceinput_1_textinput_0'}])] + self.context = {'id': '1', + 'choices': choices, + 'status': 'correct', + 'input_type': 'radio', + 'value': self.VALUE_DICT} + + super(ChoiceTextGroupTemplateTest, self).setUp() + + def test_grouping_tag(self): + """ + Tests whether we are using a section or a label to wrap choice elements. + Section is used for checkbox, so inputting text does not deselect + """ + input_tags = ('radio', 'checkbox') + self.context['status'] = 'correct' + xpath = "//section[@id='forinput1_choiceinput_0bc']" + + self.context['value'] = {} + for input_type in input_tags: + self.context['input_type'] = input_type + xml = self.render_to_xml(self.context) + self.assert_has_xpath(xml, xpath, self.context) + + def test_problem_marked_correct(self): + """Test conditions under which the entire problem + (not a particular option) is marked correct""" + + self.context['status'] = 'correct' + self.context['input_type'] = 'checkbox' + self.context['value'] = self.VALUE_DICT + + # Should mark the entire problem correct + xml = self.render_to_xml(self.context) + xpath = "//div[@class='indicator_container']/span[@class='correct']" + self.assert_has_xpath(xml, xpath, self.context) + + # Should NOT mark individual options + self.assert_no_xpath(xml, "//label[@class='choicetextgroup_incorrect']", + self.context) + + self.assert_no_xpath(xml, "//label[@class='choicetextgroup_correct']", + self.context) + + def test_problem_marked_incorrect(self): + """Test all conditions under which the entire problem + (not a particular option) is marked incorrect""" + grouping_tags = {'radio': 'label', 'checkbox': 'section'} + conditions = [ + {'status': 'incorrect', 'input_type': 'radio', 'value': {}}, + {'status': 'incorrect', 'input_type': 'checkbox', 'value': self.WRONG_CHOICE_CHECKBOX}, + {'status': 'incorrect', 'input_type': 'checkbox', 'value': self.BOTH_CHOICE_CHECKBOX}, + {'status': 'incorrect', 'input_type': 'checkbox', 'value': self.VALUE_DICT}, + {'status': 'incomplete', 'input_type': 'radio', 'value': {}}, + {'status': 'incomplete', 'input_type': 'checkbox', 'value': self.WRONG_CHOICE_CHECKBOX}, + {'status': 'incomplete', 'input_type': 'checkbox', 'value': self.BOTH_CHOICE_CHECKBOX}, + {'status': 'incomplete', 'input_type': 'checkbox', 'value': self.VALUE_DICT}] + + for test_conditions in conditions: + self.context.update(test_conditions) + xml = self.render_to_xml(self.context) + xpath = "//div[@class='indicator_container']/span[@class='incorrect']" + self.assert_has_xpath(xml, xpath, self.context) + + # Should NOT mark individual options + grouping_tag = grouping_tags[test_conditions['input_type']] + self.assert_no_xpath(xml, + "//{0}[@class='choicetextgroup_incorrect']".format(grouping_tag), + self.context) + + self.assert_no_xpath(xml, + "//{0}[@class='choicetextgroup_correct']".format(grouping_tag), + self.context) + + def test_problem_marked_unsubmitted(self): + """Test all conditions under which the entire problem + (not a particular option) is marked unanswered""" + grouping_tags = {'radio': 'label', 'checkbox': 'section'} + + conditions = [ + {'status': 'unsubmitted', 'input_type': 'radio', 'value': {}}, + {'status': 'unsubmitted', 'input_type': 'radio', 'value': self.EMPTY_DICT}, + {'status': 'unsubmitted', 'input_type': 'checkbox', 'value': {}}, + {'status': 'unsubmitted', 'input_type': 'checkbox', 'value': self.EMPTY_DICT}, + {'status': 'unsubmitted', 'input_type': 'checkbox', 'value': self.VALUE_DICT}, + {'status': 'unsubmitted', 'input_type': 'checkbox', 'value': self.BOTH_CHOICE_CHECKBOX}] + + self.context['status'] = 'unanswered' + + for test_conditions in conditions: + self.context.update(test_conditions) + xml = self.render_to_xml(self.context) + xpath = "//div[@class='indicator_container']/span[@class='unanswered']" + self.assert_has_xpath(xml, xpath, self.context) + + # Should NOT mark individual options + grouping_tag = grouping_tags[test_conditions['input_type']] + self.assert_no_xpath(xml, + "//{0}[@class='choicetextgroup_incorrect']".format(grouping_tag), + self.context) + + self.assert_no_xpath(xml, + "//{0}[@class='choicetextgroup_correct']".format(grouping_tag), + self.context) + + def test_option_marked_correct(self): + """Test conditions under which a particular option + (not the entire problem) is marked correct.""" + + conditions = [ + {'input_type': 'radio', 'value': self.VALUE_DICT}] + + self.context['status'] = 'correct' + + for test_conditions in conditions: + self.context.update(test_conditions) + xml = self.render_to_xml(self.context) + xpath = "//section[@id='forinput1_choiceinput_0bc' and\ + @class='choicetextgroup_correct']" + self.assert_has_xpath(xml, xpath, self.context) + + # Should NOT mark the whole problem + xpath = "//div[@class='indicator_container']/span" + self.assert_no_xpath(xml, xpath, self.context) + + def test_option_marked_incorrect(self): + """Test conditions under which a particular option + (not the entire problem) is marked incorrect.""" + + conditions = [ + {'input_type': 'radio', 'value': self.VALUE_DICT}] + + self.context['status'] = 'incorrect' + + for test_conditions in conditions: + self.context.update(test_conditions) + xml = self.render_to_xml(self.context) + xpath = "//section[@id='forinput1_choiceinput_0bc' and\ + @class='choicetextgroup_incorrect']" + self.assert_has_xpath(xml, xpath, self.context) + + # Should NOT mark the whole problem + xpath = "//div[@class='indicator_container']/span" + self.assert_no_xpath(xml, xpath, self.context) diff --git a/common/lib/capa/capa/tests/test_inputtypes.py b/common/lib/capa/capa/tests/test_inputtypes.py index 1b52d41890..48e34dea09 100644 --- a/common/lib/capa/capa/tests/test_inputtypes.py +++ b/common/lib/capa/capa/tests/test_inputtypes.py @@ -860,3 +860,94 @@ class AnnotationInputTest(unittest.TestCase): self.maxDiff = None self.assertDictEqual(context, expected) + + +class TestChoiceText(unittest.TestCase): + """ + Tests for checkboxtextgroup inputs + """ + @staticmethod + def build_choice_element(node_type, contents, tail_text, value): + """ + Builds a content node for a choice. + """ + # When xml is being parsed numtolerance_input and decoy_input tags map to textinput type + # in order to provide the template with correct rendering information. + if node_type in ('numtolerance_input', 'decoy_input'): + node_type = 'textinput' + choice = {'type': node_type, 'contents': contents, 'tail_text': tail_text, 'value': value} + return choice + + def check_group(self, tag, choice_tag, expected_input_type): + """ + Build a radio or checkbox group, parse it and check the resuls against the + expected output. + + `tag` should be 'checkboxtextgroup' or 'radiotextgroup' + `choice_tag` is either 'choice' for proper xml, or any other value to trigger an error. + `expected_input_type` is either 'radio' or 'checkbox'. + """ + xml_str = """ + <{tag}> + <{choice_tag} correct="false" name="choiceinput_0">this isfalse + Is a number! + + """.format(tag=tag, choice_tag=choice_tag) + element = etree.fromstring(xml_str) + state = { + 'value': '{}', + 'id': 'choicetext_input', + 'status': 'answered' + } + + first_input = self.build_choice_element('numtolerance_input', 'choiceinput_0_textinput_0', 'false', '') + second_input = self.build_choice_element('decoy_input', 'choiceinput_1_textinput_0', '', '') + first_choice_content = self.build_choice_element('text', 'this is', '', '') + second_choice_content = self.build_choice_element('text', 'Is a number', '', '') + second_choice_text = self.build_choice_element('text', "!", '', '') + + choices = [ + ('choiceinput_0', [first_choice_content, first_input]), + ('choiceinput_1', [second_choice_content, second_input, second_choice_text]) + ] + + expected = { + 'msg': '', + 'input_type': expected_input_type, + 'choices': choices, + 'show_correctness': 'always', + 'submitted_message': 'Answer received.' + } + expected.update(state) + the_input = lookup_tag(tag)(test_system(), element, state) + context = the_input._get_render_context() + self.assertEqual(context, expected) + + def test_radiotextgroup(self): + """ + Test that a properly formatted radiotextgroup problem generates + expected ouputs + """ + self.check_group('radiotextgroup', 'choice', 'radio') + + def test_checkboxtextgroup(self): + """ + Test that a properly formatted checkboxtextgroup problem generates + expected ouput + """ + self.check_group('checkboxtextgroup', 'choice', 'checkbox') + + def test_invalid_tag(self): + """ + Test to ensure that an unrecognized inputtype tag causes an error + """ + with self.assertRaises(Exception): + self.check_group('invalid', 'choice', 'checkbox') + + def test_invalid_input_tag(self): + """ + Test to ensure having a tag other than inside of + a checkbox or radiotextgroup problem raises an error. + """ + with self.assertRaisesRegexp(Exception, "Error in xml"): + self.check_group('checkboxtextgroup', 'invalid', 'checkbox') diff --git a/common/lib/capa/capa/tests/test_responsetypes.py b/common/lib/capa/capa/tests/test_responsetypes.py index 594e2ca629..a756dc640e 100644 --- a/common/lib/capa/capa/tests/test_responsetypes.py +++ b/common/lib/capa/capa/tests/test_responsetypes.py @@ -18,6 +18,8 @@ from capa.correctmap import CorrectMap from capa.util import convert_files_to_filenames from capa.xqueue_interface import dateformat +from pytz import UTC + class ResponseTest(unittest.TestCase): """ Base class for tests of capa responses.""" @@ -333,8 +335,9 @@ class SymbolicResponseTest(ResponseTest): correct_map = problem.grade_answers(input_dict) - self.assertEqual(correct_map.get_correctness('1_2_1'), - expected_correctness) + self.assertEqual( + correct_map.get_correctness('1_2_1'), expected_correctness + ) class OptionResponseTest(ResponseTest): @@ -702,7 +705,7 @@ class CodeResponseTest(ResponseTest): # Now we queue the LCP cmap = CorrectMap() for i, answer_id in enumerate(answer_ids): - queuestate = CodeResponseTest.make_queuestate(i, datetime.now()) + queuestate = CodeResponseTest.make_queuestate(i, datetime.now(UTC)) cmap.update(CorrectMap(answer_id=answer_ids[i], queuestate=queuestate)) self.problem.correct_map.update(cmap) @@ -718,7 +721,7 @@ class CodeResponseTest(ResponseTest): old_cmap = CorrectMap() for i, answer_id in enumerate(answer_ids): queuekey = 1000 + i - queuestate = CodeResponseTest.make_queuestate(queuekey, datetime.now()) + queuestate = CodeResponseTest.make_queuestate(queuekey, datetime.now(UTC)) old_cmap.update(CorrectMap(answer_id=answer_ids[i], queuestate=queuestate)) # Message format common to external graders @@ -778,13 +781,15 @@ class CodeResponseTest(ResponseTest): cmap = CorrectMap() for i, answer_id in enumerate(answer_ids): queuekey = 1000 + i - latest_timestamp = datetime.now() + latest_timestamp = datetime.now(UTC) queuestate = CodeResponseTest.make_queuestate(queuekey, latest_timestamp) cmap.update(CorrectMap(answer_id=answer_id, queuestate=queuestate)) self.problem.correct_map.update(cmap) # Queue state only tracks up to second - latest_timestamp = datetime.strptime(datetime.strftime(latest_timestamp, dateformat), dateformat) + latest_timestamp = datetime.strptime( + datetime.strftime(latest_timestamp, dateformat), dateformat + ).replace(tzinfo=UTC) self.assertEquals(self.problem.get_recentmost_queuetime(), latest_timestamp) @@ -1429,3 +1434,357 @@ class AnnotationResponseTest(ResponseTest): msg="%s should be marked %s" % (answer_id, expected_correctness)) self.assertEqual(expected_points, actual_points, msg="%s should have %d points" % (answer_id, expected_points)) + + +class ChoiceTextResponseTest(ResponseTest): + """ + Class containing setup and tests for ChoiceText responsetype. + """ + + from response_xml_factory import ChoiceTextResponseXMLFactory + xml_factory_class = ChoiceTextResponseXMLFactory + + # `TEST_INPUTS` is a dictionary mapping from + # test_name to a representation of inputs for a test problem. + TEST_INPUTS = { + "1_choice_0_input_correct": [(True, [])], + "1_choice_0_input_incorrect": [(False, [])], + "1_choice_0_input_invalid_choice": [(False, []), (True, [])], + "1_choice_1_input_correct": [(True, ["123"])], + "1_input_script_correct": [(True, ["2"])], + "1_input_script_incorrect": [(True, ["3.25"])], + "1_choice_2_inputs_correct": [(True, ["123", "456"])], + "1_choice_2_inputs_tolerance": [(True, ["123 + .5", "456 + 9"])], + "1_choice_2_inputs_1_wrong": [(True, ["0", "456"])], + "1_choice_2_inputs_both_wrong": [(True, ["0", "0"])], + "1_choice_2_inputs_inputs_blank": [(True, ["", ""])], + "1_choice_2_inputs_empty": [(False, [])], + "1_choice_2_inputs_fail_tolerance": [(True, ["123 + 1.5", "456 + 9"])], + "1_choice_1_input_within_tolerance": [(True, ["122.5"])], + "1_choice_1_input_answer_incorrect": [(True, ["345"])], + "1_choice_1_input_choice_incorrect": [(False, ["123"])], + "2_choices_0_inputs_correct": [(False, []), (True, [])], + "2_choices_0_inputs_incorrect": [(True, []), (False, [])], + "2_choices_0_inputs_blank": [(False, []), (False, [])], + "2_choices_1_input_1_correct": [(False, []), (True, ["123"])], + "2_choices_1_input_1_incorrect": [(True, []), (False, ["123"])], + "2_choices_1_input_input_wrong": [(False, []), (True, ["321"])], + "2_choices_1_input_1_blank": [(False, []), (False, [])], + "2_choices_1_input_2_correct": [(True, []), (False, ["123"])], + "2_choices_1_input_2_incorrect": [(False, []), (True, ["123"])], + "2_choices_2_inputs_correct": [(True, ["123"]), (False, [])], + "2_choices_2_inputs_wrong_choice": [(False, ["123"]), (True, [])], + "2_choices_2_inputs_wrong_input": [(True, ["321"]), (False, [])] + } + + # `TEST_SCENARIOS` is a dictionary of the form + # {Test_Name" : (Test_Problem_name, correctness)} + # correctness represents whether the problem should be graded as + # correct or incorrect when the test is run. + TEST_SCENARIOS = { + "1_choice_0_input_correct": ("1_choice_0_input", "correct"), + "1_choice_0_input_incorrect": ("1_choice_0_input", "incorrect"), + "1_choice_0_input_invalid_choice": ("1_choice_0_input", "incorrect"), + "1_input_script_correct": ("1_input_script", "correct"), + "1_input_script_incorrect": ("1_input_script", "incorrect"), + "1_choice_2_inputs_correct": ("1_choice_2_inputs", "correct"), + "1_choice_2_inputs_tolerance": ("1_choice_2_inputs", "correct"), + "1_choice_2_inputs_1_wrong": ("1_choice_2_inputs", "incorrect"), + "1_choice_2_inputs_both_wrong": ("1_choice_2_inputs", "incorrect"), + "1_choice_2_inputs_inputs_blank": ("1_choice_2_inputs", "incorrect"), + "1_choice_2_inputs_empty": ("1_choice_2_inputs", "incorrect"), + "1_choice_2_inputs_fail_tolerance": ("1_choice_2_inputs", "incorrect"), + "1_choice_1_input_correct": ("1_choice_1_input", "correct"), + "1_choice_1_input_within_tolerance": ("1_choice_1_input", "correct"), + "1_choice_1_input_answer_incorrect": ("1_choice_1_input", "incorrect"), + "1_choice_1_input_choice_incorrect": ("1_choice_1_input", "incorrect"), + "2_choices_0_inputs_correct": ("2_choices_0_inputs", "correct"), + "2_choices_0_inputs_incorrect": ("2_choices_0_inputs", "incorrect"), + "2_choices_0_inputs_blank": ("2_choices_0_inputs", "incorrect"), + "2_choices_1_input_1_correct": ("2_choices_1_input_1", "correct"), + "2_choices_1_input_1_incorrect": ("2_choices_1_input_1", "incorrect"), + "2_choices_1_input_input_wrong": ("2_choices_1_input_1", "incorrect"), + "2_choices_1_input_1_blank": ("2_choices_1_input_1", "incorrect"), + "2_choices_1_input_2_correct": ("2_choices_1_input_2", "correct"), + "2_choices_1_input_2_incorrect": ("2_choices_1_input_2", "incorrect"), + "2_choices_2_inputs_correct": ("2_choices_2_inputs", "correct"), + "2_choices_2_inputs_wrong_choice": ("2_choices_2_inputs", "incorrect"), + "2_choices_2_inputs_wrong_input": ("2_choices_2_inputs", "incorrect") + } + + # Dictionary that maps from problem_name to arguments for + # _make_problem, that will create the problem. + TEST_PROBLEM_ARGS = { + "1_choice_0_input": {"choices": ("true", {}), "script": ''}, + "1_choice_1_input": { + "choices": ("true", {"answer": "123", "tolerance": "1"}), + "script": '' + }, + + "1_input_script": { + "choices": ("true", {"answer": "$computed_response", "tolerance": "1"}), + "script": "computed_response = math.sqrt(4)" + }, + + "1_choice_2_inputs": { + "choices": [ + ( + "true", ( + {"answer": "123", "tolerance": "1"}, + {"answer": "456", "tolerance": "10"} + ) + ) + ], + "script": '' + }, + "2_choices_0_inputs": { + "choices": [("false", {}), ("true", {})], + "script": '' + + }, + "2_choices_1_input_1": { + "choices": [ + ("false", {}), ("true", {"answer": "123", "tolerance": "0"}) + ], + "script": '' + }, + "2_choices_1_input_2": { + "choices": [("true", {}), ("false", {"answer": "123", "tolerance": "0"})], + "script": '' + }, + "2_choices_2_inputs": { + "choices": [ + ("true", {"answer": "123", "tolerance": "0"}), + ("false", {"answer": "999", "tolerance": "0"}) + ], + "script": '' + } + } + + def _make_problem(self, choices, in_type='radiotextgroup', script=''): + """ + Convenience method to fill in default values for script and + type if needed, then call self.build_problem + """ + return self.build_problem( + choices=choices, + type=in_type, + script=script + ) + + def _make_answer_dict(self, choice_list): + """ + Convenience method to make generation of answers less tedious, + pass in an iterable argument with elements of the form: [bool, [ans,]] + Will generate an answer dict for those options + """ + + answer_dict = {} + for index, choice_answers_pair in enumerate(choice_list): + # Choice is whether this choice is correct + # Answers contains a list of answers to textinpts for the choice + choice, answers = choice_answers_pair + + if choice: + # Radio/Checkbox inputs in choicetext problems follow + # a naming convention that gives them names ending with "bc" + choice_id = "1_2_1_choiceinput_{index}bc".format(index=index) + choice_value = "choiceinput_{index}".format(index=index) + answer_dict[choice_id] = choice_value + # Build the names for the numtolerance_inputs and add their answers + # to `answer_dict`. + for ind, answer in enumerate(answers): + # In `answer_id` `index` represents the ordinality of the + # choice and `ind` represents the ordinality of the + # numtolerance_input inside the parent choice. + answer_id = "1_2_1_choiceinput_{index}_numtolerance_input_{ind}".format( + index=index, + ind=ind + ) + answer_dict[answer_id] = answer + + return answer_dict + + def test_invalid_xml(self): + """ + Test that build problem raises errors for invalid options + """ + with self.assertRaises(Exception): + self.build_problem(type="invalidtextgroup") + + def test_valid_xml(self): + """ + Test that `build_problem` builds valid xml + """ + self.build_problem() + self.assertTrue(True) + + def test_unchecked_input_not_validated(self): + """ + Test that a student can have a non numeric answer in an unselected + choice without causing an error to be raised when the problem is + checked. + """ + + two_choice_two_input = self._make_problem( + [ + ("true", {"answer": "123", "tolerance": "1"}), + ("false", {}) + ], + "checkboxtextgroup" + ) + + self.assert_grade( + two_choice_two_input, + self._make_answer_dict([(True, ["1"]), (False, ["Platypus"])]), + "incorrect" + ) + + def test_interpret_error(self): + """ + Test that student answers that cannot be interpeted as numbers + cause the response type to raise an error. + """ + two_choice_two_input = self._make_problem( + [ + ("true", {"answer": "123", "tolerance": "1"}), + ("false", {}) + ], + "checkboxtextgroup" + ) + + with self.assertRaisesRegexp(StudentInputError, "Could not interpret"): + # Test that error is raised for input in selected correct choice. + self.assert_grade( + two_choice_two_input, + self._make_answer_dict([(True, ["Platypus"])]), + "correct" + ) + + with self.assertRaisesRegexp(StudentInputError, "Could not interpret"): + # Test that error is raised for input in selected incorrect choice. + self.assert_grade( + two_choice_two_input, + self._make_answer_dict([(True, ["1"]), (True, ["Platypus"])]), + "correct" + ) + + def test_staff_answer_error(self): + broken_problem = self._make_problem( + [("true", {"answer": "Platypus", "tolerance": "0"}), + ("true", {"answer": "edX", "tolerance": "0"}) + ], + "checkboxtextgroup" + ) + with self.assertRaisesRegexp( + StudentInputError, + "The Staff answer could not be interpreted as a number." + ): + self.assert_grade( + broken_problem, + self._make_answer_dict( + [(True, ["1"]), (True, ["1"])] + ), + "correct" + ) + + def test_radio_grades(self): + """ + Test that confirms correct operation of grading when the inputtag is + radiotextgroup. + """ + + for name, inputs in self.TEST_INPUTS.iteritems(): + # Turn submission into the form expected when grading this problem. + submission = self._make_answer_dict(inputs) + # Lookup the problem_name, and the whether this test problem + # and inputs should be graded as correct or incorrect. + problem_name, correctness = self.TEST_SCENARIOS[name] + # Load the args needed to build the problem for this test. + problem_args = self.TEST_PROBLEM_ARGS[problem_name] + test_choices = problem_args["choices"] + test_script = problem_args["script"] + # Build the actual problem for the test. + test_problem = self._make_problem(test_choices, 'radiotextgroup', test_script) + # Make sure the actual grade matches the expected grade. + self.assert_grade( + test_problem, + submission, + correctness, + msg="{0} should be {1}".format( + name, + correctness + ) + ) + + def test_checkbox_grades(self): + """ + Test that confirms correct operation of grading when the inputtag is + checkboxtextgroup. + """ + # Dictionary from name of test_scenario to (problem_name, correctness) + # Correctness is used to test whether the problem was graded properly + scenarios = { + "2_choices_correct": ("checkbox_two_choices", "correct"), + "2_choices_incorrect": ("checkbox_two_choices", "incorrect"), + + "2_choices_2_inputs_correct": ( + "checkbox_2_choices_2_inputs", + "correct" + ), + + "2_choices_2_inputs_missing_choice": ( + "checkbox_2_choices_2_inputs", + "incorrect" + ), + + "2_choices_2_inputs_wrong_input": ( + "checkbox_2_choices_2_inputs", + "incorrect" + ) + } + # Dictionary scenario_name: test_inputs + inputs = { + "2_choices_correct": [(True, []), (True, [])], + "2_choices_incorrect": [(True, []), (False, [])], + "2_choices_2_inputs_correct": [(True, ["123"]), (True, ["456"])], + "2_choices_2_inputs_missing_choice": [ + (True, ["123"]), (False, ["456"]) + ], + "2_choices_2_inputs_wrong_input": [ + (True, ["123"]), (True, ["654"]) + ] + } + + # Two choice zero input problem with both choices being correct. + checkbox_two_choices = self._make_problem( + [("true", {}), ("true", {})], "checkboxtextgroup" + ) + # Two choice two input problem with both choices correct. + checkbox_two_choices_two_inputs = self._make_problem( + [("true", {"answer": "123", "tolerance": "0"}), + ("true", {"answer": "456", "tolerance": "0"}) + ], + "checkboxtextgroup" + ) + + # Dictionary problem_name: problem + problems = { + "checkbox_two_choices": checkbox_two_choices, + "checkbox_2_choices_2_inputs": checkbox_two_choices_two_inputs + } + + for name, inputs in inputs.iteritems(): + submission = self._make_answer_dict(inputs) + # Load the test problem's name and desired correctness + problem_name, correctness = scenarios[name] + # Load the problem + problem = problems[problem_name] + + # Make sure the actual grade matches the expected grade + self.assert_grade( + problem, + submission, + correctness, + msg="{0} should be {1}".format(name, correctness) + ) diff --git a/common/lib/capa/capa/xqueue_interface.py b/common/lib/capa/capa/xqueue_interface.py index 5cf2488af0..4da8e11d53 100644 --- a/common/lib/capa/capa/xqueue_interface.py +++ b/common/lib/capa/capa/xqueue_interface.py @@ -30,9 +30,11 @@ def make_xheader(lms_callback_url, lms_key, queue_name): 'queue_name': designate a specific queue within xqueue server, e.g. 'MITx-6.00x' (string) } """ - return json.dumps({'lms_callback_url': lms_callback_url, - 'lms_key': lms_key, - 'queue_name': queue_name}) + return json.dumps({ + 'lms_callback_url': lms_callback_url, + 'lms_key': lms_key, + 'queue_name': queue_name + }) def parse_xreply(xreply): @@ -60,7 +62,7 @@ class XQueueInterface(object): ''' def __init__(self, url, django_auth, requests_auth=None): - self.url = url + self.url = url self.auth = django_auth self.session = requests.session(auth=requests_auth) @@ -95,13 +97,13 @@ class XQueueInterface(object): return (error, msg) - def _login(self): - payload = {'username': self.auth['username'], - 'password': self.auth['password']} + payload = { + 'username': self.auth['username'], + 'password': self.auth['password'] + } return self._http_post(self.url + '/xqueue/login/', payload) - def _send_to_queue(self, header, body, files_to_upload): payload = {'xqueue_header': header, 'xqueue_body': body} @@ -112,7 +114,6 @@ class XQueueInterface(object): return self._http_post(self.url + '/xqueue/submit/', payload, files=files) - def _http_post(self, url, data, files=None): try: r = self.session.post(url, data=data, files=files) diff --git a/common/lib/xmodule/xmodule/abtest_module.py b/common/lib/xmodule/xmodule/abtest_module.py index 2e61076e94..53f080eb3a 100644 --- a/common/lib/xmodule/xmodule/abtest_module.py +++ b/common/lib/xmodule/xmodule/abtest_module.py @@ -80,8 +80,6 @@ class ABTestModule(ABTestFields, XModule): class ABTestDescriptor(ABTestFields, RawDescriptor, XmlDescriptor): module_class = ABTestModule - template_dir_name = "abtest" - @classmethod def definition_from_xml(cls, xml_object, system): """ diff --git a/common/lib/xmodule/xmodule/annotatable_module.py b/common/lib/xmodule/xmodule/annotatable_module.py index e8674360c3..f80e3e488e 100644 --- a/common/lib/xmodule/xmodule/annotatable_module.py +++ b/common/lib/xmodule/xmodule/annotatable_module.py @@ -6,12 +6,37 @@ from pkg_resources import resource_string from xmodule.x_module import XModule from xmodule.raw_module import RawDescriptor from xblock.core import Scope, String +import textwrap log = logging.getLogger(__name__) class AnnotatableFields(object): - data = String(help="XML data for the annotation", scope=Scope.content) + data = String(help="XML data for the annotation", scope=Scope.content, + default=textwrap.dedent( + """\ + + +

      Enter your (optional) instructions for the exercise in HTML format.

      +

      Annotations are specified by an <annotation> tag which may may have the following attributes:

      +
        +
      • title (optional). Title of the annotation. Defaults to Commentary if omitted.
        • +
      • body (required). Text of the annotation.
        • +
      • problem (optional). Numeric index of the problem associated with this annotation. This is a zero-based index, so the first problem on the page would have problem="0".
        • +
      • highlight (optional). Possible values: yellow, red, orange, green, blue, or purple. Defaults to yellow if this attribute is omitted.
        • +
      +       +

      Add your HTML with annotation spans here.

      +

      Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut sodales laoreet est, egestas gravida felis egestas nec. Aenean at volutpat erat. Cras commodo viverra nibh in aliquam.

      +

      Nulla facilisi. Pellentesque id vestibulum libero. Suspendisse potenti. Morbi scelerisque nisi vitae felis dictum mattis. Nam sit amet magna elit. Nullam volutpat cursus est, sit amet sagittis odio vulputate et. Curabitur euismod, orci in vulputate imperdiet, augue lorem tempor purus, id aliquet augue turpis a est. Aenean a sagittis libero. Praesent fringilla pretium magna, non condimentum risus elementum nec. Pellentesque faucibus elementum pharetra. Pellentesque vitae metus eros.

      +       + """)) + display_name = String( + display_name="Display Name", + help="Display name for this module", + scope=Scope.settings, + default='Annotation', + ) class AnnotatableModule(AnnotatableFields, XModule): @@ -125,5 +150,4 @@ class AnnotatableModule(AnnotatableFields, XModule): class AnnotatableDescriptor(AnnotatableFields, RawDescriptor): module_class = AnnotatableModule - template_dir_name = "annotatable" mako_template = "widgets/raw-edit.html" diff --git a/common/lib/xmodule/xmodule/capa_module.py b/common/lib/xmodule/xmodule/capa_module.py index eeb8f19439..8489b5f986 100644 --- a/common/lib/xmodule/xmodule/capa_module.py +++ b/common/lib/xmodule/xmodule/capa_module.py @@ -77,6 +77,14 @@ class CapaFields(object): """ Define the possible fields for a Capa problem """ + display_name = String( + display_name="Display Name", + help="This name appears in the horizontal navigation at the top of the page.", + scope=Scope.settings, + # it'd be nice to have a useful default but it screws up other things; so, + # use display_name_with_default for those + default="Blank Advanced Problem" + ) attempts = Integer(help="Number of attempts taken by the student on this problem", default=0, scope=Scope.user_state) max_attempts = Integer( @@ -94,7 +102,8 @@ class CapaFields(object): display_name="Show Answer", help=("Defines when to show the answer to the problem. " "A default value can be set in Advanced Settings."), - scope=Scope.settings, default="closed", + scope=Scope.settings, + default="finished", values=[ {"display_name": "Always", "value": "always"}, {"display_name": "Answered", "value": "answered"}, @@ -106,21 +115,24 @@ class CapaFields(object): ) force_save_button = Boolean( help="Whether to force the save button to appear on the page", - scope=Scope.settings, default=False + scope=Scope.settings, + default=False ) rerandomize = Randomization( display_name="Randomization", help="Defines how often inputs are randomized when a student loads the problem. " - "This setting only applies to problems that can have randomly generated numeric values. " - "A default value can be set in Advanced Settings.", - default="always", scope=Scope.settings, values=[ + "This setting only applies to problems that can have randomly generated numeric values. " + "A default value can be set in Advanced Settings.", + default="never", + scope=Scope.settings, + values=[ {"display_name": "Always", "value": "always"}, {"display_name": "On Reset", "value": "onreset"}, {"display_name": "Never", "value": "never"}, {"display_name": "Per Student", "value": "per_student"} ] ) - data = String(help="XML data for the problem", scope=Scope.content) + data = String(help="XML data for the problem", scope=Scope.content, default="") correct_map = Dict(help="Dictionary with the correctness of current student answers", scope=Scope.user_state, default={}) input_state = Dict(help="Dictionary for maintaining the state of inputtypes", scope=Scope.user_state) @@ -134,7 +146,7 @@ class CapaFields(object): values={"min": 0, "step": .1}, scope=Scope.settings ) - markdown = String(help="Markdown source of this module", scope=Scope.settings) + markdown = String(help="Markdown source of this module", default=None, scope=Scope.settings) source_code = String( help="Source code for LaTeX and Word problems. This feature is not well-supported.", scope=Scope.settings @@ -297,7 +309,13 @@ class CapaModule(CapaFields, XModule): d = self.get_score() score = d['score'] total = d['total'] + if total > 0: + if self.weight is not None: + # scale score and total by weight/total: + score = score * self.weight / total + total = self.weight + try: return Progress(score, total) except (TypeError, ValueError): @@ -309,11 +327,13 @@ class CapaModule(CapaFields, XModule): """ Return some html with data about the module """ + progress = self.get_progress() return self.system.render_template('problem_ajax.html', { 'element_id': self.location.html_id(), 'id': self.id, 'ajax_url': self.system.ajax_url, - 'progress': Progress.to_js_status_str(self.get_progress()) + 'progress_status': Progress.to_js_status_str(progress), + 'progress_detail': Progress.to_js_detail_str(progress), }) def check_button_name(self): @@ -473,8 +493,7 @@ class CapaModule(CapaFields, XModule): """ Return html for the problem. - Adds check, reset, save buttons as necessary based on the problem config - and state. + Adds check, reset, save buttons as necessary based on the problem config and state. """ try: @@ -504,13 +523,12 @@ class CapaModule(CapaFields, XModule): 'reset_button': self.should_show_reset_button(), 'save_button': self.should_show_save_button(), 'answer_available': self.answer_available(), - 'ajax_url': self.system.ajax_url, 'attempts_used': self.attempts, 'attempts_allowed': self.max_attempts, - 'progress': self.get_progress(), } html = self.system.render_template('problem.html', context) + if encapsulate: html = u'
      '.format( id=self.location.html_id(), ajax_url=self.system.ajax_url @@ -541,6 +559,16 @@ class CapaModule(CapaFields, XModule): 'ungraded_response': self.handle_ungraded_response } + generic_error_message = ( + "We're sorry, there was an error with processing your request. " + "Please try reloading your page and trying again." + ) + + not_found_error_message = ( + "The state of this problem has changed since you loaded this page. " + "Please refresh your page." + ) + if dispatch not in handlers: return 'Error' @@ -548,15 +576,21 @@ class CapaModule(CapaFields, XModule): try: result = handlers[dispatch](data) + + except NotFoundError as err: + _, _, traceback_obj = sys.exc_info() + raise ProcessingError, (not_found_error_message, err), traceback_obj + except Exception as err: _, _, traceback_obj = sys.exc_info() - raise ProcessingError(err.message, traceback_obj) + raise ProcessingError, (generic_error_message, err), traceback_obj after = self.get_progress() result.update({ 'progress_changed': after != before, 'progress_status': Progress.to_js_status_str(after), + 'progress_detail': Progress.to_js_detail_str(after), }) return json.dumps(result, cls=ComplexEncoder) @@ -587,6 +621,7 @@ class CapaModule(CapaFields, XModule): Problem can be completely wrong. Pressing RESET button makes this function to return False. """ + # used by conditional module return self.lcp.done def is_attempted(self): @@ -730,6 +765,7 @@ class CapaModule(CapaFields, XModule): """ return {'html': self.get_problem_html(encapsulate=False)} + @staticmethod def make_dict_of_responses(data): """ @@ -749,6 +785,13 @@ class CapaModule(CapaFields, XModule): then the output dict would contain {'1': ['test'] } (the value is a list). + Some other inputs such as ChoiceTextInput expect a dict of values in the returned + dict If the key ends with '{}' then we will assume that the value is a json + encoded dict and deserialize it. + For example, if the `data` dict contains {'input_1{}': '{"1_2_1": 1}'} + then the output dict would contain {'1': {"1_2_1": 1} } + (the value is a dictionary) + Raises an exception if: -A key in the `data` dictionary does not contain at least one underscore @@ -775,11 +818,22 @@ class CapaModule(CapaFields, XModule): # the same form input (e.g. checkbox inputs). The convention is that # if the name ends with '[]' (which looks like an array), then the # answer will be an array. + # if the name ends with '{}' (Which looks like a dict), + # then the answer will be a dict is_list_key = name.endswith('[]') - name = name[:-2] if is_list_key else name + is_dict_key = name.endswith('{}') + name = name[:-2] if is_list_key or is_dict_key else name if is_list_key: val = data.getlist(key) + elif is_dict_key: + try: + val = json.loads(data[key]) + # If the submission wasn't deserializable, raise an error. + except(KeyError, ValueError): + raise ValueError( + u"Invalid submission: {val} for {key}".format(val=data[key], key=key) + ) else: val = data[key] @@ -1072,8 +1126,12 @@ class CapaDescriptor(CapaFields, RawDescriptor): mako_template = "widgets/problem-edit.html" js = {'coffee': [resource_string(__name__, 'js/src/problem/edit.coffee')]} js_module_name = "MarkdownEditingDescriptor" - css = {'scss': [resource_string(__name__, 'css/editor/edit.scss'), - resource_string(__name__, 'css/problem/edit.scss')]} + css = { + 'scss': [ + resource_string(__name__, 'css/editor/edit.scss'), + resource_string(__name__, 'css/problem/edit.scss') + ] + } # Capa modules have some additional metadata: # TODO (vshnayder): do problems have any other metadata? Do they diff --git a/common/lib/xmodule/xmodule/combined_open_ended_module.py b/common/lib/xmodule/xmodule/combined_open_ended_module.py index 52d98f032e..e01ae49149 100644 --- a/common/lib/xmodule/xmodule/combined_open_ended_module.py +++ b/common/lib/xmodule/xmodule/combined_open_ended_module.py @@ -9,10 +9,11 @@ from xblock.core import Integer, Scope, String, List, Float, Boolean from xmodule.open_ended_grading_classes.combined_open_ended_modulev1 import CombinedOpenEndedV1Module, CombinedOpenEndedV1Descriptor from collections import namedtuple from .fields import Date +import textwrap log = logging.getLogger("mitx.courseware") -V1_SETTINGS_ATTRIBUTES = ["display_name", "attempts", "is_graded", "accept_file_upload", +V1_SETTINGS_ATTRIBUTES = ["display_name", "max_attempts", "graded", "accept_file_upload", "skip_spelling_checks", "due", "graceperiod", "weight"] V1_STUDENT_ATTRIBUTES = ["current_task_number", "task_states", "state", @@ -27,6 +28,126 @@ VERSION_TUPLES = { } DEFAULT_VERSION = 1 +DEFAULT_DATA = textwrap.dedent("""\ + + +

      Censorship in the Libraries

      + +

      'All of us can think of a book that we hope none of our children or any other children have taken off the shelf. But if I have the right to remove that book from the shelf -- that work I abhor -- then you also have exactly the same right and so does everyone else. And then we have no books left on the shelf for any of us.' --Katherine Paterson, Author +

      + +

      + Write a persuasive essay to a newspaper reflecting your vies on censorship in libraries. Do you believe that certain materials, such as books, music, movies, magazines, etc., should be removed from the shelves if they are found offensive? Support your position with convincing arguments from your own experience, observations, and/or reading. +

      + +       + + + + + Ideas + + + + + + + + + Content + + + + + + + + + Organization + + + + + + + + Style + + + + + + + + Voice + + + + + + + + + + + + + + + + Enter essay here. + This is the answer. + {"grader_settings" : "ml_grading.conf", "problem_id" : "6.002x/Welcome/OETest"} + + + + + + + + Enter essay here. + This is the answer. + {"grader_settings" : "peer_grading.conf", "problem_id" : "6.002x/Welcome/OETest"} + + + + +       +""") class VersionInteger(Integer): @@ -51,47 +172,122 @@ class CombinedOpenEndedFields(object): display_name = String( display_name="Display Name", help="This name appears in the horizontal navigation at the top of the page.", - default="Open Ended Grading", scope=Scope.settings + default="Open Response Assessment", + scope=Scope.settings ) - current_task_number = Integer(help="Current task that the student is on.", default=0, scope=Scope.user_state) - task_states = List(help="List of state dictionaries of each task within this module.", scope=Scope.user_state) - state = String(help="Which step within the current task that the student is on.", default="initial", - scope=Scope.user_state) - student_attempts = Integer(help="Number of attempts taken by the student on this problem", default=0, - scope=Scope.user_state) - ready_to_reset = Boolean( - help="If the problem is ready to be reset or not.", default=False, + current_task_number = Integer( + help="Current task that the student is on.", + default=0, scope=Scope.user_state ) - attempts = Integer( - display_name="Maximum Attempts", - help="The number of times the student can try to answer this problem.", default=1, - scope=Scope.settings, values={"min" : 1 } + task_states = List( + help="List of state dictionaries of each task within this module.", + scope=Scope.user_state + ) + state = String( + help="Which step within the current task that the student is on.", + default="initial", + scope=Scope.user_state + ) + graded = Boolean( + display_name="Graded", + help='Defines whether the student gets credit for grading this problem.', + default=False, + scope=Scope.settings + ) + student_attempts = Integer( + help="Number of attempts taken by the student on this problem", + default=0, + scope=Scope.user_state + ) + ready_to_reset = Boolean( + help="If the problem is ready to be reset or not.", + default=False, + scope=Scope.user_state + ) + max_attempts = Integer( + display_name="Maximum Attempts", + help="The number of times the student can try to answer this problem.", + default=1, + scope=Scope.settings, + values={"min" : 1 } ) - is_graded = Boolean(display_name="Graded", help="Whether or not the problem is graded.", default=False, scope=Scope.settings) accept_file_upload = Boolean( display_name="Allow File Uploads", - help="Whether or not the student can submit files as a response.", default=False, scope=Scope.settings + help="Whether or not the student can submit files as a response.", + default=False, + scope=Scope.settings ) skip_spelling_checks = Boolean( display_name="Disable Quality Filter", help="If False, the Quality Filter is enabled and submissions with poor spelling, short length, or poor grammar will not be peer reviewed.", - default=False, scope=Scope.settings + default=False, + scope=Scope.settings + ) + due = Date( + help="Date that this problem is due by", + default=None, + scope=Scope.settings ) - due = Date(help="Date that this problem is due by", default=None, scope=Scope.settings) graceperiod = String( help="Amount of time after the due date that submissions will be accepted", default=None, scope=Scope.settings ) version = VersionInteger(help="Current version number", default=DEFAULT_VERSION, scope=Scope.settings) - data = String(help="XML data for the problem", scope=Scope.content) + data = String(help="XML data for the problem", scope=Scope.content, + default=DEFAULT_DATA) weight = Float( display_name="Problem Weight", help="Defines the number of points each problem is worth. If the value is not set, each problem is worth one point.", - scope=Scope.settings, values={"min" : 0 , "step": ".1"} + scope=Scope.settings, + values={"min" : 0 , "step": ".1"}, + default=1 + ) + markdown = String( + help="Markdown source of this module", + default=textwrap.dedent("""\ + [prompt] +

      Censorship in the Libraries

      + +

      'All of us can think of a book that we hope none of our children or any other children have taken off the shelf. But if I have the right to remove that book from the shelf -- that work I abhor -- then you also have exactly the same right and so does everyone else. And then we have no books left on the shelf for any of us.' --Katherine Paterson, Author +

      + +

      + Write a persuasive essay to a newspaper reflecting your vies on censorship in libraries. Do you believe that certain materials, such as books, music, movies, magazines, etc., should be removed from the shelves if they are found offensive? Support your position with convincing arguments from your own experience, observations, and/or reading. +

      + [prompt] + [rubric] + + Ideas + - Difficult for the reader to discern the main idea. Too brief or too repetitive to establish or maintain a focus. + - Attempts a main idea. Sometimes loses focus or ineffectively displays focus. + - Presents a unifying theme or main idea, but may include minor tangents. Stays somewhat focused on topic and task. + - Presents a unifying theme or main idea without going off on tangents. Stays completely focused on topic and task. + + Content + - Includes little information with few or no details or unrelated details. Unsuccessful in attempts to explore any facets of the topic. + - Includes little information and few or no details. Explores only one or two facets of the topic. + - Includes sufficient information and supporting details. (Details may not be fully developed; ideas may be listed.) Explores some facets of the topic. + - Includes in-depth information and exceptional supporting details that are fully developed. Explores all facets of the topic. + + Organization + - Ideas organized illogically, transitions weak, and response difficult to follow. + - Attempts to logically organize ideas. Attempts to progress in an order that enhances meaning, and demonstrates use of transitions. + - Ideas organized logically. Progresses in an order that enhances meaning. Includes smooth transitions. + + Style + - Contains limited vocabulary, with many words used incorrectly. Demonstrates problems with sentence patterns. + - Contains basic vocabulary, with words that are predictable and common. Contains mostly simple sentences (although there may be an attempt at more varied sentence patterns). + - Includes vocabulary to make explanations detailed and precise. Includes varied sentence patterns, including complex sentences. + + Voice + - Demonstrates language and tone that may be inappropriate to task and reader. + - Demonstrates an attempt to adjust language and tone to task and reader. + - Demonstrates effective adjustment of language and tone to task and reader. + [rubric] + [tasks] + (Self), ({4-12}AI), ({9-12}Peer) + [tasks] + + """), + scope=Scope.settings ) - markdown = String(help="Markdown source of this module", scope=Scope.settings) class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule): @@ -143,37 +339,9 @@ class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule): def __init__(self, *args, **kwargs): """ - Definition file should have one or many task blocks, a rubric block, and a prompt block: + Definition file should have one or many task blocks, a rubric block, and a prompt block. - Sample file: - - - Blah blah rubric. - - - Some prompt. - - - - - What hint about this problem would you give to someone? - - - Save Succcesful. Thanks for participating! - - - - - - - Enter essay here. - This is the answer. - {"grader_settings" : "ml_grading.conf", - "problem_id" : "6.002x/Welcome/OETest"} - - - - + See DEFAULT_DATA for a sample. """ XModule.__init__(self, *args, **kwargs) @@ -254,6 +422,11 @@ class CombinedOpenEndedDescriptor(CombinedOpenEndedFields, RawDescriptor): js_module_name = "OpenEndedMarkdownEditingDescriptor" css = {'scss': [resource_string(__name__, 'css/editor/edit.scss'), resource_string(__name__, 'css/combinedopenended/edit.scss')]} + metadata_translations = { + 'is_graded': 'graded', + 'attempts': 'max_attempts', + } + def get_context(self): _context = RawDescriptor.get_context(self) _context.update({'markdown': self.markdown, diff --git a/common/lib/xmodule/xmodule/course_module.py b/common/lib/xmodule/xmodule/course_module.py index d75033c8a0..3c9f5c0683 100644 --- a/common/lib/xmodule/xmodule/course_module.py +++ b/common/lib/xmodule/xmodule/course_module.py @@ -15,6 +15,7 @@ import json from xblock.core import Scope, List, String, Dict, Boolean from .fields import Date +from xmodule.modulestore.locator import CourseLocator from django.utils.timezone import UTC from xmodule.util import date_utils @@ -145,16 +146,55 @@ class TextbookList(List): class CourseFields(object): - textbooks = TextbookList(help="List of pairs of (title, url) for textbooks used in this course", scope=Scope.content) + textbooks = TextbookList(help="List of pairs of (title, url) for textbooks used in this course", + default=[], scope=Scope.content) wiki_slug = String(help="Slug that points to the wiki for this course", scope=Scope.content) enrollment_start = Date(help="Date that enrollment for this class is opened", scope=Scope.settings) enrollment_end = Date(help="Date that enrollment for this class is closed", scope=Scope.settings) - start = Date(help="Start time when this module is visible", scope=Scope.settings) + start = Date(help="Start time when this module is visible", + # using now(UTC()) resulted in fractional seconds which screwed up comparisons and anyway w/b the + # time of first invocation of this stmt on the server + default=datetime.fromtimestamp(0, UTC()), + scope=Scope.settings) end = Date(help="Date that this class ends", scope=Scope.settings) advertised_start = String(help="Date that this course is advertised to start", scope=Scope.settings) - grading_policy = Dict(help="Grading policy definition for this class", scope=Scope.content) + grading_policy = Dict(help="Grading policy definition for this class", + default={"GRADER": [ + { + "type": "Homework", + "min_count": 12, + "drop_count": 2, + "short_label": "HW", + "weight": 0.15 + }, + { + "type": "Lab", + "min_count": 12, + "drop_count": 2, + "weight": 0.15 + }, + { + "type": "Midterm Exam", + "short_label": "Midterm", + "min_count": 1, + "drop_count": 0, + "weight": 0.3 + }, + { + "type": "Final Exam", + "short_label": "Final", + "min_count": 1, + "drop_count": 0, + "weight": 0.4 + } + ], + "GRADE_CUTOFFS": { + "Pass": 0.5 + }}, + scope=Scope.content) show_calculator = Boolean(help="Whether to show the calculator in this course", default=False, scope=Scope.settings) - display_name = String(help="Display name for this module", scope=Scope.settings) + display_name = String(help="Display name for this module", default="Empty", display_name="Display Name", scope=Scope.settings) + show_chat = Boolean(help="Whether to show the chat widget in this course", default=False, scope=Scope.settings) tabs = List(help="List of tabs to enable in this course", scope=Scope.settings) end_of_course_survey_url = String(help="Url for the end-of-course survey", scope=Scope.settings) discussion_blackouts = List(help="List of pairs of start/end dates for discussion blackouts", scope=Scope.settings) @@ -175,7 +215,125 @@ class CourseFields(object): allow_anonymous_to_peers = Boolean(scope=Scope.settings, default=False) advanced_modules = List(help="Beta modules used in your course", scope=Scope.settings) has_children = True - checklists = List(scope=Scope.settings) + checklists = List(scope=Scope.settings, + default=[ + {"short_description" : "Getting Started With Studio", + "items" : [{"short_description": "Add Course Team Members", + "long_description": "Grant your collaborators permission to edit your course so you can work together.", + "is_checked": False, + "action_url": "ManageUsers", + "action_text": "Edit Course Team", + "action_external": False}, + {"short_description": "Set Important Dates for Your Course", + "long_description": "Establish your course's student enrollment and launch dates on the Schedule and Details page.", + "is_checked": False, + "action_url": "SettingsDetails", + "action_text": "Edit Course Details & Schedule", + "action_external": False}, + {"short_description": "Draft Your Course's Grading Policy", + "long_description": "Set up your assignment types and grading policy even if you haven't created all your assignments.", + "is_checked": False, + "action_url": "SettingsGrading", + "action_text": "Edit Grading Settings", + "action_external": False}, + {"short_description": "Explore the Other Studio Checklists", + "long_description": "Discover other available course authoring tools, and find help when you need it.", + "is_checked": False, + "action_url": "", + "action_text": "", + "action_external": False}] + }, + {"short_description" : "Draft a Rough Course Outline", + "items" : [{"short_description": "Create Your First Section and Subsection", + "long_description": "Use your course outline to build your first Section and Subsection.", + "is_checked": False, + "action_url": "CourseOutline", + "action_text": "Edit Course Outline", + "action_external": False}, + {"short_description": "Set Section Release Dates", + "long_description": "Specify the release dates for each Section in your course. Sections become visible to students on their release dates.", + "is_checked": False, + "action_url": "CourseOutline", + "action_text": "Edit Course Outline", + "action_external": False}, + {"short_description": "Designate a Subsection as Graded", + "long_description": "Set a Subsection to be graded as a specific assignment type. Assignments within graded Subsections count toward a student's final grade.", + "is_checked": False, + "action_url": "CourseOutline", + "action_text": "Edit Course Outline", + "action_external": False}, + {"short_description": "Reordering Course Content", + "long_description": "Use drag and drop to reorder the content in your course.", + "is_checked": False, + "action_url": "CourseOutline", + "action_text": "Edit Course Outline", + "action_external": False}, + {"short_description": "Renaming Sections", + "long_description": "Rename Sections by clicking the Section name from the Course Outline.", + "is_checked": False, + "action_url": "CourseOutline", + "action_text": "Edit Course Outline", + "action_external": False}, + {"short_description": "Deleting Course Content", + "long_description": "Delete Sections, Subsections, or Units you don't need anymore. Be careful, as there is no Undo function.", + "is_checked": False, + "action_url": "CourseOutline", + "action_text": "Edit Course Outline", + "action_external": False}, + {"short_description": "Add an Instructor-Only Section to Your Outline", + "long_description": "Some course authors find using a section for unsorted, in-progress work useful. To do this, create a section and set the release date to the distant future.", + "is_checked": False, + "action_url": "CourseOutline", + "action_text": "Edit Course Outline", + "action_external": False}] + }, + {"short_description" : "Explore edX's Support Tools", + "items" : [{"short_description": "Explore the Studio Help Forum", + "long_description": "Access the Studio Help forum from the menu that appears when you click your user name in the top right corner of Studio.", + "is_checked": False, + "action_url": "http://help.edge.edx.org/", + "action_text": "Visit Studio Help", + "action_external": True}, + {"short_description": "Enroll in edX 101", + "long_description": "Register for edX 101, edX's primer for course creation.", + "is_checked": False, + "action_url": "https://edge.edx.org/courses/edX/edX101/How_to_Create_an_edX_Course/about", + "action_text": "Register for edX 101", + "action_external": True}, + {"short_description": "Download the Studio Documentation", + "long_description": "Download the searchable Studio reference documentation in PDF form.", + "is_checked": False, + "action_url": "http://files.edx.org/Getting_Started_with_Studio.pdf", + "action_text": "Download Documentation", + "action_external": True}] + }, + {"short_description" : "Draft Your Course About Page", + "items" : [{"short_description": "Draft a Course Description", + "long_description": "Courses on edX have an About page that includes a course video, description, and more. Draft the text students will read before deciding to enroll in your course.", + "is_checked": False, + "action_url": "SettingsDetails", + "action_text": "Edit Course Schedule & Details", + "action_external": False}, + {"short_description": "Add Staff Bios", + "long_description": "Showing prospective students who their instructor will be is helpful. Include staff bios on the course About page.", + "is_checked": False, + "action_url": "SettingsDetails", + "action_text": "Edit Course Schedule & Details", + "action_external": False}, + {"short_description": "Add Course FAQs", + "long_description": "Include a short list of frequently asked questions about your course.", + "is_checked": False, + "action_url": "SettingsDetails", + "action_text": "Edit Course Schedule & Details", + "action_external": False}, + {"short_description": "Add Course Prerequisites", + "long_description": "Let students know what knowledge and/or skills they should have before they enroll in your course.", + "is_checked": False, + "action_url": "SettingsDetails", + "action_text": "Edit Course Schedule & Details", + "action_external": False}] + } + ]) info_sidebar_name = String(scope=Scope.settings, default='Course Handouts') show_timezone = Boolean(help="True if timezones should be shown on dates in the courseware", scope=Scope.settings, default=True) enrollment_domain = String(help="External login method associated with user accounts allowed to register in course", @@ -208,8 +366,6 @@ class CourseFields(object): class CourseDescriptor(CourseFields, SequenceDescriptor): module_class = SequenceModule - template_dir_name = 'course' - def __init__(self, *args, **kwargs): """ Expects the same arguments as XModuleDescriptor.__init__ @@ -217,21 +373,22 @@ class CourseDescriptor(CourseFields, SequenceDescriptor): super(CourseDescriptor, self).__init__(*args, **kwargs) if self.wiki_slug is None: - self.wiki_slug = self.location.course + if isinstance(self.location, Location): + self.wiki_slug = self.location.course + elif isinstance(self.location, CourseLocator): + self.wiki_slug = self.location.course_id or self.display_name msg = None - if self.start is None: - msg = "Course loaded without a valid start date. id = %s" % self.id - self.start = datetime.now(UTC()) - log.critical(msg) - self.system.error_tracker(msg) # NOTE: relies on the modulestore to call set_grading_policy() right after # init. (Modulestore is in charge of figuring out where to load the policy from) # NOTE (THK): This is a last-minute addition for Fall 2012 launch to dynamically # disable the syllabus content for courses that do not provide a syllabus - self.syllabus_present = self.system.resources_fs.exists(path('syllabus')) + if self.system.resources_fs is None: + self.syllabus_present = False + else: + self.syllabus_present = self.system.resources_fs.exists(path('syllabus')) self._grading_policy = {} self.set_grading_policy(self.grading_policy) @@ -252,42 +409,33 @@ class CourseDescriptor(CourseFields, SequenceDescriptor): log.error(msg) continue - def default_grading_policy(self): - """ - Return a dict which is a copy of the default grading policy - """ - return {"GRADER": [ - { - "type": "Homework", - "min_count": 12, - "drop_count": 2, - "short_label": "HW", - "weight": 0.15 - }, - { - "type": "Lab", - "min_count": 12, - "drop_count": 2, - "weight": 0.15 - }, - { - "type": "Midterm Exam", - "short_label": "Midterm", - "min_count": 1, - "drop_count": 0, - "weight": 0.3 - }, - { - "type": "Final Exam", - "short_label": "Final", - "min_count": 1, - "drop_count": 0, - "weight": 0.4 - } - ], - "GRADE_CUTOFFS": { - "Pass": 0.5 - }} + # TODO check that this is still needed here and can't be by defaults. + if not self.tabs: + # When calling the various _tab methods, can omit the 'type':'blah' from the + # first arg, since that's only used for dispatch + tabs = [] + tabs.append({'type': 'courseware'}) + tabs.append({'type': 'course_info', 'name': 'Course Info'}) + + if self.syllabus_present: + tabs.append({'type': 'syllabus'}) + + tabs.append({'type': 'textbooks'}) + + # # If they have a discussion link specified, use that even if we feature + # # flag discussions off. Disabling that is mostly a server safety feature + # # at this point, and we don't need to worry about external sites. + if self.discussion_link: + tabs.append({'type': 'external_discussion', 'link': self.discussion_link}) + else: + tabs.append({'type': 'discussion', 'name': 'Discussion'}) + + tabs.append({'type': 'wiki', 'name': 'Wiki'}) + + if not self.hide_progress_tab: + tabs.append({'type': 'progress', 'name': 'Progress'}) + + self.tabs = tabs def set_grading_policy(self, course_policy): """ @@ -298,7 +446,13 @@ class CourseDescriptor(CourseFields, SequenceDescriptor): course_policy = {} # Load the global settings as a dictionary - grading_policy = self.default_grading_policy() + grading_policy = self.grading_policy + # BOY DO I HATE THIS grading_policy CODE ACROBATICS YET HERE I ADD MORE (dhm)--this fixes things persisted w/ + # defective grading policy values (but not None) + if 'GRADER' not in grading_policy: + grading_policy['GRADER'] = CourseFields.grading_policy.default['GRADER'] + if 'GRADE_CUTOFFS' not in grading_policy: + grading_policy['GRADE_CUTOFFS'] = CourseFields.grading_policy.default['GRADE_CUTOFFS'] # Override any global settings with the course settings grading_policy.update(course_policy) @@ -354,10 +508,6 @@ class CourseDescriptor(CourseFields, SequenceDescriptor): system.error_tracker("Unable to decode grading policy as json") policy = {} - # cdodge: import the grading policy information that is on disk and put into the - # descriptor 'definition' bucket as a dictionary so that it is persisted in the DB - instance.grading_policy = policy - # now set the current instance. set_grading_policy() will apply some inheritance rules instance.set_grading_policy(policy) @@ -661,6 +811,7 @@ class CourseDescriptor(CourseFields, SequenceDescriptor): if isinstance(self.advertised_start, basestring): return try_parse_iso_8601(self.advertised_start) elif self.advertised_start is None and self.start is None: + # TODO this is an impossible state since the init function forces start to have a value return 'TBD' else: return (self.advertised_start or self.start).strftime("%b %d, %Y") diff --git a/common/lib/xmodule/xmodule/css/capa/display.scss b/common/lib/xmodule/xmodule/css/capa/display.scss index c326c79b76..48912795f0 100644 --- a/common/lib/xmodule/xmodule/css/capa/display.scss +++ b/common/lib/xmodule/xmodule/css/capa/display.scss @@ -3,6 +3,7 @@ h2 { margin-bottom: 15px; &.problem-header { + display: inline-block; section.staff { margin-top: 30px; font-size: 80%; @@ -28,6 +29,13 @@ iframe[seamless]{ color: darken($error-red, 11%); } +section.problem-progress { + display: inline-block; + color: #999; + font-size: em(16); + font-weight: 100; + padding-left: 5px; +} section.problem { @media print { @@ -929,4 +937,32 @@ section.problem { } } } + .choicetextgroup{ + input[type="text"]{ + margin-bottom: 0.5em; + } + @extend .choicegroup; + + label.choicetextgroup_correct, section.choicetextgroup_correct{ + @extend label.choicegroup_correct; + + input[type="text"] { + border-color: green; + } + } + + label.choicetextgroup_incorrect, section.choicetextgroup_incorrect{ + @extend label.choicegroup_incorrect; + } + + label.choicetextgroup_show_correct, section.choicetextgroup_show_correct{ + &:after{ + content: url('../images/correct-icon.png'); + margin-left:15px; + } + } + span.mock_label{ + cursor : default; + } + } } diff --git a/common/lib/xmodule/xmodule/discussion_module.py b/common/lib/xmodule/xmodule/discussion_module.py index aef4821839..fac6a498e5 100644 --- a/common/lib/xmodule/xmodule/discussion_module.py +++ b/common/lib/xmodule/xmodule/discussion_module.py @@ -4,17 +4,27 @@ from xmodule.x_module import XModule from xmodule.raw_module import RawDescriptor from xmodule.editing_module import MetadataOnlyEditingDescriptor from xblock.core import String, Scope +from uuid import uuid4 class DiscussionFields(object): - discussion_id = String(scope=Scope.settings) + discussion_id = String(scope=Scope.settings, default="$$GUID$$") + display_name = String( + display_name="Display Name", + help="Display name for this module", + default="Discussion Tag", + scope=Scope.settings) + data = String(help="XML data for the problem", scope=Scope.content, + default="") discussion_category = String( display_name="Category", + default="Week 1", help="A category name for the discussion. This name appears in the left pane of the discussion forum for the course.", scope=Scope.settings ) discussion_target = String( display_name="Subcategory", + default="Topic-Level Student-Visible Label", help="A subcategory name for the discussion. This name appears in the left pane of the discussion forum for the course.", scope=Scope.settings ) @@ -36,9 +46,15 @@ class DiscussionModule(DiscussionFields, XModule): class DiscussionDescriptor(DiscussionFields, MetadataOnlyEditingDescriptor, RawDescriptor): - module_class = DiscussionModule - template_dir_name = "discussion" + def __init__(self, *args, **kwargs): + super(DiscussionDescriptor, self).__init__(*args, **kwargs) + # is this too late? i.e., will it get persisted and stay static w/ the first value + # any code references. I believe so. + if self.discussion_id == '$$GUID$$': + self.discussion_id = uuid4().hex + + module_class = DiscussionModule # The discussion XML format uses `id` and `for` attributes, # but these would overload other module attributes, so we prefix them # for actual use in the code diff --git a/common/lib/xmodule/xmodule/error_module.py b/common/lib/xmodule/xmodule/error_module.py index a37081c447..e7483f485a 100644 --- a/common/lib/xmodule/xmodule/error_module.py +++ b/common/lib/xmodule/xmodule/error_module.py @@ -79,8 +79,10 @@ class ErrorDescriptor(ErrorFields, JSONEditingDescriptor): @classmethod def _construct(cls, system, contents, error_msg, location): - if location.name is None: - location = location._replace( + if isinstance(location, dict) and 'course' in location: + location = Location(location) + if isinstance(location, Location) and location.name is None: + location = location.replace( category='error', # Pick a unique url_name -- the sha1 hash of the contents. # NOTE: We could try to pull out the url_name of the errored descriptor, @@ -94,8 +96,9 @@ class ErrorDescriptor(ErrorFields, JSONEditingDescriptor): model_data = { 'error_msg': str(error_msg), 'contents': contents, - 'display_name': 'Error: ' + location.name, + 'display_name': 'Error: ' + location.url(), 'location': location, + 'category': 'error' } return cls( system, @@ -109,12 +112,12 @@ class ErrorDescriptor(ErrorFields, JSONEditingDescriptor): } @classmethod - def from_json(cls, json_data, system, error_msg='Error not available'): + def from_json(cls, json_data, system, location, error_msg='Error not available'): return cls._construct( system, - json.dumps(json_data, indent=4), + json.dumps(json_data, skipkeys=False, indent=4), error_msg, - location=Location(json_data['location']), + location=location ) @classmethod diff --git a/common/lib/xmodule/xmodule/fields.py b/common/lib/xmodule/xmodule/fields.py index a9b4be4fcd..dc2f000286 100644 --- a/common/lib/xmodule/xmodule/fields.py +++ b/common/lib/xmodule/xmodule/fields.py @@ -58,8 +58,7 @@ class Date(ModelType): else: msg = "Field {0} has bad value '{1}'".format( self._name, field) - log.warning(msg) - return None + raise TypeError(msg) def to_json(self, value): """ @@ -76,9 +75,12 @@ class Date(ModelType): return value.strftime('%Y-%m-%dT%H:%M:%SZ') else: return value.isoformat() + else: + raise TypeError("Cannot convert {} to json".format(value)) TIMEDELTA_REGEX = re.compile(r'^((?P\d+?) day(?:s?))?(\s)?((?P\d+?) hour(?:s?))?(\s)?((?P\d+?) minute(?:s)?)?(\s)?((?P\d+?) second(?:s)?)?$') + class Timedelta(ModelType): def from_json(self, time_str): """ diff --git a/common/lib/xmodule/xmodule/foldit_module.py b/common/lib/xmodule/xmodule/foldit_module.py index fdab14b58d..cadf6cef0b 100644 --- a/common/lib/xmodule/xmodule/foldit_module.py +++ b/common/lib/xmodule/xmodule/foldit_module.py @@ -91,15 +91,18 @@ class FolditModule(FolditFields, XModule): PuzzleComplete.completed_puzzles(self.system.anonymous_student_id), key=lambda d: (d['set'], d['subset'])) - def puzzle_leaders(self, n=10): + def puzzle_leaders(self, n=10, courses=None): """ Returns a list of n pairs (user, score) corresponding to the top scores; the pairs are in descending order of score. """ from foldit.models import Score - leaders = [(e['username'], e['score']) for e in Score.get_tops_n(10)] - leaders.sort(key=lambda x:-x[1]) + if courses is None: + courses = [self.location.course_id] + + leaders = [(leader['username'], leader['score']) for leader in Score.get_tops_n(10, course_list=courses)] + leaders.sort(key=lambda x: -x[1]) return leaders @@ -184,7 +187,6 @@ class FolditDescriptor(FolditFields, XmlDescriptor, EditingDescriptor): filename_extension = "xml" has_score = True - template_dir_name = "foldit" js = {'coffee': [resource_string(__name__, 'js/src/html/edit.coffee')]} js_module_name = "HTMLEditingDescriptor" diff --git a/common/lib/xmodule/xmodule/gst_module.py b/common/lib/xmodule/xmodule/gst_module.py index e101d90b4c..5c902f48c2 100644 --- a/common/lib/xmodule/xmodule/gst_module.py +++ b/common/lib/xmodule/xmodule/gst_module.py @@ -19,10 +19,45 @@ from xblock.core import String, Scope log = logging.getLogger(__name__) +DEFAULT_RENDER=""" +

      Graphic slider tool: Dynamic range and implicit functions.

      + +

      You can make the range of the x axis (but not ticks of x axis) of + functions depend on a parameter value. This can be useful when the + function domain needs to be variable.

      +

      Implicit functions like a circle can be plotted as 2 separate + functions of the same color.

      +
      + + +
      + +""" +DEFAULT_CONFIGURATION=""" + + + + + Math.sqrt(r * r - x * x) + -Math.sqrt(r * r - x * x) + + + + + -r + r + + 1000 + -30, 6, 30 + -30, 6, 30 + +""" + + class GraphicalSliderToolFields(object): - render = String(scope=Scope.content) - configuration = String(scope=Scope.content) + render = String(scope=Scope.content, default=DEFAULT_RENDER) + configuration = String(scope=Scope.content, default=DEFAULT_CONFIGURATION) class GraphicalSliderToolModule(GraphicalSliderToolFields, XModule): @@ -141,7 +176,6 @@ class GraphicalSliderToolModule(GraphicalSliderToolFields, XModule): class GraphicalSliderToolDescriptor(GraphicalSliderToolFields, MakoModuleDescriptor, XmlDescriptor): module_class = GraphicalSliderToolModule - template_dir_name = 'graphical_slider_tool' @classmethod def definition_from_xml(cls, xml_object, system): diff --git a/common/lib/xmodule/xmodule/html_module.py b/common/lib/xmodule/xmodule/html_module.py index 0f7e789906..a482a86fe7 100644 --- a/common/lib/xmodule/xmodule/html_module.py +++ b/common/lib/xmodule/xmodule/html_module.py @@ -13,12 +13,21 @@ from xmodule.html_checker import check_html from xmodule.stringify import stringify_children from xmodule.x_module import XModule from xmodule.xml_module import XmlDescriptor, name_to_pathname +import textwrap log = logging.getLogger("mitx.courseware") class HtmlFields(object): - data = String(help="Html contents to display for this module", scope=Scope.content) + display_name = String( + display_name="Display Name", + help="This name appears in the horizontal navigation at the top of the page.", + scope=Scope.settings, + # it'd be nice to have a useful default but it screws up other things; so, + # use display_name_with_default for those + default="Blank HTML Page" + ) + data = String(help="Html contents to display for this module", default=u"", scope=Scope.content) source_code = String(help="Source code for LaTeX documents. This feature is not well-supported.", scope=Scope.settings) @@ -32,7 +41,7 @@ class HtmlModule(HtmlFields, XModule): css = {'scss': [resource_string(__name__, 'css/html/display.scss')]} def get_html(self): - if self.system.anonymous_student_id: + if self.system.anonymous_student_id: return self.data.replace("%%USER_ID%%", self.system.anonymous_student_id) return self.data @@ -158,9 +167,9 @@ class HtmlDescriptor(HtmlFields, XmlDescriptor, EditingDescriptor): pathname=pathname) resource_fs.makedir(os.path.dirname(filepath), recursive=True, allow_recreate=True) - with resource_fs.open(filepath, 'w') as file: + with resource_fs.open(filepath, 'w') as filestream: html_data = self.data.encode('utf-8') - file.write(html_data) + filestream.write(html_data) # write out the relative name relname = path(pathname).basename() @@ -169,26 +178,88 @@ class HtmlDescriptor(HtmlFields, XmlDescriptor, EditingDescriptor): elt.set("filename", relname) return elt +class AboutFields(object): + display_name = String( + help="Display name for this module", + scope=Scope.settings, + default="overview", + ) + data = String( + help="Html contents to display for this module", + default="", + scope=Scope.content + ) -class AboutDescriptor(HtmlDescriptor): +class AboutModule(AboutFields, HtmlModule): + """ + Overriding defaults but otherwise treated as HtmlModule. + """ + pass + +class AboutDescriptor(AboutFields, HtmlDescriptor): """ These pieces of course content are treated as HtmlModules but we need to overload where the templates are located in order to be able to create new ones """ template_dir_name = "about" + module_class = AboutModule + +class StaticTabFields(object): + """ + The overrides for Static Tabs + """ + display_name = String( + display_name="Display Name", + help="This name appears in the horizontal navigation at the top of the page.", + scope=Scope.settings, + default="Empty", + ) + data = String( + default=textwrap.dedent("""\ +

      This is where you can add additional pages to your courseware. Click the 'edit' button to begin editing.

      + """), + scope=Scope.content, + help="HTML for the additional pages" + ) -class StaticTabDescriptor(HtmlDescriptor): +class StaticTabModule(StaticTabFields, HtmlModule): + """ + Supports the field overrides + """ + pass + +class StaticTabDescriptor(StaticTabFields, HtmlDescriptor): """ These pieces of course content are treated as HtmlModules but we need to overload where the templates are located in order to be able to create new ones """ - template_dir_name = "statictab" + template_dir_name = None + module_class = StaticTabModule -class CourseInfoDescriptor(HtmlDescriptor): +class CourseInfoFields(object): + """ + Field overrides + """ + data = String( + help="Html contents to display for this module", + default="
        ", + scope=Scope.content + ) + + +class CourseInfoModule(CourseInfoFields, HtmlModule): + """ + Just to support xblock field overrides + """ + pass + + +class CourseInfoDescriptor(CourseInfoFields, HtmlDescriptor): """ These pieces of course content are treated as HtmlModules but we need to overload where the templates are located in order to be able to create new ones """ - template_dir_name = "courseinfo" + template_dir_name = None + module_class = CourseInfoModule diff --git a/common/lib/xmodule/xmodule/js/fixtures/problem.html b/common/lib/xmodule/xmodule/js/fixtures/problem.html index 525b4323b7..07e147a9e7 100644 --- a/common/lib/xmodule/xmodule/js/fixtures/problem.html +++ b/common/lib/xmodule/xmodule/js/fixtures/problem.html @@ -1,6 +1,6 @@
        diff --git a/common/lib/xmodule/xmodule/js/fixtures/problem_content.html b/common/lib/xmodule/xmodule/js/fixtures/problem_content.html index 3e580be722..5ccce952e7 100644 --- a/common/lib/xmodule/xmodule/js/fixtures/problem_content.html +++ b/common/lib/xmodule/xmodule/js/fixtures/problem_content.html @@ -1,5 +1,8 @@

        Problem Header

        +
        +
        +

        Problem Content

        diff --git a/common/lib/xmodule/xmodule/js/spec/capa/display_spec.coffee b/common/lib/xmodule/xmodule/js/spec/capa/display_spec.coffee index 1efaa6c852..33d74e2335 100644 --- a/common/lib/xmodule/xmodule/js/spec/capa/display_spec.coffee +++ b/common/lib/xmodule/xmodule/js/spec/capa/display_spec.coffee @@ -77,6 +77,25 @@ describe 'Problem', -> [@problem.updateMathML, @stubbedJax, $('#input_example_1').get(0)] ] + describe 'renderProgressState', -> + beforeEach -> + @problem = new Problem($('.xmodule_display')) + #@renderProgressState = @problem.renderProgressState + + describe 'with a status of "none"', -> + it 'reports the number of points possible', -> + @problem.el.data('progress_status', 'none') + @problem.el.data('progress_detail', '0/1') + @problem.renderProgressState() + expect(@problem.$('.problem-progress').html()).toEqual "(1 point possible)" + + describe 'with any other valid status', -> + it 'reports the current score', -> + @problem.el.data('progress_status', 'foo') + @problem.el.data('progress_detail', '1/1') + @problem.renderProgressState() + expect(@problem.$('.problem-progress').html()).toEqual "(1/1 points)" + describe 'render', -> beforeEach -> @problem = new Problem($('.xmodule_display')) @@ -223,6 +242,58 @@ describe 'Problem', -> expect($('label[for="input_1_1_3"]')).toHaveAttr 'correct_answer', 'true' expect($('label[for="input_1_2_1"]')).not.toHaveAttr 'correct_answer', 'true' + describe 'radio text question', -> + radio_text_xml=''' +
        +

        + +
        +
        + +
        +
        +
        + + +

        + +
        + + +

        +
        +
        + + +

        +
        +
        +
        +''' + beforeEach -> + # Append a radiotextresponse problem to the problem, so we can check it's javascript functionality + @problem.el.prepend(radio_text_xml) + + it 'sets the correct class on the section for the correct choice', -> + spyOn($, 'postWithPrefix').andCallFake (url, callback) -> + callback answers: "1_2_1": ["1_2_1_choiceinput_0bc"], "1_2_1_choiceinput_0bc": "3" + @problem.show() + + expect($('#forinput1_2_1_choiceinput_0bc').attr('class')).toEqual( + 'choicetextgroup_show_correct') + expect($('#answer_1_2_1_choiceinput_0bc').text()).toEqual('3') + expect($('#answer_1_2_1_choiceinput_1bc').text()).toEqual('') + expect($('#answer_1_2_1_choiceinput_2bc').text()).toEqual('') + + it 'Should not disable input fields', -> + spyOn($, 'postWithPrefix').andCallFake (url, callback) -> + callback answers: "1_2_1": ["1_2_1_choiceinput_0bc"], "1_2_1_choiceinput_0bc": "3" + @problem.show() + expect($('input#1_2_1_choiceinput_0bc').attr('disabled')).not.toEqual('disabled') + expect($('input#1_2_1_choiceinput_1bc').attr('disabled')).not.toEqual('disabled') + expect($('input#1_2_1_choiceinput_2bc').attr('disabled')).not.toEqual('disabled') + expect($('input#1_2_1').attr('disabled')).not.toEqual('disabled') + describe 'when the answers are already shown', -> beforeEach -> @problem.el.addClass 'showed' diff --git a/common/lib/xmodule/xmodule/js/spec/combinedopenended/edit_spec.coffee b/common/lib/xmodule/xmodule/js/spec/combinedopenended/edit_spec.coffee index aa077da450..d859a59dda 100644 --- a/common/lib/xmodule/xmodule/js/spec/combinedopenended/edit_spec.coffee +++ b/common/lib/xmodule/xmodule/js/spec/combinedopenended/edit_spec.coffee @@ -11,13 +11,13 @@ describe 'OpenEndedMarkdownEditingDescriptor', -> @descriptor = new OpenEndedMarkdownEditingDescriptor($('.combinedopenended-editor')) @descriptor.createXMLEditor('replace with markdown') saveResult = @descriptor.save() - expect(saveResult.metadata.markdown).toEqual(null) + expect(saveResult.nullout).toEqual(['markdown']) expect(saveResult.data).toEqual('replace with markdown') it 'saves xml from the xml editor', -> loadFixtures 'combinedopenended-without-markdown.html' @descriptor = new OpenEndedMarkdownEditingDescriptor($('.combinedopenended-editor')) saveResult = @descriptor.save() - expect(saveResult.metadata.markdown).toEqual(null) + expect(saveResult.nullout).toEqual(['markdown']) expect(saveResult.data).toEqual('xml only') describe 'insertPrompt', -> diff --git a/common/lib/xmodule/xmodule/js/spec/problem/edit_spec.coffee b/common/lib/xmodule/xmodule/js/spec/problem/edit_spec.coffee index 5161e658e7..1df9587037 100644 --- a/common/lib/xmodule/xmodule/js/spec/problem/edit_spec.coffee +++ b/common/lib/xmodule/xmodule/js/spec/problem/edit_spec.coffee @@ -11,13 +11,13 @@ describe 'MarkdownEditingDescriptor', -> @descriptor = new MarkdownEditingDescriptor($('.problem-editor')) @descriptor.createXMLEditor('replace with markdown') saveResult = @descriptor.save() - expect(saveResult.metadata.markdown).toEqual(null) + expect(saveResult.nullout).toEqual(['markdown']) expect(saveResult.data).toEqual('replace with markdown') it 'saves xml from the xml editor', -> loadFixtures 'problem-without-markdown.html' @descriptor = new MarkdownEditingDescriptor($('.problem-editor')) saveResult = @descriptor.save() - expect(saveResult.metadata.markdown).toEqual(null) + expect(saveResult.nullout).toEqual(['markdown']) expect(saveResult.data).toEqual('xml only') describe 'insertMultipleChoice', -> diff --git a/common/lib/xmodule/xmodule/js/src/capa/display.coffee b/common/lib/xmodule/xmodule/js/src/capa/display.coffee index e29276936b..09a398dd7a 100644 --- a/common/lib/xmodule/xmodule/js/src/capa/display.coffee +++ b/common/lib/xmodule/xmodule/js/src/capa/display.coffee @@ -22,7 +22,6 @@ class @Problem @$('section.action input:button').click @refreshAnswers @$('section.action input.check').click @check_fd - #@$('section.action input.check').click @check @$('section.action input.reset').click @reset @$('section.action button.show').click @show @$('section.action input.save').click @save @@ -36,15 +35,34 @@ class @Problem @$('input.math').each (index, element) => MathJax.Hub.Queue [@refreshMath, null, element] + renderProgressState: => + detail = @el.data('progress_detail') + status = @el.data('progress_status') + # i18n + progress = "(#{detail} points)" + if status == 'none' and detail? and detail.indexOf('/') > 0 + a = detail.split('/') + possible = parseInt(a[1]) + if possible == 1 + # i18n + progress = "(#{possible} point possible)" + else + # i18n + progress = "(#{possible} points possible)" + @$('.problem-progress').html(progress) + updateProgress: (response) => if response.progress_changed - @el.attr progress: response.progress_status + @el.data('progress_status', response.progress_status) + @el.data('progress_detail', response.progress_detail) @el.trigger('progressChanged') + @renderProgressState() forceUpdate: (response) => - @el.attr progress: response.progress_status + @el.data('progress_status', response.progress_status) + @el.data('progress_detail', response.progress_detail) @el.trigger('progressChanged') - + @renderProgressState() queueing: => @queued_items = @$(".xqueue") @@ -114,7 +132,7 @@ class @Problem @setupInputTypes() @bind() @queueing() - + @forceUpdate response # TODO add hooks for problem types here by inspecting response.html and doing # stuff if a div w a class is found @@ -162,9 +180,6 @@ class @Problem # maybe preferable to consolidate all dispatches to use FormData ### check_fd: => - # Calling check from check_fd will result in firing the 'problem_check' event twice, since it is also called in the check function. - #Logger.log 'problem_check', @answers - # If there are no file inputs in the problem, we can fall back on @check if $('input:file').length == 0 @check() @@ -239,6 +254,12 @@ class @Problem check: => @check_waitfor() Logger.log 'problem_check', @answers + + # Segment.io + analytics.track "Problem Checked", + problem_id: @id + answers: @answers + $.postWithPrefix "#{@url}/problem_check", @answers, (response) => switch response.success when 'incorrect', 'correct' @@ -401,6 +422,14 @@ class @Problem answer = JSON.parse(answers[answer_id]) display.showAnswer(answer) + choicetextgroup: (element, display, answers) => + element = $(element) + + input_id = element.attr('id').replace(/inputtype_/,'') + answer = answers[input_id] + for choice in answer + element.find("section#forinput#{choice}").addClass 'choicetextgroup_show_correct' + inputtypeHideAnswerMethods: choicegroup: (element, display) => element = $(element) @@ -408,3 +437,7 @@ class @Problem javascriptinput: (element, display) => display.hideAnswer() + + choicetextgroup: (element, display) => + element = $(element) + element.find("section[id^='forinput']").removeClass('choicetextgroup_show_correct') diff --git a/common/lib/xmodule/xmodule/js/src/combinedopenended/edit.coffee b/common/lib/xmodule/xmodule/js/src/combinedopenended/edit.coffee index 1b7f9bb4fb..1f3b711804 100644 --- a/common/lib/xmodule/xmodule/js/src/combinedopenended/edit.coffee +++ b/common/lib/xmodule/xmodule/js/src/combinedopenended/edit.coffee @@ -50,6 +50,10 @@ Write a persuasive essay to a newspaper reflecting your vies on censorship in li mode: null }) @setCurrentEditor(@markdown_editor) + selection = @markdown_editor.getSelection() + #Auto-add in the needed template if it isn't already in there. + if(@markdown_editor.getValue() == "") + @markdown_editor.setValue(OpenEndedMarkdownEditingDescriptor.promptTemplate + "\n" + OpenEndedMarkdownEditingDescriptor.rubricTemplate + "\n" + OpenEndedMarkdownEditingDescriptor.tasksTemplate) # Add listeners for toolbar buttons (only present for markdown editor) @element.on('click', '.xml-tab', @onShowXMLButton) @element.on('click', '.format-buttons a', @onToolbarButton) @@ -153,8 +157,7 @@ Write a persuasive essay to a newspaper reflecting your vies on censorship in li else { data: @xml_editor.getValue() - metadata: - markdown: null + nullout: ['markdown'] } @insertRubric: (selectedText) -> diff --git a/common/lib/xmodule/xmodule/js/src/problem/edit.coffee b/common/lib/xmodule/xmodule/js/src/problem/edit.coffee index b723f230e9..bd2871eb61 100644 --- a/common/lib/xmodule/xmodule/js/src/problem/edit.coffee +++ b/common/lib/xmodule/xmodule/js/src/problem/edit.coffee @@ -123,9 +123,8 @@ class @MarkdownEditingDescriptor extends XModule.Descriptor } else { - data: @xml_editor.getValue() - metadata: - markdown: null + data: @xml_editor.getValue() + nullout: ['markdown'] } @insertMultipleChoice: (selectedText) -> diff --git a/common/lib/xmodule/xmodule/js/src/sequence/display.coffee b/common/lib/xmodule/xmodule/js/src/sequence/display.coffee index 495b734785..bae1b89bab 100644 --- a/common/lib/xmodule/xmodule/js/src/sequence/display.coffee +++ b/common/lib/xmodule/xmodule/js/src/sequence/display.coffee @@ -45,7 +45,7 @@ class @Sequence new_progress = "NA" _this = this $('.problems-wrapper').each (index) -> - progress = $(this).attr 'progress' + progress = $(this).data 'progress_status' new_progress = _this.mergeProgress progress, new_progress @progressTable[@position] = new_progress diff --git a/common/lib/xmodule/xmodule/modulestore/__init__.py b/common/lib/xmodule/xmodule/modulestore/__init__.py index 2fa12e2e90..eb721dfc99 100644 --- a/common/lib/xmodule/xmodule/modulestore/__init__.py +++ b/common/lib/xmodule/xmodule/modulestore/__init__.py @@ -310,14 +310,7 @@ class ModuleStore(object): """ raise NotImplementedError - def clone_item(self, source, location): - """ - Clone a new item that is a copy of the item at the location `source` - and writes it to `location` - """ - raise NotImplementedError - - def update_item(self, location, data): + def update_item(self, location, data, allow_not_found=False): """ Set the data in the item specified by the location to data diff --git a/common/lib/xmodule/xmodule/modulestore/exceptions.py b/common/lib/xmodule/xmodule/modulestore/exceptions.py index a63efc3e43..508599b677 100644 --- a/common/lib/xmodule/xmodule/modulestore/exceptions.py +++ b/common/lib/xmodule/xmodule/modulestore/exceptions.py @@ -7,10 +7,18 @@ class ItemNotFoundError(Exception): pass +class ItemWriteConflictError(Exception): + pass + + class InsufficientSpecificationError(Exception): pass +class OverSpecificationError(Exception): + pass + + class InvalidLocationError(Exception): pass @@ -21,3 +29,13 @@ class NoPathToItem(Exception): class DuplicateItemError(Exception): pass + + +class VersionConflictError(Exception): + """ + The caller asked for either draft or published head and gave a version which conflicted with it. + """ + def __init__(self, requestedLocation, currentHead): + super(VersionConflictError, self).__init__() + self.requestedLocation = requestedLocation + self.currentHead = currentHead diff --git a/common/lib/xmodule/xmodule/modulestore/inheritance.py b/common/lib/xmodule/xmodule/modulestore/inheritance.py index a816aa9776..1314c72094 100644 --- a/common/lib/xmodule/xmodule/modulestore/inheritance.py +++ b/common/lib/xmodule/xmodule/modulestore/inheritance.py @@ -50,6 +50,8 @@ def inherit_metadata(descriptor, model_data): def own_metadata(module): + # IN SPLIT MONGO this is just ['metadata'] as it keeps ['_inherited_metadata'] separate! + # FIXME move into kvs? will that work for xml mongo? """ Return a dictionary that contains only non-inherited field keys, mapped to their values diff --git a/common/lib/xmodule/xmodule/modulestore/locator.py b/common/lib/xmodule/xmodule/modulestore/locator.py new file mode 100644 index 0000000000..928bc9f133 --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/locator.py @@ -0,0 +1,465 @@ +""" +Created on Mar 13, 2013 + +@author: dmitchell +""" +from __future__ import absolute_import +import logging +import inspect +from abc import ABCMeta, abstractmethod +from urllib import quote + +from bson.objectid import ObjectId +from bson.errors import InvalidId + +from xmodule.modulestore.exceptions import InsufficientSpecificationError, OverSpecificationError + +from .parsers import parse_url, parse_course_id, parse_block_ref + +log = logging.getLogger(__name__) + + +class Locator(object): + """ + A locator is like a URL, it refers to a course resource. + + Locator is an abstract base class: do not instantiate + """ + + __metaclass__ = ABCMeta + + @abstractmethod + def url(self): + """ + Return a string containing the URL for this location. Raises + InsufficientSpecificationError if the instance doesn't have a + complete enough specification to generate a url + """ + raise InsufficientSpecificationError() + + def quoted_url(self): + return quote(self.url(), '@;#') + + def __eq__(self, other): + return self.__dict__ == other.__dict__ + + def __repr__(self): + ''' + repr(self) returns something like this: CourseLocator("edu.mit.eecs.6002x") + ''' + classname = self.__class__.__name__ + if classname.find('.') != -1: + classname = classname.split['.'][-1] + return '%s("%s")' % (classname, unicode(self)) + + def __str__(self): + ''' + str(self) returns something like this: "edu.mit.eecs.6002x" + ''' + return unicode(self).encode('utf8') + + def __unicode__(self): + ''' + unicode(self) returns something like this: "edu.mit.eecs.6002x" + ''' + return self.url() + + @abstractmethod + def version(self): + """ + Returns the ObjectId referencing this specific location. + Raises InsufficientSpecificationError if the instance + doesn't have a complete enough specification. + """ + raise InsufficientSpecificationError() + + def set_property(self, property_name, new): + """ + Initialize property to new value. + If property has already been initialized to a different value, raise an exception. + """ + current = getattr(self, property_name) + if current and current != new: + raise OverSpecificationError('%s cannot be both %s and %s' % + (property_name, current, new)) + setattr(self, property_name, new) + + +class CourseLocator(Locator): + """ + Examples of valid CourseLocator specifications: + CourseLocator(version_guid=ObjectId('519665f6223ebd6980884f2b')) + CourseLocator(course_id='edu.mit.eecs.6002x') + CourseLocator(course_id='edu.mit.eecs.6002x;published') + CourseLocator(course_id='edu.mit.eecs.6002x', revision='published') + CourseLocator(url='edx://@519665f6223ebd6980884f2b') + CourseLocator(url='edx://edu.mit.eecs.6002x') + CourseLocator(url='edx://edu.mit.eecs.6002x;published') + + Should have at lease a specific course_id (id for the course as if it were a project w/ + versions) with optional 'revision' (must be 'draft', 'published', or None), + or version_guid (which points to a specific version). Can contain both in which case + the persistence layer may raise exceptions if the given version != the current such version + of the course. + """ + + # Default values + version_guid = None + course_id = None + revision = None + + def __unicode__(self): + """ + Return a string representing this location. + """ + if self.course_id: + result = self.course_id + if self.revision: + result += ';' + self.revision + return result + elif self.version_guid: + return '@' + str(self.version_guid) + else: + # raise InsufficientSpecificationError("missing course_id or version_guid") + return '' + + def url(self): + """ + Return a string containing the URL for this location. + """ + return 'edx://' + unicode(self) + + # -- unused args which are used via inspect + # pylint: disable= W0613 + def validate_args(self, url, version_guid, course_id, revision): + """ + Validate provided arguments. + """ + need_oneof = set(('url', 'version_guid', 'course_id')) + args, _, _, values = inspect.getargvalues(inspect.currentframe()) + provided_args = [a for a in args if a != 'self' and values[a] is not None] + if len(need_oneof.intersection(provided_args)) == 0: + raise InsufficientSpecificationError("Must provide one of these args: %s " % + list(need_oneof)) + + def is_fully_specified(self): + """ + Returns True if either version_guid is specified, or course_id+revision + are specified. + This should always return True, since this should be validated in the constructor. + """ + return self.version_guid is not None \ + or (self.course_id is not None and self.revision is not None) + + def set_course_id(self, new): + """ + Initialize course_id to new value. + If course_id has already been initialized to a different value, raise an exception. + """ + self.set_property('course_id', new) + + def set_revision(self, new): + """ + Initialize revision to new value. + If revision has already been initialized to a different value, raise an exception. + """ + self.set_property('revision', new) + + def set_version_guid(self, new): + """ + Initialize version_guid to new value. + If version_guid has already been initialized to a different value, raise an exception. + """ + self.set_property('version_guid', new) + + def as_course_locator(self): + """ + Returns a copy of itself (downcasting) as a CourseLocator. + The copy has the same CourseLocator fields as the original. + The copy does not include subclass information, such as + a usage_id (a property of BlockUsageLocator). + """ + return CourseLocator(course_id=self.course_id, + version_guid=self.version_guid, + revision=self.revision) + + def __init__(self, url=None, version_guid=None, course_id=None, revision=None): + """ + Construct a CourseLocator + Caller may provide url (but no other parameters). + Caller may provide version_guid (but no other parameters). + Caller may provide course_id (optionally provide revision). + + Resulting CourseLocator will have either a version_guid property + or a course_id (with optional revision) property, or both. + + version_guid must be an instance of bson.objectid.ObjectId or None + url, course_id, and revision must be strings or None + + """ + self.validate_args(url, version_guid, course_id, revision) + if url: + self.init_from_url(url) + if version_guid: + self.init_from_version_guid(version_guid) + if course_id or revision: + self.init_from_course_id(course_id, revision) + assert self.version_guid or self.course_id, \ + "Either version_guid or course_id should be set." + + @classmethod + def as_object_id(cls, value): + """ + Attempts to cast value as a bson.objectid.ObjectId. + If cast fails, raises ValueError + """ + if isinstance(value, ObjectId): + return value + try: + return ObjectId(value) + except InvalidId: + raise ValueError('"%s" is not a valid version_guid' % value) + + def init_from_url(self, url): + """ + url must be a string beginning with 'edx://' and containing + either a valid version_guid or course_id (with optional revision) + If a block ('#HW3') is present, it is ignored. + """ + if isinstance(url, Locator): + url = url.url() + assert isinstance(url, basestring), \ + '%s is not an instance of basestring' % url + parse = parse_url(url) + assert parse, 'Could not parse "%s" as a url' % url + if 'version_guid' in parse: + new_guid = parse['version_guid'] + self.set_version_guid(self.as_object_id(new_guid)) + else: + self.set_course_id(parse['id']) + self.set_revision(parse['revision']) + + def init_from_version_guid(self, version_guid): + """ + version_guid must be an instance of bson.objectid.ObjectId, + or able to be cast as one. + If it's a string, attempt to cast it as an ObjectId first. + """ + version_guid = self.as_object_id(version_guid) + + assert isinstance(version_guid, ObjectId), \ + '%s is not an instance of ObjectId' % version_guid + self.set_version_guid(version_guid) + + def init_from_course_id(self, course_id, explicit_revision=None): + """ + Course_id is a string like 'edu.mit.eecs.6002x' or 'edu.mit.eecs.6002x;published'. + + Revision (optional) is a string like 'published'. + It may be provided explicitly (explicit_revision) or embedded into course_id. + If revision is part of course_id ("...;published"), parse it out separately. + If revision is provided both ways, that's ok as long as they are the same value. + + If a block ('#HW3') is a part of course_id, it is ignored. + + """ + + if course_id: + if isinstance(course_id, CourseLocator): + course_id = course_id.course_id + assert course_id, "%s does not have a valid course_id" + + parse = parse_course_id(course_id) + assert parse, 'Could not parse "%s" as a course_id' % course_id + self.set_course_id(parse['id']) + rev = parse['revision'] + if rev: + self.set_revision(rev) + if explicit_revision: + self.set_revision(explicit_revision) + + def version(self): + """ + Returns the ObjectId referencing this specific location. + """ + return self.version_guid + + def html_id(self): + """ + Generate a discussion group id based on course + + To make compatible with old Location object functionality. I don't believe this behavior fits at this + place, but I have no way to override. If this is really needed, it should probably use the pretty_id to seed + the name although that's mutable. We should also clearly define the purpose and restrictions of this + (e.g., I'm assuming periods are fine). + """ + return self.course_id + + +class BlockUsageLocator(CourseLocator): + """ + Encodes a location. + + Locations address modules (aka blocks) which are definitions situated in a + course instance. Thus, a Location must identify the course and the occurrence of + the defined element in the course. Courses can be a version of an offering, the + current draft head, or the current production version. + + Locators can contain both a version and a course_id w/ revision. The split mongo functions + may raise errors if these conflict w/ the current db state (i.e., the course's revision != + the version_guid) + + Locations can express as urls as well as dictionaries. They consist of + course_identifier: course_guid | version_guid + block : guid + revision : 'draft' | 'published' (optional) + """ + + # Default value + usage_id = None + + def __init__(self, url=None, version_guid=None, course_id=None, + revision=None, usage_id=None): + """ + Construct a BlockUsageLocator + Caller may provide url, version_guid, or course_id, and optionally provide revision. + + The usage_id may be specified, either explictly or as part of + the url or course_id. If omitted, the locator is created but it + has not yet been initialized. + + Resulting BlockUsageLocator will have a usage_id property. + It will have either a version_guid property or a course_id (with optional revision) property, or both. + + version_guid must be an instance of bson.objectid.ObjectId or None + url, course_id, revision, and usage_id must be strings or None + + """ + self.validate_args(url, version_guid, course_id, revision) + if url: + self.init_block_ref_from_url(url) + if course_id: + self.init_block_ref_from_course_id(course_id) + if usage_id: + self.init_block_ref(usage_id) + CourseLocator.__init__(self, + url=url, + version_guid=version_guid, + course_id=course_id, + revision=revision) + + def is_initialized(self): + """ + Returns True if usage_id has been initialized, else returns False + """ + return self.usage_id is not None + + def version_agnostic(self): + """ + Returns a copy of itself. + If both version_guid and course_id are known, use a blank course_id in the copy. + + We don't care if the locator's version is not the current head; so, avoid version conflict + by reducing info. + + :param block_locator: + """ + if self.course_id and self.version_guid: + return BlockUsageLocator(version_guid=self.version_guid, + revision=self.revision, + usage_id=self.usage_id) + else: + return BlockUsageLocator(course_id=self.course_id, + revision=self.revision, + usage_id=self.usage_id) + + def set_usage_id(self, new): + """ + Initialize usage_id to new value. + If usage_id has already been initialized to a different value, raise an exception. + """ + self.set_property('usage_id', new) + + def init_block_ref(self, block_ref): + parse = parse_block_ref(block_ref) + assert parse, 'Could not parse "%s" as a block_ref' % block_ref + self.set_usage_id(parse['block']) + + def init_block_ref_from_url(self, url): + if isinstance(url, Locator): + url = url.url() + parse = parse_url(url) + assert parse, 'Could not parse "%s" as a url' % url + block = parse.get('block', None) + if block: + self.set_usage_id(block) + + def init_block_ref_from_course_id(self, course_id): + if isinstance(course_id, CourseLocator): + course_id = course_id.course_id + assert course_id, "%s does not have a valid course_id" + parse = parse_course_id(course_id) + assert parse, 'Could not parse "%s" as a course_id' % course_id + block = parse.get('block', None) + if block: + self.set_usage_id(block) + + def __unicode__(self): + """ + Return a string representing this location. + """ + rep = CourseLocator.__unicode__(self) + if self.usage_id is None: + # usage_id has not been initialized + return rep + '#NONE' + else: + return rep + '#' + self.usage_id + + +class DescriptionLocator(Locator): + """ + Container for how to locate a description + """ + + def __init__(self, definition_id): + self.definition_id = definition_id + + def __unicode__(self): + ''' + Return a string representing this location. + unicode(self) returns something like this: "@519665f6223ebd6980884f2b" + ''' + return '@' + str(self.definition_guid) + + def url(self): + """ + Return a string containing the URL for this location. + url(self) returns something like this: 'edx://@519665f6223ebd6980884f2b' + """ + return 'edx://' + unicode(self) + + def version(self): + """ + Returns the ObjectId referencing this specific location. + """ + return self.definition_guid + + +class VersionTree(object): + """ + Holds trees of Locators to represent version histories. + """ + def __init__(self, locator, tree_dict=None): + """ + :param locator: must be version specific (Course has version_guid or definition had id) + """ + assert isinstance(locator, Locator) and not inspect.isabstract(locator), \ + "locator must be a concrete subclass of Locator" + assert locator.version(), \ + "locator must be version specific (Course has version_guid or definition had id)" + self.locator = locator + if tree_dict is None: + self.children = [] + else: + self.children = [VersionTree(child, tree_dict) + for child in tree_dict.get(locator.version(), [])] diff --git a/common/lib/xmodule/xmodule/modulestore/mongo/base.py b/common/lib/xmodule/xmodule/modulestore/mongo/base.py index f56393d75e..bedba5d65d 100644 --- a/common/lib/xmodule/xmodule/modulestore/mongo/base.py +++ b/common/lib/xmodule/xmodule/modulestore/mongo/base.py @@ -33,7 +33,7 @@ from xblock.runtime import DbModel, KeyValueStore, InvalidScopeError from xblock.core import Scope from xmodule.modulestore import ModuleStoreBase, Location, namedtuple_to_son -from xmodule.modulestore.exceptions import ItemNotFoundError, DuplicateItemError +from xmodule.modulestore.exceptions import ItemNotFoundError from xmodule.modulestore.inheritance import own_metadata, INHERITABLE_METADATA, inherit_metadata log = logging.getLogger(__name__) @@ -62,11 +62,12 @@ class MongoKeyValueStore(KeyValueStore): A KeyValueStore that maps keyed data access to one of the 3 data areas known to the MongoModuleStore (data, children, and metadata) """ - def __init__(self, data, children, metadata, location): + def __init__(self, data, children, metadata, location, category): self._data = data self._children = children self._metadata = metadata self._location = location + self._category = category def get(self, key): if key.scope == Scope.children: @@ -78,6 +79,8 @@ class MongoKeyValueStore(KeyValueStore): elif key.scope == Scope.content: if key.field_name == 'location': return self._location + elif key.field_name == 'category': + return self._category elif key.field_name == 'data' and not isinstance(self._data, dict): return self._data else: @@ -93,6 +96,8 @@ class MongoKeyValueStore(KeyValueStore): elif key.scope == Scope.content: if key.field_name == 'location': self._location = value + elif key.field_name == 'category': + self._category = value elif key.field_name == 'data' and not isinstance(self._data, dict): self._data = value else: @@ -109,6 +114,8 @@ class MongoKeyValueStore(KeyValueStore): elif key.scope == Scope.content: if key.field_name == 'location': self._location = Location(None) + elif key.field_name == 'category': + self._category = None elif key.field_name == 'data' and not isinstance(self._data, dict): self._data = None else: @@ -123,7 +130,10 @@ class MongoKeyValueStore(KeyValueStore): return key.field_name in self._metadata elif key.scope == Scope.content: if key.field_name == 'location': + # WHY TRUE? if it's been deleted should it be False? return True + elif key.field_name == 'category': + return self._category is not None elif key.field_name == 'data' and not isinstance(self._data, dict): return True else: @@ -185,8 +195,9 @@ class CachingDescriptorSystem(MakoDescriptorSystem): else: # load the module and apply the inherited metadata try: + category = json_data['location']['category'] class_ = XModuleDescriptor.load_class( - json_data['location']['category'], + category, self.default_class ) definition = json_data.get('definition', {}) @@ -201,9 +212,12 @@ class CachingDescriptorSystem(MakoDescriptorSystem): definition.get('children', []), metadata, location, + category ) model_data = DbModel(kvs, class_, None, MongoUsage(self.course_id, location)) + model_data['category'] = category + model_data['location'] = location module = class_(self, model_data) if self.cached_metadata is not None: # parent container pointers don't differentiate between draft and non-draft @@ -217,6 +231,7 @@ class CachingDescriptorSystem(MakoDescriptorSystem): return ErrorDescriptor.from_json( json_data, self, + json_data['location'], error_msg=exc_info_to_str(sys.exc_info()) ) @@ -582,51 +597,97 @@ class MongoModuleStore(ModuleStoreBase): modules = self._load_items(list(items), depth) return modules - def clone_item(self, source, location): + def create_xmodule(self, location, definition_data=None, metadata=None, system=None): """ - Clone a new item that is a copy of the item at the location `source` - and writes it to `location` + Create the new xmodule but don't save it. Returns the new module. + + :param location: a Location--must have a category + :param definition_data: can be empty. The initial definition_data for the kvs + :param metadata: can be empty, the initial metadata for the kvs + :param system: if you already have an xmodule from the course, the xmodule.system value """ - item = None - try: - source_item = self.collection.find_one(location_to_query(source)) - - # allow for some programmatically generated substitutions in metadata, e.g. Discussion_id's should be auto-generated - for key in source_item['metadata'].keys(): - if source_item['metadata'][key] == '$$GUID$$': - source_item['metadata'][key] = uuid4().hex - - source_item['_id'] = Location(location).dict() - self.collection.insert( - source_item, - # Must include this to avoid the django debug toolbar (which defines the deprecated "safe=False") - # from overriding our default value set in the init method. - safe=self.collection.safe + if not isinstance(location, Location): + location = Location(location) + # differs from split mongo in that I believe most of this logic should be above the persistence + # layer but added it here to enable quick conversion. I'll need to reconcile these. + if metadata is None: + metadata = {} + if system is None: + system = CachingDescriptorSystem( + self, + {}, + self.default_class, + None, + self.error_tracker, + self.render_template, + {} ) - item = self._load_items([source_item])[0] + xblock_class = XModuleDescriptor.load_class(location.category, self.default_class) + if definition_data is None: + if hasattr(xblock_class, 'data') and getattr(xblock_class, 'data').default is not None: + definition_data = getattr(xblock_class, 'data').default + else: + definition_data = {} + dbmodel = self._create_new_model_data(location.category, location, definition_data, metadata) + xmodule = xblock_class(system, dbmodel) + return xmodule - # VS[compat] cdodge: This is a hack because static_tabs also have references from the course module, so - # if we add one then we need to also add it to the policy information (i.e. metadata) - # we should remove this once we can break this reference from the course to static tabs - if location.category == 'static_tab': - course = self.get_course_for_item(item.location) - existing_tabs = course.tabs or [] - existing_tabs.append({ - 'type': 'static_tab', - 'name': item.display_name, - 'url_slug': item.location.name - }) - course.tabs = existing_tabs - self.update_metadata(course.location, course._model_data._kvs._metadata) - - except pymongo.errors.DuplicateKeyError: - raise DuplicateItemError(location) + def save_xmodule(self, xmodule): + """ + Save the given xmodule (will either create or update based on whether id already exists). + Pulls out the data definition v metadata v children locally but saves it all. + :param xmodule: + """ + # Save any changes to the xmodule to the MongoKeyValueStore + xmodule.save() + # split mongo's persist_dag is more general and useful. + self.collection.save({ + '_id': xmodule.location.dict(), + 'metadata': own_metadata(xmodule), + 'definition': { + 'data': xmodule.xblock_kvs._data, + 'children': xmodule.children if xmodule.has_children else [] + } + }) # recompute (and update) the metadata inheritance tree which is cached - self.refresh_cached_metadata_inheritance_tree(Location(location)) - self.fire_updated_modulestore_signal(get_course_id_no_run(Location(location)), Location(location)) + self.refresh_cached_metadata_inheritance_tree(xmodule.location) + self.fire_updated_modulestore_signal(get_course_id_no_run(xmodule.location), xmodule.location) - return item + def create_and_save_xmodule(self, location, definition_data=None, metadata=None, system=None): + """ + Create the new xmodule and save it. Does not return the new module because if the caller + will insert it as a child, it's inherited metadata will completely change. The difference + between this and just doing create_xmodule and save_xmodule is this ensures static_tabs get + pointed to by the course. + + :param location: a Location--must have a category + :param definition_data: can be empty. The initial definition_data for the kvs + :param metadata: can be empty, the initial metadata for the kvs + :param system: if you already have an xmodule from the course, the xmodule.system value + """ + # differs from split mongo in that I believe most of this logic should be above the persistence + # layer but added it here to enable quick conversion. I'll need to reconcile these. + new_object = self.create_xmodule(location, definition_data, metadata, system) + location = new_object.location + self.save_xmodule(new_object) + + # VS[compat] cdodge: This is a hack because static_tabs also have references from the course module, so + # if we add one then we need to also add it to the policy information (i.e. metadata) + # we should remove this once we can break this reference from the course to static tabs + # TODO move this special casing to app tier (similar to attaching new element to parent) + if location.category == 'static_tab': + course = self.get_course_for_item(location) + existing_tabs = course.tabs or [] + existing_tabs.append({ + 'type': 'static_tab', + 'name': new_object.display_name, + 'url_slug': new_object.location.name + }) + course.tabs = existing_tabs + # Save any changes to the course to the MongoKeyValueStore + course.save() + self.update_metadata(course.location, course.xblock_kvs._metadata) def fire_updated_modulestore_signal(self, course_id, location): """ @@ -683,7 +744,7 @@ class MongoModuleStore(ModuleStoreBase): if result['n'] == 0: raise ItemNotFoundError(location) - def update_item(self, location, data): + def update_item(self, location, data, allow_not_found=False): """ Set the data in the item specified by the location to data @@ -691,8 +752,11 @@ class MongoModuleStore(ModuleStoreBase): location: Something that can be passed to Location data: A nested dictionary of problem data """ - - self._update_single_item(location, {'definition.data': data}) + try: + self._update_single_item(location, {'definition.data': data}) + except ItemNotFoundError: + if not allow_not_found: + raise def update_children(self, location, children): """ @@ -729,6 +793,8 @@ class MongoModuleStore(ModuleStoreBase): tab['name'] = metadata.get('display_name') break course.tabs = existing_tabs + # Save the updates to the course to the MongoKeyValueStore + course.save() self.update_metadata(course.location, own_metadata(course)) self._update_single_item(location, {'metadata': metadata}) @@ -751,6 +817,8 @@ class MongoModuleStore(ModuleStoreBase): course = self.get_course_for_item(item.location) existing_tabs = course.tabs or [] course.tabs = [tab for tab in existing_tabs if tab.get('url_slug') != location.name] + # Save the updates to the course to the MongoKeyValueStore + course.save() self.update_metadata(course.location, own_metadata(course)) # Must include this to avoid the django debug toolbar (which defines the deprecated "safe=False") @@ -775,3 +843,24 @@ class MongoModuleStore(ModuleStoreBase): are loaded on demand, rather than up front """ return {} + + def _create_new_model_data(self, category, location, definition_data, metadata): + """ + To instantiate a new xmodule which will be saved latter, set up the dbModel and kvs + """ + kvs = MongoKeyValueStore( + definition_data, + [], + metadata, + location, + category + ) + + class_ = XModuleDescriptor.load_class( + category, + self.default_class + ) + model_data = DbModel(kvs, class_, None, MongoUsage(None, location)) + model_data['category'] = category + model_data['location'] = location + return model_data diff --git a/common/lib/xmodule/xmodule/modulestore/mongo/draft.py b/common/lib/xmodule/xmodule/modulestore/mongo/draft.py index f34c3a53f9..d289e03739 100644 --- a/common/lib/xmodule/xmodule/modulestore/mongo/draft.py +++ b/common/lib/xmodule/xmodule/modulestore/mongo/draft.py @@ -8,11 +8,12 @@ and otherwise returns i4x://org/course/cat/name). from datetime import datetime -from xmodule.modulestore import Location, namedtuple_to_son -from xmodule.modulestore.exceptions import ItemNotFoundError -from xmodule.modulestore.inheritance import own_metadata from xmodule.exceptions import InvalidVersionError -from xmodule.modulestore.mongo.base import MongoModuleStore +from xmodule.modulestore import Location, namedtuple_to_son +from xmodule.modulestore.exceptions import ItemNotFoundError, DuplicateItemError +from xmodule.modulestore.inheritance import own_metadata +from xmodule.modulestore.mongo.base import location_to_query, get_course_id_no_run, MongoModuleStore +import pymongo from pytz import UTC DRAFT = 'draft' @@ -92,6 +93,21 @@ class DraftModuleStore(MongoModuleStore): except ItemNotFoundError: return wrap_draft(super(DraftModuleStore, self).get_instance(course_id, location, depth=depth)) + def create_xmodule(self, location, definition_data=None, metadata=None, system=None): + """ + Create the new xmodule but don't save it. Returns the new module with a draft locator + + :param location: a Location--must have a category + :param definition_data: can be empty. The initial definition_data for the kvs + :param metadata: can be empty, the initial metadata for the kvs + :param system: if you already have an xmodule from the course, the xmodule.system value + """ + draft_loc = as_draft(location) + if draft_loc.category in DIRECT_ONLY_CATEGORIES: + raise InvalidVersionError(location) + return super(DraftModuleStore, self).create_xmodule(draft_loc, definition_data, metadata, system) + + def get_items(self, location, course_id=None, depth=0): """ Returns a list of XModuleDescriptor instances for the items @@ -119,14 +135,26 @@ class DraftModuleStore(MongoModuleStore): ] return [wrap_draft(item) for item in draft_items + non_draft_items] - def clone_item(self, source, location): + def convert_to_draft(self, source_location): """ - Clone a new item that is a copy of the item at the location `source` - and writes it to `location` + Create a copy of the source and mark its revision as draft. + + :param source: the location of the source (its revision must be None) """ - if Location(location).category in DIRECT_ONLY_CATEGORIES: - raise InvalidVersionError(location) - return wrap_draft(super(DraftModuleStore, self).clone_item(source, as_draft(location))) + original = self.collection.find_one(location_to_query(source_location)) + draft_location = as_draft(source_location) + if draft_location.category in DIRECT_ONLY_CATEGORIES: + raise InvalidVersionError(source_location) + original['_id'] = draft_location.dict() + try: + self.collection.insert(original) + except pymongo.errors.DuplicateKeyError: + raise DuplicateItemError(original['_id']) + + self.refresh_cached_metadata_inheritance_tree(draft_location) + self.fire_updated_modulestore_signal(get_course_id_no_run(draft_location), draft_location) + + return self._load_items([original])[0] def update_item(self, location, data, allow_not_found=False): """ @@ -140,7 +168,7 @@ class DraftModuleStore(MongoModuleStore): try: draft_item = self.get_item(location) if not getattr(draft_item, 'is_draft', False): - self.clone_item(location, draft_loc) + self.convert_to_draft(location) except ItemNotFoundError, e: if not allow_not_found: raise e @@ -158,7 +186,7 @@ class DraftModuleStore(MongoModuleStore): draft_loc = as_draft(location) draft_item = self.get_item(location) if not getattr(draft_item, 'is_draft', False): - self.clone_item(location, draft_loc) + self.convert_to_draft(location) return super(DraftModuleStore, self).update_children(draft_loc, children) @@ -174,7 +202,7 @@ class DraftModuleStore(MongoModuleStore): draft_item = self.get_item(location) if not getattr(draft_item, 'is_draft', False): - self.clone_item(location, draft_loc) + self.convert_to_draft(location) if 'is_draft' in metadata: del metadata['is_draft'] @@ -218,9 +246,7 @@ class DraftModuleStore(MongoModuleStore): """ Turn the published version into a draft, removing the published version """ - if Location(location).category in DIRECT_ONLY_CATEGORIES: - raise InvalidVersionError(location) - super(DraftModuleStore, self).clone_item(location, as_draft(location)) + self.convert_to_draft(location) super(DraftModuleStore, self).delete_item(location) def _query_children_for_cache_children(self, items): diff --git a/common/lib/xmodule/xmodule/modulestore/parsers.py b/common/lib/xmodule/xmodule/modulestore/parsers.py new file mode 100644 index 0000000000..9aac3073ae --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/parsers.py @@ -0,0 +1,115 @@ +import re + +URL_RE = re.compile(r'^edx://(.+)$', re.IGNORECASE) + + +def parse_url(string): + """ + A url must begin with 'edx://' (case-insensitive match), + followed by either a version_guid or a course_id. + + Examples: + 'edx://@0123FFFF' + 'edx://edu.mit.eecs.6002x' + 'edx://edu.mit.eecs.6002x;published' + 'edx://edu.mit.eecs.6002x;published#HW3' + + This returns None if string cannot be parsed. + + If it can be parsed as a version_guid, returns a dict + with key 'version_guid' and the value, + + If it can be parsed as a course_id, returns a dict + with keys 'id' and 'revision' (value of 'revision' may be None), + + """ + match = URL_RE.match(string) + if not match: + return None + path = match.group(1) + if path[0] == '@': + return parse_guid(path[1:]) + return parse_course_id(path) + + +BLOCK_RE = re.compile(r'^\w+$', re.IGNORECASE) + + +def parse_block_ref(string): + r""" + A block_ref is a string of word_chars. + + matches one or more Unicode word characters; this includes most + characters that can be part of a word in any language, as well as numbers + and the underscore. (see definition of \w in python regular expressions, + at http://docs.python.org/dev/library/re.html) + + If string is a block_ref, returns a dict with key 'block_ref' and the value, + otherwise returns None. + """ + if len(string) > 0 and BLOCK_RE.match(string): + return {'block': string} + return None + + +GUID_RE = re.compile(r'^(?P[A-F0-9]+)(#(?P\w+))?$', re.IGNORECASE) + + +def parse_guid(string): + """ + A version_guid is a string of hex digits (0-F). + + If string is a version_guid, returns a dict with key 'version_guid' and the value, + otherwise returns None. + """ + m = GUID_RE.match(string) + if m is not None: + return m.groupdict() + else: + return None + + +COURSE_ID_RE = re.compile(r'^(?P(\w+)(\.\w+\w*)*)(;(?P\w+))?(#(?P\w+))?$', re.IGNORECASE) + + +def parse_course_id(string): + r""" + + A course_id has a main id component. + There may also be an optional revision (;published or ;draft). + There may also be an optional block (#HW3 or #Quiz2). + + Examples of valid course_ids: + + 'edu.mit.eecs.6002x' + 'edu.mit.eecs.6002x;published' + 'edu.mit.eecs.6002x#HW3' + 'edu.mit.eecs.6002x;published#HW3' + + + Syntax: + + course_id = main_id [; revision] [# block] + + main_id = name [. name]* + + revision = name + + block = name + + name = + + matches one or more Unicode word characters; this includes most + characters that can be part of a word in any language, as well as numbers + and the underscore. (see definition of \w in python regular expressions, + at http://docs.python.org/dev/library/re.html) + + If string is a course_id, returns a dict with keys 'id', 'revision', and 'block'. + Revision is optional: if missing returned_dict['revision'] is None. + Block is optional: if missing returned_dict['block'] is None. + Else returns None. + """ + match = COURSE_ID_RE.match(string) + if not match: + return None + return match.groupdict() diff --git a/common/lib/xmodule/xmodule/modulestore/split_mongo/__init__.py b/common/lib/xmodule/xmodule/modulestore/split_mongo/__init__.py new file mode 100644 index 0000000000..65a4d723dd --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/split_mongo/__init__.py @@ -0,0 +1 @@ +from split import SplitMongoModuleStore diff --git a/common/lib/xmodule/xmodule/modulestore/split_mongo/caching_descriptor_system.py b/common/lib/xmodule/xmodule/modulestore/split_mongo/caching_descriptor_system.py new file mode 100644 index 0000000000..1591757490 --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/split_mongo/caching_descriptor_system.py @@ -0,0 +1,119 @@ +import sys +import logging +from xmodule.mako_module import MakoDescriptorSystem +from xmodule.x_module import XModuleDescriptor +from xmodule.modulestore.locator import BlockUsageLocator +from xmodule.error_module import ErrorDescriptor +from xmodule.errortracker import exc_info_to_str +from xblock.runtime import DbModel +from ..exceptions import ItemNotFoundError +from .split_mongo_kvs import SplitMongoKVS, SplitMongoKVSid + +log = logging.getLogger(__name__) + +# TODO should this be here or w/ x_module or ??? +class CachingDescriptorSystem(MakoDescriptorSystem): + """ + A system that has a cache of a course version's json that it will use to load modules + from, with a backup of calling to the underlying modulestore for more data. + + Computes the metadata inheritance upon creation. + """ + def __init__(self, modulestore, course_entry, module_data, lazy, + default_class, error_tracker, render_template): + """ + Computes the metadata inheritance and sets up the cache. + + modulestore: the module store that can be used to retrieve additional + modules + + module_data: a dict mapping Location -> json that was cached from the + underlying modulestore + + default_class: The default_class to use when loading an + XModuleDescriptor from the module_data + + resources_fs: a filesystem, as per MakoDescriptorSystem + + error_tracker: a function that logs errors for later display to users + + render_template: a function for rendering templates, as per + MakoDescriptorSystem + """ + # TODO find all references to resources_fs and make handle None + super(CachingDescriptorSystem, self).__init__( + self._load_item, None, error_tracker, render_template) + self.modulestore = modulestore + self.course_entry = course_entry + self.lazy = lazy + self.module_data = module_data + self.default_class = default_class + # TODO see if self.course_id is needed: is already in course_entry but could be > 1 value + # Compute inheritance + modulestore.inherit_metadata(course_entry.get('blocks', {}), + course_entry.get('blocks', {}) + .get(course_entry.get('root'))) + + def _load_item(self, usage_id, course_entry_override=None): + # TODO ensure all callers of system.load_item pass just the id + json_data = self.module_data.get(usage_id) + if json_data is None: + # deeper than initial descendant fetch or doesn't exist + self.modulestore.cache_items(self, [usage_id], lazy=self.lazy) + json_data = self.module_data.get(usage_id) + if json_data is None: + raise ItemNotFoundError + + class_ = XModuleDescriptor.load_class( + json_data.get('category'), + self.default_class + ) + return self.xblock_from_json(class_, usage_id, json_data, course_entry_override) + + def xblock_from_json(self, class_, usage_id, json_data, course_entry_override=None): + if course_entry_override is None: + course_entry_override = self.course_entry + # most likely a lazy loader but not the id directly + definition = json_data.get('definition', {}) + metadata = json_data.get('metadata', {}) + + block_locator = BlockUsageLocator( + version_guid=course_entry_override['_id'], + usage_id=usage_id, + course_id=course_entry_override.get('course_id'), + revision=course_entry_override.get('revision') + ) + + kvs = SplitMongoKVS( + definition, + json_data.get('children', []), + metadata, + json_data.get('_inherited_metadata'), + block_locator, + json_data.get('category')) + model_data = DbModel(kvs, class_, None, + SplitMongoKVSid( + # DbModel req's that these support .url() + block_locator, + self.modulestore.definition_locator(definition))) + + try: + module = class_(self, model_data) + except Exception: + log.warning("Failed to load descriptor", exc_info=True) + if usage_id is None: + usage_id = "MISSING" + return ErrorDescriptor.from_json( + json_data, + self, + BlockUsageLocator(version_guid=course_entry_override['_id'], + usage_id=usage_id), + error_msg=exc_info_to_str(sys.exc_info()) + ) + + module.edited_by = json_data.get('edited_by') + module.edited_on = json_data.get('edited_on') + module.previous_version = json_data.get('previous_version') + module.update_version = json_data.get('update_version') + module.definition_locator = self.modulestore.definition_locator(definition) + return module diff --git a/common/lib/xmodule/xmodule/modulestore/split_mongo/definition_lazy_loader.py b/common/lib/xmodule/xmodule/modulestore/split_mongo/definition_lazy_loader.py new file mode 100644 index 0000000000..5ccaaa7ed3 --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/split_mongo/definition_lazy_loader.py @@ -0,0 +1,26 @@ +from xmodule.modulestore.locator import DescriptionLocator + + +class DefinitionLazyLoader(object): + """ + A placeholder to put into an xblock in place of its definition which + when accessed knows how to get its content. Only useful if the containing + object doesn't force access during init but waits until client wants the + definition. Only works if the modulestore is a split mongo store. + """ + def __init__(self, modulestore, definition_id): + """ + Simple placeholder for yet-to-be-fetched data + :param modulestore: the pymongo db connection with the definitions + :param definition_locator: the id of the record in the above to fetch + """ + self.modulestore = modulestore + self.definition_locator = DescriptionLocator(definition_id) + + def fetch(self): + """ + Fetch the definition. Note, the caller should replace this lazy + loader pointer with the result so as not to fetch more than once + """ + return self.modulestore.definitions.find_one( + {'_id': self.definition_locator.definition_id}) diff --git a/common/lib/xmodule/xmodule/modulestore/split_mongo/split.py b/common/lib/xmodule/xmodule/modulestore/split_mongo/split.py new file mode 100644 index 0000000000..6dd6fb480f --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/split_mongo/split.py @@ -0,0 +1,1240 @@ +import threading +import datetime +import logging +import pymongo +import re +from importlib import import_module +from path import path + +from xmodule.errortracker import null_error_tracker +from xmodule.x_module import XModuleDescriptor +from xmodule.modulestore.locator import BlockUsageLocator, DescriptionLocator, CourseLocator, VersionTree +from xmodule.modulestore.exceptions import InsufficientSpecificationError, VersionConflictError +from xmodule.modulestore import inheritance + +from .. import ModuleStoreBase +from ..exceptions import ItemNotFoundError +from .definition_lazy_loader import DefinitionLazyLoader +from .caching_descriptor_system import CachingDescriptorSystem + +log = logging.getLogger(__name__) +#============================================================================== +# Documentation is at +# https://edx-wiki.atlassian.net/wiki/display/ENG/Mongostore+Data+Structure +# +# Known issue: +# Inheritance for cached kvs doesn't work on edits. Use case. +# 1) attribute foo is inheritable +# 2) g.children = [p], p.children = [a] +# 3) g.foo = 1 on load +# 4) if g.foo > 0, if p.foo > 0, if a.foo > 0 all eval True +# 5) p.foo = -1 +# 6) g.foo > 0, p.foo <= 0 all eval True BUT +# 7) BUG: a.foo > 0 still evals True but should be False +# 8) reread and everything works right +# 9) p.del(foo), p.foo > 0 is True! works +# 10) BUG: a.foo < 0! +# Local fix wont' permanently work b/c xblock may cache a.foo... +# +#============================================================================== + + +class SplitMongoModuleStore(ModuleStoreBase): + """ + A Mongodb backed ModuleStore supporting versions, inheritance, + and sharing. + """ + def __init__(self, host, db, collection, fs_root, render_template, + port=27017, default_class=None, + error_tracker=null_error_tracker, + user=None, password=None, + **kwargs): + + ModuleStoreBase.__init__(self) + + self.db = pymongo.database.Database(pymongo.MongoClient( + host=host, + port=port, + tz_aware=True, + **kwargs + ), db) + + # TODO add caching of structures to thread_cache to prevent repeated fetches (but not index b/c + # it changes w/o having a change in id) + self.course_index = self.db[collection + '.active_versions'] + self.structures = self.db[collection + '.structures'] + self.definitions = self.db[collection + '.definitions'] + + # ??? Code review question: those familiar w/ python threading. Should I instead + # use django cache? How should I expire entries? + # _add_cache could use a lru mechanism to control the cache size? + self.thread_cache = threading.local() + + if user is not None and password is not None: + self.db.authenticate(user, password) + + # every app has write access to the db (v having a flag to indicate r/o v write) + # Force mongo to report errors, at the expense of performance + # pymongo docs suck but explanation: + # http://api.mongodb.org/java/2.10.1/com/mongodb/WriteConcern.html + self.course_index.write_concern = {'w': 1} + self.structures.write_concern = {'w': 1} + self.definitions.write_concern = {'w': 1} + + if default_class is not None: + module_path, _, class_name = default_class.rpartition('.') + class_ = getattr(import_module(module_path), class_name) + self.default_class = class_ + else: + self.default_class = None + self.fs_root = path(fs_root) + self.error_tracker = error_tracker + self.render_template = render_template + + def cache_items(self, system, base_usage_ids, depth=0, lazy=True): + ''' + Handles caching of items once inheritance and any other one time + per course per fetch operations are done. + :param system: a CachingDescriptorSystem + :param base_usage_ids: list of usage_ids to fetch + :param depth: how deep below these to prefetch + :param lazy: whether to fetch definitions or use placeholders + ''' + new_module_data = {} + for usage_id in base_usage_ids: + new_module_data = self.descendants(system.course_entry['blocks'], + usage_id, + depth, + new_module_data) + + # remove any which were already in module_data (not sure if there's a better way) + for newkey in new_module_data.iterkeys(): + if newkey in system.module_data: + del new_module_data[newkey] + + if lazy: + for block in new_module_data.itervalues(): + block['definition'] = DefinitionLazyLoader(self, + block['definition']) + else: + # Load all descendants by id + descendent_definitions = self.definitions.find({ + '_id': {'$in': [block['definition'] + for block in new_module_data.itervalues()]}}) + # turn into a map + definitions = {definition['_id']: definition + for definition in descendent_definitions} + + for block in new_module_data.itervalues(): + if block['definition'] in definitions: + block['definition'] = definitions[block['definition']] + + system.module_data.update(new_module_data) + return system.module_data + + def _load_items(self, course_entry, usage_ids, depth=0, lazy=True): + ''' + Load & cache the given blocks from the course. Prefetch down to the + given depth. Load the definitions into each block if lazy is False; + otherwise, use the lazy definition placeholder. + ''' + system = self._get_cache(course_entry['_id']) + if system is None: + system = CachingDescriptorSystem( + self, + course_entry, + {}, + lazy, + self.default_class, + self.error_tracker, + self.render_template + ) + self._add_cache(course_entry['_id'], system) + self.cache_items(system, usage_ids, depth, lazy) + return [system.load_item(usage_id, course_entry) for usage_id in usage_ids] + + def _get_cache(self, course_version_guid): + """ + Find the descriptor cache for this course if it exists + :param course_version_guid: + """ + if not hasattr(self.thread_cache, 'course_cache'): + self.thread_cache.course_cache = {} + system = self.thread_cache.course_cache + return system.get(course_version_guid) + + def _add_cache(self, course_version_guid, system): + """ + Save this cache for subsequent access + :param course_version_guid: + :param system: + """ + if not hasattr(self.thread_cache, 'course_cache'): + self.thread_cache.course_cache = {} + self.thread_cache.course_cache[course_version_guid] = system + return system + + def _clear_cache(self): + """ + Should only be used by testing or something which implements transactional boundary semantics + """ + self.thread_cache.course_cache = {} + + def _lookup_course(self, course_locator): + ''' + Decode the locator into the right series of db access. Does not + return the CourseDescriptor! It returns the actual db json from + structures. + + Semantics: if course_id and revision given, then it will get that revision. If + also give a version_guid, it will see if the current head of that revision == that guid. If not + it raises VersionConflictError (the version now differs from what it was when you got your + reference) + + :param course_locator: any subclass of CourseLocator + ''' + # NOTE: if and when this uses cache, the update if changed logic will break if the cache + # holds the same objects as the descriptors! + if not course_locator.is_fully_specified(): + raise InsufficientSpecificationError('Not fully specified: %s' % course_locator) + + if course_locator.course_id is not None and course_locator.revision is not None: + # use the course_id + index = self.course_index.find_one({'_id': course_locator.course_id}) + if index is None: + raise ItemNotFoundError(course_locator) + if course_locator.revision not in index['versions']: + raise ItemNotFoundError(course_locator) + version_guid = index['versions'][course_locator.revision] + if course_locator.version_guid is not None and version_guid != course_locator.version_guid: + # This may be a bit too touchy but it's hard to infer intent + raise VersionConflictError(course_locator, CourseLocator(course_locator, version_guid=version_guid)) + else: + # TODO should this raise an exception if revision was provided? + version_guid = course_locator.version_guid + + # cast string to ObjectId if necessary + version_guid = course_locator.as_object_id(version_guid) + entry = self.structures.find_one({'_id': version_guid}) + + # b/c more than one course can use same structure, the 'course_id' is not intrinsic to structure + # and the one assoc'd w/ it by another fetch may not be the one relevant to this fetch; so, + # fake it by explicitly setting it in the in memory structure. + + if course_locator.course_id: + entry['course_id'] = course_locator.course_id + entry['revision'] = course_locator.revision + return entry + + def get_courses(self, revision, qualifiers=None): + ''' + Returns a list of course descriptors matching any given qualifiers. + + qualifiers should be a dict of keywords matching the db fields or any + legal query for mongo to use against the active_versions collection. + + Note, this is to find the current head of the named revision type + (e.g., 'draft'). To get specific versions via guid use get_course. + ''' + if qualifiers is None: + qualifiers = {} + qualifiers.update({"versions.{}".format(revision): {"$exists": True}}) + matching = self.course_index.find(qualifiers) + + # collect ids and then query for those + version_guids = [] + id_version_map = {} + for course_entry in matching: + version_guid = course_entry['versions'][revision] + version_guids.append(version_guid) + id_version_map[version_guid] = course_entry['_id'] + + course_entries = self.structures.find({'_id': {'$in': version_guids}}) + + # get the block for the course element (s/b the root) + result = [] + for entry in course_entries: + # structures are course agnostic but the caller wants to know course, so add it in here + entry['course_id'] = id_version_map[entry['_id']] + root = entry['root'] + result.extend(self._load_items(entry, [root], 0, lazy=True)) + return result + + def get_course(self, course_locator): + ''' + Gets the course descriptor for the course identified by the locator + which may or may not be a blockLocator. + + raises InsufficientSpecificationError + ''' + course_entry = self._lookup_course(course_locator) + root = course_entry['root'] + result = self._load_items(course_entry, [root], 0, lazy=True) + return result[0] + + def get_course_for_item(self, location): + ''' + Provided for backward compatibility. Is equivalent to calling get_course + :param location: + ''' + return self.get_course(location) + + def has_item(self, block_location): + """ + Returns True if location exists in its course. Returns false if + the course or the block w/in the course do not exist for the given version. + raises InsufficientSpecificationError if the locator does not id a block + """ + if block_location.usage_id is None: + raise InsufficientSpecificationError(block_location) + try: + course_structure = self._lookup_course(block_location) + except ItemNotFoundError: + # this error only occurs if the course does not exist + return False + + return course_structure['blocks'].get(block_location.usage_id) is not None + + def get_item(self, location, depth=0): + """ + depth (int): An argument that some module stores may use to prefetch + descendants of the queried modules for more efficient results later + in the request. The depth is counted in the number of + calls to get_children() to cache. None indicates to cache all + descendants. + raises InsufficientSpecificationError or ItemNotFoundError + """ + assert isinstance(location, BlockUsageLocator) + if not location.is_initialized(): + raise InsufficientSpecificationError("Not yet initialized: %s" % location) + course = self._lookup_course(location) + items = self._load_items(course, [location.usage_id], depth, lazy=True) + if len(items) == 0: + raise ItemNotFoundError(location) + return items[0] + + # TODO refactor this and get_courses to use a constructed query + def get_items(self, locator, qualifiers): + ''' + Get all of the modules in the given course matching the qualifiers. The + qualifiers should only be fields in the structures collection (sorry). + There will be a separate search method for searching through + definitions. + + Common qualifiers are category, definition (provide definition id), + metadata: {display_name ..}, children (return + block if its children includes the one given value). If you want + substring matching use {$regex: /acme.*corp/i} type syntax. + + Although these + look like mongo queries, it is all done in memory; so, you cannot + try arbitrary queries. + + :param locator: CourseLocator or BlockUsageLocator restricting search scope + :param qualifiers: a dict restricting which elements should match + ''' + # TODO extend to only search a subdag of the course? + course = self._lookup_course(locator) + items = [] + for usage_id, value in course['blocks'].iteritems(): + if self._block_matches(value, qualifiers): + items.append(usage_id) + + if len(items) > 0: + return self._load_items(course, items, 0, lazy=True) + else: + return [] + + # What's the use case for usage_id being separate? + def get_parent_locations(self, locator, usage_id=None): + ''' + Return the locations (Locators w/ usage_ids) for the parents of this location in this + course. Could use get_items(location, {'children': usage_id}) but this is slightly faster. + NOTE: does not actually ensure usage_id exists + If usage_id is None, then the locator must specify the usage_id + ''' + if usage_id is None: + usage_id = locator.usage_id + course = self._lookup_course(locator) + items = [] + for parent_id, value in course['blocks'].iteritems(): + for child_id in value['children']: + if usage_id == child_id: + locator = locator.as_course_locator() + items.append(BlockUsageLocator(url=locator, usage_id=parent_id)) + return items + + def get_course_index_info(self, course_locator): + """ + The index records the initial creation of the indexed course and tracks the current version + heads. This function is primarily for test verification but may serve some + more general purpose. + :param course_locator: must have a course_id set + :return {'org': , 'prettyid': , + versions: {'draft': the head draft version id, + 'published': the head published version id if any, + }, + 'edited_by': who created the course originally (named edited for consistency), + 'edited_on': when the course was originally created + } + """ + if course_locator.course_id is None: + return None + index = self.course_index.find_one({'_id': course_locator.course_id}) + return index + + # TODO figure out a way to make this info accessible from the course descriptor + def get_course_history_info(self, course_locator): + """ + Because xblocks doesn't give a means to separate the course structure's meta information from + the course xblock's, this method will get that info for the structure as a whole. + :param course_locator: + :return {'original_version': the version guid of the original version of this course, + 'previous_version': the version guid of the previous version, + 'edited_by': who made the last change, + 'edited_on': when the change was made + } + """ + course = self._lookup_course(course_locator) + return {'original_version': course['original_version'], + 'previous_version': course['previous_version'], + 'edited_by': course['edited_by'], + 'edited_on': course['edited_on'] + } + + def get_definition_history_info(self, definition_locator): + """ + Because xblocks doesn't give a means to separate the definition's meta information from + the usage xblock's, this method will get that info for the definition + :return {'original_version': the version guid of the original version of this course, + 'previous_version': the version guid of the previous version, + 'edited_by': who made the last change, + 'edited_on': when the change was made + } + """ + definition = self.definitions.find_one({'_id': definition_locator.definition_id}) + if definition is None: + return None + return {'original_version': definition['original_version'], + 'previous_version': definition['previous_version'], + 'edited_by': definition['edited_by'], + 'edited_on': definition['edited_on'] + } + + def get_course_successors(self, course_locator, version_history_depth=1): + ''' + Find the version_history_depth next versions of this course. Return as a VersionTree + Mostly makes sense when course_locator uses a version_guid, but because it finds all relevant + next versions, these do include those created for other courses. + :param course_locator: + ''' + if version_history_depth < 1: + return None + if course_locator.version_guid is None: + course = self._lookup_course(course_locator) + version_guid = course.version_guid + else: + version_guid = course_locator.version_guid + + # TODO if depth is significant, it may make sense to get all that have the same original_version + # and reconstruct the subtree from version_guid + next_entries = self.structures.find({'previous_version' : version_guid}) + # must only scan cursor's once + next_versions = [struct for struct in next_entries] + result = {version_guid: [CourseLocator(version_guid=struct['_id']) for struct in next_versions]} + depth = 1 + while depth < version_history_depth and len(next_versions) > 0: + depth += 1 + next_entries = self.structures.find({'previous_version': + {'$in': [struct['_id'] for struct in next_versions]}}) + next_versions = [struct for struct in next_entries] + for course_structure in next_versions: + result.setdefault(course_structure['previous_version'], []).append( + CourseLocator(version_guid=struct['_id'])) + return VersionTree(CourseLocator(course_locator, version_guid=version_guid), result) + + + def get_block_generations(self, block_locator): + ''' + Find the history of this block. Return as a VersionTree of each place the block changed (except + deletion). + + The block's history tracks its explicit changes; so, changes in descendants won't be reflected + as new iterations. + ''' + block_locator = block_locator.version_agnostic() + course_struct = self._lookup_course(block_locator) + usage_id = block_locator.usage_id + update_version_field = 'blocks.{}.update_version'.format(usage_id) + all_versions_with_block = self.structures.find({'original_version': course_struct['original_version'], + update_version_field: {'$exists': True}}) + # find (all) root versions and build map previous: [successors] + possible_roots = [] + result = {} + for version in all_versions_with_block: + if version['_id'] == version['blocks'][usage_id]['update_version']: + if version['blocks'][usage_id].get('previous_version') is None: + possible_roots.append(version['blocks'][usage_id]['update_version']) + else: + result.setdefault(version['blocks'][usage_id]['previous_version'], set()).add( + version['blocks'][usage_id]['update_version']) + # more than one possible_root means usage was added and deleted > 1x. + if len(possible_roots) > 1: + # find the history segment including block_locator's version + element_to_find = course_struct['blocks'][usage_id]['update_version'] + if element_to_find in possible_roots: + possible_roots = [element_to_find] + for possibility in possible_roots: + if self._find_local_root(element_to_find, possibility, result): + possible_roots = [possibility] + break + elif len(possible_roots) == 0: + return None + # convert the results value sets to locators + for k, versions in result.iteritems(): + result[k] = [BlockUsageLocator(version_guid=version, usage_id=usage_id) + for version in versions] + return VersionTree(BlockUsageLocator(version_guid=possible_roots[0], usage_id=usage_id), result) + + def get_definition_successors(self, definition_locator, version_history_depth=1): + ''' + Find the version_history_depth next versions of this definition. Return as a VersionTree + ''' + # TODO implement + pass + + def create_definition_from_data(self, new_def_data, category, user_id): + """ + Pull the definition fields out of descriptor and save to the db as a new definition + w/o a predecessor and return the new id. + + :param user_id: request.user object + """ + document = {"category" : category, + "data": new_def_data, + "edited_by": user_id, + "edited_on": datetime.datetime.utcnow(), + "previous_version": None, + "original_version": None} + new_id = self.definitions.insert(document) + definition_locator = DescriptionLocator(new_id) + document['original_version'] = new_id + self.definitions.update({'_id': new_id}, {'$set': {"original_version": new_id}}) + return definition_locator + + def update_definition_from_data(self, definition_locator, new_def_data, user_id): + """ + See if new_def_data differs from the persisted version. If so, update + the persisted version and return the new id. + + :param user_id: request.user + """ + def needs_saved(): + if isinstance(new_def_data, dict): + for key, value in new_def_data.iteritems(): + if key not in old_definition['data'] or value != old_definition['data'][key]: + return True + for key, value in old_definition['data'].iteritems(): + if key not in new_def_data: + return True + else: + return new_def_data != old_definition['data'] + + # if this looks in cache rather than fresh fetches, then it will probably not detect + # actual change b/c the descriptor and cache probably point to the same objects + old_definition = self.definitions.find_one({'_id': definition_locator.definition_id}) + if old_definition is None: + raise ItemNotFoundError(definition_locator.url()) + del old_definition['_id'] + + if needs_saved(): + old_definition['data'] = new_def_data + old_definition['edited_by'] = user_id + old_definition['edited_on'] = datetime.datetime.utcnow() + old_definition['previous_version'] = definition_locator.definition_id + new_id = self.definitions.insert(old_definition) + return DescriptionLocator(new_id), True + else: + return definition_locator, False + + def _generate_usage_id(self, course_blocks, category): + """ + Generate a somewhat readable block id unique w/in this course using the category + :param course_blocks: the current list of blocks. + :param category: + """ + # NOTE: a potential bug is that a block is deleted and another created which gets the old + # block's id. a possible fix is to cache the last serial in a dict in the structure + # {category: last_serial...} + # A potential confusion is if the name incorporates the parent's name, then if the child + # moves, its id won't change and will be confusing + serial = 1 + while category + str(serial) in course_blocks: + serial += 1 + return category + str(serial) + + def _generate_course_id(self, id_root): + """ + Generate a somewhat readable course id unique w/in this db using the id_root + :param course_blocks: the current list of blocks. + :param category: + """ + existing_uses = self.course_index.find({"_id": {"$regex": id_root}}) + if existing_uses.count() > 0: + max_found = 0 + matcher = re.compile(id_root + r'(\d+)') + for entry in existing_uses: + serial = re.search(matcher, entry['_id']) + if serial is not None and serial.groups > 0: + value = int(serial.group(1)) + if value > max_found: + max_found = value + return id_root + str(max_found + 1) + else: + return id_root + + # TODO I would love to write this to take a real descriptor and persist it BUT descriptors, kvs, and dbmodel + # all assume locators are set and unique! Having this take the model contents piecemeal breaks the separation + # of model from persistence layer + def create_item(self, course_or_parent_locator, category, user_id, definition_locator=None, new_def_data=None, + metadata=None, force=False): + """ + Add a descriptor to persistence as the last child of the optional parent_location or just as an element + of the course (if no parent provided). Return the resulting post saved version with populated locators. + + If the locator is a BlockUsageLocator, then it's assumed to be the parent. If it's a CourseLocator, then it's + merely the containing course. + + raises InsufficientSpecificationError if there is no course locator. + raises VersionConflictError if course_id and version_guid given and the current version head != version_guid + and force is not True. + force: fork the structure and don't update the course draftVersion if the above + + The incoming definition_locator should either be None to indicate this is a brand new definition or + a pointer to the existing definition to which this block should point or from which this was derived. + If new_def_data is None, then definition_locator must have a value meaning that this block points + to the existing definition. If new_def_data is not None and definition_location is not None, then + new_def_data is assumed to be a new payload for definition_location. + + Creates a new version of the course structure, creates and inserts the new block, makes the block point + to the definition which may be new or a new version of an existing or an existing. + Rules for course locator: + + * If the course locator specifies a course_id and either it doesn't + specify version_guid or the one it specifies == the current draft, it progresses the course to point + to the new draft and sets the active version to point to the new draft + * If the locator has a course_id but its version_guid != current draft, it raises VersionConflictError. + + NOTE: using a version_guid will end up creating a new version of the course. Your new item won't be in + the course id'd by version_guid but instead in one w/ a new version_guid. Ensure in this case that you get + the new version_guid from the locator in the returned object! + """ + # find course_index entry if applicable and structures entry + index_entry = self._get_index_if_valid(course_or_parent_locator, force) + structure = self._lookup_course(course_or_parent_locator) + + # persist the definition if persisted != passed + if (definition_locator is None or definition_locator.definition_id is None): + definition_locator = self.create_definition_from_data(new_def_data, category, user_id) + elif new_def_data is not None: + definition_locator, _ = self.update_definition_from_data(definition_locator, new_def_data, user_id) + + # copy the structure and modify the new one + new_structure = self._version_structure(structure, user_id) + # generate an id + new_usage_id = self._generate_usage_id(new_structure['blocks'], category) + update_version_keys = ['blocks.{}.update_version'.format(new_usage_id)] + if isinstance(course_or_parent_locator, BlockUsageLocator) and course_or_parent_locator.usage_id is not None: + parent = new_structure['blocks'][course_or_parent_locator.usage_id] + parent['children'].append(new_usage_id) + parent['edited_on'] = datetime.datetime.utcnow() + parent['edited_by'] = user_id + parent['previous_version'] = parent['update_version'] + update_version_keys.append('blocks.{}.update_version'.format(course_or_parent_locator.usage_id)) + new_structure['blocks'][new_usage_id] = { + "children": [], + "category": category, + "definition": definition_locator.definition_id, + "metadata": metadata if metadata else {}, + 'edited_on': datetime.datetime.utcnow(), + 'edited_by': user_id, + 'previous_version': None + } + new_id = self.structures.insert(new_structure) + update_version_payload = {key: new_id for key in update_version_keys} + self.structures.update({'_id': new_id}, + {'$set': update_version_payload}) + + # update the index entry if appropriate + if index_entry is not None: + self._update_head(index_entry, course_or_parent_locator.revision, new_id) + course_parent = course_or_parent_locator.as_course_locator() + else: + course_parent = None + + # fetch and return the new item--fetching is unnecessary but a good qc step + return self.get_item(BlockUsageLocator(course_id=course_parent, + usage_id=new_usage_id, + version_guid=new_id)) + + def create_course(self, org, prettyid, user_id, id_root=None, metadata=None, course_data=None, + master_version='draft', versions_dict=None, root_category='course'): + """ + Create a new entry in the active courses index which points to an existing or new structure. Returns + the course root of the resulting entry (the location has the course id) + + id_root: allows the caller to specify the course_id. It's a root in that, if it's already taken, + this method will append things to the root to make it unique. (defaults to org) + + metadata: if provided, will set the metadata of the root course object in the new draft course. If both + metadata and a starting version are provided, it will generate a successor version to the given version, + and update the metadata with any provided values (via update not setting). + + course_data: if provided, will update the data of the new course xblock definition to this. Like metadata, + if provided, this will cause a new version of any given version as well as a new version of the + definition (which will point to the existing one if given a version). If not provided and given + a draft_version, it will reuse the same definition as the draft course (obvious since it's reusing the draft + course). If not provided and no draft is given, it will be empty and get the field defaults (hopefully) when + loaded. + + master_version: the tag (key) for the version name in the dict which is the 'draft' version. Not the actual + version guid, but what to call it. + + versions_dict: the starting version ids where the keys are the tags such as 'draft' and 'published' + and the values are structure guids. If provided, the new course will reuse this version (unless you also + provide any overrides such as metadata, see above). if not provided, will create a mostly empty course + structure with just a category course root xblock. + """ + if metadata is None: + metadata = {} + # build from inside out: definition, structure, index entry + # if building a wholly new structure + if versions_dict is None or master_version not in versions_dict: + # create new definition and structure + if course_data is None: + course_data = {} + definition_entry = { + 'category': root_category, + 'data': course_data, + 'edited_by': user_id, + 'edited_on': datetime.datetime.utcnow(), + 'previous_version': None, + } + definition_id = self.definitions.insert(definition_entry) + definition_entry['original_version'] = definition_id + self.definitions.update({'_id': definition_id}, {'$set': {"original_version": definition_id}}) + + draft_structure = { + 'root': 'course', + 'previous_version': None, + 'edited_by': user_id, + 'edited_on': datetime.datetime.utcnow(), + 'blocks': { + 'course': { + 'children':[], + 'category': 'course', + 'definition': definition_id, + 'metadata': metadata, + 'edited_on': datetime.datetime.utcnow(), + 'edited_by': user_id, + 'previous_version': None}}} + new_id = self.structures.insert(draft_structure) + draft_structure['original_version'] = new_id + self.structures.update({'_id': new_id}, + {'$set': {"original_version": new_id, + 'blocks.course.update_version': new_id}}) + if versions_dict is None: + versions_dict = {master_version: new_id} + else: + versions_dict[master_version] = new_id + + else: + # just get the draft_version structure + draft_version = CourseLocator(version_guid=versions_dict[master_version]) + draft_structure = self._lookup_course(draft_version) + if course_data is not None or metadata: + draft_structure = self._version_structure(draft_structure, user_id) + root_block = draft_structure['blocks'][draft_structure['root']] + if metadata is not None: + root_block['metadata'].update(metadata) + if course_data is not None: + definition = self.definitions.find_one({'_id': root_block['definition']}) + definition['data'].update(course_data) + definition['previous_version'] = definition['_id'] + definition['edited_by'] = user_id + definition['edited_on'] = datetime.datetime.utcnow() + del definition['_id'] + root_block['definition'] = self.definitions.insert(definition) + root_block['edited_on'] = datetime.datetime.utcnow() + root_block['edited_by'] = user_id + root_block['previous_version'] = root_block.get('update_version') + # insert updates the '_id' in draft_structure + new_id = self.structures.insert(draft_structure) + versions_dict[master_version] = new_id + self.structures.update({'_id': new_id}, + {'$set': {'blocks.{}.update_version'.format(draft_structure['root']): new_id}}) + # create the index entry + if id_root is None: + id_root = org + new_id = self._generate_course_id(id_root) + + index_entry = { + '_id': new_id, + 'org': org, + 'prettyid': prettyid, + 'edited_by': user_id, + 'edited_on': datetime.datetime.utcnow(), + 'versions': versions_dict} + new_id = self.course_index.insert(index_entry) + return self.get_course(CourseLocator(course_id=new_id, revision=master_version)) + + def update_item(self, descriptor, user_id, force=False): + """ + Save the descriptor's definition, metadata, & children references (i.e., it doesn't descend the tree). + Return the new descriptor (updated location). + + raises ItemNotFoundError if the location does not exist. + + Creates a new course version. If the descriptor's location has a course_id, it moves the course head + pointer. If the version_guid of the descriptor points to a non-head version and there's been an intervening + change to this item, it raises a VersionConflictError unless force is True. In the force case, it forks + the course but leaves the head pointer where it is (this change will not be in the course head). + + The implementation tries to detect which, if any changes, actually need to be saved and thus won't version + the definition, structure, nor course if they didn't change. + """ + original_structure = self._lookup_course(descriptor.location) + index_entry = self._get_index_if_valid(descriptor.location, force) + + descriptor.definition_locator, is_updated = self.update_definition_from_data( + descriptor.definition_locator, descriptor.xblock_kvs.get_data(), user_id) + # check children + original_entry = original_structure['blocks'][descriptor.location.usage_id] + if (not is_updated and descriptor.has_children + and not self._xblock_lists_equal(original_entry['children'], descriptor.children)): + is_updated = True + # check metadata + if not is_updated: + is_updated = self._compare_metadata(descriptor.xblock_kvs.get_own_metadata(), original_entry['metadata']) + + # if updated, rev the structure + if is_updated: + new_structure = self._version_structure(original_structure, user_id) + block_data = new_structure['blocks'][descriptor.location.usage_id] + if descriptor.has_children: + block_data["children"] = [self._usage_id(child) for child in descriptor.children] + + block_data["definition"] = descriptor.definition_locator.definition_id + block_data["metadata"] = descriptor.xblock_kvs.get_own_metadata() + block_data['edited_on'] = datetime.datetime.utcnow() + block_data['edited_by'] = user_id + block_data['previous_version'] = block_data['update_version'] + new_id = self.structures.insert(new_structure) + self.structures.update({'_id': new_id}, + {'$set': {'blocks.{}.update_version'.format(descriptor.location.usage_id): new_id}}) + + # update the index entry if appropriate + if index_entry is not None: + self._update_head(index_entry, descriptor.location.revision, new_id) + + # fetch and return the new item--fetching is unnecessary but a good qc step + return self.get_item(BlockUsageLocator(descriptor.location, version_guid=new_id)) + else: + # nothing changed, just return the one sent in + return descriptor + + def persist_xblock_dag(self, xblock, user_id, force=False): + """ + create or update the xblock and all of its children. The xblock's location must specify a course. + If it doesn't specify a usage_id, then it's presumed to be new and need creation. This function + descends the children performing the same operation for any that are xblocks. Any children which + are usage_ids just update the children pointer. + + All updates go into the same course version (bulk updater). + + Updates the objects which came in w/ updated location and definition_location info. + + returns the post-persisted version of the incoming xblock. Note that its children will be ids not + objects. + + :param xblock: + :param user_id: + """ + # find course_index entry if applicable and structures entry + index_entry = self._get_index_if_valid(xblock.location, force) + structure = self._lookup_course(xblock.location) + new_structure = self._version_structure(structure, user_id) + + changed_blocks = self._persist_subdag(xblock, user_id, new_structure['blocks']) + + if changed_blocks: + new_id = self.structures.insert(new_structure) + update_command = {} + for usage_id in changed_blocks: + update_command['blocks.{}.update_version'.format(usage_id)] = new_id + self.structures.update({'_id': new_id}, {'$set': update_command}) + + # update the index entry if appropriate + if index_entry is not None: + self._update_head(index_entry, xblock.location.revision, new_id) + + # fetch and return the new item--fetching is unnecessary but a good qc step + return self.get_item(BlockUsageLocator(xblock.location, version_guid=new_id)) + else: + return xblock + + def _persist_subdag(self, xblock, user_id, structure_blocks): + # persist the definition if persisted != passed + new_def_data = xblock.xblock_kvs.get_data() + if (xblock.definition_locator is None or xblock.definition_locator.definition_id is None): + xblock.definition_locator = self.create_definition_from_data(new_def_data, + xblock.category, user_id) + is_updated = True + elif new_def_data is not None: + xblock.definition_locator, is_updated = self.update_definition_from_data(xblock.definition_locator, + new_def_data, user_id) + + if xblock.location.usage_id is None: + # generate an id + is_new = True + is_updated = True + usage_id = self._generate_usage_id(structure_blocks, xblock.category) + xblock.location.usage_id = usage_id + else: + is_new = False + usage_id = xblock.location.usage_id + if (not is_updated and xblock.has_children + and not self._xblock_lists_equal(structure_blocks[usage_id]['children'], xblock.children)): + is_updated = True + + children = [] + updated_blocks = [] + if xblock.has_children: + for child in xblock.children: + if isinstance(child, XModuleDescriptor): + updated_blocks += self._persist_subdag(child, user_id, structure_blocks) + children.append(child.location.usage_id) + else: + children.append(child) + + is_updated = is_updated or updated_blocks + metadata = xblock.xblock_kvs.get_own_metadata() + if not is_new and not is_updated: + is_updated = self._compare_metadata(metadata, structure_blocks[usage_id]['metadata']) + + if is_updated: + structure_blocks[usage_id] = { + "children": children, + "category": xblock.category, + "definition": xblock.definition_locator.definition_id, + "metadata": metadata if metadata else {}, + 'previous_version': structure_blocks.get(usage_id, {}).get('update_version'), + 'edited_by': user_id, + 'edited_on': datetime.datetime.utcnow() + } + updated_blocks.append(usage_id) + + return updated_blocks + + def _compare_metadata(self, metadata, original_metadata): + original_keys = original_metadata.keys() + if len(metadata) != len(original_keys): + return True + else: + new_keys = metadata.keys() + for key in original_keys: + if key not in new_keys or original_metadata[key] != metadata[key]: + return True + + # TODO change all callers to update_item + def update_children(self, course_id, location, children): + raise NotImplementedError() + + # TODO change all callers to update_item + def update_metadata(self, course_id, location, metadata): + raise NotImplementedError() + + def update_course_index(self, course_locator, new_values_dict, update_versions=False): + """ + Change the given course's index entry for the given fields. new_values_dict + should be a subset of the dict returned by get_course_index_info. + It cannot include '_id' (will raise IllegalArgument). + Provide update_versions=True if you intend this to replace the versions hash. + Note, this operation can be dangerous and break running courses. + + If the dict includes versions and not update_versions, it will raise an exception. + + If the dict includes edited_on or edited_by, it will raise an exception + + Does not return anything useful. + """ + # TODO how should this log the change? edited_on and edited_by for this entry + # has the semantic of who created the course and when; so, changing those will lose + # that information. + if '_id' in new_values_dict: + raise ValueError("Cannot override _id") + if 'edited_on' in new_values_dict or 'edited_by' in new_values_dict: + raise ValueError("Cannot set edited_on or edited_by") + if not update_versions and 'versions' in new_values_dict: + raise ValueError("Cannot override versions without setting update_versions") + self.course_index.update({'_id': course_locator.course_id}, + {'$set': new_values_dict}) + + def delete_item(self, usage_locator, user_id, force=False): + """ + Delete the tree rooted at block and any references w/in the course to the block + from a new version of the course structure. + + returns CourseLocator for new version + + raises ItemNotFoundError if the location does not exist. + raises ValueError if usage_locator points to the structure root + + Creates a new course version. If the descriptor's location has a course_id, it moves the course head + pointer. If the version_guid of the descriptor points to a non-head version and there's been an intervening + change to this item, it raises a VersionConflictError unless force is True. In the force case, it forks + the course but leaves the head pointer where it is (this change will not be in the course head). + """ + assert isinstance(usage_locator, BlockUsageLocator) and usage_locator.is_initialized() + original_structure = self._lookup_course(usage_locator) + if original_structure['root'] == usage_locator.usage_id: + raise ValueError("Cannot delete the root of a course") + index_entry = self._get_index_if_valid(usage_locator, force) + new_structure = self._version_structure(original_structure, user_id) + new_blocks = new_structure['blocks'] + parents = self.get_parent_locations(usage_locator) + update_version_keys = [] + for parent in parents: + parent_block = new_blocks[parent.usage_id] + parent_block['children'].remove(usage_locator.usage_id) + parent_block['edited_on'] = datetime.datetime.utcnow() + parent_block['edited_by'] = user_id + parent_block['previous_version'] = parent_block['update_version'] + update_version_keys.append('blocks.{}.update_version'.format(parent.usage_id)) + # remove subtree + def remove_subtree(usage_id): + for child in new_blocks[usage_id]['children']: + remove_subtree(child) + del new_blocks[usage_id] + remove_subtree(usage_locator.usage_id) + + # update index if appropriate and structures + new_id = self.structures.insert(new_structure) + if update_version_keys: + update_version_payload = {key: new_id for key in update_version_keys} + self.structures.update({'_id': new_id}, {'$set': update_version_payload}) + + result = CourseLocator(version_guid=new_id) + + # update the index entry if appropriate + if index_entry is not None: + self._update_head(index_entry, usage_locator.revision, new_id) + result.course_id = usage_locator.course_id + result.revision = usage_locator.revision + + return result + + def delete_course(self, course_id): + """ + Remove the given course from the course index. + + Only removes the course from the index. The data remains. You can use create_course + with a versions hash to restore the course; however, the edited_on and + edited_by won't reflect the originals, of course. + + :param course_id: uses course_id rather than locator to emphasize its global effect + """ + index = self.course_index.find_one({'_id': course_id}) + if index is None: + raise ItemNotFoundError(course_id) + # this is the only real delete in the system. should it do something else? + self.course_index.remove(index['_id']) + + # TODO remove all callers and then this + def get_errored_courses(self): + """ + This function doesn't make sense for the mongo modulestore, as structures + are loaded on demand, rather than up front + """ + return {} + + def inherit_metadata(self, block_map, block, inheriting_metadata=None): + """ + Updates block with any value + that exist in inheriting_metadata and don't appear in block['metadata'], + and then inherits block['metadata'] to all of the children in + block['children']. Filters by inheritance.INHERITABLE_METADATA + """ + if block is None: + return + + if inheriting_metadata is None: + inheriting_metadata = {} + + # the currently passed down values take precedence over any previously cached ones + # NOTE: this should show the values which all fields would have if inherited: i.e., + # not set to the locally defined value but to value set by nearest ancestor who sets it + block.setdefault('_inherited_metadata', {}).update(inheriting_metadata) + + # update the inheriting w/ what should pass to children + inheriting_metadata = block['_inherited_metadata'].copy() + for field in inheritance.INHERITABLE_METADATA: + if field in block['metadata']: + inheriting_metadata[field] = block['metadata'][field] + + for child in block.get('children', []): + self.inherit_metadata(block_map, block_map[child], inheriting_metadata) + + def descendants(self, block_map, usage_id, depth, descendent_map): + """ + adds block and its descendants out to depth to descendent_map + Depth specifies the number of levels of descendants to return + (0 => this usage only, 1 => this usage and its children, etc...) + A depth of None returns all descendants + """ + if usage_id not in block_map: + return descendent_map + + if usage_id not in descendent_map: + descendent_map[usage_id] = block_map[usage_id] + + if depth is None or depth > 0: + depth = depth - 1 if depth is not None else None + for child in block_map[usage_id].get('children', []): + descendent_map = self.descendants(block_map, child, depth, + descendent_map) + + return descendent_map + + def definition_locator(self, definition): + ''' + Pull the id out of the definition w/ correct semantics for its + representation + ''' + if isinstance(definition, DefinitionLazyLoader): + return definition.definition_locator + elif '_id' not in definition: + return None + else: + return DescriptionLocator(definition['_id']) + + def _block_matches(self, value, qualifiers): + ''' + Return True or False depending on whether the value (block contents) + matches the qualifiers as per get_items + :param value: + :param qualifiers: + ''' + for key, criteria in qualifiers.iteritems(): + if key in value: + target = value[key] + if not self._value_matches(target, criteria): + return False + elif criteria is not None: + return False + return True + + def _value_matches(self, target, criteria): + ''' helper for _block_matches ''' + if isinstance(target, list): + return any(self._value_matches(ele, criteria) + for ele in target) + elif isinstance(criteria, dict): + if '$regex' in criteria: + return re.search(criteria['$regex'], target) is not None + elif not isinstance(target, dict): + return False + else: + return (isinstance(target, dict) and + self._block_matches(target, criteria)) + else: + return criteria == target + + def _xblock_lists_equal(self, lista, listb): + """ + Do the 2 lists refer to the same xblocks in the same order (presumes they're from the + same course) + + :param lista: + :param listb: + """ + if len(lista) != len(listb): + return False + for idx in enumerate(lista): + if lista[idx] != listb[idx]: + itema = self._usage_id(lista[idx]) + if itema != self._usage_id(listb[idx]): + return False + return True + + def _usage_id(self, xblock_or_id): + """ + arg is either an xblock or an id. If an xblock, get the usage_id from its location. Otherwise, return itself. + :param xblock_or_id: + """ + if isinstance(xblock_or_id, XModuleDescriptor): + return xblock_or_id.location.usage_id + else: + return xblock_or_id + + def _get_index_if_valid(self, locator, force=False): + """ + If the locator identifies a course and points to its draft (or plausibly its draft), + then return the index entry. + + raises VersionConflictError if not the right version + + :param locator: + """ + if locator.course_id is None or locator.revision is None: + return None + else: + index_entry = self.course_index.find_one({'_id': locator.course_id}) + if (locator.version_guid is not None + and index_entry['versions'][locator.revision] != locator.version_guid + and not force): + raise VersionConflictError( + locator, + CourseLocator( + course_id=index_entry['_id'], + version_guid=index_entry['versions'][locator.revision], + revision=locator.revision)) + else: + return index_entry + + def _version_structure(self, structure, user_id): + """ + Copy the structure and update the history info (edited_by, edited_on, previous_version) + :param structure: + :param user_id: + """ + new_structure = structure.copy() + new_structure['blocks'] = new_structure['blocks'].copy() + del new_structure['_id'] + new_structure['previous_version'] = structure['_id'] + new_structure['edited_by'] = user_id + new_structure['edited_on'] = datetime.datetime.utcnow() + return new_structure + + def _find_local_root(self, element_to_find, possibility, tree): + if possibility not in tree: + return False + if element_to_find in tree[possibility]: + return True + for subtree in tree[possibility]: + if self._find_local_root(element_to_find, subtree, tree): + return True + return False + + + def _update_head(self, index_entry, revision, new_id): + """ + Update the active index for the given course's revision to point to new_id + + :param index_entry: + :param course_locator: + :param new_id: + """ + self.course_index.update( + {"_id": index_entry["_id"]}, + {"$set": {"versions.{}".format(revision): new_id}}) diff --git a/common/lib/xmodule/xmodule/modulestore/split_mongo/split_mongo_kvs.py b/common/lib/xmodule/xmodule/modulestore/split_mongo/split_mongo_kvs.py new file mode 100644 index 0000000000..843c1ce364 --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/split_mongo/split_mongo_kvs.py @@ -0,0 +1,163 @@ +import copy +from xblock.core import Scope +from collections import namedtuple +from xblock.runtime import KeyValueStore, InvalidScopeError +from .definition_lazy_loader import DefinitionLazyLoader + +# id is a BlockUsageLocator, def_id is the definition's guid +SplitMongoKVSid = namedtuple('SplitMongoKVSid', 'id, def_id') + + +# TODO should this be here or w/ x_module or ??? +class SplitMongoKVS(KeyValueStore): + """ + A KeyValueStore that maps keyed data access to one of the 3 data areas + known to the MongoModuleStore (data, children, and metadata) + """ + def __init__(self, definition, children, metadata, _inherited_metadata, location, category): + """ + + :param definition: + :param children: + :param metadata: the locally defined value for each metadata field + :param _inherited_metadata: the value of each inheritable field from above this. + Note, metadata may override and disagree w/ this b/c this says what the value + should be if metadata is undefined for this field. + """ + # ensure kvs's don't share objects w/ others so that changes can't appear in separate ones + # the particular use case was that changes to kvs's were polluting caches. My thinking was + # that kvs's should be independent thus responsible for the isolation. + if isinstance(definition, DefinitionLazyLoader): + self._definition = definition + else: + self._definition = copy.copy(definition) + self._children = copy.copy(children) + self._metadata = copy.copy(metadata) + self._inherited_metadata = _inherited_metadata + self._location = location + self._category = category + + def get(self, key): + if key.scope == Scope.children: + return self._children + elif key.scope == Scope.parent: + return None + elif key.scope == Scope.settings: + if key.field_name in self._metadata: + return self._metadata[key.field_name] + elif key.field_name in self._inherited_metadata: + return self._inherited_metadata[key.field_name] + else: + raise KeyError() + elif key.scope == Scope.content: + if key.field_name == 'location': + return self._location + elif key.field_name == 'category': + return self._category + else: + if isinstance(self._definition, DefinitionLazyLoader): + self._definition = self._definition.fetch() + if (key.field_name == 'data' and + not isinstance(self._definition.get('data'), dict)): + return self._definition.get('data') + elif 'data' not in self._definition or key.field_name not in self._definition['data']: + raise KeyError() + else: + return self._definition['data'][key.field_name] + else: + raise InvalidScopeError(key.scope) + + def set(self, key, value): + # TODO cache db update implications & add method to invoke + if key.scope == Scope.children: + self._children = value + # TODO remove inheritance from any orphaned exchildren + # TODO add inheritance to any new children + elif key.scope == Scope.settings: + # TODO if inheritable, push down to children who don't override + self._metadata[key.field_name] = value + elif key.scope == Scope.content: + if key.field_name == 'location': + self._location = value + elif key.field_name == 'category': + self._category = value + else: + if isinstance(self._definition, DefinitionLazyLoader): + self._definition = self._definition.fetch() + if (key.field_name == 'data' and + not isinstance(self._definition.get('data'), dict)): + self._definition.get('data') + else: + self._definition.setdefault('data', {})[key.field_name] = value + else: + raise InvalidScopeError(key.scope) + + def delete(self, key): + # TODO cache db update implications & add method to invoke + if key.scope == Scope.children: + self._children = [] + elif key.scope == Scope.settings: + # TODO if inheritable, ensure _inherited_metadata has value from above and + # revert children to that value + if key.field_name in self._metadata: + del self._metadata[key.field_name] + elif key.scope == Scope.content: + # don't allow deletion of location nor category + if key.field_name == 'location': + pass + elif key.field_name == 'category': + pass + else: + if isinstance(self._definition, DefinitionLazyLoader): + self._definition = self._definition.fetch() + if (key.field_name == 'data' and + not isinstance(self._definition.get('data'), dict)): + self._definition.setdefault('data', None) + else: + try: + del self._definition['data'][key.field_name] + except KeyError: + pass + else: + raise InvalidScopeError(key.scope) + + def has(self, key): + if key.scope in (Scope.children, Scope.parent): + return True + elif key.scope == Scope.settings: + return key.field_name in self._metadata or key.field_name in self._inherited_metadata + elif key.scope == Scope.content: + if key.field_name == 'location': + return True + elif key.field_name == 'category': + return self._category is not None + else: + if isinstance(self._definition, DefinitionLazyLoader): + self._definition = self._definition.fetch() + if (key.field_name == 'data' and + not isinstance(self._definition.get('data'), dict)): + return self._definition.get('data') is not None + else: + return key.field_name in self._definition.get('data', {}) + else: + return False + + def get_data(self): + """ + Intended only for use by persistence layer to get the native definition['data'] rep + """ + if isinstance(self._definition, DefinitionLazyLoader): + self._definition = self._definition.fetch() + return self._definition.get('data') + + def get_own_metadata(self): + """ + Get the metadata explicitly set on this element. + """ + return self._metadata + + def get_inherited_metadata(self): + """ + Get the metadata set by the ancestors (which own metadata may override or not) + """ + return self._inherited_metadata diff --git a/common/lib/xmodule/xmodule/modulestore/tests/django_utils.py b/common/lib/xmodule/xmodule/modulestore/tests/django_utils.py index 6c5c1f66ca..4f998d57fb 100644 --- a/common/lib/xmodule/xmodule/modulestore/tests/django_utils.py +++ b/common/lib/xmodule/xmodule/modulestore/tests/django_utils.py @@ -5,7 +5,6 @@ from django.test import TestCase from django.conf import settings import xmodule.modulestore.django -from xmodule.templates import update_templates from unittest.util import safe_repr @@ -48,7 +47,7 @@ def draft_mongo_store_config(data_dir): return { 'default': { - 'ENGINE': 'xmodule.modulestore.mongo.DraftMongoModuleStore', + 'ENGINE': 'xmodule.modulestore.mongo.draft.DraftModuleStore', 'OPTIONS': modulestore_options }, 'direct': { @@ -110,22 +109,6 @@ class ModuleStoreTestCase(TestCase): modulestore.collection.remove(query) modulestore.collection.drop() - @staticmethod - def load_templates_if_necessary(): - """ - Load templates into the direct modulestore only if they do not already exist. - We need the templates, because they are copied to create - XModules such as sections and problems. - """ - modulestore = xmodule.modulestore.django.modulestore('direct') - - # Count the number of templates - query = {"_id.course": "templates"} - num_templates = modulestore.collection.find(query).count() - - if num_templates < 1: - update_templates(modulestore) - @classmethod def setUpClass(cls): """ @@ -169,9 +152,6 @@ class ModuleStoreTestCase(TestCase): # Flush anything that is not a template ModuleStoreTestCase.flush_mongo_except_templates() - # Check that we have templates loaded; if not, load them - ModuleStoreTestCase.load_templates_if_necessary() - # Call superclass implementation super(ModuleStoreTestCase, self)._pre_setup() @@ -185,34 +165,31 @@ class ModuleStoreTestCase(TestCase): # Call superclass implementation super(ModuleStoreTestCase, self)._post_teardown() + def assert2XX(self, status_code, msg=None): """ Assert that the given value is a success status (between 200 and 299) """ - if not 200 <= status_code < 300: - msg = self._formatMessage(msg, "%s is not a success status" % safe_repr(status_code)) - raise self.failureExecption(msg) + msg = self._formatMessage(msg, "%s is not a success status" % safe_repr(status_code)) + self.assertTrue(status_code >= 200 and status_code < 300, msg=msg) def assert3XX(self, status_code, msg=None): """ Assert that the given value is a redirection status (between 300 and 399) """ - if not 300 <= status_code < 400: - msg = self._formatMessage(msg, "%s is not a redirection status" % safe_repr(status_code)) - raise self.failureExecption(msg) + msg = self._formatMessage(msg, "%s is not a redirection status" % safe_repr(status_code)) + self.assertTrue(status_code >= 300 and status_code < 400, msg=msg) def assert4XX(self, status_code, msg=None): """ Assert that the given value is a client error status (between 400 and 499) """ - if not 400 <= status_code < 500: - msg = self._formatMessage(msg, "%s is not a client error status" % safe_repr(status_code)) - raise self.failureExecption(msg) + msg = self._formatMessage(msg, "%s is not a client error status" % safe_repr(status_code)) + self.assertTrue(status_code >= 400 and status_code < 500, msg=msg) def assert5XX(self, status_code, msg=None): """ Assert that the given value is a server error status (between 500 and 599) """ - if not 500 <= status_code < 600: - msg = self._formatMessage(msg, "%s is not a server error status" % safe_repr(status_code)) - raise self.failureExecption(msg) + msg = self._formatMessage(msg, "%s is not a server error status" % safe_repr(status_code)) + self.assertTrue(status_code >= 500 and status_code < 600, msg=msg) diff --git a/common/lib/xmodule/xmodule/modulestore/tests/factories.py b/common/lib/xmodule/xmodule/modulestore/tests/factories.py index 457a88482a..f2e4017114 100644 --- a/common/lib/xmodule/xmodule/modulestore/tests/factories.py +++ b/common/lib/xmodule/xmodule/modulestore/tests/factories.py @@ -1,15 +1,14 @@ -from factory import Factory, lazy_attribute_sequence, lazy_attribute -from uuid import uuid4 import datetime +from factory import Factory, LazyAttributeSequence +from uuid import uuid4 +from pytz import UTC + from xmodule.modulestore import Location from xmodule.modulestore.django import modulestore -from xmodule.modulestore.inheritance import own_metadata -from xmodule.x_module import ModuleSystem -from mitxmako.shortcuts import render_to_string -from xblock.runtime import InvalidScopeError -from pytz import UTC - +from xmodule.course_module import CourseDescriptor +from xblock.core import Scope +from xmodule.x_module import XModuleDescriptor class XModuleCourseFactory(Factory): """ @@ -21,9 +20,8 @@ class XModuleCourseFactory(Factory): @classmethod def _create(cls, target_class, **kwargs): - template = Location('i4x', 'edx', 'templates', 'course', 'Empty') org = kwargs.pop('org', None) - number = kwargs.pop('number', None) + number = kwargs.pop('number', kwargs.pop('course', None)) display_name = kwargs.pop('display_name', None) location = Location('i4x', org, number, 'course', Location.clean(display_name)) @@ -33,36 +31,20 @@ class XModuleCourseFactory(Factory): store = modulestore() # Write the data to the mongo datastore - new_course = store.clone_item(template, location) + new_course = store.create_xmodule(location) # This metadata code was copied from cms/djangoapps/contentstore/views.py if display_name is not None: new_course.display_name = display_name - new_course.lms.start = datetime.datetime.now(UTC) - new_course.tabs = kwargs.pop( - 'tabs', - [ - {"type": "courseware"}, - {"type": "course_info", "name": "Course Info"}, - {"type": "discussion", "name": "Discussion"}, - {"type": "wiki", "name": "Wiki"}, - {"type": "progress", "name": "Progress"} - ] - ) + new_course.lms.start = datetime.datetime.now(UTC).replace(microsecond=0) # The rest of kwargs become attributes on the course: for k, v in kwargs.iteritems(): setattr(new_course, k, v) # Update the data in the mongo datastore - store.update_metadata(new_course.location, own_metadata(new_course)) - store.update_item(new_course.location, new_course._model_data._kvs._data) - - # update_item updates the the course as it exists in the modulestore, but doesn't - # update the instance we are working with, so have to refetch the course after updating it. - new_course = store.get_instance(new_course.id, new_course.location) - + store.save_xmodule(new_course) return new_course @@ -73,7 +55,6 @@ class Course: class CourseFactory(XModuleCourseFactory): FACTORY_FOR = Course - template = 'i4x://edx/templates/course/Empty' org = 'MITx' number = '999' display_name = 'Robot Super Course' @@ -86,76 +67,71 @@ class XModuleItemFactory(Factory): ABSTRACT_FACTORY = True - display_name = None + parent_location = 'i4x://MITx/999/course/Robot_Super_Course' + category = 'problem' + display_name = LazyAttributeSequence(lambda o, n: "{} {}".format(o.category, n)) - @lazy_attribute - def category(attr): - template = Location(attr.template) - return template.category - - @lazy_attribute - def location(attr): - parent = Location(attr.parent_location) - dest_name = attr.display_name.replace(" ", "_") if attr.display_name is not None else uuid4().hex - return parent._replace(category=attr.category, name=dest_name) + @staticmethod + def location(parent, category, display_name): + dest_name = display_name.replace(" ", "_") if display_name is not None else uuid4().hex + return Location(parent).replace(category=category, name=dest_name) @classmethod def _create(cls, target_class, **kwargs): """ - Uses *kwargs*: + Uses ``**kwargs``: - *parent_location* (required): the location of the parent module + :parent_location: (required): the location of the parent module (e.g. the parent course or section) - *template* (required): the template to create the item from - (e.g. i4x://templates/section/Empty) + :category: the category of the resulting item. - *data* (optional): the data for the item + :data: (optional): the data for the item (e.g. XML problem definition for a problem item) - *display_name* (optional): the display name of the item + :display_name: (optional): the display name of the item - *metadata* (optional): dictionary of metadata attributes + :metadata: (optional): dictionary of metadata attributes - *target_class* is ignored + :boilerplate: (optional) the boilerplate for overriding field values + + :target_class: is ignored """ DETACHED_CATEGORIES = ['about', 'static_tab', 'course_info'] - + # catch any old style users before they get into trouble + assert not 'template' in kwargs parent_location = Location(kwargs.get('parent_location')) - template = Location(kwargs.get('template')) data = kwargs.get('data') + category = kwargs.get('category') display_name = kwargs.get('display_name') metadata = kwargs.get('metadata', {}) + location = kwargs.get('location', XModuleItemFactory.location(parent_location, category, display_name)) + assert location != parent_location + if kwargs.get('boilerplate') is not None: + template_id = kwargs.get('boilerplate') + clz = XModuleDescriptor.load_class(category) + template = clz.get_template(template_id) + assert template is not None + metadata.update(template.get('metadata', {})) + if not isinstance(data, basestring): + data.update(template.get('data')) store = modulestore('direct') # This code was based off that in cms/djangoapps/contentstore/views.py parent = store.get_item(parent_location) - new_item = store.clone_item(template, kwargs.get('location')) - # replace the display name with an optional parameter passed in from the caller if display_name is not None: - new_item.display_name = display_name + metadata['display_name'] = display_name + store.create_and_save_xmodule(location, metadata=metadata, definition_data=data) - # Add additional metadata or override current metadata - item_metadata = own_metadata(new_item) - item_metadata.update(metadata) - store.update_metadata(new_item.location.url(), item_metadata) + if location.category not in DETACHED_CATEGORIES: + parent.children.append(location.url()) + store.update_children(parent_location, parent.children) - # replace the data with the optional *data* parameter - if data is not None: - store.update_item(new_item.location, data) - - if new_item.location.category not in DETACHED_CATEGORIES: - store.update_children(parent_location, parent.children + [new_item.location.url()]) - - # update_children updates the the item as it exists in the modulestore, but doesn't - # update the instance we are working with, so have to refetch the item after updating it. - new_item = store.get_item(new_item.location) - - return new_item + return store.get_item(location) class Item: @@ -164,40 +140,4 @@ class Item: class ItemFactory(XModuleItemFactory): FACTORY_FOR = Item - - parent_location = 'i4x://MITx/999/course/Robot_Super_Course' - template = 'i4x://edx/templates/chapter/Empty' - - @lazy_attribute_sequence - def display_name(attr, n): - return "{} {}".format(attr.category.title(), n) - - -def get_test_xmodule_for_descriptor(descriptor): - """ - Attempts to create an xmodule which responds usually correctly from the descriptor. Not guaranteed. - - :param descriptor: - """ - module_sys = ModuleSystem( - ajax_url='', - track_function=None, - get_module=None, - render_template=render_to_string, - replace_urls=None, - xblock_model_data=_test_xblock_model_data_accessor(descriptor) - ) - return descriptor.xmodule(module_sys) - - -def _test_xblock_model_data_accessor(descriptor): - simple_map = {} - for field in descriptor.fields: - try: - simple_map[field.name] = getattr(descriptor, field.name) - except InvalidScopeError: - simple_map[field.name] = field.default - for field in descriptor.module_class.fields: - if field.name not in simple_map: - simple_map[field.name] = field.default - return lambda o: simple_map + category = 'chapter' diff --git a/common/lib/xmodule/xmodule/modulestore/tests/persistent_factories.py b/common/lib/xmodule/xmodule/modulestore/tests/persistent_factories.py new file mode 100644 index 0000000000..5e46f5a318 --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/tests/persistent_factories.py @@ -0,0 +1,96 @@ +from xmodule.modulestore.django import modulestore +from xmodule.course_module import CourseDescriptor +from xmodule.x_module import XModuleDescriptor +import factory + + +# [dhm] I'm not sure why we're using factory_boy if we're not following its pattern. If anyone +# assumes they can call build, it will completely fail, for example. +# pylint: disable=W0232 +class PersistentCourseFactory(factory.Factory): + """ + Create a new course (not a new version of a course, but a whole new index entry). + + keywords: + * org: defaults to textX + * prettyid: defaults to 999 + * display_name + * user_id + * data (optional) the data payload to save in the course item + * metadata (optional) the metadata payload. If display_name is in the metadata, that takes + precedence over any display_name provided directly. + """ + FACTORY_FOR = CourseDescriptor + + org = 'testX' + prettyid = '999' + display_name = 'Robot Super Course' + user_id = "test_user" + data = None + metadata = None + master_version = 'draft' + + # pylint: disable=W0613 + @classmethod + def _create(cls, target_class, *args, **kwargs): + + org = kwargs.get('org') + prettyid = kwargs.get('prettyid') + display_name = kwargs.get('display_name') + user_id = kwargs.get('user_id') + data = kwargs.get('data') + metadata = kwargs.get('metadata', {}) + if metadata is None: + metadata = {} + if 'display_name' not in metadata: + metadata['display_name'] = display_name + + # Write the data to the mongo datastore + new_course = modulestore('split').create_course( + org, prettyid, user_id, metadata=metadata, course_data=data, id_root=prettyid, + master_version=kwargs.get('master_version')) + + return new_course + + @classmethod + def _build(cls, target_class, *args, **kwargs): + raise NotImplementedError() + + +class ItemFactory(factory.Factory): + FACTORY_FOR = XModuleDescriptor + + category = 'chapter' + user_id = 'test_user' + display_name = factory.LazyAttributeSequence(lambda o, n: "{} {}".format(o.category, n)) + + # pylint: disable=W0613 + @classmethod + def _create(cls, target_class, *args, **kwargs): + """ + Uses *kwargs*: + + *parent_location* (required): the location of the course & possibly parent + + *category* (defaults to 'chapter') + + *data* (optional): the data for the item + + definition_locator (optional): the DescriptorLocator for the definition this uses or branches + + *display_name* (optional): the display name of the item + + *metadata* (optional): dictionary of metadata attributes (display_name here takes + precedence over the above attr) + """ + metadata = kwargs.get('metadata', {}) + if 'display_name' not in metadata and 'display_name' in kwargs: + metadata['display_name'] = kwargs['display_name'] + + return modulestore('split').create_item(kwargs['parent_location'], kwargs['category'], + kwargs['user_id'], definition_locator=kwargs.get('definition_locator'), + new_def_data=kwargs.get('data'), metadata=metadata) + + @classmethod + def _build(cls, target_class, *args, **kwargs): + raise NotImplementedError() diff --git a/common/lib/xmodule/xmodule/modulestore/tests/test_locators.py b/common/lib/xmodule/xmodule/modulestore/tests/test_locators.py new file mode 100644 index 0000000000..2626b6692d --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/tests/test_locators.py @@ -0,0 +1,539 @@ +''' +Created on Mar 14, 2013 + +@author: dmitchell +''' +from unittest import TestCase +from nose.plugins.skip import SkipTest + +from bson.objectid import ObjectId +from xmodule.modulestore.locator import Locator, CourseLocator, BlockUsageLocator +from xmodule.modulestore.exceptions import InvalidLocationError, \ + InsufficientSpecificationError, OverSpecificationError + + +class LocatorTest(TestCase): + + def test_cant_instantiate_abstract_class(self): + self.assertRaises(TypeError, Locator) + + def test_course_constructor_overspecified(self): + self.assertRaises( + OverSpecificationError, + CourseLocator, + url='edx://edu.mit.eecs.6002x', + course_id='edu.harvard.history', + revision='published', + version_guid=ObjectId()) + self.assertRaises( + OverSpecificationError, + CourseLocator, + url='edx://edu.mit.eecs.6002x', + course_id='edu.harvard.history', + version_guid=ObjectId()) + self.assertRaises( + OverSpecificationError, + CourseLocator, + url='edx://edu.mit.eecs.6002x;published', + revision='draft') + self.assertRaises( + OverSpecificationError, + CourseLocator, + course_id='edu.mit.eecs.6002x;published', + revision='draft') + + def test_course_constructor_underspecified(self): + self.assertRaises(InsufficientSpecificationError, CourseLocator) + self.assertRaises(InsufficientSpecificationError, CourseLocator, revision='published') + + def test_course_constructor_bad_version_guid(self): + self.assertRaises(ValueError, CourseLocator, version_guid="012345") + self.assertRaises(InsufficientSpecificationError, CourseLocator, version_guid=None) + + def test_course_constructor_version_guid(self): + # generate a random location + test_id_1 = ObjectId() + test_id_1_loc = str(test_id_1) + testobj_1 = CourseLocator(version_guid=test_id_1) + self.check_course_locn_fields(testobj_1, 'version_guid', version_guid=test_id_1) + self.assertEqual(str(testobj_1.version_guid), test_id_1_loc) + self.assertEqual(str(testobj_1), '@' + test_id_1_loc) + self.assertEqual(testobj_1.url(), 'edx://@' + test_id_1_loc) + + # Test using a given string + test_id_2_loc = '519665f6223ebd6980884f2b' + test_id_2 = ObjectId(test_id_2_loc) + testobj_2 = CourseLocator(version_guid=test_id_2) + self.check_course_locn_fields(testobj_2, 'version_guid', version_guid=test_id_2) + self.assertEqual(str(testobj_2.version_guid), test_id_2_loc) + self.assertEqual(str(testobj_2), '@' + test_id_2_loc) + self.assertEqual(testobj_2.url(), 'edx://@' + test_id_2_loc) + + def test_course_constructor_bad_course_id(self): + """ + Test all sorts of badly-formed course_ids (and urls with those course_ids) + """ + for bad_id in ('edu.mit.', + ' edu.mit.eecs', + 'edu.mit.eecs ', + '@edu.mit.eecs', + '#edu.mit.eecs', + 'edu.mit.ee cs', + 'edu.mit.ee,cs', + 'edu.mit.ee/cs', + 'edu.mit.ee$cs', + 'edu.mit.ee&cs', + 'edu.mit.ee()cs', + ';this', + 'edu.mit.eecs;', + 'edu.mit.eecs;this;that', + 'edu.mit.eecs;this;', + 'edu.mit.eecs;this ', + 'edu.mit.eecs;th%is ', + ): + self.assertRaises(AssertionError, CourseLocator, course_id=bad_id) + self.assertRaises(AssertionError, CourseLocator, url='edx://' + bad_id) + + def test_course_constructor_bad_url(self): + for bad_url in ('edx://', + 'edx:/edu.mit.eecs', + 'http://edu.mit.eecs', + 'edu.mit.eecs', + 'edx//edu.mit.eecs'): + self.assertRaises(AssertionError, CourseLocator, url=bad_url) + + def test_course_constructor_redundant_001(self): + testurn = 'edu.mit.eecs.6002x' + testobj = CourseLocator(course_id=testurn, url='edx://' + testurn) + self.check_course_locn_fields(testobj, 'course_id', course_id=testurn) + + def test_course_constructor_redundant_002(self): + testurn = 'edu.mit.eecs.6002x;published' + expected_urn = 'edu.mit.eecs.6002x' + expected_rev = 'published' + testobj = CourseLocator(course_id=testurn, url='edx://' + testurn) + self.check_course_locn_fields(testobj, 'course_id', + course_id=expected_urn, + revision=expected_rev) + + def test_course_constructor_course_id_no_revision(self): + testurn = 'edu.mit.eecs.6002x' + testobj = CourseLocator(course_id=testurn) + self.check_course_locn_fields(testobj, 'course_id', course_id=testurn) + self.assertEqual(testobj.course_id, testurn) + self.assertEqual(str(testobj), testurn) + self.assertEqual(testobj.url(), 'edx://' + testurn) + + def test_course_constructor_course_id_with_revision(self): + testurn = 'edu.mit.eecs.6002x;published' + expected_id = 'edu.mit.eecs.6002x' + expected_revision = 'published' + testobj = CourseLocator(course_id=testurn) + self.check_course_locn_fields(testobj, 'course_id with revision', + course_id=expected_id, + revision=expected_revision, + ) + self.assertEqual(testobj.course_id, expected_id) + self.assertEqual(testobj.revision, expected_revision) + self.assertEqual(str(testobj), testurn) + self.assertEqual(testobj.url(), 'edx://' + testurn) + + def test_course_constructor_course_id_separate_revision(self): + test_id = 'edu.mit.eecs.6002x' + test_revision = 'published' + expected_urn = 'edu.mit.eecs.6002x;published' + testobj = CourseLocator(course_id=test_id, revision=test_revision) + self.check_course_locn_fields(testobj, 'course_id with separate revision', + course_id=test_id, + revision=test_revision, + ) + self.assertEqual(testobj.course_id, test_id) + self.assertEqual(testobj.revision, test_revision) + self.assertEqual(str(testobj), expected_urn) + self.assertEqual(testobj.url(), 'edx://' + expected_urn) + + def test_course_constructor_course_id_repeated_revision(self): + """ + The same revision appears in the course_id and the revision field. + """ + test_id = 'edu.mit.eecs.6002x;published' + test_revision = 'published' + expected_id = 'edu.mit.eecs.6002x' + expected_urn = 'edu.mit.eecs.6002x;published' + testobj = CourseLocator(course_id=test_id, revision=test_revision) + self.check_course_locn_fields(testobj, 'course_id with repeated revision', + course_id=expected_id, + revision=test_revision, + ) + self.assertEqual(testobj.course_id, expected_id) + self.assertEqual(testobj.revision, test_revision) + self.assertEqual(str(testobj), expected_urn) + self.assertEqual(testobj.url(), 'edx://' + expected_urn) + + def test_block_constructor(self): + testurn = 'edu.mit.eecs.6002x;published#HW3' + expected_id = 'edu.mit.eecs.6002x' + expected_revision = 'published' + expected_block_ref = 'HW3' + testobj = BlockUsageLocator(course_id=testurn) + self.check_block_locn_fields(testobj, 'test_block constructor', + course_id=expected_id, + revision=expected_revision, + block=expected_block_ref) + self.assertEqual(str(testobj), testurn) + self.assertEqual(testobj.url(), 'edx://' + testurn) + + # ------------------------------------------------------------ + # Disabled tests + + def test_course_urls(self): + ''' + Test constructor and property accessors. + ''' + raise SkipTest() + self.assertRaises(TypeError, CourseLocator, 'empty constructor') + + # url inits + testurn = 'edx://org/course/category/name' + self.assertRaises(InvalidLocationError, CourseLocator, url=testurn) + testurn = 'unknown/versionid/blockid' + self.assertRaises(InvalidLocationError, CourseLocator, url=testurn) + + testurn = 'cvx/versionid' + testobj = CourseLocator(testurn) + self.check_course_locn_fields(testobj, testurn, 'versionid') + self.assertEqual(testobj, CourseLocator(testobj), + 'initialization from another instance') + + testurn = 'cvx/versionid/' + testobj = CourseLocator(testurn) + self.check_course_locn_fields(testobj, testurn, 'versionid') + + testurn = 'cvx/versionid/blockid' + testobj = CourseLocator(testurn) + self.check_course_locn_fields(testobj, testurn, 'versionid') + + testurn = 'cvx/versionid/blockid/extraneousstuff?including=args' + testobj = CourseLocator(testurn) + self.check_course_locn_fields(testobj, testurn, 'versionid') + + testurn = 'cvx://versionid/blockid' + testobj = CourseLocator(testurn) + self.check_course_locn_fields(testobj, testurn, 'versionid') + + testurn = 'crx/courseid/blockid' + testobj = CourseLocator(testurn) + self.check_course_locn_fields(testobj, testurn, course_id='courseid') + + testurn = 'crx/courseid@revision/blockid' + testobj = CourseLocator(testurn) + self.check_course_locn_fields(testobj, testurn, course_id='courseid', + revision='revision') + self.assertEqual(testobj, CourseLocator(testobj), + 'run initialization from another instance') + + def test_course_keyword_setters(self): + raise SkipTest() + # arg list inits + testobj = CourseLocator(version_guid='versionid') + self.check_course_locn_fields(testobj, 'versionid arg', 'versionid') + + testobj = CourseLocator(course_id='courseid') + self.check_course_locn_fields(testobj, 'courseid arg', + course_id='courseid') + + testobj = CourseLocator(course_id='courseid', revision='rev') + self.check_course_locn_fields(testobj, 'rev arg', + course_id='courseid', + revision='rev') + # ignores garbage + testobj = CourseLocator(course_id='courseid', revision='rev', + potato='spud') + self.check_course_locn_fields(testobj, 'extra keyword arg', + course_id='courseid', + revision='rev') + + # url w/ keyword override + testurn = 'crx/courseid@revision/blockid' + testobj = CourseLocator(testurn, revision='rev') + self.check_course_locn_fields(testobj, 'rev override', + course_id='courseid', + revision='rev') + + def test_course_dict(self): + raise SkipTest() + # dict init w/ keyword overwrites + testobj = CourseLocator({"version_guid": 'versionid'}) + self.check_course_locn_fields(testobj, 'versionid dict', 'versionid') + + testobj = CourseLocator({"course_id": 'courseid'}) + self.check_course_locn_fields(testobj, 'courseid dict', + course_id='courseid') + + testobj = CourseLocator({"course_id": 'courseid', "revision": 'rev'}) + self.check_course_locn_fields(testobj, 'rev dict', + course_id='courseid', + revision='rev') + # ignores garbage + testobj = CourseLocator({"course_id": 'courseid', "revision": 'rev', + "potato": 'spud'}) + self.check_course_locn_fields(testobj, 'extra keyword dict', + course_id='courseid', + revision='rev') + testobj = CourseLocator({"course_id": 'courseid', "revision": 'rev'}, + revision='alt') + self.check_course_locn_fields(testobj, 'rev dict', + course_id='courseid', + revision='alt') + + # urn init w/ dict & keyword overwrites + testobj = CourseLocator('crx/notcourse@notthis', + {"course_id": 'courseid'}, + revision='alt') + self.check_course_locn_fields(testobj, 'rev dict', + course_id='courseid', + revision='alt') + + def test_url(self): + ''' + Ensure CourseLocator generates expected urls. + ''' + raise SkipTest() + + testobj = CourseLocator(version_guid='versionid') + self.assertEqual(testobj.url(), 'cvx/versionid', 'versionid') + self.assertEqual(testobj, CourseLocator(testobj.url()), + 'versionid conversion through url') + + testobj = CourseLocator(course_id='courseid') + self.assertEqual(testobj.url(), 'crx/courseid', 'courseid') + self.assertEqual(testobj, CourseLocator(testobj.url()), + 'courseid conversion through url') + + testobj = CourseLocator(course_id='courseid', revision='rev') + self.assertEqual(testobj.url(), 'crx/courseid@rev', 'rev') + self.assertEqual(testobj, CourseLocator(testobj.url()), + 'rev conversion through url') + + def test_html(self): + ''' + Ensure CourseLocator generates expected urls. + ''' + raise SkipTest() + testobj = CourseLocator(version_guid='versionid') + self.assertEqual(testobj.html_id(), 'cvx/versionid', 'versionid') + self.assertEqual(testobj, CourseLocator(testobj.html_id()), + 'versionid conversion through html_id') + + testobj = CourseLocator(course_id='courseid') + self.assertEqual(testobj.html_id(), 'crx/courseid', 'courseid') + self.assertEqual(testobj, CourseLocator(testobj.html_id()), + 'courseid conversion through html_id') + + testobj = CourseLocator(course_id='courseid', revision='rev') + self.assertEqual(testobj.html_id(), 'crx/courseid%40rev', 'rev') + self.assertEqual(testobj, CourseLocator(testobj.html_id()), + 'rev conversion through html_id') + + def test_block_locator(self): + ''' + Test constructor and property accessors. + ''' + raise SkipTest() + self.assertIsInstance(BlockUsageLocator(), BlockUsageLocator, + 'empty constructor') + + # url inits + testurn = 'edx://org/course/category/name' + self.assertRaises(InvalidLocationError, BlockUsageLocator, testurn) + testurn = 'unknown/versionid/blockid' + self.assertRaises(InvalidLocationError, BlockUsageLocator, testurn) + + testurn = 'cvx/versionid' + testobj = BlockUsageLocator(testurn) + self.check_block_locn_fields(testobj, testurn, 'versionid') + self.assertEqual(testobj, BlockUsageLocator(testobj), + 'initialization from another instance') + + testurn = 'cvx/versionid/' + testobj = BlockUsageLocator(testurn) + self.check_block_locn_fields(testobj, testurn, 'versionid') + + testurn = 'cvx/versionid/blockid' + testobj = BlockUsageLocator(testurn) + self.check_block_locn_fields(testobj, testurn, 'versionid', + block='blockid') + + testurn = 'cvx/versionid/blockid/extraneousstuff?including=args' + testobj = BlockUsageLocator(testurn) + self.check_block_locn_fields(testobj, testurn, 'versionid', + block='blockid') + + testurn = 'cvx://versionid/blockid' + testobj = BlockUsageLocator(testurn) + self.check_block_locn_fields(testobj, testurn, 'versionid', + block='blockid') + + testurn = 'crx/courseid/blockid' + testobj = BlockUsageLocator(testurn) + self.check_block_locn_fields(testobj, testurn, course_id='courseid', + block='blockid') + + testurn = 'crx/courseid@revision/blockid' + testobj = BlockUsageLocator(testurn) + self.check_block_locn_fields(testobj, testurn, course_id='courseid', + revision='revision', block='blockid') + self.assertEqual(testobj, BlockUsageLocator(testobj), + 'run initialization from another instance') + + def test_block_keyword_init(self): + # arg list inits + raise SkipTest() + testobj = BlockUsageLocator(version_guid='versionid') + self.check_block_locn_fields(testobj, 'versionid arg', 'versionid') + + testobj = BlockUsageLocator(version_guid='versionid', usage_id='myblock') + self.check_block_locn_fields(testobj, 'versionid arg', 'versionid', + block='myblock') + + testobj = BlockUsageLocator(course_id='courseid') + self.check_block_locn_fields(testobj, 'courseid arg', + course_id='courseid') + + testobj = BlockUsageLocator(course_id='courseid', revision='rev') + self.check_block_locn_fields(testobj, 'rev arg', + course_id='courseid', + revision='rev') + # ignores garbage + testobj = BlockUsageLocator(course_id='courseid', revision='rev', + usage_id='this_block', potato='spud') + self.check_block_locn_fields(testobj, 'extra keyword arg', + course_id='courseid', block='this_block', revision='rev') + + # url w/ keyword override + testurn = 'crx/courseid@revision/blockid' + testobj = BlockUsageLocator(testurn, revision='rev') + self.check_block_locn_fields(testobj, 'rev override', + course_id='courseid', block='blockid', + revision='rev') + + def test_block_keywords(self): + # dict init w/ keyword overwrites + raise SkipTest() + testobj = BlockUsageLocator({"version_guid": 'versionid', + 'usage_id': 'dictblock'}) + self.check_block_locn_fields(testobj, 'versionid dict', 'versionid', + block='dictblock') + + testobj = BlockUsageLocator({"course_id": 'courseid', + 'usage_id': 'dictblock'}) + self.check_block_locn_fields(testobj, 'courseid dict', + block='dictblock', course_id='courseid') + + testobj = BlockUsageLocator({"course_id": 'courseid', "revision": 'rev', + 'usage_id': 'dictblock'}) + self.check_block_locn_fields(testobj, 'rev dict', + course_id='courseid', block='dictblock', + revision='rev') + # ignores garbage + testobj = BlockUsageLocator({"course_id": 'courseid', "revision": 'rev', + 'usage_id': 'dictblock', "potato": 'spud'}) + self.check_block_locn_fields(testobj, 'extra keyword dict', + course_id='courseid', block='dictblock', + revision='rev') + testobj = BlockUsageLocator({"course_id": 'courseid', "revision": 'rev', + 'usage_id': 'dictblock'}, revision='alt', usage_id='anotherblock') + self.check_block_locn_fields(testobj, 'rev dict', + course_id='courseid', block='anotherblock', + revision='alt') + + # urn init w/ dict & keyword overwrites + testobj = BlockUsageLocator('crx/notcourse@notthis/northis', + {"course_id": 'courseid'}, revision='alt', usage_id='anotherblock') + self.check_block_locn_fields(testobj, 'rev dict', + course_id='courseid', block='anotherblock', + revision='alt') + + def test_ensure_fully_specd(self): + ''' + Test constructor and property accessors. + ''' + raise SkipTest() + self.assertRaises(InsufficientSpecificationError, + BlockUsageLocator.ensure_fully_specified, BlockUsageLocator()) + + # url inits + testurn = 'edx://org/course/category/name' + self.assertRaises(InvalidLocationError, + BlockUsageLocator.ensure_fully_specified, testurn) + testurn = 'unknown/versionid/blockid' + self.assertRaises(InvalidLocationError, + BlockUsageLocator.ensure_fully_specified, testurn) + + testurn = 'cvx/versionid' + self.assertRaises(InsufficientSpecificationError, + BlockUsageLocator.ensure_fully_specified, testurn) + + testurn = 'cvx/versionid/' + self.assertRaises(InsufficientSpecificationError, + BlockUsageLocator.ensure_fully_specified, testurn) + + testurn = 'cvx/versionid/blockid' + self.assertIsInstance(BlockUsageLocator.ensure_fully_specified(testurn), + BlockUsageLocator, testurn) + + testurn = 'cvx/versionid/blockid/extraneousstuff?including=args' + self.assertIsInstance(BlockUsageLocator.ensure_fully_specified(testurn), + BlockUsageLocator, testurn) + + testurn = 'cvx://versionid/blockid' + self.assertIsInstance(BlockUsageLocator.ensure_fully_specified(testurn), + BlockUsageLocator, testurn) + + testurn = 'crx/courseid/blockid' + self.assertIsInstance(BlockUsageLocator.ensure_fully_specified(testurn), + BlockUsageLocator, testurn) + + testurn = 'crx/courseid@revision/blockid' + self.assertIsInstance(BlockUsageLocator.ensure_fully_specified(testurn), + BlockUsageLocator, testurn) + + def test_ensure_fully_via_keyword(self): + # arg list inits + raise SkipTest() + testobj = BlockUsageLocator(version_guid='versionid') + self.assertRaises(InsufficientSpecificationError, + BlockUsageLocator.ensure_fully_specified, testobj) + + testurn = 'crx/courseid@revision/blockid' + testobj = BlockUsageLocator(version_guid='versionid', usage_id='myblock') + self.assertIsInstance(BlockUsageLocator.ensure_fully_specified(testurn), + BlockUsageLocator, testurn) + + testobj = BlockUsageLocator(course_id='courseid') + self.assertRaises(InsufficientSpecificationError, + BlockUsageLocator.ensure_fully_specified, testobj) + + testobj = BlockUsageLocator(course_id='courseid', revision='rev') + self.assertRaises(InsufficientSpecificationError, + BlockUsageLocator.ensure_fully_specified, testobj) + + testobj = BlockUsageLocator(course_id='courseid', revision='rev', + usage_id='this_block') + self.assertIsInstance(BlockUsageLocator.ensure_fully_specified(testurn), + BlockUsageLocator, testurn) + + # ------------------------------------------------------------------ + # Utilities + + def check_course_locn_fields(self, testobj, msg, version_guid=None, + course_id=None, revision=None): + self.assertEqual(testobj.version_guid, version_guid, msg) + self.assertEqual(testobj.course_id, course_id, msg) + self.assertEqual(testobj.revision, revision, msg) + + def check_block_locn_fields(self, testobj, msg, version_guid=None, + course_id=None, revision=None, block=None): + self.check_course_locn_fields(testobj, msg, version_guid, course_id, + revision) + self.assertEqual(testobj.usage_id, block) diff --git a/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py b/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py index 44e69fb0ed..c149724cc7 100644 --- a/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py +++ b/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py @@ -9,7 +9,6 @@ from xblock.runtime import KeyValueStore, InvalidScopeError from xmodule.modulestore import Location from xmodule.modulestore.mongo import MongoModuleStore, MongoKeyValueStore from xmodule.modulestore.xml_importer import import_from_xml -from xmodule.templates import update_templates from .test_modulestore import check_path_to_location from . import DATA_DIR @@ -51,7 +50,6 @@ class TestMongoModuleStore(object): # Explicitly list the courses to load (don't want the big one) courses = ['toy', 'simple'] import_from_xml(store, DATA_DIR, courses) - update_templates(store) return store @staticmethod @@ -126,7 +124,7 @@ class TestMongoKeyValueStore(object): self.location = Location('i4x://org/course/category/name@version') self.children = ['i4x://org/course/child/a', 'i4x://org/course/child/b'] self.metadata = {'meta': 'meta_val'} - self.kvs = MongoKeyValueStore(self.data, self.children, self.metadata, self.location) + self.kvs = MongoKeyValueStore(self.data, self.children, self.metadata, self.location, 'category') def _check_read(self, key, expected_value): assert_equals(expected_value, self.kvs.get(key)) diff --git a/common/lib/xmodule/xmodule/modulestore/tests/test_split_modulestore.py b/common/lib/xmodule/xmodule/modulestore/tests/test_split_modulestore.py new file mode 100644 index 0000000000..29f6cce919 --- /dev/null +++ b/common/lib/xmodule/xmodule/modulestore/tests/test_split_modulestore.py @@ -0,0 +1,992 @@ +''' +Created on Mar 25, 2013 + +@author: dmitchell +''' +import datetime +import subprocess +import unittest +import uuid +from importlib import import_module + +from xblock.core import Scope +from xmodule.course_module import CourseDescriptor +from xmodule.modulestore.exceptions import InsufficientSpecificationError, ItemNotFoundError, VersionConflictError +from xmodule.modulestore.locator import CourseLocator, BlockUsageLocator, VersionTree, DescriptionLocator +from pytz import UTC +from path import path +import re + + +class SplitModuleTest(unittest.TestCase): + ''' + The base set of tests manually populates a db w/ courses which have + versions. It creates unique collection names and removes them after all + tests finish. + ''' + # Snippet of what would be in the django settings envs file + modulestore_options = { + 'default_class': 'xmodule.raw_module.RawDescriptor', + 'host': 'localhost', + 'db': 'test_xmodule', + 'collection': 'modulestore{0}'.format(uuid.uuid4().hex), + 'fs_root': '', + } + + MODULESTORE = { + 'ENGINE': 'xmodule.modulestore.split_mongo.SplitMongoModuleStore', + 'OPTIONS': modulestore_options + } + + # don't create django dependency; so, duplicates common.py in envs + match = re.search(r'(.*?/common)(?:$|/)', path(__file__)) + COMMON_ROOT = match.group(1) + + modulestore = None + + # These version_guids correspond to values hard-coded in fixture files + # used for these tests. The files live in mitx/fixtures/splitmongo_json/* + + GUID_D0 = "1d00000000000000dddd0000" # v12345d + GUID_D1 = "1d00000000000000dddd1111" # v12345d1 + GUID_D2 = "1d00000000000000dddd2222" # v23456d + GUID_D3 = "1d00000000000000dddd3333" # v12345d0 + GUID_D4 = "1d00000000000000dddd4444" # v23456d0 + GUID_D5 = "1d00000000000000dddd5555" # v345679d + GUID_P = "1d00000000000000eeee0000" # v23456p + + @staticmethod + def bootstrapDB(): + ''' + Loads the initial data into the db ensuring the collection name is + unique. + ''' + collection_prefix = SplitModuleTest.MODULESTORE['OPTIONS']['collection'] + '.' + dbname = SplitModuleTest.MODULESTORE['OPTIONS']['db'] + processes = [ + subprocess.Popen([ + 'mongoimport', '-d', dbname, '-c', + collection_prefix + collection, '--jsonArray', + '--file', + SplitModuleTest.COMMON_ROOT + '/test/data/splitmongo_json/' + collection + '.json' + ]) + for collection in ('active_versions', 'structures', 'definitions')] + for p in processes: + if p.wait() != 0: + raise Exception("DB did not init correctly") + + @classmethod + def tearDownClass(cls): + collection_prefix = SplitModuleTest.MODULESTORE['OPTIONS']['collection'] + '.' + if SplitModuleTest.modulestore: + for collection in ('active_versions', 'structures', 'definitions'): + modulestore().db.drop_collection(collection_prefix + collection) + # drop the modulestore to force re init + SplitModuleTest.modulestore = None + + def findByIdInResult(self, collection, _id): + """ + Result is a collection of descriptors. Find the one whose block id + matches the _id. + """ + for element in collection: + if element.location.usage_id == _id: + return element + + +class SplitModuleCourseTests(SplitModuleTest): + ''' + Course CRUD operation tests + ''' + + def test_get_courses(self): + courses = modulestore().get_courses('draft') + # should have gotten 3 draft courses + self.assertEqual(len(courses), 3, "Wrong number of courses") + # check metadata -- NOTE no promised order + course = self.findByIdInResult(courses, "head12345") + self.assertEqual(course.location.course_id, "GreekHero") + self.assertEqual( + str(course.location.version_guid), self.GUID_D0, + "course version mismatch" + ) + self.assertEqual(course.category, 'course', 'wrong category') + self.assertEqual(len(course.tabs), 6, "wrong number of tabs") + self.assertEqual( + course.display_name, "The Ancient Greek Hero", + "wrong display name" + ) + self.assertEqual( + course.advertised_start, "Fall 2013", + "advertised_start" + ) + self.assertEqual( + len(course.children), 3, + "children") + self.assertEqual(course.definition_locator.definition_id, "head12345_12") + # check dates and graders--forces loading of descriptor + self.assertEqual(course.edited_by, "testassist@edx.org") + self.assertEqual(str(course.previous_version), self.GUID_D1) + self.assertDictEqual(course.grade_cutoffs, {"Pass": 0.45}) + + def test_revision_requests(self): + # query w/ revision qualifier (both draft and published) + courses_published = modulestore().get_courses('published') + self.assertEqual(len(courses_published), 1, len(courses_published)) + course = self.findByIdInResult(courses_published, "head23456") + self.assertIsNotNone(course, "published courses") + self.assertEqual(course.location.course_id, "wonderful") + self.assertEqual(str(course.location.version_guid), self.GUID_P, + course.location.version_guid) + self.assertEqual(course.category, 'course', 'wrong category') + self.assertEqual(len(course.tabs), 4, "wrong number of tabs") + self.assertEqual(course.display_name, "The most wonderful course", + course.display_name) + self.assertIsNone(course.advertised_start) + self.assertEqual(len(course.children), 0, + "children") + + def test_search_qualifiers(self): + # query w/ search criteria + courses = modulestore().get_courses('draft', qualifiers={'org': 'testx'}) + self.assertEqual(len(courses), 2) + self.assertIsNotNone(self.findByIdInResult(courses, "head12345")) + self.assertIsNotNone(self.findByIdInResult(courses, "head23456")) + + courses = modulestore().get_courses( + 'draft', + qualifiers={'edited_on': {"$lt": datetime.datetime(2013, 3, 28, 15)}}) + self.assertEqual(len(courses), 2) + + courses = modulestore().get_courses( + 'draft', + qualifiers={'org': 'testx', "prettyid": "test_course"}) + self.assertEqual(len(courses), 1) + self.assertIsNotNone(self.findByIdInResult(courses, "head12345")) + + def test_get_course(self): + ''' + Test the various calling forms for get_course + ''' + locator = CourseLocator(version_guid=self.GUID_D1) + course = modulestore().get_course(locator) + self.assertIsNone(course.location.course_id) + self.assertEqual(str(course.location.version_guid), self.GUID_D1) + self.assertEqual(course.category, 'course') + self.assertEqual(len(course.tabs), 6) + self.assertEqual(course.display_name, "The Ancient Greek Hero") + self.assertIsNone(course.advertised_start) + self.assertEqual(len(course.children), 0) + self.assertEqual(course.definition_locator.definition_id, "head12345_11") + # check dates and graders--forces loading of descriptor + self.assertEqual(course.edited_by, "testassist@edx.org") + self.assertDictEqual(course.grade_cutoffs, {"Pass": 0.55}) + + locator = CourseLocator(course_id='GreekHero', revision='draft') + course = modulestore().get_course(locator) + self.assertEqual(course.location.course_id, "GreekHero") + self.assertEqual(str(course.location.version_guid), self.GUID_D0) + self.assertEqual(course.category, 'course') + self.assertEqual(len(course.tabs), 6) + self.assertEqual(course.display_name, "The Ancient Greek Hero") + self.assertEqual(course.advertised_start, "Fall 2013") + self.assertEqual(len(course.children), 3) + # check dates and graders--forces loading of descriptor + self.assertEqual(course.edited_by, "testassist@edx.org") + self.assertDictEqual(course.grade_cutoffs, {"Pass": 0.45}) + + locator = CourseLocator(course_id='wonderful', revision='published') + course = modulestore().get_course(locator) + self.assertEqual(course.location.course_id, "wonderful") + self.assertEqual(str(course.location.version_guid), self.GUID_P) + + locator = CourseLocator(course_id='wonderful', revision='draft') + course = modulestore().get_course(locator) + self.assertEqual(str(course.location.version_guid), self.GUID_D2) + + def test_get_course_negative(self): + # Now negative testing + self.assertRaises(InsufficientSpecificationError, + modulestore().get_course, CourseLocator(course_id='edu.meh.blah')) + self.assertRaises(ItemNotFoundError, + modulestore().get_course, CourseLocator(course_id='nosuchthing', revision='draft')) + self.assertRaises(ItemNotFoundError, + modulestore().get_course, + CourseLocator(course_id='GreekHero', revision='published')) + + def test_course_successors(self): + """ + get_course_successors(course_locator, version_history_depth=1) + """ + locator = CourseLocator(version_guid=self.GUID_D3) + result = modulestore().get_course_successors(locator) + self.assertIsInstance(result, VersionTree) + self.assertIsNone(result.locator.course_id) + self.assertEqual(str(result.locator.version_guid), self.GUID_D3) + self.assertEqual(len(result.children), 1) + self.assertEqual(str(result.children[0].locator.version_guid), self.GUID_D1) + self.assertEqual(len(result.children[0].children), 0, "descended more than one level") + result = modulestore().get_course_successors(locator, version_history_depth=2) + self.assertEqual(len(result.children), 1) + self.assertEqual(str(result.children[0].locator.version_guid), self.GUID_D1) + self.assertEqual(len(result.children[0].children), 1) + result = modulestore().get_course_successors(locator, version_history_depth=99) + self.assertEqual(len(result.children), 1) + self.assertEqual(str(result.children[0].locator.version_guid), self.GUID_D1) + self.assertEqual(len(result.children[0].children), 1) + + +class SplitModuleItemTests(SplitModuleTest): + ''' + Item read tests including inheritance + ''' + + def test_has_item(self): + ''' + has_item(BlockUsageLocator) + ''' + # positive tests of various forms + locator = BlockUsageLocator(version_guid=self.GUID_D1, usage_id='head12345') + self.assertTrue(modulestore().has_item(locator), + "couldn't find in %s" % self.GUID_D1) + + locator = BlockUsageLocator(course_id='GreekHero', usage_id='head12345', revision='draft') + self.assertTrue( + modulestore().has_item(locator), + "couldn't find in 12345" + ) + self.assertTrue( + modulestore().has_item(BlockUsageLocator( + course_id=locator.course_id, + revision='draft', + usage_id=locator.usage_id + )), + "couldn't find in draft 12345" + ) + self.assertFalse( + modulestore().has_item(BlockUsageLocator( + course_id=locator.course_id, + revision='published', + usage_id=locator.usage_id)), + "found in published 12345" + ) + locator.revision = 'draft' + self.assertTrue( + modulestore().has_item(locator), + "not found in draft 12345" + ) + + # not a course obj + locator = BlockUsageLocator(course_id='GreekHero', usage_id='chapter1', revision='draft') + self.assertTrue( + modulestore().has_item(locator), + "couldn't find chapter1" + ) + + # in published course + locator = BlockUsageLocator(course_id="wonderful", usage_id="head23456", revision='draft') + self.assertTrue(modulestore().has_item(BlockUsageLocator(course_id=locator.course_id, + usage_id=locator.usage_id, + revision='published')), + "couldn't find in 23456") + locator.revision = 'published' + self.assertTrue(modulestore().has_item(locator), "couldn't find in 23456") + + def test_negative_has_item(self): + # negative tests--not found + # no such course or block + locator = BlockUsageLocator(course_id="doesnotexist", usage_id="head23456", revision='draft') + self.assertFalse(modulestore().has_item(locator)) + locator = BlockUsageLocator(course_id="wonderful", usage_id="doesnotexist", revision='draft') + self.assertFalse(modulestore().has_item(locator)) + + # negative tests--insufficient specification + self.assertRaises(InsufficientSpecificationError, BlockUsageLocator) + self.assertRaises(InsufficientSpecificationError, + modulestore().has_item, BlockUsageLocator(version_guid=self.GUID_D1)) + self.assertRaises(InsufficientSpecificationError, + modulestore().has_item, BlockUsageLocator(course_id='GreekHero')) + + def test_get_item(self): + ''' + get_item(blocklocator) + ''' + # positive tests of various forms + locator = BlockUsageLocator(version_guid=self.GUID_D1, usage_id='head12345') + block = modulestore().get_item(locator) + self.assertIsInstance(block, CourseDescriptor) + + locator = BlockUsageLocator(course_id='GreekHero', usage_id='head12345', revision='draft') + block = modulestore().get_item(locator) + self.assertEqual(block.location.course_id, "GreekHero") + # look at this one in detail + self.assertEqual(len(block.tabs), 6, "wrong number of tabs") + self.assertEqual(block.display_name, "The Ancient Greek Hero") + self.assertEqual(block.advertised_start, "Fall 2013") + self.assertEqual(len(block.children), 3) + self.assertEqual(block.definition_locator.definition_id, "head12345_12") + # check dates and graders--forces loading of descriptor + self.assertEqual(block.edited_by, "testassist@edx.org") + self.assertDictEqual( + block.grade_cutoffs, {"Pass": 0.45}, + ) + + # try to look up other revisions + self.assertRaises(ItemNotFoundError, + modulestore().get_item, + BlockUsageLocator(course_id=locator.as_course_locator(), + usage_id=locator.usage_id, + revision='published')) + locator.revision = 'draft' + self.assertIsInstance( + modulestore().get_item(locator), + CourseDescriptor + ) + + def test_get_non_root(self): + # not a course obj + locator = BlockUsageLocator(course_id='GreekHero', usage_id='chapter1', revision='draft') + block = modulestore().get_item(locator) + self.assertEqual(block.location.course_id, "GreekHero") + self.assertEqual(block.category, 'chapter') + self.assertEqual(block.definition_locator.definition_id, "chapter12345_1") + self.assertEqual(block.display_name, "Hercules") + self.assertEqual(block.edited_by, "testassist@edx.org") + + # in published course + locator = BlockUsageLocator(course_id="wonderful", usage_id="head23456", revision='published') + self.assertIsInstance( + modulestore().get_item(locator), + CourseDescriptor + ) + + # negative tests--not found + # no such course or block + locator = BlockUsageLocator(course_id="doesnotexist", usage_id="head23456", revision='draft') + with self.assertRaises(ItemNotFoundError): + modulestore().get_item(locator) + locator = BlockUsageLocator(course_id="wonderful", usage_id="doesnotexist", revision='draft') + with self.assertRaises(ItemNotFoundError): + modulestore().get_item(locator) + + # negative tests--insufficient specification + with self.assertRaises(InsufficientSpecificationError): + modulestore().get_item(BlockUsageLocator(version_guid=self.GUID_D1)) + with self.assertRaises(InsufficientSpecificationError): + modulestore().get_item(BlockUsageLocator(course_id='GreekHero', revision='draft')) + + # pylint: disable=W0212 + def test_matching(self): + ''' + test the block and value matches help functions + ''' + self.assertTrue(modulestore()._value_matches('help', 'help')) + self.assertFalse(modulestore()._value_matches('help', 'Help')) + self.assertTrue(modulestore()._value_matches(['distract', 'help', 'notme'], 'help')) + self.assertFalse(modulestore()._value_matches(['distract', 'Help', 'notme'], 'help')) + self.assertFalse(modulestore()._value_matches({'field': ['distract', 'Help', 'notme']}, {'field': 'help'})) + self.assertFalse(modulestore()._value_matches(['distract', 'Help', 'notme'], {'field': 'help'})) + self.assertTrue(modulestore()._value_matches( + {'field': ['distract', 'help', 'notme'], + 'irrelevant': 2}, + {'field': 'help'})) + self.assertTrue(modulestore()._value_matches('I need some help', {'$regex': 'help'})) + self.assertTrue(modulestore()._value_matches(['I need some help', 'today'], {'$regex': 'help'})) + self.assertFalse(modulestore()._value_matches('I need some help', {'$regex': 'Help'})) + self.assertFalse(modulestore()._value_matches(['I need some help', 'today'], {'$regex': 'Help'})) + + self.assertTrue(modulestore()._block_matches({'a': 1, 'b': 2}, {'a': 1})) + self.assertTrue(modulestore()._block_matches({'a': 1, 'b': 2}, {'c': None})) + self.assertTrue(modulestore()._block_matches({'a': 1, 'b': 2}, {'a': 1, 'c': None})) + self.assertFalse(modulestore()._block_matches({'a': 1, 'b': 2}, {'a': 2})) + self.assertFalse(modulestore()._block_matches({'a': 1, 'b': 2}, {'c': 1})) + self.assertFalse(modulestore()._block_matches({'a': 1, 'b': 2}, {'a': 1, 'c': 1})) + + def test_get_items(self): + ''' + get_items(locator, qualifiers, [revision]) + ''' + locator = CourseLocator(version_guid=self.GUID_D0) + # get all modules + matches = modulestore().get_items(locator, {}) + self.assertEqual(len(matches), 6) + matches = modulestore().get_items(locator, {'category': 'chapter'}) + self.assertEqual(len(matches), 3) + matches = modulestore().get_items(locator, {'category': 'garbage'}) + self.assertEqual(len(matches), 0) + matches = modulestore().get_items( + locator, + { + 'category': 'chapter', + 'metadata': {'display_name': {'$regex': 'Hera'}} + } + ) + self.assertEqual(len(matches), 2) + + matches = modulestore().get_items(locator, {'children': 'chapter2'}) + self.assertEqual(len(matches), 1) + self.assertEqual(matches[0].location.usage_id, 'head12345') + + def test_get_parents(self): + ''' + get_parent_locations(locator, [usage_id], [revision]): [BlockUsageLocator] + ''' + locator = CourseLocator(course_id="GreekHero", revision='draft') + parents = modulestore().get_parent_locations(locator, usage_id='chapter1') + self.assertEqual(len(parents), 1) + self.assertEqual(parents[0].usage_id, 'head12345') + self.assertEqual(parents[0].course_id, "GreekHero") + locator.usage_id = 'chapter2' + parents = modulestore().get_parent_locations(locator) + self.assertEqual(len(parents), 1) + self.assertEqual(parents[0].usage_id, 'head12345') + parents = modulestore().get_parent_locations(locator, usage_id='nosuchblock') + self.assertEqual(len(parents), 0) + + def test_get_children(self): + """ + Test the existing get_children method on xdescriptors + """ + locator = BlockUsageLocator(course_id="GreekHero", usage_id="head12345", revision='draft') + block = modulestore().get_item(locator) + children = block.get_children() + expected_ids = [ + "chapter1", "chapter2", "chapter3" + ] + for child in children: + self.assertEqual(child.category, "chapter") + self.assertIn(child.location.usage_id, expected_ids) + expected_ids.remove(child.location.usage_id) + self.assertEqual(len(expected_ids), 0) + + +class TestItemCrud(SplitModuleTest): + """ + Test create update and delete of items + """ + # TODO do I need to test this case which I believe won't work: + # 1) fetch a course and some of its blocks + # 2) do a series of CRUD operations on those previously fetched elements + # The problem here will be that the version_guid of the items will be the version at time of fetch. + # Each separate save will change the head version; so, the 2nd piecemeal change will flag the version + # conflict. That is, if versions are v0..vn and start as v0 in initial fetch, the first CRUD op will + # say it's changing an object from v0, splitMongo will process it and make the current head v1, the next + # crud op will pass in its v0 element and splitMongo will flag the version conflict. + # What I don't know is how realistic this test is and whether to wrap the modulestore with a higher level + # transactional operation which manages the version change or make the threading cache reason out whether or + # not the changes are independent and additive and thus non-conflicting. + # A use case I expect is + # (client) change this metadata + # (server) done, here's the new info which, btw, updates the course version to v1 + # (client) add these children to this other node (which says it came from v0 or + # will the client have refreshed the version before doing the op?) + # In this case, having a server side transactional model won't help b/c the bug is a long-transaction on the + # on the client where it would be a mistake for the server to assume anything about client consistency. The best + # the server could do would be to see if the parent's children changed at all since v0. + + def test_create_minimal_item(self): + """ + create_item(course_or_parent_locator, category, user, definition_locator=None, new_def_data=None, + metadata=None): new_desciptor + """ + # grab link to course to ensure new versioning works + locator = CourseLocator(course_id="GreekHero", revision='draft') + premod_course = modulestore().get_course(locator) + premod_time = datetime.datetime.now(UTC) - datetime.timedelta(seconds=1) + # add minimal one w/o a parent + category = 'sequential' + new_module = modulestore().create_item( + locator, category, 'user123', + metadata={'display_name': 'new sequential'} + ) + # check that course version changed and course's previous is the other one + self.assertEqual(new_module.location.course_id, "GreekHero") + self.assertNotEqual(new_module.location.version_guid, premod_course.location.version_guid) + self.assertIsNone(locator.version_guid, "Version inadvertently filled in") + current_course = modulestore().get_course(locator) + self.assertEqual(new_module.location.version_guid, current_course.location.version_guid) + + history_info = modulestore().get_course_history_info(current_course.location) + self.assertEqual(history_info['previous_version'], premod_course.location.version_guid) + self.assertEqual(str(history_info['original_version']), self.GUID_D3) + self.assertEqual(history_info['edited_by'], "user123") + self.assertGreaterEqual(history_info['edited_on'], premod_time) + self.assertLessEqual(history_info['edited_on'], datetime.datetime.now(UTC)) + # check block's info: category, definition_locator, and display_name + self.assertEqual(new_module.category, 'sequential') + self.assertIsNotNone(new_module.definition_locator) + self.assertEqual(new_module.display_name, 'new sequential') + # check that block does not exist in previous version + locator = BlockUsageLocator( + version_guid=premod_course.location.version_guid, + usage_id=new_module.location.usage_id + ) + self.assertRaises(ItemNotFoundError, modulestore().get_item, locator) + + def test_create_parented_item(self): + """ + Test create_item w/ specifying the parent of the new item + """ + locator = BlockUsageLocator(course_id="wonderful", usage_id="head23456", revision='draft') + premod_course = modulestore().get_course(locator) + category = 'chapter' + new_module = modulestore().create_item( + locator, category, 'user123', + metadata={'display_name': 'new chapter'}, + definition_locator=DescriptionLocator("chapter12345_2") + ) + # check that course version changed and course's previous is the other one + self.assertNotEqual(new_module.location.version_guid, premod_course.location.version_guid) + parent = modulestore().get_item(locator) + self.assertIn(new_module.location.usage_id, parent.children) + self.assertEqual(new_module.definition_locator.definition_id, "chapter12345_2") + + def test_unique_naming(self): + """ + Check that 2 modules of same type get unique usage_ids. Also check that if creation provides + a definition id and new def data that it branches the definition in the db. + Actually, this tries to test all create_item features not tested above. + """ + locator = BlockUsageLocator(course_id="contender", usage_id="head345679", revision='draft') + category = 'problem' + premod_time = datetime.datetime.now(UTC) - datetime.timedelta(seconds=1) + new_payload = "empty" + new_module = modulestore().create_item( + locator, category, 'anotheruser', + metadata={'display_name': 'problem 1'}, + new_def_data=new_payload + ) + another_payload = "not empty" + another_module = modulestore().create_item( + locator, category, 'anotheruser', + metadata={'display_name': 'problem 2'}, + definition_locator=DescriptionLocator("problem12345_3_1"), + new_def_data=another_payload + ) + # check that course version changed and course's previous is the other one + parent = modulestore().get_item(locator) + self.assertNotEqual(new_module.location.usage_id, another_module.location.usage_id) + self.assertIn(new_module.location.usage_id, parent.children) + self.assertIn(another_module.location.usage_id, parent.children) + self.assertEqual(new_module.data, new_payload) + self.assertEqual(another_module.data, another_payload) + # check definition histories + new_history = modulestore().get_definition_history_info(new_module.definition_locator) + self.assertIsNone(new_history['previous_version']) + self.assertEqual(new_history['original_version'], new_module.definition_locator.definition_id) + self.assertEqual(new_history['edited_by'], "anotheruser") + self.assertLessEqual(new_history['edited_on'], datetime.datetime.now(UTC)) + self.assertGreaterEqual(new_history['edited_on'], premod_time) + another_history = modulestore().get_definition_history_info(another_module.definition_locator) + self.assertEqual(another_history['previous_version'], 'problem12345_3_1') + # TODO check that default fields are set + + def test_update_metadata(self): + """ + test updating an items metadata ensuring the definition doesn't version but the course does if it should + """ + locator = BlockUsageLocator(course_id="GreekHero", usage_id="problem3_2", revision='draft') + problem = modulestore().get_item(locator) + pre_def_id = problem.definition_locator.definition_id + pre_version_guid = problem.location.version_guid + self.assertIsNotNone(pre_def_id) + self.assertIsNotNone(pre_version_guid) + premod_time = datetime.datetime.now(UTC) - datetime.timedelta(seconds=1) + self.assertNotEqual(problem.max_attempts, 4, "Invalidates rest of test") + + problem.max_attempts = 4 + updated_problem = modulestore().update_item(problem, 'changeMaven') + # check that course version changed and course's previous is the other one + self.assertEqual(updated_problem.definition_locator.definition_id, pre_def_id) + self.assertNotEqual(updated_problem.location.version_guid, pre_version_guid) + self.assertEqual(updated_problem.max_attempts, 4) + # refetch to ensure original didn't change + original_location = BlockUsageLocator( + version_guid=pre_version_guid, + usage_id=problem.location.usage_id + ) + problem = modulestore().get_item(original_location) + self.assertNotEqual(problem.max_attempts, 4, "original changed") + + current_course = modulestore().get_course(locator) + self.assertEqual(updated_problem.location.version_guid, current_course.location.version_guid) + + history_info = modulestore().get_course_history_info(current_course.location) + self.assertEqual(history_info['previous_version'], pre_version_guid) + self.assertEqual(str(history_info['original_version']), self.GUID_D3) + self.assertEqual(history_info['edited_by'], "changeMaven") + self.assertGreaterEqual(history_info['edited_on'], premod_time) + self.assertLessEqual(history_info['edited_on'], datetime.datetime.now(UTC)) + + def test_update_children(self): + """ + test updating an item's children ensuring the definition doesn't version but the course does if it should + """ + locator = BlockUsageLocator(course_id="GreekHero", usage_id="chapter3", revision='draft') + block = modulestore().get_item(locator) + pre_def_id = block.definition_locator.definition_id + pre_version_guid = block.location.version_guid + + # reorder children + self.assertGreater(len(block.children), 0, "meaningless test") + moved_child = block.children.pop() + updated_problem = modulestore().update_item(block, 'childchanger') + # check that course version changed and course's previous is the other one + self.assertEqual(updated_problem.definition_locator.definition_id, pre_def_id) + self.assertNotEqual(updated_problem.location.version_guid, pre_version_guid) + self.assertEqual(updated_problem.children, block.children) + self.assertNotIn(moved_child, updated_problem.children) + locator.usage_id = "chapter1" + other_block = modulestore().get_item(locator) + other_block.children.append(moved_child) + other_updated = modulestore().update_item(other_block, 'childchanger') + self.assertIn(moved_child, other_updated.children) + + def test_update_definition(self): + """ + test updating an item's definition: ensure it gets versioned as well as the course getting versioned + """ + locator = BlockUsageLocator(course_id="GreekHero", usage_id="head12345", revision='draft') + block = modulestore().get_item(locator) + pre_def_id = block.definition_locator.definition_id + pre_version_guid = block.location.version_guid + + block.grading_policy['GRADER'][0]['min_count'] = 13 + updated_block = modulestore().update_item(block, 'definition_changer') + + self.assertNotEqual(updated_block.definition_locator.definition_id, pre_def_id) + self.assertNotEqual(updated_block.location.version_guid, pre_version_guid) + self.assertEqual(updated_block.grading_policy['GRADER'][0]['min_count'], 13) + + def test_update_manifold(self): + """ + Test updating metadata, children, and definition in a single call ensuring all the versioning occurs + """ + # first add 2 children to the course for the update to manipulate + locator = BlockUsageLocator(course_id="contender", usage_id="head345679", revision='draft') + category = 'problem' + new_payload = "empty" + modulestore().create_item( + locator, category, 'test_update_manifold', + metadata={'display_name': 'problem 1'}, + new_def_data=new_payload + ) + another_payload = "not empty" + modulestore().create_item( + locator, category, 'test_update_manifold', + metadata={'display_name': 'problem 2'}, + definition_locator=DescriptionLocator("problem12345_3_1"), + new_def_data=another_payload + ) + # pylint: disable=W0212 + modulestore()._clear_cache() + + # now begin the test + block = modulestore().get_item(locator) + pre_def_id = block.definition_locator.definition_id + pre_version_guid = block.location.version_guid + + self.assertNotEqual(block.grading_policy['GRADER'][0]['min_count'], 13) + block.grading_policy['GRADER'][0]['min_count'] = 13 + block.children = block.children[1:] + [block.children[0]] + block.advertised_start = "Soon" + + updated_block = modulestore().update_item(block, "test_update_manifold") + self.assertNotEqual(updated_block.definition_locator.definition_id, pre_def_id) + self.assertNotEqual(updated_block.location.version_guid, pre_version_guid) + self.assertEqual(updated_block.grading_policy['GRADER'][0]['min_count'], 13) + self.assertEqual(updated_block.children[0], block.children[0]) + self.assertEqual(updated_block.advertised_start, "Soon") + + def test_delete_item(self): + course = self.create_course_for_deletion() + self.assertRaises(ValueError, + modulestore().delete_item, + course.location, + 'deleting_user') + reusable_location = BlockUsageLocator( + course_id=course.location.course_id, + usage_id=course.location.usage_id, + revision='draft') + + # delete a leaf + problems = modulestore().get_items(reusable_location, {'category': 'problem'}) + locn_to_del = problems[0].location + new_course_loc = modulestore().delete_item(locn_to_del, 'deleting_user') + deleted = BlockUsageLocator(course_id=reusable_location.course_id, + revision=reusable_location.revision, + usage_id=locn_to_del.usage_id) + self.assertFalse(modulestore().has_item(deleted)) + self.assertRaises(VersionConflictError, modulestore().has_item, locn_to_del) + locator = BlockUsageLocator( + version_guid=locn_to_del.version_guid, + usage_id=locn_to_del.usage_id + ) + self.assertTrue(modulestore().has_item(locator)) + self.assertNotEqual(new_course_loc.version_guid, course.location.version_guid) + + # delete a subtree + nodes = modulestore().get_items(reusable_location, {'category': 'chapter'}) + new_course_loc = modulestore().delete_item(nodes[0].location, 'deleting_user') + # check subtree + + def check_subtree(node): + if node: + node_loc = node.location + self.assertFalse(modulestore().has_item( + BlockUsageLocator( + course_id=node_loc.course_id, + revision=node_loc.revision, + usage_id=node.location.usage_id))) + locator = BlockUsageLocator( + version_guid=node.location.version_guid, + usage_id=node.location.usage_id) + self.assertTrue(modulestore().has_item(locator)) + if node.has_children: + for sub in node.get_children(): + check_subtree(sub) + check_subtree(nodes[0]) + + def create_course_for_deletion(self): + course = modulestore().create_course('nihilx', 'deletion', 'deleting_user') + root = BlockUsageLocator( + course_id=course.location.course_id, + usage_id=course.location.usage_id, + revision='draft') + for _ in range(4): + self.create_subtree_for_deletion(root, ['chapter', 'vertical', 'problem']) + return modulestore().get_item(root) + + def create_subtree_for_deletion(self, parent, category_queue): + if not category_queue: + return + node = modulestore().create_item(parent, category_queue[0], 'deleting_user') + node_loc = BlockUsageLocator(parent.as_course_locator(), usage_id=node.location.usage_id) + for _ in range(4): + self.create_subtree_for_deletion(node_loc, category_queue[1:]) + + +class TestCourseCreation(SplitModuleTest): + """ + Test create_course, duh :-) + """ + def test_simple_creation(self): + """ + The simplest case but probing all expected results from it. + """ + # Oddly getting differences of 200nsec + pre_time = datetime.datetime.now(UTC) - datetime.timedelta(milliseconds=1) + new_course = modulestore().create_course('test_org', 'test_course', 'create_user') + new_locator = new_course.location + # check index entry + index_info = modulestore().get_course_index_info(new_locator) + self.assertEqual(index_info['org'], 'test_org') + self.assertEqual(index_info['prettyid'], 'test_course') + self.assertGreaterEqual(index_info["edited_on"], pre_time) + self.assertLessEqual(index_info["edited_on"], datetime.datetime.now(UTC)) + self.assertEqual(index_info['edited_by'], 'create_user') + # check structure info + structure_info = modulestore().get_course_history_info(new_locator) + self.assertEqual(structure_info['original_version'], index_info['versions']['draft']) + self.assertIsNone(structure_info['previous_version']) + self.assertGreaterEqual(structure_info["edited_on"], pre_time) + self.assertLessEqual(structure_info["edited_on"], datetime.datetime.now(UTC)) + self.assertEqual(structure_info['edited_by'], 'create_user') + # check the returned course object + self.assertIsInstance(new_course, CourseDescriptor) + self.assertEqual(new_course.category, 'course') + self.assertFalse(new_course.show_calculator) + self.assertTrue(new_course.allow_anonymous) + self.assertEqual(len(new_course.children), 0) + self.assertEqual(new_course.edited_by, "create_user") + self.assertEqual(len(new_course.grading_policy['GRADER']), 4) + self.assertDictEqual(new_course.grade_cutoffs, {"Pass": 0.5}) + + def test_cloned_course(self): + """ + Test making a course which points to an existing draft and published but not making any changes to either. + """ + pre_time = datetime.datetime.now(UTC) + original_locator = CourseLocator(course_id="wonderful", revision='draft') + original_index = modulestore().get_course_index_info(original_locator) + new_draft = modulestore().create_course( + 'leech', 'best_course', 'leech_master', id_root='best', + versions_dict=original_index['versions']) + new_draft_locator = new_draft.location + self.assertRegexpMatches(new_draft_locator.course_id, r'best.*') + # the edited_by and other meta fields on the new course will be the original author not this one + self.assertEqual(new_draft.edited_by, 'test@edx.org') + self.assertLess(new_draft.edited_on, pre_time) + self.assertEqual(new_draft.location.version_guid, original_index['versions']['draft']) + # however the edited_by and other meta fields on course_index will be this one + new_index = modulestore().get_course_index_info(new_draft_locator) + self.assertGreaterEqual(new_index["edited_on"], pre_time) + self.assertLessEqual(new_index["edited_on"], datetime.datetime.now(UTC)) + self.assertEqual(new_index['edited_by'], 'leech_master') + + new_published_locator = CourseLocator(course_id=new_draft_locator.course_id, revision='published') + new_published = modulestore().get_course(new_published_locator) + self.assertEqual(new_published.edited_by, 'test@edx.org') + self.assertLess(new_published.edited_on, pre_time) + self.assertEqual(new_published.location.version_guid, original_index['versions']['published']) + + # changing this course will not change the original course + # using new_draft.location will insert the chapter under the course root + new_item = modulestore().create_item( + new_draft.location, 'chapter', 'leech_master', + metadata={'display_name': 'new chapter'} + ) + new_draft_locator.version_guid = None + new_index = modulestore().get_course_index_info(new_draft_locator) + self.assertNotEqual(new_index['versions']['draft'], original_index['versions']['draft']) + new_draft = modulestore().get_course(new_draft_locator) + self.assertEqual(new_item.edited_by, 'leech_master') + self.assertGreaterEqual(new_item.edited_on, pre_time) + self.assertNotEqual(new_item.location.version_guid, original_index['versions']['draft']) + self.assertNotEqual(new_draft.location.version_guid, original_index['versions']['draft']) + structure_info = modulestore().get_course_history_info(new_draft_locator) + self.assertGreaterEqual(structure_info["edited_on"], pre_time) + self.assertLessEqual(structure_info["edited_on"], datetime.datetime.now(UTC)) + self.assertEqual(structure_info['edited_by'], 'leech_master') + + original_course = modulestore().get_course(original_locator) + self.assertEqual(original_course.location.version_guid, original_index['versions']['draft']) + self.assertFalse( + modulestore().has_item(BlockUsageLocator( + original_locator, + usage_id=new_item.location.usage_id + )) + ) + + def test_derived_course(self): + """ + Create a new course which overrides metadata and course_data + """ + pre_time = datetime.datetime.now(UTC) + original_locator = CourseLocator(course_id="contender", revision='draft') + original = modulestore().get_course(original_locator) + original_index = modulestore().get_course_index_info(original_locator) + data_payload = {} + metadata_payload = {} + for field in original.fields: + if field.scope == Scope.content and field.name != 'location': + data_payload[field.name] = getattr(original, field.name) + elif field.scope == Scope.settings: + metadata_payload[field.name] = getattr(original, field.name) + data_payload['grading_policy']['GRADE_CUTOFFS'] = {'A': .9, 'B': .8, 'C': .65} + metadata_payload['display_name'] = 'Derivative' + new_draft = modulestore().create_course( + 'leech', 'derivative', 'leech_master', id_root='counter', + versions_dict={'draft': original_index['versions']['draft']}, + course_data=data_payload, + metadata=metadata_payload + ) + new_draft_locator = new_draft.location + self.assertRegexpMatches(new_draft_locator.course_id, r'counter.*') + # the edited_by and other meta fields on the new course will be the original author not this one + self.assertEqual(new_draft.edited_by, 'leech_master') + self.assertGreaterEqual(new_draft.edited_on, pre_time) + self.assertNotEqual(new_draft.location.version_guid, original_index['versions']['draft']) + # however the edited_by and other meta fields on course_index will be this one + new_index = modulestore().get_course_index_info(new_draft_locator) + self.assertGreaterEqual(new_index["edited_on"], pre_time) + self.assertLessEqual(new_index["edited_on"], datetime.datetime.now(UTC)) + self.assertEqual(new_index['edited_by'], 'leech_master') + self.assertEqual(new_draft.display_name, metadata_payload['display_name']) + self.assertDictEqual( + new_draft.grading_policy['GRADE_CUTOFFS'], + data_payload['grading_policy']['GRADE_CUTOFFS'] + ) + + def test_update_course_index(self): + """ + Test changing the org, pretty id, etc of a course. Test that it doesn't allow changing the id, etc. + """ + locator = CourseLocator(course_id="GreekHero", revision='draft') + modulestore().update_course_index(locator, {'org': 'funkyU'}) + course_info = modulestore().get_course_index_info(locator) + self.assertEqual(course_info['org'], 'funkyU') + + modulestore().update_course_index(locator, {'org': 'moreFunky', 'prettyid': 'Ancient Greek Demagods'}) + course_info = modulestore().get_course_index_info(locator) + self.assertEqual(course_info['org'], 'moreFunky') + self.assertEqual(course_info['prettyid'], 'Ancient Greek Demagods') + + self.assertRaises(ValueError, modulestore().update_course_index, locator, {'_id': 'funkygreeks'}) + + with self.assertRaises(ValueError): + modulestore().update_course_index( + locator, + {'edited_on': datetime.datetime.now(UTC)} + ) + with self.assertRaises(ValueError): + modulestore().update_course_index( + locator, + {'edited_by': 'sneak'} + ) + + self.assertRaises(ValueError, modulestore().update_course_index, locator, + {'versions': {'draft': self.GUID_D1}}) + + # an allowed but not necessarily recommended way to revert the draft version + versions = course_info['versions'] + versions['draft'] = self.GUID_D1 + modulestore().update_course_index(locator, {'versions': versions}, update_versions=True) + course = modulestore().get_course(locator) + self.assertEqual(str(course.location.version_guid), self.GUID_D1) + + # an allowed but not recommended way to publish a course + versions['published'] = self.GUID_D1 + modulestore().update_course_index(locator, {'versions': versions}, update_versions=True) + course = modulestore().get_course(CourseLocator(course_id=locator.course_id, revision="published")) + self.assertEqual(str(course.location.version_guid), self.GUID_D1) + + +class TestInheritance(SplitModuleTest): + """ + Test the metadata inheritance mechanism. + """ + def test_inheritance(self): + """ + The actual test + """ + # Note, not testing value where defined (course) b/c there's no + # defined accessor for it on CourseDescriptor. + locator = BlockUsageLocator(course_id="GreekHero", usage_id="problem3_2", revision='draft') + node = modulestore().get_item(locator) + # inherited + self.assertEqual(node.graceperiod, datetime.timedelta(hours=2)) + locator = BlockUsageLocator(course_id="GreekHero", usage_id="problem1", revision='draft') + node = modulestore().get_item(locator) + # overridden + self.assertEqual(node.graceperiod, datetime.timedelta(hours=4)) + + # TODO test inheritance after set and delete of attrs + + +#=========================================== +# This mocks the django.modulestore() function and is intended purely to disentangle +# the tests from django +def modulestore(): + def load_function(path): + module_path, _, name = path.rpartition('.') + return getattr(import_module(module_path), name) + + if SplitModuleTest.modulestore is None: + SplitModuleTest.bootstrapDB() + class_ = load_function(SplitModuleTest.MODULESTORE['ENGINE']) + + options = {} + + options.update(SplitModuleTest.MODULESTORE['OPTIONS']) + options['render_template'] = render_to_template_mock + + # pylint: disable=W0142 + SplitModuleTest.modulestore = class_(**options) + + return SplitModuleTest.modulestore + + +# pylint: disable=W0613 +def render_to_template_mock(*args): + pass diff --git a/common/lib/xmodule/xmodule/modulestore/xml.py b/common/lib/xmodule/xmodule/modulestore/xml.py index 26c8b9bfca..8bc3142c77 100644 --- a/common/lib/xmodule/xmodule/modulestore/xml.py +++ b/common/lib/xmodule/xmodule/modulestore/xml.py @@ -194,6 +194,10 @@ class ImportSystem(XMLParsingSystem, MakoDescriptorSystem): if hasattr(descriptor, 'children'): for child in descriptor.get_children(): parent_tracker.add_parent(child.location, descriptor.location) + + # After setting up the descriptor, save any changes that we have + # made to attributes on the descriptor to the underlying KeyValueStore. + descriptor.save() return descriptor render_template = lambda: '' @@ -463,7 +467,10 @@ class XMLModuleStore(ModuleStoreBase): # tabs are referenced in policy.json through a 'slug' which is just the filename without the .html suffix slug = os.path.splitext(os.path.basename(filepath))[0] loc = Location('i4x', course_descriptor.location.org, course_descriptor.location.course, category, slug) - module = HtmlDescriptor(system, {'data': html, 'location': loc}) + module = HtmlDescriptor( + system, + {'data': html, 'location': loc, 'category': category} + ) # VS[compat]: # Hack because we need to pull in the 'display_name' for static tabs (because we need to edit them) # from the course policy diff --git a/common/lib/xmodule/xmodule/modulestore/xml_importer.py b/common/lib/xmodule/xmodule/modulestore/xml_importer.py index 71c6983644..8df768f442 100644 --- a/common/lib/xmodule/xmodule/modulestore/xml_importer.py +++ b/common/lib/xmodule/xmodule/modulestore/xml_importer.py @@ -138,7 +138,7 @@ def import_module_from_xml(modulestore, static_content_store, course_data_path, # For example, what I'm seeing is -> # Note the dropped element closing tag. This causes the LMS to fail when rendering modules - that's # no good, so we have to do this kludge - if isinstance(module.data, str) or isinstance(module.data, unicode): # some module 'data' fields are non strings which blows up the link traversal code + if isinstance(module.data, str) or isinstance(module.data, unicode): # some module 'data' fields are non strings which blows up the link traversal code lxml_rewrite_links(module.data, lambda link: verify_content_links(module, course_data_path, static_content_store, link, remap_dict)) for key in remap_dict.keys(): @@ -315,7 +315,7 @@ def import_module(module, store, course_data_path, static_content_store, allow_n # For example, what I'm seeing is -> # Note the dropped element closing tag. This causes the LMS to fail when rendering modules - that's # no good, so we have to do this kludge - if isinstance(module_data, str) or isinstance(module_data, unicode): # some module 'data' fields are non strings which blows up the link traversal code + if isinstance(module_data, str) or isinstance(module_data, unicode): # some module 'data' fields are non strings which blows up the link traversal code lxml_rewrite_links(module_data, lambda link: verify_content_links(module, course_data_path, static_content_store, link, remap_dict)) for key in remap_dict.keys(): @@ -523,6 +523,26 @@ def validate_data_source_paths(data_dir, course_dir): return err_cnt, warn_cnt +def validate_course_policy(module_store, course_id): + """ + Validate that the course explicitly sets values for any fields whose defaults may have changed between + the export and the import. + + Does not add to error count as these are just warnings. + """ + # is there a reliable way to get the module location just given the course_id? + warn_cnt = 0 + for module in module_store.modules[course_id].itervalues(): + if module.location.category == 'course': + if not 'rerandomize' in module._model_data: + warn_cnt += 1 + print 'WARN: course policy does not specify value for "rerandomize" whose default is now "never". The behavior of your course may change.' + if not 'showanswer' in module._model_data: + warn_cnt += 1 + print 'WARN: course policy does not specify value for "showanswer" whose default is now "finished". The behavior of your course may change.' + return warn_cnt + + def perform_xlint(data_dir, course_dirs, default_class='xmodule.raw_module.RawDescriptor', load_error_modules=True): @@ -568,6 +588,8 @@ def perform_xlint(data_dir, course_dirs, err_cnt += validate_category_hierarchy(module_store, course_id, "chapter", "sequential") # constrain that sequentials only have 'verticals' err_cnt += validate_category_hierarchy(module_store, course_id, "sequential", "vertical") + # validate the course policy overrides any defaults which have changed over time + warn_cnt += validate_course_policy(module_store, course_id) # don't allow metadata on verticals, since we can't edit them in studio err_cnt += validate_no_non_editable_metadata(module_store, course_id, "vertical") # don't allow metadata on chapters, since we can't edit them in studio diff --git a/common/lib/xmodule/xmodule/open_ended_grading_classes/combined_open_ended_modulev1.py b/common/lib/xmodule/xmodule/open_ended_grading_classes/combined_open_ended_modulev1.py index 1fe62035e6..933eb0b5bb 100644 --- a/common/lib/xmodule/xmodule/open_ended_grading_classes/combined_open_ended_modulev1.py +++ b/common/lib/xmodule/xmodule/open_ended_grading_classes/combined_open_ended_modulev1.py @@ -78,37 +78,7 @@ class CombinedOpenEndedV1Module(): instance_state=None, shared_state=None, metadata=None, static_data=None, **kwargs): """ - Definition file should have one or many task blocks, a rubric block, and a prompt block: - - Sample file: - - - Blah blah rubric. - - - Some prompt. - - - - - What hint about this problem would you give to someone? - - - Save Succcesful. Thanks for participating! - - - - - - - Enter essay here. - This is the answer. - {"grader_settings" : "ml_grading.conf", - "problem_id" : "6.002x/Welcome/OETest"} - - - - + Definition file should have one or many task blocks, a rubric block, and a prompt block. See DEFAULT_DATA in combined_open_ended_module for a sample. """ @@ -131,14 +101,14 @@ class CombinedOpenEndedV1Module(): # Allow reset is true if student has failed the criteria to move to the next child task self.ready_to_reset = instance_state.get('ready_to_reset', False) - self.attempts = self.instance_state.get('attempts', MAX_ATTEMPTS) - self.is_scored = self.instance_state.get('is_graded', IS_SCORED) in TRUE_DICT - self.accept_file_upload = self.instance_state.get('accept_file_upload', ACCEPT_FILE_UPLOAD) in TRUE_DICT - self.skip_basic_checks = self.instance_state.get('skip_spelling_checks', SKIP_BASIC_CHECKS) in TRUE_DICT + self.max_attempts = instance_state.get('max_attempts', MAX_ATTEMPTS) + self.is_scored = instance_state.get('graded', IS_SCORED) in TRUE_DICT + self.accept_file_upload = instance_state.get('accept_file_upload', ACCEPT_FILE_UPLOAD) in TRUE_DICT + self.skip_basic_checks = instance_state.get('skip_spelling_checks', SKIP_BASIC_CHECKS) in TRUE_DICT - due_date = self.instance_state.get('due', None) + due_date = instance_state.get('due', None) - grace_period_string = self.instance_state.get('graceperiod', None) + grace_period_string = instance_state.get('graceperiod', None) try: self.timeinfo = TimeInfo(due_date, grace_period_string) except Exception: @@ -153,7 +123,7 @@ class CombinedOpenEndedV1Module(): # Static data is passed to the child modules to render self.static_data = { 'max_score': self._max_score, - 'max_attempts': self.attempts, + 'max_attempts': self.max_attempts, 'prompt': definition['prompt'], 'rubric': definition['rubric'], 'display_name': self.display_name, @@ -643,15 +613,18 @@ class CombinedOpenEndedV1Module(): if not self.ready_to_reset: return self.out_of_sync_error(data) - if self.student_attempts > self.attempts: + if self.student_attempts >= self.max_attempts-1: + if self.student_attempts==self.max_attempts-1: + self.student_attempts +=1 return { 'success': False, # This is a student_facing_error 'error': ( 'You have attempted this question {0} times. ' 'You are only allowed to attempt it {1} times.' - ).format(self.student_attempts, self.attempts) + ).format(self.student_attempts, self.max_attempts) } + self.student_attempts +=1 self.state = self.INITIAL self.ready_to_reset = False for i in xrange(0, len(self.task_xml)): @@ -726,7 +699,12 @@ class CombinedOpenEndedV1Module(): """ max_score = None score = None - if self.is_scored and self.weight is not None: + + #The old default was None, so set to 1 if it is the old default weight + weight = self.weight + if weight is None: + weight = 1 + if self.is_scored: # Finds the maximum score of all student attempts and keeps it. score_mat = [] for i in xrange(0, len(self.task_states)): @@ -739,7 +717,7 @@ class CombinedOpenEndedV1Module(): for z in xrange(0, len(score)): if score[z] is None: score[z] = 0 - score[z] *= float(self.weight) + score[z] *= float(weight) score_mat.append(score) if len(score_mat) > 0: @@ -753,7 +731,7 @@ class CombinedOpenEndedV1Module(): if max_score is not None: # Weight the max score if it is not None - max_score *= float(self.weight) + max_score *= float(weight) else: # Without a max_score, we cannot have a score! score = None @@ -810,7 +788,6 @@ class CombinedOpenEndedV1Descriptor(): filename_extension = "xml" has_score = True - template_dir_name = "combinedopenended" def __init__(self, system): self.system = system diff --git a/common/lib/xmodule/xmodule/open_ended_grading_classes/open_ended_module.py b/common/lib/xmodule/xmodule/open_ended_grading_classes/open_ended_module.py index 0f0851fbf7..2e7a3eaf89 100644 --- a/common/lib/xmodule/xmodule/open_ended_grading_classes/open_ended_module.py +++ b/common/lib/xmodule/xmodule/open_ended_grading_classes/open_ended_module.py @@ -19,6 +19,7 @@ import openendedchild from numpy import median from datetime import datetime +from pytz import UTC from .combined_open_ended_rubric import CombinedOpenEndedRubric @@ -170,7 +171,7 @@ class OpenEndedModule(openendedchild.OpenEndedChild): if xqueue is None: return {'success': False, 'msg': "Couldn't submit feedback."} qinterface = xqueue['interface'] - qtime = datetime.strftime(datetime.now(), xqueue_interface.dateformat) + qtime = datetime.strftime(datetime.now(UTC), xqueue_interface.dateformat) anonymous_student_id = system.anonymous_student_id queuekey = xqueue_interface.make_hashkey(str(system.seed) + qtime + anonymous_student_id + @@ -224,7 +225,7 @@ class OpenEndedModule(openendedchild.OpenEndedChild): if xqueue is None: return False qinterface = xqueue['interface'] - qtime = datetime.strftime(datetime.now(), xqueue_interface.dateformat) + qtime = datetime.strftime(datetime.now(UTC), xqueue_interface.dateformat) anonymous_student_id = system.anonymous_student_id @@ -730,7 +731,6 @@ class OpenEndedDescriptor(): filename_extension = "xml" has_score = True - template_dir_name = "openended" def __init__(self, system): self.system = system diff --git a/common/lib/xmodule/xmodule/open_ended_grading_classes/openendedchild.py b/common/lib/xmodule/xmodule/open_ended_grading_classes/openendedchild.py index 047ab0244c..10f939b270 100644 --- a/common/lib/xmodule/xmodule/open_ended_grading_classes/openendedchild.py +++ b/common/lib/xmodule/xmodule/open_ended_grading_classes/openendedchild.py @@ -5,6 +5,7 @@ import re import open_ended_image_submission from xmodule.progress import Progress +import capa.xqueue_interface as xqueue_interface from capa.util import * from .peer_grading_service import PeerGradingService, MockPeerGradingService import controller_query_service @@ -334,12 +335,15 @@ class OpenEndedChild(object): log.exception("Could not create image and check it.") if image_ok: - image_key = image_data.name + datetime.now().strftime("%Y%m%d%H%M%S") + image_key = image_data.name + datetime.now(UTC).strftime( + xqueue_interface.dateformat + ) try: image_data.seek(0) - success, s3_public_url = open_ended_image_submission.upload_to_s3(image_data, image_key, - self.s3_interface) + success, s3_public_url = open_ended_image_submission.upload_to_s3( + image_data, image_key, self.s3_interface + ) except: log.exception("Could not upload image to S3.") diff --git a/common/lib/xmodule/xmodule/open_ended_grading_classes/self_assessment_module.py b/common/lib/xmodule/xmodule/open_ended_grading_classes/self_assessment_module.py index a5498289e2..1262e1f68f 100644 --- a/common/lib/xmodule/xmodule/open_ended_grading_classes/self_assessment_module.py +++ b/common/lib/xmodule/xmodule/open_ended_grading_classes/self_assessment_module.py @@ -287,7 +287,6 @@ class SelfAssessmentDescriptor(): filename_extension = "xml" has_score = True - template_dir_name = "selfassessment" def __init__(self, system): self.system = system diff --git a/common/lib/xmodule/xmodule/peer_grading_module.py b/common/lib/xmodule/xmodule/peer_grading_module.py index 7df444a892..09cac9a6b4 100644 --- a/common/lib/xmodule/xmodule/peer_grading_module.py +++ b/common/lib/xmodule/xmodule/peer_grading_module.py @@ -9,6 +9,7 @@ from .capa_module import ComplexEncoder from .x_module import XModule from xmodule.raw_module import RawDescriptor from xmodule.modulestore.django import modulestore +from xmodule.modulestore.exceptions import ItemNotFoundError from .timeinfo import TimeInfo from xblock.core import Dict, String, Scope, Boolean, Integer, Float from xmodule.fields import Date @@ -19,36 +20,37 @@ from django.utils.timezone import UTC log = logging.getLogger(__name__) -USE_FOR_SINGLE_LOCATION = False -LINK_TO_LOCATION = "" -MAX_SCORE = 1 -IS_GRADED = False EXTERNAL_GRADER_NO_CONTACT_ERROR = "Failed to contact external graders. Please notify course staff." - class PeerGradingFields(object): use_for_single_location = Boolean( display_name="Show Single Problem", help='When True, only the single problem specified by "Link to Problem Location" is shown. ' 'When False, a panel is displayed with all problems available for peer grading.', - default=USE_FOR_SINGLE_LOCATION, scope=Scope.settings + default=False, + scope=Scope.settings ) link_to_location = String( display_name="Link to Problem Location", help='The location of the problem being graded. Only used when "Show Single Problem" is True.', - default=LINK_TO_LOCATION, scope=Scope.settings + default="", + scope=Scope.settings ) - is_graded = Boolean( + graded = Boolean( display_name="Graded", help='Defines whether the student gets credit for grading this problem. Only used when "Show Single Problem" is True.', - default=IS_GRADED, scope=Scope.settings + default=False, + scope=Scope.settings ) - due_date = Date(help="Due date that should be displayed.", default=None, scope=Scope.settings) - grace_period_string = String(help="Amount of grace to give on the due date.", default=None, scope=Scope.settings) - max_grade = Integer( - help="The maximum grade that a student can receive for this problem.", default=MAX_SCORE, - scope=Scope.settings, values={"min": 0} + due = Date( + help="Due date that should be displayed.", + default=None, + scope=Scope.settings) + grace_period_string = String( + help="Amount of grace to give on the due date.", + default=None, + scope=Scope.settings ) student_data_for_location = Dict( help="Student data for a given peer grading problem.", @@ -57,8 +59,18 @@ class PeerGradingFields(object): weight = Float( display_name="Problem Weight", help="Defines the number of points each problem is worth. If the value is not set, each problem is worth one point.", - scope=Scope.settings, values={"min": 0, "step": ".1"} + scope=Scope.settings, values={"min": 0, "step": ".1"}, + default=1 ) + display_name = String( + display_name="Display Name", + help="Display name for this module", + scope=Scope.settings, + default="Peer Grading Interface" + ) + data = String(help="Html contents to display for this module", + default='', + scope=Scope.content) class PeerGradingModule(PeerGradingFields, XModule): @@ -89,35 +101,31 @@ class PeerGradingModule(PeerGradingFields, XModule): if self.use_for_single_location: try: self.linked_problem = modulestore().get_instance(self.system.course_id, self.link_to_location) - except: + except ItemNotFoundError: log.error("Linked location {0} for peer grading module {1} does not exist".format( self.link_to_location, self.location)) raise - due_date = self.linked_problem._model_data.get('peer_grading_due', None) + due_date = self.linked_problem._model_data.get('due', None) if due_date: self._model_data['due'] = due_date try: - self.timeinfo = TimeInfo(self.due_date, self.grace_period_string) - except: - log.error("Error parsing due date information in location {0}".format(location)) + self.timeinfo = TimeInfo(self.due, self.grace_period_string) + except Exception: + log.error("Error parsing due date information in location {0}".format(self.location)) raise self.display_due_date = self.timeinfo.display_due_date try: self.student_data_for_location = json.loads(self.student_data_for_location) - except: + except Exception: pass self.ajax_url = self.system.ajax_url if not self.ajax_url.endswith("/"): self.ajax_url = self.ajax_url + "/" - # Integer could return None, so keep this check. - if not isinstance(self.max_grade, int): - raise TypeError("max_grade needs to be an integer.") - def closed(self): return self._closed(self.timeinfo) @@ -201,11 +209,16 @@ class PeerGradingModule(PeerGradingFields, XModule): def get_score(self): max_score = None score = None + weight = self.weight + + #The old default was None, so set to 1 if it is the old default weight + if weight is None: + weight = 1 score_dict = { 'score': score, 'total': max_score, } - if not self.use_for_single_location or not self.is_graded: + if not self.use_for_single_location or not self.graded: return score_dict try: @@ -225,11 +238,10 @@ class PeerGradingModule(PeerGradingFields, XModule): # Ensures that once a student receives a final score for peer grading, that it does not change. self.student_data_for_location = response - if self.weight is not None: - score = int(count_graded >= count_required and count_graded > 0) * float(self.weight) - total = self.max_grade * float(self.weight) - score_dict['score'] = score - score_dict['total'] = total + score = int(count_graded >= count_required and count_graded > 0) * float(weight) + total = float(weight) + score_dict['score'] = score + score_dict['total'] = total return score_dict @@ -240,8 +252,8 @@ class PeerGradingModule(PeerGradingFields, XModule): randomization, and 5/7 on another ''' max_grade = None - if self.use_for_single_location and self.is_graded: - max_grade = self.max_grade + if self.use_for_single_location and self.graded: + max_grade = self.weight return max_grade def get_next_submission(self, data): @@ -521,7 +533,7 @@ class PeerGradingModule(PeerGradingFields, XModule): problem_location = problem['location'] descriptor = _find_corresponding_module_for_location(problem_location) if descriptor: - problem['due'] = descriptor._model_data.get('peer_grading_due', None) + problem['due'] = descriptor._model_data.get('due', None) grace_period_string = descriptor._model_data.get('graceperiod', None) try: problem_timeinfo = TimeInfo(problem['due'], grace_period_string) @@ -604,14 +616,18 @@ class PeerGradingDescriptor(PeerGradingFields, RawDescriptor): has_score = True always_recalculate_grades = True - template_dir_name = "peer_grading" #Specify whether or not to pass in open ended interface needs_open_ended_interface = True + metadata_translations = { + 'is_graded': 'graded', + 'attempts': 'max_attempts', + 'due_data' : 'due' + } + @property def non_editable_metadata_fields(self): non_editable_fields = super(PeerGradingDescriptor, self).non_editable_metadata_fields - non_editable_fields.extend([PeerGradingFields.due_date, PeerGradingFields.grace_period_string, - PeerGradingFields.max_grade]) + non_editable_fields.extend([PeerGradingFields.due, PeerGradingFields.grace_period_string]) return non_editable_fields diff --git a/common/lib/xmodule/xmodule/poll_module.py b/common/lib/xmodule/xmodule/poll_module.py index ca12f239ab..8e7407752a 100644 --- a/common/lib/xmodule/xmodule/poll_module.py +++ b/common/lib/xmodule/xmodule/poll_module.py @@ -140,7 +140,6 @@ class PollDescriptor(PollFields, MakoModuleDescriptor, XmlDescriptor): _child_tag_name = 'answer' module_class = PollModule - template_dir_name = 'poll' @classmethod def definition_from_xml(cls, xml_object, system): diff --git a/common/lib/xmodule/xmodule/raw_module.py b/common/lib/xmodule/xmodule/raw_module.py index 554be73926..4c6c719224 100644 --- a/common/lib/xmodule/xmodule/raw_module.py +++ b/common/lib/xmodule/xmodule/raw_module.py @@ -13,7 +13,7 @@ class RawDescriptor(XmlDescriptor, XMLEditingDescriptor): Module that provides a raw editing view of its data and children. It requires that the definition xml is valid. """ - data = String(help="XML data for the module", scope=Scope.content) + data = String(help="XML data for the module", default="", scope=Scope.content) @classmethod def definition_from_xml(cls, xml_object, system): diff --git a/common/lib/xmodule/xmodule/schematic_module.py b/common/lib/xmodule/xmodule/schematic_module.py deleted file mode 100644 index 83bcc5351d..0000000000 --- a/common/lib/xmodule/xmodule/schematic_module.py +++ /dev/null @@ -1,11 +0,0 @@ - -from .x_module import XModule, XModuleDescriptor - - -class ModuleDescriptor(XModuleDescriptor): - pass - - -class Module(XModule): - def get_html(self): - return ''.format(item_id=self.item_id) diff --git a/common/lib/xmodule/xmodule/templates.py b/common/lib/xmodule/xmodule/templates.py index 6479b3df24..8e350bb618 100644 --- a/common/lib/xmodule/xmodule/templates.py +++ b/common/lib/xmodule/xmodule/templates.py @@ -1,34 +1,18 @@ """ -This module handles loading xmodule templates from disk into the modulestore. -These templates are used by the CMS to provide baseline content that -can be cloned when adding new modules to a course. +This module handles loading xmodule templates +These templates are used by the CMS to provide content that overrides xmodule defaults for +samples. -`Template`s are defined in x_module. They contain 3 attributes: - metadata: A dictionary with the template metadata. This should contain - any values for fields - * with scope Scope.settings - * that have values different than the field defaults - * and that are to be editable in Studio - data: A JSON value that defines the template content. This should be a dictionary - containing values for fields - * with scope Scope.content - * that have values different than the field defaults - * and that are to be editable in Studio - or, if the module uses a single Scope.content String field named `data`, this - should be a string containing the contents of that field - children: A list of Location urls that define the template children - -Templates are defined on XModuleDescriptor types, in the template attribute. +``Template``s are defined in x_module. They contain 2 attributes: + :metadata: A dictionary with the template metadata + :data: A JSON value that defines the template content """ - +# should this move to cms since it's really only for module crud? import logging -from fs.memoryfs import MemoryFS from collections import defaultdict from .x_module import XModuleDescriptor -from .mako_module import MakoDescriptorSystem -from .modulestore import Location log = logging.getLogger(__name__) @@ -37,73 +21,9 @@ def all_templates(): """ Returns all templates for enabled modules, grouped by descriptor type """ - + # TODO use memcache to memoize w/ expiration templates = defaultdict(list) for category, descriptor in XModuleDescriptor.load_classes(): templates[category] = descriptor.templates() return templates - - -class TemplateTestSystem(MakoDescriptorSystem): - """ - This system exists to help verify that XModuleDescriptors can be instantiated - from their defined templates before we load the templates into the modulestore. - """ - def __init__(self): - super(TemplateTestSystem, self).__init__( - lambda *a, **k: None, - MemoryFS(), - lambda msg: None, - render_template=lambda *a, **k: None, - ) - - -def update_templates(modulestore): - """ - Updates the set of templates in the modulestore with all templates currently - available from the installed plugins - """ - - # cdodge: build up a list of all existing templates. This will be used to determine which - # templates have been removed from disk - and thus we need to remove from the DB - templates_to_delete = modulestore.get_items(['i4x', 'edx', 'templates', None, None, None]) - - for category, templates in all_templates().items(): - for template in templates: - if 'display_name' not in template.metadata: - log.warning('No display_name specified in template {0}, skipping'.format(template)) - continue - - template_location = Location('i4x', 'edx', 'templates', category, Location.clean_for_url_name(template.metadata['display_name'])) - - try: - json_data = { - 'definition': { - 'data': template.data, - 'children': template.children - }, - 'metadata': template.metadata - } - json_data['location'] = template_location.dict() - - XModuleDescriptor.load_from_json(json_data, TemplateTestSystem()) - except: - log.warning('Unable to instantiate {cat} from template {template}, skipping'.format( - cat=category, - template=template - ), exc_info=True) - continue - - modulestore.update_item(template_location, template.data) - modulestore.update_children(template_location, template.children) - modulestore.update_metadata(template_location, template.metadata) - - # remove template from list of templates to delete - templates_to_delete = [t for t in templates_to_delete if t.location != template_location] - - # now remove all templates which appear to have removed from disk - if len(templates_to_delete) > 0: - logging.debug('deleting dangling templates = {0}'.format(templates_to_delete)) - for template in templates_to_delete: - modulestore.delete_item(template.location) diff --git a/common/lib/xmodule/xmodule/templates/about/empty.yaml b/common/lib/xmodule/xmodule/templates/about/empty.yaml deleted file mode 100644 index fa3ed606bd..0000000000 --- a/common/lib/xmodule/xmodule/templates/about/empty.yaml +++ /dev/null @@ -1,5 +0,0 @@ ---- -metadata: - display_name: Empty -data: "

        This is where you can add additional information about your course.

        " -children: [] \ No newline at end of file diff --git a/common/lib/xmodule/xmodule/templates/about/overview.yaml b/common/lib/xmodule/xmodule/templates/about/overview.yaml index 0031ebffaf..9b2e895526 100644 --- a/common/lib/xmodule/xmodule/templates/about/overview.yaml +++ b/common/lib/xmodule/xmodule/templates/about/overview.yaml @@ -3,51 +3,50 @@ metadata: display_name: overview data: | -
        -

        About This Course

        -

        Include your long course description here. The long course description should contain 150-400 words.

        +
        +

        About This Course

        +

        Include your long course description here. The long course description should contain 150-400 words.

        -

        This is paragraph 2 of the long course description. Add more paragraphs as needed. Make sure to enclose them in paragraph tags.

        -
        +

        This is paragraph 2 of the long course description. Add more paragraphs as needed. Make sure to enclose them in paragraph tags.

        +
        -
        -

        Prerequisites

        -

        Add information about course prerequisites here.

        -
        +
        +

        Prerequisites

        +

        Add information about course prerequisites here.

        +
        -
        -

        Course Staff

        -
        -
        - -
        +
        +

        Course Staff

        +
        +
        + +
        -

        Staff Member #1

        -

        Biography of instructor/staff member #1

        -
        +

        Staff Member #1

        +

        Biography of instructor/staff member #1

        +
        -
        -
        - -
        +
        +
        + +
        -

        Staff Member #2

        -

        Biography of instructor/staff member #2

        -
        -
        +

        Staff Member #2

        +

        Biography of instructor/staff member #2

        + +
        -
        -
        -

        Frequently Asked Questions

        -
        -

        Do I need to buy a textbook?

        -

        No, a free online version of Chemistry: Principles, Patterns, and Applications, First Edition by Bruce Averill and Patricia Eldredge will be available, though you can purchase a printed version (published by FlatWorld Knowledge) if you’d like.

        -
        +
        +
        +

        Frequently Asked Questions

        +
        +

        Do I need to buy a textbook?

        +

        No, a free online version of Chemistry: Principles, Patterns, and Applications, First Edition by Bruce Averill and Patricia Eldredge will be available, though you can purchase a printed version (published by FlatWorld Knowledge) if you’d like.

        +
        -
        -

        Question #2

        -

        Your answer would be displayed here.

        -
        -
        -
        -children: [] +
        +

        Question #2

        +

        Your answer would be displayed here.

        +
        +
        +
        diff --git a/common/lib/xmodule/xmodule/templates/annotatable/default.yaml b/common/lib/xmodule/xmodule/templates/annotatable/default.yaml deleted file mode 100644 index 31dd489fb4..0000000000 --- a/common/lib/xmodule/xmodule/templates/annotatable/default.yaml +++ /dev/null @@ -1,20 +0,0 @@ ---- -metadata: - display_name: 'Annotation' -data: | - - -

        Enter your (optional) instructions for the exercise in HTML format.

        -

        Annotations are specified by an <annotation> tag which may may have the following attributes:

        -
          -
        • title (optional). Title of the annotation. Defaults to Commentary if omitted.
          • -
        • body (required). Text of the annotation.
          • -
        • problem (optional). Numeric index of the problem associated with this annotation. This is a zero-based index, so the first problem on the page would have problem="0".
          • -
        • highlight (optional). Possible values: yellow, red, orange, green, blue, or purple. Defaults to yellow if this attribute is omitted.
          • -
        -         -

        Add your HTML with annotation spans here.

        -

        Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut sodales laoreet est, egestas gravida felis egestas nec. Aenean at volutpat erat. Cras commodo viverra nibh in aliquam.

        -

        Nulla facilisi. Pellentesque id vestibulum libero. Suspendisse potenti. Morbi scelerisque nisi vitae felis dictum mattis. Nam sit amet magna elit. Nullam volutpat cursus est, sit amet sagittis odio vulputate et. Curabitur euismod, orci in vulputate imperdiet, augue lorem tempor purus, id aliquet augue turpis a est. Aenean a sagittis libero. Praesent fringilla pretium magna, non condimentum risus elementum nec. Pellentesque faucibus elementum pharetra. Pellentesque vitae metus eros.

        -         -children: [] diff --git a/common/lib/xmodule/xmodule/templates/combinedopenended/default.yaml b/common/lib/xmodule/xmodule/templates/combinedopenended/default.yaml deleted file mode 100644 index f7d639ebfb..0000000000 --- a/common/lib/xmodule/xmodule/templates/combinedopenended/default.yaml +++ /dev/null @@ -1,37 +0,0 @@ ---- -metadata: - display_name: Open Ended Response - markdown: "" -data: | - - - - - Category 1 - - - - - - -

        Why is the sky blue?

        -         - - - - - - - Enter essay here. - This is the answer. - {"grader_settings" : "peer_grading.conf", "problem_id" : "700x/Demo"} - - - -         - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/course/empty.yaml b/common/lib/xmodule/xmodule/templates/course/empty.yaml deleted file mode 100644 index 89f1bfcf21..0000000000 --- a/common/lib/xmodule/xmodule/templates/course/empty.yaml +++ /dev/null @@ -1,124 +0,0 @@ ---- -metadata: - display_name: Empty - start: 2020-10-10T10:00 - checklists: [ - {"short_description" : "Getting Started With Studio", - "items" : [{"short_description": "Add Course Team Members", - "long_description": "Grant your collaborators permission to edit your course so you can work together.", - "is_checked": false, - "action_url": "ManageUsers", - "action_text": "Edit Course Team", - "action_external": false}, - {"short_description": "Set Important Dates for Your Course", - "long_description": "Establish your course's student enrollment and launch dates on the Schedule and Details page.", - "is_checked": false, - "action_url": "SettingsDetails", - "action_text": "Edit Course Details & Schedule", - "action_external": false}, - {"short_description": "Draft Your Course's Grading Policy", - "long_description": "Set up your assignment types and grading policy even if you haven't created all your assignments.", - "is_checked": false, - "action_url": "SettingsGrading", - "action_text": "Edit Grading Settings", - "action_external": false}, - {"short_description": "Explore the Other Studio Checklists", - "long_description": "Discover other available course authoring tools, and find help when you need it.", - "is_checked": false, - "action_url": "", - "action_text": "", - "action_external": false}] - }, - {"short_description" : "Draft a Rough Course Outline", - "items" : [{"short_description": "Create Your First Section and Subsection", - "long_description": "Use your course outline to build your first Section and Subsection.", - "is_checked": false, - "action_url": "CourseOutline", - "action_text": "Edit Course Outline", - "action_external": false}, - {"short_description": "Set Section Release Dates", - "long_description": "Specify the release dates for each Section in your course. Sections become visible to students on their release dates.", - "is_checked": false, - "action_url": "CourseOutline", - "action_text": "Edit Course Outline", - "action_external": false}, - {"short_description": "Designate a Subsection as Graded", - "long_description": "Set a Subsection to be graded as a specific assignment type. Assignments within graded Subsections count toward a student's final grade.", - "is_checked": false, - "action_url": "CourseOutline", - "action_text": "Edit Course Outline", - "action_external": false}, - {"short_description": "Reordering Course Content", - "long_description": "Use drag and drop to reorder the content in your course.", - "is_checked": false, - "action_url": "CourseOutline", - "action_text": "Edit Course Outline", - "action_external": false}, - {"short_description": "Renaming Sections", - "long_description": "Rename Sections by clicking the Section name from the Course Outline.", - "is_checked": false, - "action_url": "CourseOutline", - "action_text": "Edit Course Outline", - "action_external": false}, - {"short_description": "Deleting Course Content", - "long_description": "Delete Sections, Subsections, or Units you don't need anymore. Be careful, as there is no Undo function.", - "is_checked": false, - "action_url": "CourseOutline", - "action_text": "Edit Course Outline", - "action_external": false}, - {"short_description": "Add an Instructor-Only Section to Your Outline", - "long_description": "Some course authors find using a section for unsorted, in-progress work useful. To do this, create a section and set the release date to the distant future.", - "is_checked": false, - "action_url": "CourseOutline", - "action_text": "Edit Course Outline", - "action_external": false}] - }, - {"short_description" : "Explore edX's Support Tools", - "items" : [{"short_description": "Explore the Studio Help Forum", - "long_description": "Access the Studio Help forum from the menu that appears when you click your user name in the top right corner of Studio.", - "is_checked": false, - "action_url": "http://help.edge.edx.org/", - "action_text": "Visit Studio Help", - "action_external": true}, - {"short_description": "Enroll in edX 101", - "long_description": "Register for edX 101, edX's primer for course creation.", - "is_checked": false, - "action_url": "https://edge.edx.org/courses/edX/edX101/How_to_Create_an_edX_Course/about", - "action_text": "Register for edX 101", - "action_external": true}, - {"short_description": "Download the Studio Documentation", - "long_description": "Download the searchable Studio reference documentation in PDF form.", - "is_checked": false, - "action_url": "http://files.edx.org/Getting_Started_with_Studio.pdf", - "action_text": "Download Documentation", - "action_external": true}] - }, - {"short_description" : "Draft Your Course About Page", - "items" : [{"short_description": "Draft a Course Description", - "long_description": "Courses on edX have an About page that includes a course video, description, and more. Draft the text students will read before deciding to enroll in your course.", - "is_checked": false, - "action_url": "SettingsDetails", - "action_text": "Edit Course Schedule & Details", - "action_external": false}, - {"short_description": "Add Staff Bios", - "long_description": "Showing prospective students who their instructor will be is helpful. Include staff bios on the course About page.", - "is_checked": false, - "action_url": "SettingsDetails", - "action_text": "Edit Course Schedule & Details", - "action_external": false}, - {"short_description": "Add Course FAQs", - "long_description": "Include a short list of frequently asked questions about your course.", - "is_checked": false, - "action_url": "SettingsDetails", - "action_text": "Edit Course Schedule & Details", - "action_external": false}, - {"short_description": "Add Course Prerequisites", - "long_description": "Let students know what knowledge and/or skills they should have before they enroll in your course.", - "is_checked": false, - "action_url": "SettingsDetails", - "action_text": "Edit Course Schedule & Details", - "action_external": false}] - } - ] -data: { 'textbooks' : [ ], 'wiki_slug' : null } -children: [] diff --git a/common/lib/xmodule/xmodule/templates/courseinfo/empty.yaml b/common/lib/xmodule/xmodule/templates/courseinfo/empty.yaml deleted file mode 100644 index c6958ed887..0000000000 --- a/common/lib/xmodule/xmodule/templates/courseinfo/empty.yaml +++ /dev/null @@ -1,5 +0,0 @@ ---- -metadata: - display_name: Empty -data: "
          " -children: [] \ No newline at end of file diff --git a/common/lib/xmodule/xmodule/templates/default/empty.yaml b/common/lib/xmodule/xmodule/templates/default/empty.yaml deleted file mode 100644 index a2fb2b5832..0000000000 --- a/common/lib/xmodule/xmodule/templates/default/empty.yaml +++ /dev/null @@ -1,5 +0,0 @@ ---- -metadata: - display_name: Empty -data: "" -children: [] diff --git a/common/lib/xmodule/xmodule/templates/discussion/default.yaml b/common/lib/xmodule/xmodule/templates/discussion/default.yaml deleted file mode 100644 index 049e34b3e7..0000000000 --- a/common/lib/xmodule/xmodule/templates/discussion/default.yaml +++ /dev/null @@ -1,9 +0,0 @@ ---- -metadata: - display_name: Discussion Tag - for: Topic-Level Student-Visible Label - id: $$GUID$$ - discussion_category: Week 1 -data: | - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/html/announcement.yaml b/common/lib/xmodule/xmodule/templates/html/announcement.yaml index 82fe8fbc03..c0ecc61524 100644 --- a/common/lib/xmodule/xmodule/templates/html/announcement.yaml +++ b/common/lib/xmodule/xmodule/templates/html/announcement.yaml @@ -1,7 +1,6 @@ --- metadata: - display_name: Announcement - + display_name: Announcement data: |
          1. @@ -22,4 +21,3 @@ data: |
        -children: [] diff --git a/common/lib/xmodule/xmodule/templates/html/empty.yaml b/common/lib/xmodule/xmodule/templates/html/empty.yaml deleted file mode 100644 index 40b005af28..0000000000 --- a/common/lib/xmodule/xmodule/templates/html/empty.yaml +++ /dev/null @@ -1,7 +0,0 @@ ---- -metadata: - display_name: Blank HTML Page - -data: | - -children: [] \ No newline at end of file diff --git a/common/lib/xmodule/xmodule/templates/html/everything.yaml b/common/lib/xmodule/xmodule/templates/html/everything.yaml deleted file mode 100644 index 348ce64fa1..0000000000 --- a/common/lib/xmodule/xmodule/templates/html/everything.yaml +++ /dev/null @@ -1,33 +0,0 @@ ---- -metadata: - display_name: Announcement - -data: | -

        Heading of document

        -

        First subheading

        -

        This is a paragraph. It will take care of line breaks for you.

        HTML only parses the location - - of tags for inserting line breaks into your doc, not - line - breaks - you - add - yourself. -

        -

        Links

        -

        You can refer to other parts of the internet with a link, to other parts of your course by prepending your link with /course/

        -

        Now a list:

        -
          -
        • An item
          • -
        • Another item
          • -
        • And yet another
          • -
        -

        This list has an ordering

        -
          -
        1. An item
          2. -
        2. Another item
          3. -
        3. Yet another item
          4. -
        -

        Note, we have a lot of standard edX styles, so please try to avoid any custom styling, and make sure that you make a note of any custom styling that you do yourself so that we can incorporate it into - tools that other people can use.

        -children: [] diff --git a/common/lib/xmodule/xmodule/templates/html/latex_html.yaml b/common/lib/xmodule/xmodule/templates/html/latex_html.yaml index ff92f2aead..2db7e98c65 100644 --- a/common/lib/xmodule/xmodule/templates/html/latex_html.yaml +++ b/common/lib/xmodule/xmodule/templates/html/latex_html.yaml @@ -1,16 +1,16 @@ --- metadata: display_name: E-text Written in LaTeX - source_code: | - \subsection{Example of E-text in LaTeX} +source_code: | + \subsection{Example of E-text in LaTeX} - It is very convenient to write complex equations in LaTeX. + It is very convenient to write complex equations in LaTeX. - \begin{equation} - x = \frac{-b\pm\sqrt{b^2-4*a*c}}{2a} - \end{equation} + \begin{equation} + x = \frac{-b\pm\sqrt{b^2-4*a*c}}{2a} + \end{equation} - Seize the moment. + Seize the moment. data: | @@ -19,4 +19,3 @@ data: | It is very convenient to write complex equations in LaTeX.

        -children: [] diff --git a/common/lib/xmodule/xmodule/templates/peer_grading/default.yaml b/common/lib/xmodule/xmodule/templates/peer_grading/default.yaml deleted file mode 100644 index 5d88a18ad8..0000000000 --- a/common/lib/xmodule/xmodule/templates/peer_grading/default.yaml +++ /dev/null @@ -1,9 +0,0 @@ ---- -metadata: - display_name: Peer Grading Interface - max_grade: 1 -data: | - - - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/blank_common.yaml b/common/lib/xmodule/xmodule/templates/problem/blank_common.yaml new file mode 100644 index 0000000000..3dcac29aba --- /dev/null +++ b/common/lib/xmodule/xmodule/templates/problem/blank_common.yaml @@ -0,0 +1,5 @@ +--- +metadata: + display_name: Blank Common Problem + markdown: "" +data: "" diff --git a/common/lib/xmodule/xmodule/templates/problem/circuitschematic.yaml b/common/lib/xmodule/xmodule/templates/problem/circuitschematic.yaml index 56f802a6a3..051dfe1912 100644 --- a/common/lib/xmodule/xmodule/templates/problem/circuitschematic.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/circuitschematic.yaml @@ -1,63 +1,60 @@ - --- metadata: display_name: Circuit Schematic Builder - rerandomize: never - showanswer: finished + markdown: !!null data: | - - Please make a voltage divider that splits the provided voltage evenly. - - -
        - -
        - - dc_value = "dc analysis not found" - for response in submission[0]: - if response[0] == 'dc': - for node in response[1:]: - dc_value = node['output'] - - if dc_value == .5: - correct = ['correct'] - else: - correct = ['incorrect'] - -         - -

        Make a high pass filter

        -
        - -
        - - ac_values = None - for response in submission[0]: - if response[0] == 'ac': - for node in response[1:]: - ac_values = node['NodeA'] - print "the ac analysis value:", ac_values - if ac_values == None: - correct = ['incorrect'] - elif ac_values[0][1] < ac_values[1][1]: - correct = ['correct'] - else: - correct = ['incorrect'] - -         - - -
        -

        Explanation

        -

        A voltage divider that evenly divides the input voltage can be formed with two identically valued resistors, with the sampled voltage taken in between the two.

        -

        -

        A simple high-pass filter without any further constaints can be formed by simply putting a resister in series with a capacitor. The actual values of the components do not really matter in order to meet the constraints of the problem.

        -

        -
        -         -         -children: [] + + Please make a voltage divider that splits the provided voltage evenly. + + +
        + +
        + + dc_value = "dc analysis not found" + for response in submission[0]: + if response[0] == 'dc': + for node in response[1:]: + dc_value = node['output'] + + if dc_value == .5: + correct = ['correct'] + else: + correct = ['incorrect'] + +         + +

        Make a high pass filter

        +
        + +
        + + ac_values = None + for response in submission[0]: + if response[0] == 'ac': + for node in response[1:]: + ac_values = node['NodeA'] + print "the ac analysis value:", ac_values + if ac_values == None: + correct = ['incorrect'] + elif ac_values[0][1] < ac_values[1][1]: + correct = ['correct'] + else: + correct = ['incorrect'] + +         + + +
        +

        Explanation

        +

        A voltage divider that evenly divides the input voltage can be formed with two identically valued resistors, with the sampled voltage taken in between the two.

        +

        +

        A simple high-pass filter without any further constaints can be formed by simply putting a resister in series with a capacitor. The actual values of the components do not really matter in order to meet the constraints of the problem.

        +

        +
        +         +         diff --git a/common/lib/xmodule/xmodule/templates/problem/customgrader.yaml b/common/lib/xmodule/xmodule/templates/problem/customgrader.yaml index 48feef481b..05de74f28c 100644 --- a/common/lib/xmodule/xmodule/templates/problem/customgrader.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/customgrader.yaml @@ -1,50 +1,47 @@ --- metadata: display_name: Custom Python-Evaluated Input - rerandomize: never - showanswer: finished + markdown: !!null data: | - -

        - A custom python-evaluated input problem accepts one or more lines of text input from the - student, and evaluates the inputs for correctness based on evaluation using a - python script embedded within the problem. -

        + +

        + A custom python-evaluated input problem accepts one or more lines of text input from the + student, and evaluates the inputs for correctness based on evaluation using a + python script embedded within the problem. +

        - + -

        Enter two integers which sum to 10:

        - -
        - -         +

        Enter two integers which sum to 10:

        + +
        + +         -

        Enter two integers which sum to 20:

        - -
        - -         - - -
        -

        Explanation

        -

        Any set of integers on the line \(y = 10 - x\) and \(y = 20 - x\) satisfy these constraints.

        - -
        -         -         - -children: [] +

        Enter two integers which sum to 20:

        + +
        + +         + + +
        +

        Explanation

        +

        Any set of integers on the line \(y = 10 - x\) and \(y = 20 - x\) satisfy these constraints.

        + +
        +         +         diff --git a/common/lib/xmodule/xmodule/templates/problem/drag_and_drop.yaml b/common/lib/xmodule/xmodule/templates/problem/drag_and_drop.yaml new file mode 100644 index 0000000000..678b75716b --- /dev/null +++ b/common/lib/xmodule/xmodule/templates/problem/drag_and_drop.yaml @@ -0,0 +1,41 @@ +--- +metadata: + display_name: Drag and Drop + markdown: !!null +data: | + + Here's an example of a "Drag and Drop" question set. Click and drag each word in the scrollbar below, up to the numbered bucket which matches the number of letters in the word. + + + + + + + + + + + + + + + + correct_answer = { + '1': [[70, 150], 121], + '6': [[190, 150], 121], + '8': [[190, 150], 121], + '2': [[310, 150], 121], + '9': [[310, 150], 121], + '11': [[310, 150], 121], + '4': [[420, 150], 121], + '7': [[420, 150], 121], + '3': [[550, 150], 121], + '5': [[550, 150], 121], + '10': [[550, 150], 121]} + if draganddrop.grade(submission[0], correct_answer): + correct = ['correct'] + else: + correct = ['incorrect'] + + + diff --git a/common/lib/xmodule/xmodule/templates/problem/empty.yaml b/common/lib/xmodule/xmodule/templates/problem/empty.yaml deleted file mode 100644 index 97a2aef423..0000000000 --- a/common/lib/xmodule/xmodule/templates/problem/empty.yaml +++ /dev/null @@ -1,11 +0,0 @@ ---- -metadata: - display_name: Blank Common Problem - rerandomize: never - showanswer: finished - markdown: "" -data: | - - - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/emptyadvanced.yaml b/common/lib/xmodule/xmodule/templates/problem/emptyadvanced.yaml deleted file mode 100644 index 3d696ec2fd..0000000000 --- a/common/lib/xmodule/xmodule/templates/problem/emptyadvanced.yaml +++ /dev/null @@ -1,10 +0,0 @@ ---- -metadata: - display_name: Blank Advanced Problem - rerandomize: never - showanswer: finished -data: | - - - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/forumularesponse.yaml b/common/lib/xmodule/xmodule/templates/problem/forumularesponse.yaml index 0401a01c31..807a7833e4 100644 --- a/common/lib/xmodule/xmodule/templates/problem/forumularesponse.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/forumularesponse.yaml @@ -1,8 +1,7 @@ --- metadata: display_name: Math Expression Input - rerandomize: never - showanswer: finished + markdown: !!null data: |

        @@ -43,5 +42,3 @@ data: |

      - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/imageresponse.yaml b/common/lib/xmodule/xmodule/templates/problem/imageresponse.yaml index ab1f22e3b2..1bc481ba4e 100644 --- a/common/lib/xmodule/xmodule/templates/problem/imageresponse.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/imageresponse.yaml @@ -1,26 +1,22 @@ --- metadata: display_name: Image Mapped Input - rerandomize: never - showanswer: finished + markdown: !!null data: | -

      - An image mapped input problem presents an image for the student. Input is - given by the location of mouse clicks on the image. Correctness of input can be evaluated based on expected dimensions of a rectangle. -

      - -

      Which object in this image is required by the fire code?

      - - - - -
      -

      Explanation

      -

      The fire code requires that all exits be clearly marked, so the red exit sign is the correct answer.

      -
      -       +

      + An image mapped input problem presents an image for the student. + Input is given by the location of mouse clicks on the image. + Correctness of input can be evaluated based on expected dimensions of a rectangle. +

      +

      Which animal shown below is a kitten?

      + + + + +
      +

      Explanation

      +

      The animal on the right is a kitten. The animal on the left is a puppy, not a kitten.

      +
      +       - - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/latex_problem.yaml b/common/lib/xmodule/xmodule/templates/problem/latex_problem.yaml index 82d7e8c1ae..097055cfe3 100644 --- a/common/lib/xmodule/xmodule/templates/problem/latex_problem.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/latex_problem.yaml @@ -85,6 +85,7 @@ metadata: can contain equations: $\alpha = \frac{2}{\sqrt{1+\gamma}}$ } This is some text after the showhide example. + markdown: !!null data: | @@ -214,4 +215,3 @@ data: |

      -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/multiplechoice.yaml b/common/lib/xmodule/xmodule/templates/problem/multiplechoice.yaml index 10d51de280..68845564d5 100644 --- a/common/lib/xmodule/xmodule/templates/problem/multiplechoice.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/multiplechoice.yaml @@ -1,35 +1,25 @@ --- metadata: display_name: Multiple Choice - rerandomize: never - showanswer: finished - markdown: - "A multiple choice problem presents radio buttons for student input. Students can only select a single + markdown: | + A multiple choice problem presents radio buttons for student input. Students can only select a single option presented. Multiple Choice questions have been the subject of many areas of research due to the early invention and adoption of bubble sheets. - One of the main elements that goes into a good multiple choice question is the existence of good distractors. That is, each of the alternate responses presented to the student should be the result of a plausible mistake that a student might make. - What Apple device competed with the portable CD player? - ( ) The iPad - ( ) Napster - (x) The iPod - ( ) The vegetable peeler - [explanation] The release of the iPod allowed consumers to carry their entire music library with them in a format that did not rely on fragile and energy-intensive spinning disks. [explanation] - " data: |

      @@ -54,4 +44,3 @@ data: |

      -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/numericalresponse.yaml b/common/lib/xmodule/xmodule/templates/problem/numericalresponse.yaml index 548fd94fab..e97a54b460 100644 --- a/common/lib/xmodule/xmodule/templates/problem/numericalresponse.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/numericalresponse.yaml @@ -1,45 +1,33 @@ --- metadata: display_name: Numerical Input - rerandomize: never - showanswer: finished - markdown: - "A numerical input problem accepts a line of text input from the + markdown: | + A numerical input problem accepts a line of text input from the student, and evaluates the input for correctness based on its numerical value. - The answer is correct if it is within a specified numerical tolerance of the expected answer. - Enter the numerical value of Pi: - = 3.14159 +- .02 - Enter the approximate value of 502*9: - = 4518 +- 15% - - + Enter the number of fingers on a human hand: - = 5 - [explanation] Pi, or the the ratio between a circle's circumference to its diameter, is an irrational number known to extreme precision. It is value is approximately equal to 3.14. - + Although you can get an exact value by typing 502*9 into a calculator, the result will be close to 500*10, or 5,000. The grader accepts any response within 15% of the true value, 4518, so that you can use any estimation technique that you like. - + If you look at your hand, you can count that you have five fingers. [explanation] - " - data: |

      @@ -83,5 +71,3 @@ data: |

      - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/optionresponse.yaml b/common/lib/xmodule/xmodule/templates/problem/optionresponse.yaml index c2edfb1cbc..44af7a850a 100644 --- a/common/lib/xmodule/xmodule/templates/problem/optionresponse.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/optionresponse.yaml @@ -1,21 +1,16 @@ --- metadata: display_name: Dropdown - rerandomize: never - showanswer: finished - markdown: - "Dropdown problems give a limited set of options for students to respond with, and present those options + markdown: | + Dropdown problems give a limited set of options for students to respond with, and present those options in a format that encourages them to search for a specific answer rather than being immediately presented with options from which to recognize the correct answer. - The answer options and the identification of the correct answer is defined in the optioninput tag. - Translation between Dropdown and __________ is extremely straightforward: - - [[(Multiple Choice), Text Input, Numerical Input, External Response, Image Response]] + [[(Multiple Choice), Text Input, Numerical Input, External Response, Image Response]] [explanation] Multiple Choice also allows students to select from a variety of pre-written responses, although the @@ -23,7 +18,6 @@ metadata: slightly because students are more likely to think of an answer and then search for it rather than relying purely on recognition to answer the question. [explanation] - " data: |

      Dropdown problems give a limited set of options for students to respond with, and present those options @@ -45,4 +39,3 @@ data: | -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/problem_with_hint.yaml b/common/lib/xmodule/xmodule/templates/problem/problem_with_hint.yaml index 73a94ed941..aa1000a93a 100644 --- a/common/lib/xmodule/xmodule/templates/problem/problem_with_hint.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/problem_with_hint.yaml @@ -46,9 +46,8 @@ metadata: enter your answer in upper or lower case, with or without quotes. \edXabox{type="custom" cfn='test_str' expect='python' hintfn='hint_fn'} - + markdown: !!null data: | -

      @@ -92,4 +91,3 @@ data: |

      -children: [] diff --git a/common/lib/xmodule/xmodule/templates/problem/string_response.yaml b/common/lib/xmodule/xmodule/templates/problem/string_response.yaml index 64e3dc062f..cf95fe8331 100644 --- a/common/lib/xmodule/xmodule/templates/problem/string_response.yaml +++ b/common/lib/xmodule/xmodule/templates/problem/string_response.yaml @@ -1,18 +1,13 @@ --- metadata: display_name: Text Input - rerandomize: never - showanswer: finished - # Note, the extra newlines are needed to make the yaml parser add blank lines instead of folding - markdown: - "A text input problem accepts a line of text from the + markdown: | + A text input problem accepts a line of text from the student, and evaluates the input for correctness based on an expected answer. - The answer is correct if it matches every character of the expected answer. This can be a problem with international spelling, dates, or anything where the format of the answer is not clear. - Which US state has Lansing as its capital? @@ -23,9 +18,8 @@ metadata: Lansing is the capital of Michigan, although it is not Michgan's largest city, or even the seat of the county in which it resides. [explanation] - " data: | - +

      A text input problem accepts a line of text from the @@ -46,4 +40,3 @@ data: | -children: [] diff --git a/common/lib/xmodule/xmodule/templates/sequence/with_video.yaml b/common/lib/xmodule/xmodule/templates/sequence/with_video.yaml deleted file mode 100644 index a56d44ebff..0000000000 --- a/common/lib/xmodule/xmodule/templates/sequence/with_video.yaml +++ /dev/null @@ -1,7 +0,0 @@ ---- -metadata: - display_name: Sequence with Video - data_dir: a_made_up_name -data: '' -children: - - 'i4x://edx/templates/video/default' diff --git a/common/lib/xmodule/xmodule/templates/statictab/empty.yaml b/common/lib/xmodule/xmodule/templates/statictab/empty.yaml deleted file mode 100644 index 410e1496c2..0000000000 --- a/common/lib/xmodule/xmodule/templates/statictab/empty.yaml +++ /dev/null @@ -1,5 +0,0 @@ ---- -metadata: - display_name: Empty -data: "

      This is where you can add additional pages to your courseware. Click the 'edit' button to begin editing.

      " -children: [] \ No newline at end of file diff --git a/common/lib/xmodule/xmodule/templates/video/default.yaml b/common/lib/xmodule/xmodule/templates/video/default.yaml deleted file mode 100644 index 048e7396c7..0000000000 --- a/common/lib/xmodule/xmodule/templates/video/default.yaml +++ /dev/null @@ -1,5 +0,0 @@ ---- -metadata: - display_name: default -data: "" -children: [] diff --git a/common/lib/xmodule/xmodule/templates/videoalpha/default.yaml b/common/lib/xmodule/xmodule/templates/videoalpha/default.yaml deleted file mode 100644 index 1c25b272a3..0000000000 --- a/common/lib/xmodule/xmodule/templates/videoalpha/default.yaml +++ /dev/null @@ -1,11 +0,0 @@ ---- -metadata: - display_name: Video Alpha - version: 1 -data: | - - - - - -children: [] diff --git a/common/lib/xmodule/xmodule/templates/word_cloud/default.yaml b/common/lib/xmodule/xmodule/templates/word_cloud/default.yaml deleted file mode 100644 index 53e9eeaae4..0000000000 --- a/common/lib/xmodule/xmodule/templates/word_cloud/default.yaml +++ /dev/null @@ -1,5 +0,0 @@ ---- -metadata: - display_name: Word cloud -data: {} -children: [] diff --git a/common/lib/xmodule/xmodule/tests/test_capa_module.py b/common/lib/xmodule/xmodule/tests/test_capa_module.py index 1e84174291..80c4e41e8f 100644 --- a/common/lib/xmodule/xmodule/tests/test_capa_module.py +++ b/common/lib/xmodule/xmodule/tests/test_capa_module.py @@ -636,10 +636,10 @@ class CapaModuleTest(unittest.TestCase): # Expect that the problem was reset module.new_lcp.assert_called_once_with(None) - module.choose_new_seed.assert_called_once_with() def test_reset_problem_closed(self): - module = CapaFactory.create() + # pre studio default + module = CapaFactory.create(rerandomize="always") # Simulate that the problem is closed with patch('xmodule.capa_module.CapaModule.closed') as mock_closed: @@ -900,13 +900,13 @@ class CapaModuleTest(unittest.TestCase): module = CapaFactory.create(done=False) self.assertFalse(module.should_show_reset_button()) - # Otherwise, DO show the reset button - module = CapaFactory.create(done=True) + # pre studio default value, DO show the reset button + module = CapaFactory.create(rerandomize="always", done=True) self.assertTrue(module.should_show_reset_button()) # If survey question for capa (max_attempts = 0), # DO show the reset button - module = CapaFactory.create(max_attempts=0, done=True) + module = CapaFactory.create(rerandomize="always", max_attempts=0, done=True) self.assertTrue(module.should_show_reset_button()) def test_should_show_save_button(self): @@ -940,8 +940,8 @@ class CapaModuleTest(unittest.TestCase): module = CapaFactory.create(max_attempts=None, rerandomize="per_student", done=True) self.assertFalse(module.should_show_save_button()) - # Otherwise, DO show the save button - module = CapaFactory.create(done=False) + # pre-studio default, DO show the save button + module = CapaFactory.create(rerandomize="always", done=False) self.assertTrue(module.should_show_save_button()) # If we're not randomizing and we have limited attempts, then we can save @@ -1233,6 +1233,37 @@ class CapaModuleTest(unittest.TestCase): mock_log.exception.assert_called_once_with('Got bad progress') mock_log.reset_mock() + @patch('xmodule.capa_module.Progress') + def test_get_progress_calculate_progress_fraction(self, mock_progress): + """ + Check that score and total are calculated correctly for the progress fraction. + """ + module = CapaFactory.create() + module.weight = 1 + module.get_progress() + mock_progress.assert_called_with(0, 1) + + other_module = CapaFactory.create(correct=True) + other_module.weight = 1 + other_module.get_progress() + mock_progress.assert_called_with(1, 1) + + def test_get_html(self): + """ + Check that get_html() calls get_progress() with no arguments. + """ + module = CapaFactory.create() + module.get_progress = Mock(wraps=module.get_progress) + module.get_html() + module.get_progress.assert_called_once_with() + + def test_get_problem(self): + """ + Check that get_problem() returns the expected dictionary. + """ + module = CapaFactory.create() + self.assertEquals(module.get_problem("data"), {'html': module.get_problem_html(encapsulate=False)}) + class ComplexEncoderTest(unittest.TestCase): def test_default(self): diff --git a/common/lib/xmodule/xmodule/tests/test_combined_open_ended.py b/common/lib/xmodule/xmodule/tests/test_combined_open_ended.py index e1f8d135de..4fd0ddccf7 100644 --- a/common/lib/xmodule/xmodule/tests/test_combined_open_ended.py +++ b/common/lib/xmodule/xmodule/tests/test_combined_open_ended.py @@ -14,6 +14,7 @@ from xmodule.modulestore import Location from lxml import etree import capa.xqueue_interface as xqueue_interface from datetime import datetime +from pytz import UTC import logging log = logging.getLogger(__name__) @@ -212,7 +213,7 @@ class OpenEndedModuleTest(unittest.TestCase): 'submission_id': '1', 'grader_id': '1', 'score': 3} - qtime = datetime.strftime(datetime.now(), xqueue_interface.dateformat) + qtime = datetime.strftime(datetime.now(UTC), xqueue_interface.dateformat) student_info = {'anonymous_student_id': self.test_system.anonymous_student_id, 'submission_time': qtime} contents = { @@ -233,7 +234,7 @@ class OpenEndedModuleTest(unittest.TestCase): def test_send_to_grader(self): submission = "This is a student submission" - qtime = datetime.strftime(datetime.now(), xqueue_interface.dateformat) + qtime = datetime.strftime(datetime.now(UTC), xqueue_interface.dateformat) student_info = {'anonymous_student_id': self.test_system.anonymous_student_id, 'submission_time': qtime} contents = self.openendedmodule.payload.copy() @@ -335,7 +336,7 @@ class CombinedOpenEndedModuleTest(unittest.TestCase): 's3_interface': test_util_open_ended.S3_INTERFACE, 'open_ended_grading_interface': test_util_open_ended.OPEN_ENDED_GRADING_INTERFACE, 'skip_basic_checks': False, - 'is_graded': True, + 'graded': True, } oeparam = etree.XML(''' @@ -453,7 +454,7 @@ class CombinedOpenEndedModuleTest(unittest.TestCase): self.assertFalse(changed) def test_get_score_realistic(self): - instance_state = r"""{"ready_to_reset": false, "skip_spelling_checks": true, "current_task_number": 1, "weight": 5.0, "graceperiod": "1 day 12 hours 59 minutes 59 seconds", "is_graded": "True", "task_states": ["{\"child_created\": false, \"child_attempts\": 4, \"version\": 1, \"child_history\": [{\"answer\": \"The students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the group\\u2019s procedure, describe what additional information you would need in order to replicate the expe\", \"post_assessment\": \"{\\\"submission_id\\\": 3097, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: More grammar errors than average.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"the students data are recorded in the table below . starting mass g ending mass g difference in mass g marble . . . limestone . . . wood . . . plastic . . . after reading the groups procedure , describe what additional information you would need in order to replicate the expe\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3233, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"After 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the\", \"post_assessment\": \"[3]\", \"score\": 3}, {\"answer\": \"To replicate the experiment, the procedure would require more detail. One piece of information that is omitted is the amount of vinegar used in the experiment. It is also important to know what temperature the experiment was kept at during the 24 hours. Finally, the procedure needs to include details about the experiment, for example if the whole sample must be submerged.\", \"post_assessment\": \"[3]\", \"score\": 3}, {\"answer\": \"e the mass of four different samples.\\r\\nPour vinegar in each of four separate, but identical, containers.\\r\\nPlace a sample of one material into one container and label. Repeat with remaining samples, placing a single sample into a single container.\\r\\nAfter 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\", \"post_assessment\": \"[3]\", \"score\": 3}, {\"answer\": \"\", \"post_assessment\": \"[3]\", \"score\": 3}], \"max_score\": 3, \"child_state\": \"done\"}", "{\"child_created\": false, \"child_attempts\": 0, \"version\": 1, \"child_history\": [{\"answer\": \"The students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the group\\u2019s procedure, describe what additional information you would need in order to replicate the expe\", \"post_assessment\": \"{\\\"submission_id\\\": 3097, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: More grammar errors than average.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"the students data are recorded in the table below . starting mass g ending mass g difference in mass g marble . . . limestone . . . wood . . . plastic . . . after reading the groups procedure , describe what additional information you would need in order to replicate the expe\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3233, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"After 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the\", \"post_assessment\": \"{\\\"submission_id\\\": 3098, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"after hours , remove the samples from the containers and rinse each sample with distilled water . allow the samples to sit and dry for minutes . determine the mass of each sample . the students data are recorded in the table below . starting mass g ending mass g difference in mass g marble . . . limestone . . . wood . . . plastic . . . after reading the\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3235, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"To replicate the experiment, the procedure would require more detail. One piece of information that is omitted is the amount of vinegar used in the experiment. It is also important to know what temperature the experiment was kept at during the 24 hours. Finally, the procedure needs to include details about the experiment, for example if the whole sample must be submerged.\", \"post_assessment\": \"{\\\"submission_id\\\": 3099, \\\"score\\\": 3, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"to replicate the experiment , the procedure would require more detail . one piece of information that is omitted is the amount of vinegar used in the experiment . it is also important to know what temperature the experiment was kept at during the hours . finally , the procedure needs to include details about the experiment , for example if the whole sample must be submerged .\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3237, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality3\\\"}\", \"score\": 3}, {\"answer\": \"e the mass of four different samples.\\r\\nPour vinegar in each of four separate, but identical, containers.\\r\\nPlace a sample of one material into one container and label. Repeat with remaining samples, placing a single sample into a single container.\\r\\nAfter 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\", \"post_assessment\": \"{\\\"submission_id\\\": 3100, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"e the mass of four different samples . pour vinegar in each of four separate , but identical , containers . place a sample of one material into one container and label . repeat with remaining samples , placing a single sample into a single container . after hours , remove the samples from the containers and rinse each sample with distilled water . allow the samples to sit and dry for minutes . determine the mass of each sample . the students data are recorded in the table below . \\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3239, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"\", \"post_assessment\": \"{\\\"submission_id\\\": 3101, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"invalid essay .\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3241, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}], \"max_score\": 3, \"child_state\": \"done\"}"], "attempts": "10000", "student_attempts": 0, "due": null, "state": "done", "accept_file_upload": false, "display_name": "Science Question -- Machine Assessed"}""" + instance_state = r"""{"ready_to_reset": false, "skip_spelling_checks": true, "current_task_number": 1, "weight": 5.0, "graceperiod": "1 day 12 hours 59 minutes 59 seconds", "graded": "True", "task_states": ["{\"child_created\": false, \"child_attempts\": 4, \"version\": 1, \"child_history\": [{\"answer\": \"The students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the group\\u2019s procedure, describe what additional information you would need in order to replicate the expe\", \"post_assessment\": \"{\\\"submission_id\\\": 3097, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: More grammar errors than average.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"the students data are recorded in the table below . starting mass g ending mass g difference in mass g marble . . . limestone . . . wood . . . plastic . . . after reading the groups procedure , describe what additional information you would need in order to replicate the expe\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3233, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"After 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the\", \"post_assessment\": \"[3]\", \"score\": 3}, {\"answer\": \"To replicate the experiment, the procedure would require more detail. One piece of information that is omitted is the amount of vinegar used in the experiment. It is also important to know what temperature the experiment was kept at during the 24 hours. Finally, the procedure needs to include details about the experiment, for example if the whole sample must be submerged.\", \"post_assessment\": \"[3]\", \"score\": 3}, {\"answer\": \"e the mass of four different samples.\\r\\nPour vinegar in each of four separate, but identical, containers.\\r\\nPlace a sample of one material into one container and label. Repeat with remaining samples, placing a single sample into a single container.\\r\\nAfter 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\", \"post_assessment\": \"[3]\", \"score\": 3}, {\"answer\": \"\", \"post_assessment\": \"[3]\", \"score\": 3}], \"max_score\": 3, \"child_state\": \"done\"}", "{\"child_created\": false, \"child_attempts\": 0, \"version\": 1, \"child_history\": [{\"answer\": \"The students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the group\\u2019s procedure, describe what additional information you would need in order to replicate the expe\", \"post_assessment\": \"{\\\"submission_id\\\": 3097, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: More grammar errors than average.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"the students data are recorded in the table below . starting mass g ending mass g difference in mass g marble . . . limestone . . . wood . . . plastic . . . after reading the groups procedure , describe what additional information you would need in order to replicate the expe\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3233, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"After 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\\r\\nStarting Mass (g)\\tEnding Mass (g)\\tDifference in Mass (g)\\r\\nMarble\\t 9.8\\t 9.4\\t\\u20130.4\\r\\nLimestone\\t10.4\\t 9.1\\t\\u20131.3\\r\\nWood\\t11.2\\t11.2\\t 0.0\\r\\nPlastic\\t 7.2\\t 7.1\\t\\u20130.1\\r\\nAfter reading the\", \"post_assessment\": \"{\\\"submission_id\\\": 3098, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"after hours , remove the samples from the containers and rinse each sample with distilled water . allow the samples to sit and dry for minutes . determine the mass of each sample . the students data are recorded in the table below . starting mass g ending mass g difference in mass g marble . . . limestone . . . wood . . . plastic . . . after reading the\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3235, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"To replicate the experiment, the procedure would require more detail. One piece of information that is omitted is the amount of vinegar used in the experiment. It is also important to know what temperature the experiment was kept at during the 24 hours. Finally, the procedure needs to include details about the experiment, for example if the whole sample must be submerged.\", \"post_assessment\": \"{\\\"submission_id\\\": 3099, \\\"score\\\": 3, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"to replicate the experiment , the procedure would require more detail . one piece of information that is omitted is the amount of vinegar used in the experiment . it is also important to know what temperature the experiment was kept at during the hours . finally , the procedure needs to include details about the experiment , for example if the whole sample must be submerged .\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3237, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality3\\\"}\", \"score\": 3}, {\"answer\": \"e the mass of four different samples.\\r\\nPour vinegar in each of four separate, but identical, containers.\\r\\nPlace a sample of one material into one container and label. Repeat with remaining samples, placing a single sample into a single container.\\r\\nAfter 24 hours, remove the samples from the containers and rinse each sample with distilled water.\\r\\nAllow the samples to sit and dry for 30 minutes.\\r\\nDetermine the mass of each sample.\\r\\nThe students\\u2019 data are recorded in the table below.\\r\\n\", \"post_assessment\": \"{\\\"submission_id\\\": 3100, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"e the mass of four different samples . pour vinegar in each of four separate , but identical , containers . place a sample of one material into one container and label . repeat with remaining samples , placing a single sample into a single container . after hours , remove the samples from the containers and rinse each sample with distilled water . allow the samples to sit and dry for minutes . determine the mass of each sample . the students data are recorded in the table below . \\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3239, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}, {\"answer\": \"\", \"post_assessment\": \"{\\\"submission_id\\\": 3101, \\\"score\\\": 0, \\\"feedback\\\": \\\"{\\\\\\\"spelling\\\\\\\": \\\\\\\"Spelling: Ok.\\\\\\\", \\\\\\\"grammar\\\\\\\": \\\\\\\"Grammar: Ok.\\\\\\\", \\\\\\\"markup-text\\\\\\\": \\\\\\\"invalid essay .\\\\\\\"}\\\", \\\"success\\\": true, \\\"grader_id\\\": 3241, \\\"grader_type\\\": \\\"ML\\\", \\\"rubric_scores_complete\\\": true, \\\"rubric_xml\\\": \\\"Response Quality0\\\"}\", \"score\": 0}], \"max_score\": 3, \"child_state\": \"done\"}"], "attempts": "10000", "student_attempts": 0, "due": null, "state": "done", "accept_file_upload": false, "display_name": "Science Question -- Machine Assessed"}""" instance_state = json.loads(instance_state) rubric = """ @@ -504,11 +505,13 @@ class OpenEndedModuleXmlTest(unittest.TestCase, DummyModulestore): See if we can load the module and save an answer @return: """ - #Load the module + # Load the module module = self.get_module_from_location(self.problem_location, COURSE) - #Try saving an answer + # Try saving an answer module.handle_ajax("save_answer", {"student_answer": self.answer}) + # Save our modifications to the underlying KeyValueStore so they can be persisted + module.save() task_one_json = json.loads(module.task_states[0]) self.assertEqual(task_one_json['child_history'][0]['answer'], self.answer) @@ -629,3 +632,59 @@ class OpenEndedModuleXmlTest(unittest.TestCase, DummyModulestore): #reset the problem module.handle_ajax("reset", {}) self.assertEqual(module.state, "initial") + + +class OpenEndedModuleXmlAttemptTest(unittest.TestCase, DummyModulestore): + """ + Test if student is able to reset the problem + """ + problem_location = Location(["i4x", "edX", "open_ended", "combinedopenended", "SampleQuestion1Attempt"]) + answer = "blah blah" + assessment = [0, 1] + hint = "blah" + + def setUp(self): + self.test_system = get_test_system() + self.test_system.xqueue['interface'] = Mock( + send_to_queue=Mock(side_effect=[1, "queued"]) + ) + self.setup_modulestore(COURSE) + + def test_reset_fail(self): + """ + Test the flow of the module if we complete the self assessment step and then reset + Since the problem only allows one attempt, should fail. + @return: + """ + assessment = [0, 1] + module = self.get_module_from_location(self.problem_location, COURSE) + + #Simulate a student saving an answer + module.handle_ajax("save_answer", {"student_answer": self.answer}) + status = module.handle_ajax("get_status", {}) + self.assertTrue(isinstance(status, basestring)) + + #Mock a student submitting an assessment + assessment_dict = MockQueryDict() + assessment_dict.update({'assessment': sum(assessment), 'score_list[]': assessment}) + module.handle_ajax("save_assessment", assessment_dict) + task_one_json = json.loads(module.task_states[0]) + self.assertEqual(json.loads(task_one_json['child_history'][0]['post_assessment']), assessment) + status = module.handle_ajax("get_status", {}) + self.assertTrue(isinstance(status, basestring)) + + #Move to the next step in the problem + module.handle_ajax("next_problem", {}) + self.assertEqual(module.current_task_number, 0) + + html = module.get_html() + self.assertTrue(isinstance(html, basestring)) + + #Module should now be done + rubric = module.handle_ajax("get_combined_rubric", {}) + self.assertTrue(isinstance(rubric, basestring)) + self.assertEqual(module.state, "done") + + #Try to reset, should fail because only 1 attempt is allowed + reset_data = json.loads(module.handle_ajax("reset", {})) + self.assertEqual(reset_data['success'], False) diff --git a/common/lib/xmodule/xmodule/tests/test_conditional.py b/common/lib/xmodule/xmodule/tests/test_conditional.py index abb8b4941a..b28d236369 100644 --- a/common/lib/xmodule/xmodule/tests/test_conditional.py +++ b/common/lib/xmodule/xmodule/tests/test_conditional.py @@ -217,8 +217,11 @@ class ConditionalModuleXmlTest(unittest.TestCase): html = ajax['html'] self.assertFalse(any(['This is a secret' in item for item in html])) - # now change state of the capa problem to make it completed - inner_get_module(Location('i4x://HarvardX/ER22x/problem/choiceprob')).attempts = 1 + # Now change state of the capa problem to make it completed + inner_module = inner_get_module(Location('i4x://HarvardX/ER22x/problem/choiceprob')) + inner_module.attempts = 1 + # Save our modifications to the underlying KeyValueStore so they can be persisted + inner_module.save() ajax = json.loads(module.handle_ajax('', '')) print "post-attempt ajax: ", ajax diff --git a/common/lib/xmodule/xmodule/tests/test_fields.py b/common/lib/xmodule/xmodule/tests/test_fields.py index f0eb082dcf..8453adaa20 100644 --- a/common/lib/xmodule/xmodule/tests/test_fields.py +++ b/common/lib/xmodule/xmodule/tests/test_fields.py @@ -44,7 +44,8 @@ class DateTest(unittest.TestCase): def test_return_None(self): self.assertIsNone(DateTest.date.from_json("")) self.assertIsNone(DateTest.date.from_json(None)) - self.assertIsNone(DateTest.date.from_json(['unknown value'])) + with self.assertRaises(TypeError): + DateTest.date.from_json(['unknown value']) def test_old_due_date_format(self): current = datetime.datetime.today() @@ -83,6 +84,8 @@ class DateTest(unittest.TestCase): DateTest.date.to_json( DateTest.date.from_json("2012-12-31T23:00:01-01:00")), "2012-12-31T23:00:01-01:00") + with self.assertRaises(TypeError): + DateTest.date.to_json('2012-12-31T23:00:01-01:00') class TimedeltaTest(unittest.TestCase): diff --git a/common/lib/xmodule/xmodule/tests/test_import.py b/common/lib/xmodule/xmodule/tests/test_import.py index 30c8939b5b..2fe9d70627 100644 --- a/common/lib/xmodule/xmodule/tests/test_import.py +++ b/common/lib/xmodule/xmodule/tests/test_import.py @@ -156,11 +156,7 @@ class ImportTestCase(BaseCourseTestCase): child = descriptor.get_children()[0] self.assertEqual(child.lms.due, ImportTestCase.date.from_json(v)) self.assertEqual(child._inheritable_metadata, child._inherited_metadata) - self.assertEqual(2, len(child._inherited_metadata)) - self.assertLessEqual( - ImportTestCase.date.from_json(child._inherited_metadata['start']), - datetime.datetime.now(UTC()) - ) + self.assertEqual(1, len(child._inherited_metadata)) self.assertEqual(v, child._inherited_metadata['due']) # Now export and check things @@ -218,10 +214,8 @@ class ImportTestCase(BaseCourseTestCase): self.assertEqual(child.lms.due, None) # pylint: disable=W0212 self.assertEqual(child._inheritable_metadata, child._inherited_metadata) - self.assertEqual(1, len(child._inherited_metadata)) - # why do these tests look in the internal structure v just calling child.start? self.assertLessEqual( - ImportTestCase.date.from_json(child._inherited_metadata['start']), + child.lms.start, datetime.datetime.now(UTC()) ) @@ -249,12 +243,7 @@ class ImportTestCase(BaseCourseTestCase): self.assertEqual(descriptor.lms.due, ImportTestCase.date.from_json(course_due)) self.assertEqual(child.lms.due, ImportTestCase.date.from_json(child_due)) # Test inherited metadata. Due does not appear here (because explicitly set on child). - self.assertEqual(1, len(child._inherited_metadata)) - self.assertLessEqual( - ImportTestCase.date.from_json(child._inherited_metadata['start']), - datetime.datetime.now(UTC())) - # Test inheritable metadata. This has the course inheritable value for due. - self.assertEqual(2, len(child._inheritable_metadata)) + self.assertEqual(1, len(child._inheritable_metadata)) self.assertEqual(course_due, child._inheritable_metadata['due']) def test_is_pointer_tag(self): diff --git a/common/lib/xmodule/xmodule/tests/test_logic.py b/common/lib/xmodule/xmodule/tests/test_logic.py index 9be533885c..5fe7aa2832 100644 --- a/common/lib/xmodule/xmodule/tests/test_logic.py +++ b/common/lib/xmodule/xmodule/tests/test_logic.py @@ -28,7 +28,8 @@ class LogicTest(unittest.TestCase): def setUp(self): class EmptyClass: """Empty object.""" - pass + url_name = '' + category = 'test' self.system = get_test_system() self.descriptor = EmptyClass() diff --git a/common/lib/xmodule/xmodule/tests/test_xblock_wrappers.py b/common/lib/xmodule/xmodule/tests/test_xblock_wrappers.py new file mode 100644 index 0000000000..d4df2e1054 --- /dev/null +++ b/common/lib/xmodule/xmodule/tests/test_xblock_wrappers.py @@ -0,0 +1,160 @@ +""" +Tests for the wrapping layer that provides the XBlock API using XModule/Descriptor +functionality +""" + +from nose.tools import assert_equal +from unittest.case import SkipTest +from mock import Mock + +from xmodule.annotatable_module import AnnotatableDescriptor +from xmodule.capa_module import CapaDescriptor +from xmodule.course_module import CourseDescriptor +from xmodule.combined_open_ended_module import CombinedOpenEndedDescriptor +from xmodule.discussion_module import DiscussionDescriptor +from xmodule.gst_module import GraphicalSliderToolDescriptor +from xmodule.html_module import HtmlDescriptor +from xmodule.peer_grading_module import PeerGradingDescriptor +from xmodule.poll_module import PollDescriptor +from xmodule.video_module import VideoDescriptor +from xmodule.word_cloud_module import WordCloudDescriptor +from xmodule.crowdsource_hinter import CrowdsourceHinterDescriptor +from xmodule.videoalpha_module import VideoAlphaDescriptor +from xmodule.seq_module import SequenceDescriptor +from xmodule.conditional_module import ConditionalDescriptor +from xmodule.randomize_module import RandomizeDescriptor +from xmodule.vertical_module import VerticalDescriptor +from xmodule.wrapper_module import WrapperDescriptor + +LEAF_XMODULES = ( + AnnotatableDescriptor, + CapaDescriptor, + CombinedOpenEndedDescriptor, + DiscussionDescriptor, + GraphicalSliderToolDescriptor, + HtmlDescriptor, + PeerGradingDescriptor, + PollDescriptor, + VideoDescriptor, + # This is being excluded because it has dependencies on django + #VideoAlphaDescriptor, + WordCloudDescriptor, +) + + +CONTAINER_XMODULES = ( + CrowdsourceHinterDescriptor, + CourseDescriptor, + SequenceDescriptor, + ConditionalDescriptor, + RandomizeDescriptor, + VerticalDescriptor, + WrapperDescriptor, + CourseDescriptor, +) + + +class TestXBlockWrapper(object): + + @property + def leaf_module_runtime(self): + runtime = Mock() + runtime.render_template = lambda *args, **kwargs: unicode((args, kwargs)) + runtime.anonymous_student_id = 'anonymous_student_id' + runtime.open_ended_grading_interface = {} + runtime.seed = 5 + runtime.get = lambda x: getattr(runtime, x) + runtime.position = 2 + runtime.ajax_url = 'ajax_url' + runtime.xblock_model_data = lambda d: d._model_data + return runtime + + @property + def leaf_descriptor_runtime(self): + runtime = Mock() + runtime.render_template = lambda *args, **kwargs: unicode((args, kwargs)) + return runtime + + def leaf_descriptor(self, descriptor_cls): + return descriptor_cls( + self.leaf_descriptor_runtime, + {'location': 'i4x://org/course/catagory/name'} + ) + + def leaf_module(self, descriptor_cls): + return self.leaf_descriptor(descriptor_cls).xmodule(self.leaf_module_runtime) + + def container_module_runtime(self, depth): + runtime = self.leaf_module_runtime + if depth == 0: + runtime.get_module.side_effect = lambda x: self.leaf_module(HtmlDescriptor) + else: + runtime.get_module.side_effect = lambda x: self.container_module(VerticalDescriptor, depth-1) + return runtime + + @property + def container_descriptor_runtime(self): + runtime = Mock() + runtime.render_template = lambda *args, **kwargs: unicode((args, kwargs)) + return runtime + + def container_descriptor(self, descriptor_cls): + return descriptor_cls( + self.container_descriptor_runtime, + { + 'location': 'i4x://org/course/catagory/name', + 'children': range(3) + } + ) + + def container_module(self, descriptor_cls, depth): + return self.container_descriptor(descriptor_cls).xmodule(self.container_module_runtime(depth)) + +class TestStudentView(TestXBlockWrapper): + + # Test that for all of the leaf XModule Descriptors, + # the student_view wrapper returns the same thing in its content + # as get_html returns + def test_student_view_leaf_node(self): + for descriptor_cls in LEAF_XMODULES: + yield self.check_student_view_leaf_node, descriptor_cls + + # Check that when an xmodule is instantiated from descriptor_cls + # it generates the same thing from student_view that it does from get_html + def check_student_view_leaf_node(self, descriptor_cls): + xmodule = self.leaf_module(descriptor_cls) + assert_equal(xmodule.get_html(), xmodule.student_view(None).content) + + + # Test that for all container XModule Descriptors, + # their corresponding XModule renders the same thing using student_view + # as it does using get_html, under the following conditions: + # a) All of its descendents are xmodules + # b) Some of its descendents are xmodules and some are xblocks + # c) All of its descendents are xblocks + def test_student_view_container_node(self): + for descriptor_cls in CONTAINER_XMODULES: + yield self.check_student_view_container_node_xmodules_only, descriptor_cls + yield self.check_student_view_container_node_mixed, descriptor_cls + yield self.check_student_view_container_node_xblocks_only, descriptor_cls + + + # Check that when an xmodule is generated from descriptor_cls + # with only xmodule children, it generates the same html from student_view + # as it does using get_html + def check_student_view_container_node_xmodules_only(self, descriptor_cls): + xmodule = self.container_module(descriptor_cls, 2) + assert_equal(xmodule.get_html(), xmodule.student_view(None).content) + + # Check that when an xmodule is generated from descriptor_cls + # with mixed xmodule and xblock children, it generates the same html from student_view + # as it does using get_html + def check_student_view_container_node_mixed(self, descriptor_cls): + raise SkipTest("XBlock support in XDescriptor not yet fully implemented") + + # Check that when an xmodule is generated from descriptor_cls + # with only xblock children, it generates the same html from student_view + # as it does using get_html + def check_student_view_container_node_xblocks_only(self, descriptor_cls): + raise SkipTest("XBlock support in XModules not yet fully implemented") + diff --git a/common/lib/xmodule/xmodule/tests/test_xml_module.py b/common/lib/xmodule/xmodule/tests/test_xml_module.py index 7ccc71dd96..a277ff2900 100644 --- a/common/lib/xmodule/xmodule/tests/test_xml_module.py +++ b/common/lib/xmodule/xmodule/tests/test_xml_module.py @@ -141,6 +141,7 @@ class EditableMetadataFieldsTest(unittest.TestCase): def get_xml_editable_fields(self, model_data): system = get_test_system() system.render_template = Mock(return_value="
      Test Template HTML
      ") + model_data['category'] = 'test' return XmlDescriptor(runtime=system, model_data=model_data).editable_metadata_fields def get_descriptor(self, model_data): diff --git a/common/lib/xmodule/xmodule/video_module.py b/common/lib/xmodule/xmodule/video_module.py index 3c6203107d..1d9ad35135 100644 --- a/common/lib/xmodule/xmodule/video_module.py +++ b/common/lib/xmodule/xmodule/video_module.py @@ -21,6 +21,17 @@ log = logging.getLogger(__name__) class VideoFields(object): """Fields for `VideoModule` and `VideoDescriptor`.""" + display_name = String( + display_name="Display Name", + help="This name appears in the horizontal navigation at the top of the page.", + scope=Scope.settings, + # it'd be nice to have a useful default but it screws up other things; so, + # use display_name_with_default for those + default="Video Title" + ) + data = String(help="XML data for the problem", + default='', + scope=Scope.content) position = Integer(help="Current position in the video", scope=Scope.user_state, default=0) show_captions = Boolean(help="This controls whether or not captions are shown by default.", display_name="Show Captions", scope=Scope.settings, default=True) youtube_id_1_0 = String(help="This is the Youtube ID reference for the normal speed video.", display_name="Default Speed", scope=Scope.settings, default="OEoXaMPEzfM") @@ -86,7 +97,6 @@ class VideoDescriptor(VideoFields, MetadataOnlyEditingDescriptor, RawDescriptor): module_class = VideoModule - template_dir_name = "video" def __init__(self, *args, **kwargs): super(VideoDescriptor, self).__init__(*args, **kwargs) @@ -115,9 +125,16 @@ class VideoDescriptor(VideoFields, url identifiers """ video = super(VideoDescriptor, cls).from_xml(xml_data, system, org, course) - _parse_video_xml(video, xml_data) + _parse_video_xml(video, video.data) return video + def definition_to_xml(self, resource_fs): + """ + Override the base implementation. We don't actually have anything in the 'data' field + (it's an empty string), so we just return a simple XML element + """ + return etree.Element('video') + def _parse_video_xml(video, xml_data): """ diff --git a/common/lib/xmodule/xmodule/videoalpha_module.py b/common/lib/xmodule/xmodule/videoalpha_module.py index 3b5b90e674..d8ed8949f1 100644 --- a/common/lib/xmodule/xmodule/videoalpha_module.py +++ b/common/lib/xmodule/xmodule/videoalpha_module.py @@ -28,15 +28,27 @@ from xblock.core import Integer, Scope, String import datetime import time +import textwrap log = logging.getLogger(__name__) class VideoAlphaFields(object): """Fields for `VideoAlphaModule` and `VideoAlphaDescriptor`.""" - data = String(help="XML data for the problem", scope=Scope.content) + data = String(help="XML data for the problem", + default=textwrap.dedent('''\ + + + + + '''), + scope=Scope.content) position = Integer(help="Current position in the video", scope=Scope.user_state, default=0) - display_name = String(help="Display name for this module", scope=Scope.settings) + display_name = String( + display_name="Display Name", help="Display name for this module", + default="Video Alpha", + scope=Scope.settings + ) class VideoAlphaModule(VideoAlphaFields, XModule): @@ -167,4 +179,3 @@ class VideoAlphaModule(VideoAlphaFields, XModule): class VideoAlphaDescriptor(VideoAlphaFields, RawDescriptor): """Descriptor for `VideoAlphaModule`.""" module_class = VideoAlphaModule - template_dir_name = "videoalpha" diff --git a/common/lib/xmodule/xmodule/word_cloud_module.py b/common/lib/xmodule/xmodule/word_cloud_module.py index a7f3f92795..004e6ed320 100644 --- a/common/lib/xmodule/xmodule/word_cloud_module.py +++ b/common/lib/xmodule/xmodule/word_cloud_module.py @@ -14,7 +14,7 @@ from xmodule.raw_module import RawDescriptor from xmodule.editing_module import MetadataOnlyEditingDescriptor from xmodule.x_module import XModule -from xblock.core import Scope, Dict, Boolean, List, Integer +from xblock.core import Scope, Dict, Boolean, List, Integer, String log = logging.getLogger(__name__) @@ -31,6 +31,12 @@ def pretty_bool(value): class WordCloudFields(object): """XFields for word cloud.""" + display_name = String( + display_name="Display Name", + help="Display name for this module", + scope=Scope.settings, + default="Word cloud" + ) num_inputs = Integer( display_name="Inputs", help="Number of text boxes available for students to input words/sentences.", @@ -234,7 +240,7 @@ class WordCloudModule(WordCloudFields, XModule): return self.content -class WordCloudDescriptor(MetadataOnlyEditingDescriptor, RawDescriptor, WordCloudFields): +class WordCloudDescriptor(WordCloudFields, MetadataOnlyEditingDescriptor, RawDescriptor): """Descriptor for WordCloud Xmodule.""" module_class = WordCloudModule template_dir_name = 'word_cloud' diff --git a/common/lib/xmodule/xmodule/x_module.py b/common/lib/xmodule/xmodule/x_module.py index 0f5bbf4f2e..47642e9f90 100644 --- a/common/lib/xmodule/xmodule/x_module.py +++ b/common/lib/xmodule/xmodule/x_module.py @@ -7,10 +7,12 @@ from lxml import etree from collections import namedtuple from pkg_resources import resource_listdir, resource_string, resource_isdir -from xmodule.modulestore import Location -from xmodule.modulestore.exceptions import ItemNotFoundError +from xmodule.modulestore import inheritance, Location +from xmodule.modulestore.exceptions import ItemNotFoundError, InsufficientSpecificationError, InvalidLocationError from xblock.core import XBlock, Scope, String, Integer, Float, ModelType +from xblock.fragment import Fragment +from xmodule.modulestore.locator import BlockUsageLocator log = logging.getLogger(__name__) @@ -27,7 +29,13 @@ class LocationField(ModelType): """ Parse the json value as a Location """ - return Location(value) + try: + return Location(value) + except InvalidLocationError: + if isinstance(value, BlockUsageLocator): + return value + else: + return BlockUsageLocator(value) def to_json(self, value): """ @@ -101,6 +109,8 @@ class XModuleFields(object): display_name="Display Name", help="This name appears in the horizontal navigation at the top of the page.", scope=Scope.settings, + # it'd be nice to have a useful default but it screws up other things; so, + # use display_name_with_default for those default=None ) @@ -113,6 +123,14 @@ class XModuleFields(object): scope=Scope.content, default=Location(None), ) + # Please note that in order to be compatible with XBlocks more generally, + # the LMS and CMS shouldn't be using this field. It's only for internal + # consumption by the XModules themselves + category = String( + display_name="xmodule category", + help="This is the category id for the XModule. It's for internal use only", + scope=Scope.content, + ) class XModule(XModuleFields, HTMLSnippet, XBlock): @@ -148,8 +166,20 @@ class XModule(XModuleFields, HTMLSnippet, XBlock): self._model_data = model_data self.system = runtime self.descriptor = descriptor - self.url_name = self.location.name - self.category = self.location.category + # LMS tests don't require descriptor but really it's required + if descriptor: + self.url_name = descriptor.url_name + # don't need to set category as it will automatically get from descriptor + elif isinstance(self.location, Location): + self.url_name = self.location.name + if not hasattr(self, 'category'): + self.category = self.location.category + elif isinstance(self.location, BlockUsageLocator): + self.url_name = self.location.usage_id + if not hasattr(self, 'category'): + raise InsufficientSpecificationError() + else: + raise InsufficientSpecificationError() self._loaded_children = None @property @@ -173,6 +203,13 @@ class XModule(XModuleFields, HTMLSnippet, XBlock): ''' if self._loaded_children is None: child_descriptors = self.get_child_descriptors() + + # This deliberately uses system.get_module, rather than runtime.get_block, + # because we're looking at XModule children, rather than XModuleDescriptor children. + # That means it can use the deprecated XModule apis, rather than future XBlock apis + + # TODO: Once we're in a system where this returns a mix of XModuleDescriptors + # and XBlocks, we're likely to have to change this more children = [self.system.get_module(descriptor) for descriptor in child_descriptors] # get_module returns None if the current user doesn't have access # to the location. @@ -278,6 +315,19 @@ class XModule(XModuleFields, HTMLSnippet, XBlock): return "" + # ~~~~~~~~~~~~~~~ XBlock API Wrappers ~~~~~~~~~~~~~~~~ + def student_view(self, context): + """ + Return a fragment with the html from this XModule + + Doesn't yet add any of the javascript to the fragment, nor the css. + Also doesn't expect any javascript binding, yet. + + Makes no use of the context parameter + """ + return Fragment(self.get_html()) + + def policy_key(location): """ Get the key for a location in a policy file. (Since the policy file is @@ -290,36 +340,67 @@ Template = namedtuple("Template", "metadata data children") class ResourceTemplates(object): + """ + Gets the templates associated w/ a containing cls. The cls must have a 'template_dir_name' attribute. + It finds the templates as directly in this directory under 'templates'. + """ @classmethod def templates(cls): """ - Returns a list of Template objects that describe possible templates that can be used - to create a module of this type. - If no templates are provided, there will be no way to create a module of - this type + Returns a list of dictionary field: value objects that describe possible templates that can be used + to seed a module of this type. Expects a class attribute template_dir_name that defines the directory inside the 'templates' resource directory to pull templates from """ templates = [] - dirname = os.path.join('templates', cls.template_dir_name) - if not resource_isdir(__name__, dirname): - log.warning("No resource directory {dir} found when loading {cls_name} templates".format( - dir=dirname, - cls_name=cls.__name__, - )) - return [] - - for template_file in resource_listdir(__name__, dirname): - if not template_file.endswith('.yaml'): - log.warning("Skipping unknown template file %s" % template_file) - continue - template_content = resource_string(__name__, os.path.join(dirname, template_file)) - template = yaml.safe_load(template_content) - templates.append(Template(**template)) + dirname = cls.get_template_dir() + if dirname is not None: + for template_file in resource_listdir(__name__, dirname): + if not template_file.endswith('.yaml'): + log.warning("Skipping unknown template file %s", template_file) + continue + template_content = resource_string(__name__, os.path.join(dirname, template_file)) + template = yaml.safe_load(template_content) + template['template_id'] = template_file + templates.append(template) return templates + @classmethod + def get_template_dir(cls): + if getattr(cls, 'template_dir_name', None): + dirname = os.path.join('templates', getattr(cls, 'template_dir_name')) + if not resource_isdir(__name__, dirname): + log.warning("No resource directory {dir} found when loading {cls_name} templates".format( + dir=dirname, + cls_name=cls.__name__, + )) + return None + else: + return dirname + else: + return None + + @classmethod + def get_template(cls, template_id): + """ + Get a single template by the given id (which is the file name identifying it w/in the class's + template_dir_name) + + """ + dirname = cls.get_template_dir() + if dirname is not None: + try: + template_content = resource_string(__name__, os.path.join(dirname, template_id)) + except IOError: + return None + template = yaml.safe_load(template_content) + template['template_id'] = template_id + return template + else: + return None + class XModuleDescriptor(XModuleFields, HTMLSnippet, ResourceTemplates, XBlock): """ @@ -346,9 +427,6 @@ class XModuleDescriptor(XModuleFields, HTMLSnippet, ResourceTemplates, XBlock): # be equal equality_attributes = ('_model_data', 'location') - # Name of resource directory to load templates from - template_dir_name = "default" - # Class level variable # True if this descriptor always requires recalculation of grades, for @@ -386,8 +464,21 @@ class XModuleDescriptor(XModuleFields, HTMLSnippet, ResourceTemplates, XBlock): """ super(XModuleDescriptor, self).__init__(*args, **kwargs) self.system = self.runtime - self.url_name = self.location.name - self.category = self.location.category + if isinstance(self.location, Location): + self.url_name = self.location.name + if not hasattr(self, 'category'): + self.category = self.location.category + elif isinstance(self.location, BlockUsageLocator): + self.url_name = self.location.usage_id + if not hasattr(self, 'category'): + raise InsufficientSpecificationError() + else: + raise InsufficientSpecificationError() + # update_version is the version which last updated this xblock v prev being the penultimate updater + # leaving off original_version since it complicates creation w/o any obv value yet and is computable + # by following previous until None + # definition_locator is only used by mongostores which separate definitions from blocks + self.edited_by = self.edited_on = self.previous_version = self.update_version = self.definition_locator = None self._child_instances = None @property @@ -419,11 +510,14 @@ class XModuleDescriptor(XModuleFields, HTMLSnippet, ResourceTemplates, XBlock): if self._child_instances is None: self._child_instances = [] for child_loc in self.children: - try: - child = self.system.load_item(child_loc) - except ItemNotFoundError: - log.exception('Unable to load item {loc}, skipping'.format(loc=child_loc)) - continue + if isinstance(child_loc, XModuleDescriptor): + child = child_loc + else: + try: + child = self.runtime.get_block(child_loc) + except ItemNotFoundError: + log.exception('Unable to load item {loc}, skipping'.format(loc=child_loc)) + continue self._child_instances.append(child) return self._child_instances @@ -461,22 +555,30 @@ class XModuleDescriptor(XModuleFields, HTMLSnippet, ResourceTemplates, XBlock): # ================================= JSON PARSING =========================== @staticmethod - def load_from_json(json_data, system, default_class=None): + def load_from_json(json_data, system, default_class=None, parent_xblock=None): """ This method instantiates the correct subclass of XModuleDescriptor based - on the contents of json_data. + on the contents of json_data. It does not persist it and can create one which + has no usage id. - json_data must contain a 'location' element, and must be suitable to be - passed into the subclasses `from_json` method as model_data + parent_xblock is used to compute inherited metadata as well as to append the new xblock. + + json_data: + - 'location' : must have this field + - 'category': the xmodule category (required or location must be a Location) + - 'metadata': a dict of locally set metadata (not inherited) + - 'children': a list of children's usage_ids w/in this course + - 'definition': + - '_id' (optional): the usage_id of this. Will generate one if not given one. """ class_ = XModuleDescriptor.load_class( - json_data['location']['category'], + json_data.get('category', json_data.get('location', {}).get('category')), default_class ) - return class_.from_json(json_data, system) + return class_.from_json(json_data, system, parent_xblock) @classmethod - def from_json(cls, json_data, system): + def from_json(cls, json_data, system, parent_xblock=None): """ Creates an instance of this descriptor from the supplied json_data. This may be overridden by subclasses @@ -494,28 +596,25 @@ class XModuleDescriptor(XModuleFields, HTMLSnippet, ResourceTemplates, XBlock): Otherwise, it contains the single field 'data' 4) Any value later in this list overrides a value earlier in this list - system: A DescriptorSystem for interacting with external resources + json_data: + - 'category': the xmodule category (required) + - 'metadata': a dict of locally set metadata (not inherited) + - 'children': a list of children's usage_ids w/in this course + - 'definition': + - '_id' (optional): the usage_id of this. Will generate one if not given one. """ - model_data = {} + usage_id = json_data.get('_id', None) + if not '_inherited_metadata' in json_data and parent_xblock is not None: + json_data['_inherited_metadata'] = parent_xblock.xblock_kvs.get_inherited_metadata().copy() + json_metadata = json_data.get('metadata', {}) + for field in inheritance.INHERITABLE_METADATA: + if field in json_metadata: + json_data['_inherited_metadata'][field] = json_metadata[field] - for key, value in json_data.get('metadata', {}).items(): - model_data[cls._translate(key)] = value - - model_data.update(json_data.get('metadata', {})) - - definition = json_data.get('definition', {}) - if 'children' in definition: - model_data['children'] = definition['children'] - - if 'data' in definition: - if isinstance(definition['data'], dict): - model_data.update(definition['data']) - else: - model_data['data'] = definition['data'] - - model_data['location'] = json_data['location'] - - return cls(system, model_data) + new_block = system.xblock_from_json(cls, usage_id, json_data) + if parent_xblock is not None: + parent_xblock.children.append(new_block) + return new_block @classmethod def _translate(cls, key): @@ -591,6 +690,15 @@ class XModuleDescriptor(XModuleFields, HTMLSnippet, ResourceTemplates, XBlock): """ return [('{}', '{}')] + @property + def xblock_kvs(self): + """ + Use w/ caution. Really intended for use by the persistence layer. + """ + # if caller wants kvs, caller's assuming it's up to date; so, decache it + self.save() + return self._model_data._kvs + # =============================== BUILTIN METHODS ========================== def __eq__(self, other): eq = (self.__class__ == other.__class__ and @@ -720,6 +828,10 @@ class DescriptorSystem(object): self.resources_fs = resources_fs self.error_tracker = error_tracker + def get_block(self, block_id): + """See documentation for `xblock.runtime:Runtime.get_block`""" + return self.load_item(block_id) + class XMLParsingSystem(DescriptorSystem): def __init__(self, load_item, resources_fs, error_tracker, process_xml, policy, **kwargs): @@ -808,8 +920,8 @@ class ModuleSystem(object): publish(event) - A function that allows XModules to publish events (such as grade changes) - xblock_model_data - A dict-like object containing the all data available to this - xblock + xblock_model_data - A function that constructs a model_data for an xblock from its + corresponding descriptor cache - A cache object with two methods: .get(key) returns an object from the cache or None. diff --git a/common/lib/xmodule/xmodule/xml_module.py b/common/lib/xmodule/xmodule/xml_module.py index 0528bbfb6c..5b8d2c8aee 100644 --- a/common/lib/xmodule/xmodule/xml_module.py +++ b/common/lib/xmodule/xmodule/xml_module.py @@ -306,6 +306,7 @@ class XmlDescriptor(XModuleDescriptor): org and course are optional strings that will be used in the generated modules url identifiers """ + xml_object = etree.fromstring(xml_data) # VS[compat] -- just have the url_name lookup, once translation is done url_name = xml_object.get('url_name', xml_object.get('slug')) @@ -318,7 +319,8 @@ class XmlDescriptor(XModuleDescriptor): filepath = cls._format_filepath(xml_object.tag, name_to_pathname(url_name)) definition_xml = cls.load_file(filepath, system.resources_fs, location) else: - definition_xml = xml_object # this is just a pointer, not the real definition content + definition_xml = xml_object + filepath = None definition, children = cls.load_definition(definition_xml, system, location) # note this removes metadata @@ -356,6 +358,7 @@ class XmlDescriptor(XModuleDescriptor): if key not in set(f.name for f in cls.fields + cls.lms.fields): model_data['xml_attributes'][key] = value model_data['location'] = location + model_data['category'] = xml_object.tag return cls( system, diff --git a/common/static/css/pdfviewer.css b/common/static/css/pdfviewer.css index 656bc47c29..8b0253261b 100644 --- a/common/static/css/pdfviewer.css +++ b/common/static/css/pdfviewer.css @@ -100,7 +100,7 @@ select { .toolbar { /* position: absolute; */ left: 0; - right: 0; + right: 0; height: 32px; z-index: 9999; cursor: default; @@ -185,6 +185,7 @@ select { margin: 0; } +.splitToolbarButton > .toolbarButton, /*added */ .splitToolbarButton:hover > .toolbarButton, .splitToolbarButton:focus > .toolbarButton, .splitToolbarButton.toggled > .toolbarButton, diff --git a/common/static/js/capa/choicetextinput.js b/common/static/js/capa/choicetextinput.js new file mode 100644 index 0000000000..4d7540f938 --- /dev/null +++ b/common/static/js/capa/choicetextinput.js @@ -0,0 +1,75 @@ +(function () { + var update = function () { + // Whenever a value changes create a new serialized version of this + // problem's inputs and set the hidden input fields value to equal it. + var parent = $(this).closest('.problems-wrapper'); + // find the closest parent problems-wrapper and use that as the problem + // grab the input id from the input + // real_input is the hidden input field + var real_input = $('input.choicetextvalue', parent); + var all_inputs = $('.choicetextinput .ctinput', parent); + var user_inputs = {}; + $(all_inputs).each(function (index, elt) { + var node = $(elt); + var name = node.attr('id'); + var val = node.val(); + var radio_value = node.attr('value'); + var type = node.attr('type'); + var is_checked = node.attr('checked'); + if (type === "radio" || type === "checkbox") { + if (is_checked === "checked" || is_checked === "true") { + user_inputs[name] = radio_value; + } + } else { + user_inputs[name] = val; + } + }); + var val_string = JSON.stringify(user_inputs); + //this is what gets submitted as the answer, we deserialize it later + real_input.val(val_string); + }; + + var check_parent = function (event) { + // This looks for the containing choice of a textinput + // and sets it to be checked. + var elt = $(event.target); + var parent_container = elt.closest('section[id^="forinput"]'); + var choice = parent_container.find("input[type='checkbox'], input[type='radio']"); + choice.attr("checked", "checked"); + choice.change(); + //need to check it then trigger the change event + }; + + var imitate_label = function (event) { + // This causes a section to check and uncheck + // a radiobutton/checkbox whenever a user clicks on it + // If the button/checkbox is disabled, nothing happens + var elt = $(event.target); + var parent_container = elt.closest('section[id^="forinput"]'); + var choice = parent_container.find("input[type='checkbox'], input[type='radio']"); + if (choice.attr("type") === "radio") { + choice.attr("checked", "checked"); + } else { + if (choice.attr('checked')) { + choice.prop("checked", false); + } else { + choice.prop("checked", true); + } + + } + choice.change(); + update(); + + }; + var choices = $('.mock_label'); + var inputs = $('.choicetextinput .ctinput'); + var text_inputs = $('.choicetextinput .ctinput[type="text"]'); + // update on load + inputs.each(update); + // and on every change + // This allows text inside of choices to behave as if they were part of + // a label for the choice's button/checkbox + choices.click(imitate_label); + inputs.bind("change", update); + text_inputs.click(check_parent); +}).call(this); diff --git a/common/templates/mathjax_include.html b/common/templates/mathjax_include.html index 803f2145a4..0ddbd68eee 100644 --- a/common/templates/mathjax_include.html +++ b/common/templates/mathjax_include.html @@ -33,4 +33,4 @@ - + diff --git a/common/test/data/open_ended/combinedopenended/SampleQuestion1Attempt.xml b/common/test/data/open_ended/combinedopenended/SampleQuestion1Attempt.xml new file mode 100644 index 0000000000..9bfabca191 --- /dev/null +++ b/common/test/data/open_ended/combinedopenended/SampleQuestion1Attempt.xml @@ -0,0 +1,24 @@ + + + + + Writing Applications + + + + + Language Conventions + + + + + + +

      Censorship in the Libraries

      +

      "All of us can think of a book that we hope none of our children or any other children have taken off the shelf. But if I have the right to remove that book from the shelf -- that work I abhor -- then you also have exactly the same right and so does everyone else. And then we have no books left on the shelf for any of us." --Katherine Paterson, Author

      +

      Write a persuasive essay to a newspaper reflecting your vies on censorship in libraries. Do you believe that certain materials, such as books, music, movies, magazines, etc., should be removed from the shelves if they are found offensive? Support your position with convincing arguments from your own experience, observations, and/or reading.

      +       + + + +       \ No newline at end of file diff --git a/common/test/data/open_ended/course/2012_Fall.xml b/common/test/data/open_ended/course/2012_Fall.xml index 32c810174b..609d12f94c 100644 --- a/common/test/data/open_ended/course/2012_Fall.xml +++ b/common/test/data/open_ended/course/2012_Fall.xml @@ -1,6 +1,7 @@ + diff --git a/common/test/data/splitmongo_json/active_versions.json b/common/test/data/splitmongo_json/active_versions.json new file mode 100644 index 0000000000..b41440e0e7 --- /dev/null +++ b/common/test/data/splitmongo_json/active_versions.json @@ -0,0 +1,27 @@ +[{"_id" : "GreekHero", + "org" : "testx", + "prettyid" : "test_course", + "versions" : { + "draft" : { "$oid" : "1d00000000000000dddd0000" } + }, + "edited_on" : {"$date" : 1364481713238}, + "edited_by" : "test@edx.org"}, + + {"_id" : "wonderful", + "org" : "testx", + "prettyid" : "another_course", + "versions" : { + "draft" : { "$oid" : "1d00000000000000dddd2222" }, + "published" : { "$oid" : "1d00000000000000eeee0000" } + }, + "edited_on" : {"$date" : 1364481313238}, + "edited_by" : "test@edx.org"}, + + {"_id" : "contender", + "org" : "guestx", + "prettyid" : "test_course", + "versions" : { + "draft" : { "$oid" : "1d00000000000000dddd5555" }}, + "edited_on" : {"$date" : 1364491313238}, + "edited_by" : "test@guestx.edu"} +] diff --git a/common/test/data/splitmongo_json/definitions.json b/common/test/data/splitmongo_json/definitions.json new file mode 100644 index 0000000000..0ed42112aa --- /dev/null +++ b/common/test/data/splitmongo_json/definitions.json @@ -0,0 +1,334 @@ +[ + { + "_id":"head12345_12", + "category":"course", + "data":{ + "textbooks":[ + + ], + "grading_policy":{ + "GRADER":[ + { + "min_count":4, + "weight":0.15, + "type":"Homework", + "drop_count":2, + "short_label":"HWa" + }, + { + "short_label":"", + "min_count":12, + "type":"Lab", + "drop_count":2, + "weight":0.15 + }, + { + "short_label":"Midterm", + "min_count":1, + "type":"Midterm Exam", + "drop_count":0, + "weight":0.3 + }, + { + "short_label":"Final", + "min_count":1, + "type":"Final Exam", + "drop_count":0, + "weight":0.4 + } + ], + "GRADE_CUTOFFS":{ + "Pass":0.45 + } + }, + "wiki_slug":null + }, + "edited_by":"testassist@edx.org", + "edited_on":{"$date" : 1364481713238}, + "previous_version":"head12345_11", + "original_version":"head12345_10" + }, + { + "_id":"head12345_11", + "category":"course", + "data":{ + "textbooks":[ + + ], + "grading_policy":{ + "GRADER":[ + { + "min_count":5, + "weight":0.15, + "type":"Homework", + "drop_count":1, + "short_label":"HWa" + }, + { + "short_label":"", + "min_count":12, + "type":"Lab", + "drop_count":2, + "weight":0.15 + }, + { + "short_label":"Midterm", + "min_count":1, + "type":"Midterm Exam", + "drop_count":0, + "weight":0.3 + }, + { + "short_label":"Final", + "min_count":1, + "type":"Final Exam", + "drop_count":0, + "weight":0.4 + } + ], + "GRADE_CUTOFFS":{ + "Pass":0.55 + } + }, + "wiki_slug":null + }, + "edited_by":"testassist@edx.org", + "edited_on":{"$date" : 1364481713238}, + "previous_version":"head12345_10", + "original_version":"head12345_10" + }, + { + "_id":"head12345_10", + "category":"course", + "data":{ + "textbooks":[ + + ], + "grading_policy":{ + "GRADER":[ + { + "min_count":5, + "weight":0.15, + "type":"Homework", + "drop_count":1, + "short_label":"HWa" + }, + { + "short_label":"", + "min_count":2, + "type":"Lab", + "drop_count":0, + "weight":0.15 + }, + { + "short_label":"Midterm", + "min_count":1, + "type":"Midterm Exam", + "drop_count":0, + "weight":0.3 + }, + { + "short_label":"Final", + "min_count":1, + "type":"Final Exam", + "drop_count":0, + "weight":0.4 + } + ], + "GRADE_CUTOFFS":{ + "Pass":0.75 + } + }, + "wiki_slug":null + }, + "edited_by":"test@edx.org", + "edited_on":{"$date": 1364473713238}, + "previous_version":null, + "original_version":"head12345_10" + }, + { + "_id":"head23456_1", + "category":"course", + "data":{ + "textbooks":[ + + ], + "grading_policy":{ + "GRADER":[ + { + "min_count":14, + "weight":0.25, + "type":"Homework", + "drop_count":1, + "short_label":"HWa" + }, + { + "short_label":"", + "min_count":12, + "type":"Lab", + "drop_count":2, + "weight":0.25 + }, + { + "short_label":"Midterm", + "min_count":1, + "type":"Midterm Exam", + "drop_count":0, + "weight":0.2 + }, + { + "short_label":"Final", + "min_count":1, + "type":"Final Exam", + "drop_count":0, + "weight":0.3 + } + ], + "GRADE_CUTOFFS":{ + "Pass":0.45 + } + }, + "wiki_slug":null + }, + "edited_by":"test@edx.org", + "edited_on":{"$date": 1364481313238}, + "previous_version":"head23456_0", + "original_version":"head23456_0" + }, + { + "_id":"head23456_0", + "category":"course", + "data":{ + "textbooks":[ + + ], + "grading_policy":{ + "GRADER":[ + { + "min_count":14, + "weight":0.25, + "type":"Homework", + "drop_count":1, + "short_label":"HWa" + }, + { + "short_label":"", + "min_count":12, + "type":"Lab", + "drop_count":2, + "weight":0.25 + }, + { + "short_label":"Midterm", + "min_count":1, + "type":"Midterm Exam", + "drop_count":0, + "weight":0.2 + }, + { + "short_label":"Final", + "min_count":1, + "type":"Final Exam", + "drop_count":0, + "weight":0.3 + } + ], + "GRADE_CUTOFFS":{ + "Pass":0.95 + } + }, + "wiki_slug":null + }, + "edited_by":"test@edx.org", + "edited_on":{"$date" : 1364481313238}, + "previous_version":null, + "original_version":"head23456_0" + }, + { + "_id":"head345679_1", + "category":"course", + "data":{ + "textbooks":[ + + ], + "grading_policy":{ + "GRADER":[ + { + "min_count":4, + "weight":0.25, + "type":"Homework", + "drop_count":0, + "short_label":"HW" + }, + { + "short_label":"Midterm", + "min_count":1, + "type":"Midterm Exam", + "drop_count":0, + "weight":0.4 + }, + { + "short_label":"Final", + "min_count":1, + "type":"Final Exam", + "drop_count":0, + "weight":0.35 + } + ], + "GRADE_CUTOFFS":{ + "Pass":0.25 + } + }, + "wiki_slug":null + }, + "edited_by":"test@edx.org", + "edited_on":{"$date" : 1364481313238}, + "previous_version":null, + "original_version":"head23456_0" + }, + { + "_id":"chapter12345_1", + "category":"chapter", + "data":null, + "edited_by":"testassist@edx.org", + "edited_on":{"$date" : 1364483713238}, + "previous_version":null, + "original_version":"chapter12345_1" + }, + { + "_id":"chapter12345_2", + "category":"chapter", + "data":null, + "edited_by":"testassist@edx.org", + "edited_on":{"$date" : 1364483713238}, + "previous_version":null, + "original_version":"chapter12345_2" + }, + { + "_id":"chapter12345_3", + "category":"chapter", + "data":null, + "edited_by":"testassist@edx.org", + "edited_on":{"$date" : 1364483713238}, + "previous_version":null, + "original_version":"chapter12345_3" + }, + { + "_id":"problem12345_3_1", + "category":"problem", + "data":"", + "edited_by":"testassist@edx.org", + "edited_on":{"$date" : 1364483713238}, + "previous_version":null, + "original_version":"problem12345_3_1" + }, + { + "_id":"problem12345_3_2", + "category":"problem", + "data":"", + "edited_by":"testassist@edx.org", + "edited_on":{"$date" : 1364483713238}, + "previous_version":null, + "original_version":"problem12345_3_2" + } +] \ No newline at end of file diff --git a/common/test/data/splitmongo_json/structures.json b/common/test/data/splitmongo_json/structures.json new file mode 100644 index 0000000000..0021225213 --- /dev/null +++ b/common/test/data/splitmongo_json/structures.json @@ -0,0 +1,471 @@ +[ + { + "_id": { "$oid" : "1d00000000000000dddd0000"}, + "root":"head12345", + "original_version":{ "$oid" : "1d00000000000000dddd3333" }, + "previous_version":{ "$oid" : "1d00000000000000dddd1111" }, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364483713238 + }, + "blocks":{ + "head12345":{ + "children":[ + "chapter1", + "chapter2", + "chapter3" + ], + "category":"course", + "definition":"head12345_12", + "metadata":{ + "end":"2013-06-13T04:30", + "tabs":[ + { + "type":"courseware" + }, + { + "type":"course_info", + "name":"Course Info" + }, + { + "type":"discussion", + "name":"Discussion" + }, + { + "type":"wiki", + "name":"Wiki" + }, + { + "type":"static_tab", + "name":"Syllabus", + "url_slug":"01356a17b5924b17a04b7fc2426a3798" + }, + { + "type":"static_tab", + "name":"Advice for Students", + "url_slug":"57e9991c0d794ff58f7defae3e042e39" + } + ], + "enrollment_start":"2013-01-01T05:00", + "graceperiod":"2 hours 0 minutes 0 seconds", + "start":"2013-02-14T05:00", + "enrollment_end":"2013-03-02T05:00", + "data_dir":"MITx-2-Base", + "advertised_start":"Fall 2013", + "display_name":"The Ancient Greek Hero" + }, + "update_version":{ "$oid" : "1d00000000000000dddd0000" }, + "previous_version":{ "$oid" : "1d00000000000000dddd1111" }, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364483713238 + } + }, + "chapter1":{ + "children":[ + + ], + "category":"chapter", + "definition":"chapter12345_1", + "metadata":{ + "display_name":"Hercules" + }, + "update_version":{ "$oid" : "1d00000000000000dddd0000" }, + "previous_version":null, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364483713238 + } + }, + "chapter2":{ + "children":[ + + ], + "category":"chapter", + "definition":"chapter12345_2", + "metadata":{ + "display_name":"Hera heckles Hercules" + }, + "update_version":{ "$oid" : "1d00000000000000dddd0000" }, + "previous_version":null, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364483713238 + } + }, + "chapter3":{ + "children":[ + "problem1", + "problem3_2" + ], + "category":"chapter", + "definition":"chapter12345_3", + "metadata":{ + "display_name":"Hera cuckolds Zeus" + }, + "update_version":{ "$oid" : "1d00000000000000dddd0000" }, + "previous_version":null, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364483713238 + } + }, + "problem1":{ + "children":[ + + ], + "category":"problem", + "definition":"problem12345_3_1", + "metadata":{ + "display_name":"Problem 3.1", + "graceperiod":"4 hours 0 minutes 0 seconds" + }, + "update_version":{ "$oid" : "1d00000000000000dddd0000" }, + "previous_version":null, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364483713238 + } + }, + "problem3_2":{ + "children":[ + + ], + "category":"problem", + "definition":"problem12345_3_2", + "metadata":{ + "display_name":"Problem 3.2" + }, + "update_version":{ "$oid" : "1d00000000000000dddd0000" }, + "previous_version":null, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364483713238 + } + } + } + }, + { + "_id": { "$oid" : "1d00000000000000dddd1111"}, + "root":"head12345", + "original_version":{ "$oid" : "1d00000000000000dddd3333" }, + "previous_version":{ "$oid" : "1d00000000000000dddd3333" }, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364481713238 + }, + "blocks":{ + "head12345":{ + "children":[ + + ], + "category":"course", + "definition":"head12345_11", + "metadata":{ + "end":"2013-04-13T04:30", + "tabs":[ + { + "type":"courseware" + }, + { + "type":"course_info", + "name":"Course Info" + }, + { + "type":"discussion", + "name":"Discussion" + }, + { + "type":"wiki", + "name":"Wiki" + }, + { + "type":"static_tab", + "name":"Syllabus", + "url_slug":"01356a17b5924b17a04b7fc2426a3798" + }, + { + "type":"static_tab", + "name":"Advice for Students", + "url_slug":"57e9991c0d794ff58f7defae3e042e39" + } + ], + "enrollment_start":null, + "graceperiod":"2 hours 0 minutes 0 seconds", + "start":"2013-02-14T05:00", + "enrollment_end":null, + "data_dir":"MITx-2-Base", + "advertised_start":null, + "display_name":"The Ancient Greek Hero" + }, + "update_version":{ "$oid" : "1d00000000000000dddd1111" }, + "previous_version":{ "$oid" : "1d00000000000000dddd3333" }, + "edited_by":"testassist@edx.org", + "edited_on":{ + "$date":1364481713238 + } + } + } + }, + { + "_id": { "$oid" : "1d00000000000000dddd3333"}, + "root":"head12345", + "original_version":{ "$oid" : "1d00000000000000dddd3333" }, + "previous_version":null, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364473713238 + }, + "blocks":{ + "head12345":{ + "children":[ + + ], + "category":"course", + "definition":"head12345_10", + "metadata":{ + "end":null, + "tabs":[ + { + "type":"courseware" + }, + { + "type":"course_info", + "name":"Course Info" + }, + { + "type":"discussion", + "name":"Discussion" + }, + { + "type":"wiki", + "name":"Wiki" + } + ], + "enrollment_start":null, + "graceperiod":null, + "start":"2013-02-14T05:00", + "enrollment_end":null, + "data_dir":"MITx-2-Base", + "advertised_start":null, + "display_name":"The Ancient Greek Hero" + }, + "update_version":{ "$oid" : "1d00000000000000dddd3333" }, + "previous_version":null, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364473713238 + } + } + } + }, + { + "_id": { "$oid" : "1d00000000000000dddd2222"}, + "root":"head23456", + "original_version":{ "$oid" : "1d00000000000000dddd4444" }, + "previous_version":{ "$oid" : "1d00000000000000dddd4444" }, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364481313238 + }, + "blocks":{ + "head23456":{ + "children":[ + + ], + "category":"course", + "definition":"head23456_1", + "metadata":{ + "end":null, + "tabs":[ + { + "type":"courseware" + }, + { + "type":"course_info", + "name":"Course Info" + }, + { + "type":"discussion", + "name":"Discussion" + }, + { + "type":"wiki", + "name":"Wiki" + } + ], + "enrollment_start":null, + "graceperiod":null, + "start":"2013-02-14T05:00", + "enrollment_end":null, + "data_dir":"MITx-2-Base", + "advertised_start":null, + "display_name":"The most wonderful course" + }, + "update_version":{ "$oid" : "1d00000000000000dddd2222" }, + "previous_version":{ "$oid" : "1d00000000000000dddd4444" }, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364481313238 + } + + } + } + }, + { + "_id": { "$oid" : "1d00000000000000dddd4444"}, + "root":"head23456", + "original_version":{ "$oid" : "1d00000000000000dddd4444" }, + "previous_version":null, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364480313238 + }, + "blocks":{ + "head23456":{ + "children":[ + + ], + "category":"course", + "definition":"head23456_0", + "metadata":{ + "end":null, + "tabs":[ + { + "type":"courseware" + }, + { + "type":"course_info", + "name":"Course Info" + }, + { + "type":"discussion", + "name":"Discussion" + }, + { + "type":"wiki", + "name":"Wiki" + } + ], + "enrollment_start":null, + "graceperiod":null, + "start":"2013-02-14T05:00", + "enrollment_end":null, + "data_dir":"MITx-2-Base", + "advertised_start":null, + "display_name":"A wonderful course" + }, + "update_version":{ "$oid" : "1d00000000000000dddd4444" }, + "previous_version":null, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364480313238 + } + } + } + }, + { + "_id": { "$oid" : "1d00000000000000eeee0000"}, + "root":"head23456", + "original_version":{ "$oid" : "1d00000000000000eeee0000" }, + "previous_version":null, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364481333238 + }, + "blocks":{ + "head23456":{ + "children":[ + + ], + "category":"course", + "definition":"head23456_1", + "metadata":{ + "end":null, + "tabs":[ + { + "type":"courseware" + }, + { + "type":"course_info", + "name":"Course Info" + }, + { + "type":"discussion", + "name":"Discussion" + }, + { + "type":"wiki", + "name":"Wiki" + } + ], + "enrollment_start":null, + "graceperiod":null, + "start":"2013-02-14T05:00", + "enrollment_end":null, + "data_dir":"MITx-2-Base", + "advertised_start":null, + "display_name":"The most wonderful course" + }, + "update_version":{ "$oid" : "1d00000000000000eeee0000" }, + "previous_version":null, + "edited_by":"test@edx.org", + "edited_on":{ + "$date":1364481333238 + } + } + } + }, + { + "_id": { "$oid" : "1d00000000000000dddd5555"}, + "root":"head345679", + "original_version":{ "$oid" : "1d00000000000000dddd5555" }, + "previous_version":null, + "edited_by":"test@guestx.edu", + "edited_on":{ + "$date":1364491313238 + }, + "blocks":{ + "head345679":{ + "children":[ + + ], + "category":"course", + "definition":"head345679_1", + "metadata":{ + "end":null, + "tabs":[ + { + "type":"courseware" + }, + { + "type":"course_info", + "name":"Course Info" + }, + { + "type":"discussion", + "name":"Discussion" + }, + { + "type":"wiki", + "name":"Wiki" + } + ], + "enrollment_start":null, + "graceperiod":null, + "start":"2013-03-14T05:00", + "enrollment_end":null, + "data_dir":"MITx-3-Base", + "advertised_start":null, + "display_name":"Yet another contender" + }, + "update_version":{ "$oid" : "1d00000000000000dddd5555" }, + "previous_version":null, + "edited_by":"test@guestx.edu", + "edited_on":{ + "$date":1364491313238 + } + } + } + } +] diff --git a/common/test/data/toy/chapter/secret/magic.xml b/common/test/data/toy/chapter/secret/magic.xml index 62756b8896..f85d2e75da 100644 --- a/common/test/data/toy/chapter/secret/magic.xml +++ b/common/test/data/toy/chapter/secret/magic.xml @@ -1,3 +1,3 @@ - diff --git a/common/test/data/toy/course/2012_Fall.xml b/common/test/data/toy/course/2012_Fall.xml index c3500040c2..679f7bbfdb 100644 --- a/common/test/data/toy/course/2012_Fall.xml +++ b/common/test/data/toy/course/2012_Fall.xml @@ -3,10 +3,11 @@ - - diff --git a/common/test/data/toy/html/toyjumpto.html b/common/test/data/toy/html/toyjumpto.html new file mode 100644 index 0000000000..df2a5a42aa --- /dev/null +++ b/common/test/data/toy/html/toyjumpto.html @@ -0,0 +1 @@ +This is a link to another page and some Chinese 四節比分和七年前

      Some more Chinese 四節比分和七年前

      diff --git a/common/test/data/toy/html/toyjumpto.xml b/common/test/data/toy/html/toyjumpto.xml new file mode 100644 index 0000000000..af9cdf4f74 --- /dev/null +++ b/common/test/data/toy/html/toyjumpto.xml @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/common/test/data/toy/vertical/vertical_test.xml b/common/test/data/toy/vertical/vertical_test.xml index e801a4ac86..68c5745f37 100644 --- a/common/test/data/toy/vertical/vertical_test.xml +++ b/common/test/data/toy/vertical/vertical_test.xml @@ -1,4 +1,6 @@ +
      +% if show_chat: +
      +
      + Open Chat + Close Chat +
      +
      + ## The Candy.js plugin wants to render in an element with #candy +
      +
      +
      +% endif % if course.show_calculator:
      diff --git a/lms/templates/courseware/submission_history.html b/lms/templates/courseware/submission_history.html index 683c61c5a0..b4ce5ed2a6 100644 --- a/lms/templates/courseware/submission_history.html +++ b/lms/templates/courseware/submission_history.html @@ -1,5 +1,5 @@ <% import json %> -

      ${username} > ${course_id} > ${location}

      +

      ${username | h} > ${course_id | h} > ${location | h}

      % for i, entry in enumerate(history_entries):
      diff --git a/lms/templates/login.html b/lms/templates/login.html index 671ce5a3e7..5ab63a86c2 100644 --- a/lms/templates/login.html +++ b/lms/templates/login.html @@ -38,13 +38,12 @@ toggleSubmitButton(false); }); - $('#login-form').on('ajax:complete', function() { + $('#login-form').on('ajax:error', function() { toggleSubmitButton(true); }); $('#login-form').on('ajax:success', function(event, json, xhr) { if(json.success) { - $('.message.submission-error').removeClass('is-shown'); var u=decodeURI(window.location.search); next=u.split("next=")[1]; if (next) { @@ -53,6 +52,7 @@ location.href="${reverse('dashboard')}"; } } else { + toggleSubmitButton(true); $('.message.submission-error').addClass('is-shown').focus(); $('.message.submission-error .message-copy').html(json.value); } diff --git a/lms/templates/main_django.html b/lms/templates/main_django.html index f5ee06d280..da3af1935c 100644 --- a/lms/templates/main_django.html +++ b/lms/templates/main_django.html @@ -7,7 +7,9 @@ {% compressed_css 'application' %} + {% block main_vendor_js %} {% compressed_js 'main_vendor' %} + {% endblock %} {% block headextra %}{% endblock %} {% render_block "css" %} diff --git a/lms/templates/problem.html b/lms/templates/problem.html index fc49ab7ce7..f4f8e78b66 100644 --- a/lms/templates/problem.html +++ b/lms/templates/problem.html @@ -1,11 +1,11 @@ <%namespace name='static' file='static_content.html'/>

      ${ problem['name'] } - % if problem['weight'] != 1 and problem['weight'] is not None: - : ${ problem['weight'] } points - % endif

      +
      +
      +
      ${ problem['html'] } diff --git a/lms/templates/problem_ajax.html b/lms/templates/problem_ajax.html index 42cd18c4e3..1babda1ae2 100644 --- a/lms/templates/problem_ajax.html +++ b/lms/templates/problem_ajax.html @@ -1 +1 @@ -
      +
      diff --git a/lms/templates/register.html b/lms/templates/register.html index 1c2490c0a0..57a9ffa843 100644 --- a/lms/templates/register.html +++ b/lms/templates/register.html @@ -45,15 +45,15 @@ toggleSubmitButton(false); }); - $('#register-form').on('ajax:complete', function() { + $('#register-form').on('ajax:error', function() { toggleSubmitButton(true); }); $('#register-form').on('ajax:success', function(event, json, xhr) { if(json.success) { - $('.message.submission-error').removeClass('is-shown'); location.href="${reverse('dashboard')}"; } else { + toggleSubmitButton(true); $('.status.message.submission-error').addClass('is-shown').focus(); $('.status.message.submission-error .message-copy').html(json.value).stop().css("display", "block"); $(".field-error").removeClass('field-error'); diff --git a/lms/templates/static_pdfbook.html b/lms/templates/static_pdfbook.html index 565a59977a..754e519113 100644 --- a/lms/templates/static_pdfbook.html +++ b/lms/templates/static_pdfbook.html @@ -1,3 +1,4 @@ +<%! from django.utils.translation import ugettext as _ %> <%inherit file="main.html" /> <%namespace name='static' file='static_content.html'/> <%block name="title"> @@ -11,7 +12,7 @@ <%static:js group='courseware'/> - + <%block name="js_extra"> @@ -35,10 +36,10 @@ %if page is not None: options.pageNum = ${page}; %endif - + $('#outerContainer').PDFViewer(options); }); - + <%include file="/courseware/course_navigation.html" args="active_page='pdftextbook/{0}'.format(book_index)" /> @@ -91,40 +92,43 @@
      %if 'chapters' in textbook: -
      -
        - <%def name="print_entry(entry, index_value)"> -
      • - - ${entry.get('title')} - -
        • - +
        +
        +
        +
          + <%def name="print_entry(entry, index_value)"> +
        • + + ${entry.get('title')} + +
          • + - % for (index, entry) in enumerate(textbook['chapters']): - ${print_entry(entry, index+1)} - % endfor -
        -
        + % for (index, entry) in enumerate(textbook['chapters']): + ${print_entry(entry, index+1)} + % endfor +
      +
      %endif -
      +
      - -
      -
      -
      - + +
      +
      +
      + + diff --git a/lms/templates/unsubscribe.html b/lms/templates/unsubscribe.html new file mode 100644 index 0000000000..6f8c042d76 --- /dev/null +++ b/lms/templates/unsubscribe.html @@ -0,0 +1,17 @@ +<%! from django.core.urlresolvers import reverse %> +<%inherit file="main.html" /> + +<%namespace name='static' file='static_content.html'/> + +
      + +
      +

      Unsubscribe Successful!

      +
      + +

      + You will no longer receive notification emails from edX. + Click here to return to your dashboard. +

      +
      +
      diff --git a/lms/urls.py b/lms/urls.py index 085a35b9f4..9670a67a4a 100644 --- a/lms/urls.py +++ b/lms/urls.py @@ -59,6 +59,8 @@ urlpatterns = ('', # nopep8 name='auth_password_reset_done'), url(r'^heartbeat$', include('heartbeat.urls')), + + url(r'^user_api/', include('user_api.urls')), ) # University profiles only make sense in the default edX context @@ -175,6 +177,8 @@ if settings.COURSEWARE_ENABLED: urlpatterns += ( url(r'^courses/(?P[^/]+/[^/]+/[^/]+)/jump_to/(?P.*)$', 'courseware.views.jump_to', name="jump_to"), + url(r'^courses/(?P[^/]+/[^/]+/[^/]+)/jump_to_id/(?P.*)$', + 'courseware.views.jump_to_id', name="jump_to_id"), url(r'^courses/(?P[^/]+/[^/]+/[^/]+)/modx/(?P.*?)/(?P[^/]*)$', 'courseware.module_render.modx_dispatch', name='modx_dispatch'), @@ -329,7 +333,10 @@ if settings.COURSEWARE_ENABLED: url(r'^courses/(?P[^/]+/[^/]+/[^/]+)/news$', 'courseware.views.news', name="news"), url(r'^courses/(?P[^/]+/[^/]+/[^/]+)/discussion/', - include('django_comment_client.urls')) + include('django_comment_client.urls')), + url(r'^notification_prefs/enable/', 'notification_prefs.views.ajax_enable'), + url(r'^notification_prefs/disable/', 'notification_prefs.views.ajax_disable'), + url(r'^notification_prefs/unsubscribe/(?P[a-zA-Z0-9-_=]+)/', 'notification_prefs.views.unsubscribe'), ) urlpatterns += ( # This MUST be the last view in the courseware--it's a catch-all for custom tabs. diff --git a/lms/xmodule_namespace.py b/lms/xmodule_namespace.py index aaef0b76db..d57ad9ce52 100644 --- a/lms/xmodule_namespace.py +++ b/lms/xmodule_namespace.py @@ -3,6 +3,8 @@ Namespace that defines fields common to all blocks used in the LMS """ from xblock.core import Namespace, Boolean, Scope, String, Float from xmodule.fields import Date, Timedelta +from datetime import datetime +from pytz import UTC class LmsNamespace(Namespace): @@ -25,7 +27,11 @@ class LmsNamespace(Namespace): scope=Scope.settings, ) - start = Date(help="Start time when this module is visible", scope=Scope.settings) + start = Date( + help="Start time when this module is visible", + default=datetime.fromtimestamp(0, UTC), + scope=Scope.settings + ) due = Date(help="Date that this problem is due by", scope=Scope.settings) source_file = String(help="source file name (eg for latex)", scope=Scope.settings) giturl = String(help="url root for course data git repository", scope=Scope.settings) @@ -35,8 +41,16 @@ class LmsNamespace(Namespace): help="Amount of time after the due date that submissions will be accepted", scope=Scope.settings ) - showanswer = String(help="When to show the problem answer to the student", scope=Scope.settings, default="closed") - rerandomize = String(help="When to rerandomize the problem", default="always", scope=Scope.settings) + showanswer = String( + help="When to show the problem answer to the student", + scope=Scope.settings, + default="finished" + ) + rerandomize = String( + help="When to rerandomize the problem", + default="never", + scope=Scope.settings + ) days_early_for_beta = Float( help="Number of days early to show content to beta users", default=None, diff --git a/manage.py b/manage.py new file mode 100755 index 0000000000..948bf147d4 --- /dev/null +++ b/manage.py @@ -0,0 +1,90 @@ +#!/usr/bin/env python +""" +Usage: manage.py {lms|cms} [--settings env] ... + +Run django management commands. Because edx-platform contains multiple django projects, +the first argument specifies which project to run (cms [Studio] or lms [Learning Management System]). + +By default, those systems run in with a settings file appropriate for development. However, +by passing the --settings flag, you can specify what environment specific settings file to use. + +Any arguments not understood by this manage.py will be passed to django-admin.py +""" + +import os +import sys +import glob2 +import imp +from argparse import ArgumentParser + +def parse_args(): + """Parse edx specific arguments to manage.py""" + parser = ArgumentParser() + subparsers = parser.add_subparsers(title='system', description='edx service to run') + + lms = subparsers.add_parser( + 'lms', + help='Learning Management System', + add_help=False, + usage='%(prog)s [options] ...' + ) + lms.add_argument('-h', '--help', action='store_true', help='show this help message and exit') + lms.add_argument( + '--settings', + help="Which django settings module to use from inside of lms.envs. If not provided, the DJANGO_SETTINGS_MODULE " + "environment variable will be used if it is set, otherwise will default to lms.envs.dev") + lms.add_argument( + '-s', '--service-variant', + choices=['lms', 'lms-xml', 'lms-preview'], + default='lms', + help='Which service variant to run, when using the aws environment') + lms.set_defaults( + help_string=lms.format_help(), + settings_base='lms/envs', + default_settings='lms.envs.dev' + ) + + cms = subparsers.add_parser( + 'cms', + help='Studio', + add_help=False, + usage='%(prog)s [options] ...' + ) + cms.add_argument( + '--settings', + help="Which django settings module to use from inside cms.envs. If not provided, the DJANGO_SETTINGS_MODULE " + "environment variable will be used if it is set, otherwise will default to cms.envs.dev") + cms.add_argument('-h', '--help', action='store_true', help='show this help message and exit') + cms.set_defaults( + help_string=cms.format_help(), + settings_base='cms/envs', + default_settings='cms.envs.dev', + service_variant='cms' + ) + + + edx_args, django_args = parser.parse_known_args() + + if edx_args.help: + print "edX:" + print edx_args.help_string + + return edx_args, django_args + + +if __name__ == "__main__": + edx_args, django_args = parse_args() + + if edx_args.settings: + os.environ["DJANGO_SETTINGS_MODULE"] = edx_args.settings_base.replace('/', '.') + "." + edx_args.settings + else: + os.environ.setdefault("DJANGO_SETTINGS_MODULE", edx_args.default_settings) + os.environ.setdefault("SERVICE_VARIANT", edx_args.service_variant) + if edx_args.help: + print "Django:" + # This will trigger django-admin.py to print out its help + django_args.insert(0, '--help') + + from django.core.management import execute_from_command_line + + execute_from_command_line([sys.argv[0]] + django_args) diff --git a/rakelib/django.rake b/rakelib/django.rake index eeb8135d4d..40005e19e3 100644 --- a/rakelib/django.rake +++ b/rakelib/django.rake @@ -11,8 +11,7 @@ end task :fastlms do # this is >2 times faster that rake [lms], and does not need web, good for local dev - django_admin = ENV['DJANGO_ADMIN_PATH'] || select_executable('django-admin.py', 'django-admin') - sh("#{django_admin} runserver --traceback --settings=lms.envs.dev --pythonpath=.") + sh("./manage.py lms runserver --traceback") end # Start :system locally with the specified :env and :options. @@ -36,9 +35,8 @@ end desc "Start #{system} Celery worker" task "#{system}_worker", [:options] => [:predjango] do |t, args| args.with_defaults(:options => default_options[system]) - django_admin = ENV['DJANGO_ADMIN_PATH'] || select_executable('django-admin.py', 'django-admin') command = 'celery worker' - sh("#{django_admin} #{command} --loglevel=INFO --settings=#{system}.envs.dev_with_worker --pythonpath=. #{args.join(' ')}") + sh("./manage.py #{system} --settings dev_with_worker #{command} --loglevel=INFO #{args.join(' ')}") end # Per environment tasks @@ -111,11 +109,6 @@ namespace :cms do end end - desc "Imports all the templates from the code pack" - task :update_templates do - sh(django_admin(:cms, :dev, :update_templates)) - end - desc "Import course data within the given DATA_DIR variable" task :xlint do if ENV['DATA_DIR'] and ENV['COURSE_DIR'] diff --git a/rakelib/helpers.rb b/rakelib/helpers.rb index 3373214a19..6826570a18 100644 --- a/rakelib/helpers.rb +++ b/rakelib/helpers.rb @@ -12,8 +12,7 @@ def select_executable(*cmds) end def django_admin(system, env, command, *args) - django_admin = ENV['DJANGO_ADMIN_PATH'] || select_executable('django-admin.py', 'django-admin') - return "#{django_admin} #{command} --traceback --settings=#{system}.envs.#{env} --pythonpath=. #{args.join(' ')}" + return "./manage.py #{system} --settings #{env} #{command} --traceback #{args.join(' ')}" end def report_dir_path(dir) @@ -52,8 +51,14 @@ end # Runs Process.spawn, and kills the process at the end of the rake process # Expects the same arguments as Process.spawn -def background_process(*command) - pid = Process.spawn({}, *command, {:pgroup => true}) +def background_process(command, logfile=nil) + spawn_opts = {:pgroup => true} + if !logfile.nil? + puts "Running '#{command.join(' ')}', redirecting output to #{logfile}".red + spawn_opts[[:err, :out]] = [logfile, 'a'] + end + pid = Process.spawn({}, *command, spawn_opts) + command = [*command] at_exit do puts "Ending process and children" @@ -88,9 +93,10 @@ end # Runs a command as a background process, as long as no other processes # tagged with the same tag are running -def singleton_process(*command) +def singleton_process(command, logfile=nil) + command = [*command] if Sys::ProcTable.ps.select {|proc| proc.cmdline.include?(command.join(' '))}.empty? - background_process(*command) + background_process(command, logfile) else puts "Process '#{command.join(' ')} already running, skipping".blue end diff --git a/rakelib/jasmine.rake b/rakelib/jasmine.rake index ff72161937..5a0c4acedc 100644 --- a/rakelib/jasmine.rake +++ b/rakelib/jasmine.rake @@ -8,6 +8,11 @@ PREFERRED_METHOD = PHANTOMJS_PATH.nil? ? 'browser' : 'phantomjs' if PHANTOMJS_PATH.nil? puts("phantomjs not found on path. Set $PHANTOMJS_PATH. Using browser for jasmine tests".blue) end +LOGDIR = 'logs/jasmine' + +CLOBBER.include(LOGDIR) + +directory LOGDIR def django_for_jasmine(system, django_reload) if !django_reload @@ -17,7 +22,7 @@ def django_for_jasmine(system, django_reload) port = 10000 + rand(40000) jasmine_url = "http://localhost:#{port}/_jasmine/" - background_process(*django_admin(system, 'jasmine', 'runserver', '-v', '0', port.to_s, reload_arg).split(' ')) + background_process(django_admin(system, 'jasmine', 'runserver', '-v', '0', port.to_s, reload_arg).split(' '), "#{LOGDIR}/django.log") up = false start_time = Time.now @@ -80,7 +85,7 @@ end namespace :jasmine do namespace system do desc "Open jasmine tests for #{system} in your default browser" - task :browser => [:clean_reports_dir] do + task :browser => [:clean_reports_dir, LOGDIR] do Rake::Task[:assets].invoke(system, 'jasmine') django_for_jasmine(system, true) do |jasmine_url| jasmine_browser(jasmine_url) @@ -88,7 +93,7 @@ end end desc "Open jasmine tests for #{system} in your default browser, and dynamically recompile coffeescript" - task :'browser:watch' => [:clean_reports_dir, :'assets:coffee:_watch'] do + task :'browser:watch' => [:clean_reports_dir, :'assets:coffee:_watch', LOGDIR] do django_for_jasmine(system, true) do |jasmine_url| jasmine_browser(jasmine_url, jitter=0, wait=0) end @@ -97,7 +102,7 @@ end end desc "Use phantomjs to run jasmine tests for #{system} from the console" - task :phantomjs => [:clean_reports_dir] do + task :phantomjs => [:clean_reports_dir, LOGDIR] do Rake::Task[:assets].invoke(system, 'jasmine') phantomjs = ENV['PHANTOMJS_PATH'] || 'phantomjs' django_for_jasmine(system, false) do |jasmine_url| diff --git a/rakelib/quality.rake b/rakelib/quality.rake index b9254528cf..7cbe10ce1f 100644 --- a/rakelib/quality.rake +++ b/rakelib/quality.rake @@ -43,3 +43,17 @@ end end task :pep8 => :"pep8:#{system}" end + +dquality_dir = File.join(REPORT_DIR, "diff_quality") +directory dquality_dir + +desc "Build the html diff quality reports, and print the reports to the console." +task :quality => dquality_dir do + # Generage diff-quality html report for pep8, and print to console + sh("diff-quality --violations=pep8 --html-report #{dquality_dir}/diff_quality_pep8.html") + sh("diff-quality --violations=pep8") + + # Generage diff-quality html report for pylint, and print to console + sh("diff-quality --violations=pylint --html-report #{dquality_dir}/diff_quality_pylint.html") + sh("diff-quality --violations=pylint") +end \ No newline at end of file diff --git a/rakelib/tests.rake b/rakelib/tests.rake index 7e3e672f39..57861902bc 100644 --- a/rakelib/tests.rake +++ b/rakelib/tests.rake @@ -161,4 +161,4 @@ task :coverage => :report_dirs do if not found_coverage_info puts "No coverage info found. Run `rake test` before running `rake coverage`." end -end +end \ No newline at end of file diff --git a/requirements/edx/base.txt b/requirements/edx/base.txt index 910b6f3def..dd9d8b5a3e 100644 --- a/requirements/edx/base.txt +++ b/requirements/edx/base.txt @@ -7,6 +7,7 @@ celery==3.0.19 distribute>=0.6.28 django-celery==3.0.17 django-countries==1.5 +django-filter==0.6.0 django-followit==0.0.3 django-keyedcache==1.4-6 django-kombu==0.9.4 @@ -20,6 +21,7 @@ django-ses==0.4.1 django-storages==1.1.5 django-threaded-multihost==1.4-1 django-method-override==0.1.0 +djangorestframework==2.3.5 django==1.4.5 feedparser==5.1.3 fs==0.4.0 @@ -35,6 +37,7 @@ path.py==3.0.1 Pillow==1.7.8 pip>=1.3 polib==1.0.3 +pycrypto>=2.6 pygments==1.5 pygraphviz==1.1 pymongo==2.4.1 diff --git a/requirements/edx/github.txt b/requirements/edx/github.txt index f64568dc10..79e3af89a1 100644 --- a/requirements/edx/github.txt +++ b/requirements/edx/github.txt @@ -3,11 +3,11 @@ # Third-party: -e git://github.com/edx/django-staticfiles.git@6d2504e5c8#egg=django-staticfiles -e git://github.com/edx/django-pipeline.git#egg=django-pipeline --e git://github.com/edx/django-wiki.git@ac906abe#egg=django-wiki +-e git://github.com/edx/django-wiki.git@41815e2ef1b0323f92900f8e60711b0f0c37766b#egg=django-wiki -e git://github.com/dementrock/pystache_custom.git@776973740bdaad83a3b029f96e415a7d1e8bec2f#egg=pystache_custom-dev -e git://github.com/eventbrite/zendesk.git@d53fe0e81b623f084e91776bcf6369f8b7b63879#egg=zendesk # Our libraries: --e git+https://github.com/edx/XBlock.git@4d8735e883#egg=XBlock --e git+https://github.com/edx/codejail.git@0a1b468#egg=codejail --e git+https://github.com/edx/diff-cover.git@v0.1.3#egg=diff_cover +-e git+https://github.com/edx/XBlock.git@b697bebd45deebd0f868613fab6722a0460ca0c1#egg=XBlock +-e git+https://github.com/edx/codejail.git@c08967fb44d1bcdb259d3ec58812e3ac592539c2#egg=codejail +-e git+https://github.com/edx/diff-cover.git@v0.2.0#egg=diff_cover diff --git a/requirements/edx/local.txt b/requirements/edx/local.txt index f5ba60e21b..04a1f7f2c6 100644 --- a/requirements/edx/local.txt +++ b/requirements/edx/local.txt @@ -2,6 +2,7 @@ -e common/lib/calc -e common/lib/capa -e common/lib/chem +-e common/lib/sandbox-packages -e common/lib/symmath -e common/lib/xmodule -e . diff --git a/scripts/create-dev-env.sh b/scripts/create-dev-env.sh index a28ee1a8b4..25e1b7520b 100755 --- a/scripts/create-dev-env.sh +++ b/scripts/create-dev-env.sh @@ -495,9 +495,9 @@ mkdir -p "$BASE/log" mkdir -p "$BASE/db" mkdir -p "$BASE/data" -rake django-admin[syncdb,lms,dev,--noinput] -rake django-admin[migrate] -rake cms:update_templates +./manage.py lms syncdb --noinput --migrate +./manage.py cms syncdb --noinput --migrate + # Configure Git output "Fixing your git default settings" @@ -520,18 +520,13 @@ if [[ ! $quiet ]]; then $ workon mitx - To initialize Django - - $ rake django-admin[syncdb] - $ rake django-admin[migrate] - To start the Django on port 8000 $ rake lms Or to start Django on a different - $ rake django-admin[runserver,lms,dev,] + $ ./manage.py lms runserver If the Django development server starts properly you should see: diff --git a/scripts/release-email-list.sh b/scripts/release-email-list.sh index 64fa7c00d1..f54018f4f5 100755 --- a/scripts/release-email-list.sh +++ b/scripts/release-email-list.sh @@ -4,27 +4,36 @@ LOG_CMD="git --no-pager log $1..$2" RESPONSIBLE=$(sort -u <($LOG_CMD --format='tformat:%ae' && $LOG_CMD --format='tformat:%ce')) +echo "~~~~ Email ~~~~~" + echo -n 'To: ' echo ${RESPONSIBLE} | sed "s/ /, /g" echo echo "You've made changes that are about to be released. All of the commits that you either authored or committed are listed below. Please verify them on -\$ENVIRONMENT" +\$ENVIRONMENT. + +Please record your notes on https://edx-wiki.atlassian.net/wiki/display/ENG/Release+Page%3A+\$DATE +and add any bugs found to the Release Candidate Bugs section" echo +echo "~~~~~ Wiki Table ~~~~~" +echo "Type Ctrl+Shift+D on Confluence to embed the following table in your release wiki page" +echo + +echo '||Author||Changes||Commit Link||Testing Notes||' + for EMAIL in $RESPONSIBLE; do AUTHORED_BY="$LOG_CMD --author=<${EMAIL}>" COMMITTED_BY="$LOG_CMD --committer=<${EMAIL}>" COMMITTED_NOT_AUTHORED="$COMMITTED_BY $($AUTHORED_BY --format='tformat:^%h')" - echo $EMAIL "authored the following commits:" - $AUTHORED_BY --format='tformat: %s - https://github.com/edx/edx-platform/commit/%h' - echo + $AUTHORED_BY --format="tformat:|$EMAIL|%s|[commit|https://github.com/edx/edx-platform/commit/%h]| |" | head -n 1 + $AUTHORED_BY --format="tformat:| |%s|[commit|https://github.com/edx/edx-platform/commit/%h]| |" | tail -n +2 if [[ $($COMMITTED_NOT_AUTHORED) != "" ]]; then - echo $EMAIL "committed but didn't author the following commits:" - $COMMITTED_NOT_AUTHORED --format='tformat: %s - https://github.com/edx/edx-platform/commit/%h' - echo + $COMMITTED_NOT_AUTHORED --format="tformat:|$EMAIL|%s|[commit|https://github.com/edx/edx-platform/commit/%h]|Committed, didn't author|" | head -n 1 + $COMMITTED_NOT_AUTHORED --format="tformat:| |%s|[commit|https://github.com/edx/edx-platform/commit/%h]| |" | tail -n +2 fi done \ No newline at end of file diff --git a/scripts/runone.py b/scripts/runone.py index a644aa077b..b403b09ff9 100755 --- a/scripts/runone.py +++ b/scripts/runone.py @@ -41,18 +41,17 @@ def main(argv): test_py_path = find_full_path(test_py_path) test_spec = "%s:%s.%s" % (test_py_path, test_class, test_method) - settings = None + system = None if test_py_path.startswith('cms'): - settings = 'cms.envs.test' + system = 'cms' elif test_py_path.startswith('lms'): - settings = 'lms.envs.test' + system = 'lms' - if settings: + if system: # Run as a django test suite from django.core import management - django_args = ["django-admin.py", "test", "--pythonpath=."] - django_args.append("--settings=%s" % settings) + django_args = ["./manage.py", system, "--settings", "test", "test"] if args.nocapture: django_args.append("-s") django_args.append(test_spec) diff --git a/scripts/vagrant-provisioning.sh b/scripts/vagrant-provisioning.sh index d4f627b2c4..0243cd36ae 100755 --- a/scripts/vagrant-provisioning.sh +++ b/scripts/vagrant-provisioning.sh @@ -4,6 +4,7 @@ # # Authors: Xavier Antoviaque # David Baumgold +# Yarko Tymciurak # # This software's license gives you freedom; you can copy, convey, # propagate, redistribute and/or modify this program under the terms of @@ -30,102 +31,112 @@ # # This script is ran by `$ vagrant up`, see the README for more explanations +on_create() +{ + # APT - Packages ############################################################## -# APT - Packages ############################################################## - -apt-get update -apt-get install -y python-software-properties vim + apt-get update + apt-get install -y python-software-properties vim -# Curl - No progress bar ###################################################### + # Curl - No progress bar ###################################################### -[[ -f ~vagrant/.curlrc ]] || echo "silent show-error" > ~vagrant/.curlrc -chown vagrant.vagrant ~vagrant/.curlrc + [[ -f ~vagrant/.curlrc ]] || echo "silent show-error" > ~vagrant/.curlrc + chown vagrant.vagrant ~vagrant/.curlrc -# SSH - Known hosts ########################################################### + # SSH - Known hosts ########################################################### -# Github -([[ -f ~vagrant/.ssh/known_hosts ]] && grep "zBX7bKA= ssh" ~vagrant/.ssh/known_hosts) || { - echo "|1|4DtBcMsTM4zgl/jTS7h3ZkmS/Vc=|XkRnn2xEhr8ixOxeskJAzBX7bKA= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==" >> ~vagrant/.ssh/known_hosts -} -([[ -f ~vagrant/.ssh/known_hosts ]] && grep "jO3J5bvw= ssh" ~vagrant/.ssh/known_hosts) || { - echo "|1|9rANf/qOAPgKH/TXpGuZCAgGxMs=|x9VYWEDI8kiotbhhNXqjO3J5bvw= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==" >> ~vagrant/.ssh/known_hosts -} -chown vagrant.vagrant ~vagrant/.ssh/known_hosts + # Github + ([[ -f ~vagrant/.ssh/known_hosts ]] && grep "zBX7bKA= ssh" ~vagrant/.ssh/known_hosts) || { + echo "|1|4DtBcMsTM4zgl/jTS7h3ZkmS/Vc=|XkRnn2xEhr8ixOxeskJAzBX7bKA= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==" >> ~vagrant/.ssh/known_hosts + } + ([[ -f ~vagrant/.ssh/known_hosts ]] && grep "jO3J5bvw= ssh" ~vagrant/.ssh/known_hosts) || { + echo "|1|9rANf/qOAPgKH/TXpGuZCAgGxMs=|x9VYWEDI8kiotbhhNXqjO3J5bvw= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==" >> ~vagrant/.ssh/known_hosts + } + chown vagrant.vagrant ~vagrant/.ssh/known_hosts -# edX - Development environment ############################################### + # edX - Development environment ############################################### -# Node modules require a filesystem with symlinks (Windows support) -mkdir -p /opt/edx/node_modules /opt/edx/edx-platform/node_modules -([[ -f /etc/fstab ]] && grep '/opt/edx/node_modules' /etc/fstab) || { - echo '/opt/edx/node_modules /opt/edx/edx-platform/node_modules none bind,noauto 0 0' >> /etc/fstab - mount /opt/edx/node_modules -} -# Must be mounted *after* the NFS mount, made manually by Vagrant -([[ -f /etc/cron.d/nodemodules ]] && grep '/opt/edx/node_modules' /etc/cron.d/nodemodules) || { - echo '@reboot root until [ -n "`mount |grep "/opt/edx/edx-platform type"`" ]; do sleep 1; done; mount /opt/edx/node_modules' > /etc/cron.d/nodemodules -} + # Node modules require a filesystem with symlinks (Windows support) + mkdir -p /opt/edx/node_modules /opt/edx/edx-platform/node_modules + ([[ -f /etc/fstab ]] && grep '/opt/edx/node_modules' /etc/fstab) || { + echo '/opt/edx/node_modules /opt/edx/edx-platform/node_modules none bind,noauto 0 0' >> /etc/fstab + mount /opt/edx/node_modules + } + # Must be mounted *after* the NFS mount, made manually by Vagrant + ([[ -f /etc/cron.d/nodemodules ]] && grep '/opt/edx/node_modules' /etc/cron.d/nodemodules) || { + echo '@reboot root until [ -n "`mount |grep "/opt/edx/edx-platform type"`" ]; do sleep 1; done; mount /opt/edx/node_modules' > /etc/cron.d/nodemodules + } -# Force rechecking all prerequisites (could have been fetched outside of the VM) -rm -rf /opt/edx/edx-platform/.prereqs_cache + # Force rechecking all prerequisites (could have been fetched outside of the VM) + rm -rf /opt/edx/edx-platform/.prereqs_cache -# Permissions -chown vagrant.vagrant /opt/edx /opt/edx/node_modules /opt/edx/edx-platform/node_modules + # Permissions + chown vagrant.vagrant /opt/edx /opt/edx/node_modules /opt/edx/edx-platform/node_modules -# For convenience with `vagrant ssh`, the `edx-platform` virtualenv is always -# loaded after the first run, so we need to deactivate that behavior to run -# `create-dev-env.sh`. -[[ -f ~vagrant/.bash_profile ]] && { - mv ~vagrant/.bash_profile ~vagrant/.bash_profile.bak -} -sudo -u vagrant -i bash -c "cd /opt/edx/edx-platform && PROJECT_HOME=/opt/edx ./scripts/create-dev-env.sh -ynq" + # For convenience with `vagrant ssh`, the `edx-platform` virtualenv is always + # loaded after the first run, so we need to deactivate that behavior to run + # `create-dev-env.sh`. + [[ -f ~vagrant/.bash_profile ]] && { + mv ~vagrant/.bash_profile ~vagrant/.bash_profile.bak + } + sudo -u vagrant -i bash -c "cd /opt/edx/edx-platform && PROJECT_HOME=/opt/edx ./scripts/create-dev-env.sh -ynq" -# Load .bashrc ################################################################ -([[ -f ~vagrant/.bash_profile ]] && grep ".bashrc" ~vagrant/.bash_profile) || { - echo -e "\n. /home/vagrant/.bashrc\n" >> ~vagrant/.bash_profile -} + # Load .bashrc ################################################################ + ([[ -f ~vagrant/.bash_profile ]] && grep ".bashrc" ~vagrant/.bash_profile) || { + echo ". /home/vagrant/.bashrc" >> ~vagrant/.bash_profile + } -# Virtualenv - Always load #################################################### + # Virtualenv - Always load #################################################### -([[ -f ~vagrant/.bash_profile ]] && grep "edx-platform/bin/activate" ~vagrant/.bash_profile) || { - echo -e "\n. /home/vagrant/.virtualenvs/edx-platform/bin/activate\n" >> ~vagrant/.bash_profile -} + ([[ -f ~vagrant/.bash_profile ]] && grep "edx-platform/bin/activate" ~vagrant/.bash_profile) || { + echo ". /home/vagrant/.virtualenvs/edx-platform/bin/activate" >> ~vagrant/.bash_profile + } -# Directory ################################################################### + # Directory ################################################################### -grep "cd /opt/edx/edx-platform" ~vagrant/.bash_profile || { - echo -e "\ncd /opt/edx/edx-platform\n" >> ~vagrant/.bash_profile -} + grep "cd /opt/edx/edx-platform" ~vagrant/.bash_profile || { + echo "cd /opt/edx/edx-platform" >> ~vagrant/.bash_profile + } + # Permissions + chown vagrant.vagrant ~vagrant/.bash_profile -# End ######################################################################### - -cat << EOF + cat << EOF ============================================================================== -Success! +Success - Created your development environment! ============================================================================== -Now, from the virtual machine (connect with "vagrant ssh" if vagrant didn't -log you in already), you can start Studio & LMS with the following commands: - -- Learning management system (LMS): - $ rake lms[cms.dev,0.0.0.0:8000] - - => http://192.168.20.40:8000/ - -- Studio: - $ rake cms[dev,0.0.0.0:8001] - - => http://192.168.20.40:8001/ - - -See the README for details. - - EOF +} # End on_create() ######################################################## +## only initialize / setup the development environment once: +# we create node_modules, so that's a good test: +[[ -d /opt/edx/node_modules ]] || on_create +# grab what the Vagrantfile spec'd our IP to be: +# expecting: +# - relevant ip on eth1; +# - line of interest to look like: +# inet 192.168.20.40/24 brd 192.168.20.255 scope global eth1 +MY_IP=$(ip addr show dev eth1 | sed -n '/inet /{s/.*[ ]\(.*\)\/.*/\1/;p}') + +cat << EOF +Connect to your virtual machine with "vagrant ssh". +Some examples you can use from your virtual machine: + +- Start Learning management system (LMS): + $ rake lms[cms.dev,0.0.0.0:8000] + => http://${MY_IP}:8000/ + +- Start Studio: + $ rake cms[dev,0.0.0.0:8001] + => http://${MY_IP}:8001/ + +See the README for more. + +EOF