diff --git a/.gitignore b/.gitignore index 83e2b67655..69bc47afdd 100644 --- a/.gitignore +++ b/.gitignore @@ -10,6 +10,8 @@ .AppleDouble database.sqlite requirements/private.txt +lms/envs/private.py +cms/envs/private.py courseware/static/js/mathjax/* flushdb.sh build diff --git a/AUTHORS b/AUTHORS index 154b0c9b98..7d6397629f 100644 --- a/AUTHORS +++ b/AUTHORS @@ -75,3 +75,4 @@ Frances Botsford Jonah Stanley Slater Victoroff Peter Fogg +Renzo Lucioni \ No newline at end of file diff --git a/CHANGELOG.rst b/CHANGELOG.rst new file mode 100644 index 0000000000..03a3b8c809 --- /dev/null +++ b/CHANGELOG.rst @@ -0,0 +1,93 @@ +Change Log +---------- + +These are notable changes in edx-platform. This is a rolling list of changes, +in roughly chronological order, most recent first. Add your entries at or near +the top. Include a label indicating the component affected. + +Studio, LMS: Make ModelTypes more strict about their expected content (for +instance, Boolean, Integer, String), but also allow them to hold either the +typed value, or a String that can be converted to their typed value. For example, +an Integer can contain 3 or '3'. This changed an update to the xblock library. + +Blades: Video Alpha bug fix for speed changing to 1.0 in Firefox. + +Blades: Additional event tracking added to Video Alpha: fullscreen switch, show/hide +captions. + +LMS: Some errors handling Non-ASCII data in XML courses have been fixed. + +LMS: Add page-load tracking using segment-io (if SEGMENT_IO_LMS_KEY and +SEGMENT_IO_LMS feature flag is on) + +Blades: Simplify calc.py (which is used for the Numerical/Formula responses); add trig/other functions. + +LMS: Background colors on login, register, and courseware have been corrected +back to white. + +LMS: Accessibility improvements have been made to several courseware and +navigation elements. + +LMS: Small design/presentation changes to login and register views. + +LMS: Functionality added to instructor enrollment tab in LMS such that invited +student can be auto-enrolled in course or when activating if not current +student. + +Blades: Staff debug info is now accessible for Graphical Slider Tool problems. + +Blades: For Video Alpha the events ready, play, pause, seek, and speed change +are logged on the server (in the logs). + +Common: Developers can now have private Django settings files. + +Common: Safety code added to prevent anything above the vertical level in the +course tree from being marked as version='draft'. It will raise an exception if +the code tries to so mark a node. We need the backtraces to figure out where +this very infrequent intermittent marking was occurring. It was making courses +look different in Studio than in LMS. + +Deploy: MKTG_URLS is now read from env.json. + +Common: Theming makes it possible to change the look of the site, from +Stanford. + +Common: Accessibility UI fixes. + +Common: The "duplicate email" error message is more informative. + +Studio: Component metadata settings editor. + +Studio: Autoplay for Video Alpha is disabled (only in Studio). + +Studio: Single-click creation for video and discussion components. + +Studio: fixed a bad link in the activation page. + +LMS: Changed the help button text. + +LMS: Fixed failing numeric response (decimal but no trailing digits). + +LMS: XML Error module no longer shows students a stack trace. + +Blades: Videoalpha. + +XModules: Added partial credit for foldit module. + +XModules: Added "randomize" XModule to list of XModule types. + +XModules: Show errors with full descriptors. + +XQueue: Fixed (hopefully) worker crash when the connection to RabbitMQ is +dropped suddenly. + +XQueue: Upload file submissions to a specially named bucket in S3. + +Common: Removed request debugger. + +Common: Updated Django to version 1.4.5. + +Common: Updated CodeJail. + +Common: Allow setting of authentication session cookie name. + diff --git a/README.md b/README.md index 3a6236ea70..92a4116354 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,7 @@ CMS templates. Fortunately, `rake` will do all of this for you! Just run: If you are running these commands using the [`zsh`](http://www.zsh.org/) shell, zsh will assume that you are doing -[shell globbing](https://en.wikipedia.org/wiki/Glob_(programming)), search for +[shell globbing](https://en.wikipedia.org/wiki/Glob_%28programming%29), search for a file in your directory named `django-adminsyncdb` or `django-adminmigrate`, and fail. To fix this, just surround the argument with quotation marks, so that you're running `rake "django-admin[syncdb]"`. diff --git a/cms/djangoapps/contentstore/features/advanced-settings.feature b/cms/djangoapps/contentstore/features/advanced-settings.feature index 558294e890..13600f2086 100644 --- a/cms/djangoapps/contentstore/features/advanced-settings.feature +++ b/cms/djangoapps/contentstore/features/advanced-settings.feature @@ -28,11 +28,18 @@ Feature: Advanced (manual) course policy Scenario: Test how multi-line input appears Given I am on the Advanced Course Settings page in Studio - When I create a JSON object as a value + When I create a JSON object as a value for "discussion_topics" Then it is displayed as formatted And I reload the page Then it is displayed as formatted + Scenario: Test error if value supplied is of the wrong type + Given I am on the Advanced Course Settings page in Studio + When I create a JSON object as a value for "display_name" + Then I get an error on save + And I reload the page + Then the policy key value is unchanged + Scenario: Test automatic quoting of non-JSON values Given I am on the Advanced Course Settings page in Studio When I create a non-JSON value not in quotes diff --git a/cms/djangoapps/contentstore/features/advanced-settings.py b/cms/djangoapps/contentstore/features/advanced-settings.py index eb00c06ba9..049103db27 100644 --- a/cms/djangoapps/contentstore/features/advanced-settings.py +++ b/cms/djangoapps/contentstore/features/advanced-settings.py @@ -2,8 +2,7 @@ #pylint: disable=W0621 from lettuce import world, step -from common import * -from nose.tools import assert_false, assert_equal +from nose.tools import assert_false, assert_equal, assert_regexp_matches """ http://selenium.googlecode.com/svn/trunk/docs/api/py/webdriver/selenium.webdriver.common.keys.html @@ -52,9 +51,9 @@ def edit_the_value_of_a_policy_key_and_save(step): change_display_name_value(step, '"foo"') -@step('I create a JSON object as a value$') -def create_JSON_object(step): - change_display_name_value(step, '{"key": "value", "key_2": "value_2"}') +@step('I create a JSON object as a value for "(.*)"$') +def create_JSON_object(step, key): + change_value(step, key, '{"key": "value", "key_2": "value_2"}') @step('I create a non-JSON value not in quotes$') @@ -82,7 +81,12 @@ def they_are_alphabetized(step): @step('it is displayed as formatted$') def it_is_formatted(step): - assert_policy_entries([DISPLAY_NAME_KEY], ['{\n "key": "value",\n "key_2": "value_2"\n}']) + assert_policy_entries(['discussion_topics'], ['{\n "key": "value",\n "key_2": "value_2"\n}']) + + +@step('I get an error on save$') +def error_on_save(step): + assert_regexp_matches(world.css_text('#notification-error-description'), 'Incorrect setting format') @step('it is displayed as a string') @@ -124,11 +128,16 @@ def get_display_name_value(): def change_display_name_value(step, new_value): + change_value(step, DISPLAY_NAME_KEY, new_value) - world.css_find(".CodeMirror")[get_index_of(DISPLAY_NAME_KEY)].click() + +def change_value(step, key, new_value): + index = get_index_of(key) + world.css_find(".CodeMirror")[index].click() g = world.css_find("div.CodeMirror.CodeMirror-focused > div > textarea") - display_name = get_display_name_value() - for count in range(len(display_name)): + current_value = world.css_find(VALUE_CSS)[index].value + g._element.send_keys(Keys.CONTROL + Keys.END) + for count in range(len(current_value)): g._element.send_keys(Keys.END, Keys.BACK_SPACE) # Must delete "" before typing the JSON value g._element.send_keys(Keys.END, Keys.BACK_SPACE, Keys.BACK_SPACE, new_value) diff --git a/cms/djangoapps/contentstore/features/problem-editor.feature b/cms/djangoapps/contentstore/features/problem-editor.feature index 6ed8c1619b..bde350d8a3 100644 --- a/cms/djangoapps/contentstore/features/problem-editor.feature +++ b/cms/djangoapps/contentstore/features/problem-editor.feature @@ -52,7 +52,7 @@ Feature: Problem Editor Scenario: User cannot type out of range values in an integer number field Given I have created a Blank Common Problem And I edit and select Settings - Then if I set the max attempts to "-3", it displays initially as "-3", and is persisted as "1" + Then if I set the max attempts to "-3", it displays initially as "-3", and is persisted as "0" Scenario: Settings changes are not saved on Cancel Given I have created a Blank Common Problem diff --git a/cms/djangoapps/contentstore/features/problem-editor.py b/cms/djangoapps/contentstore/features/problem-editor.py index 5dfcf55046..7679128beb 100644 --- a/cms/djangoapps/contentstore/features/problem-editor.py +++ b/cms/djangoapps/contentstore/features/problem-editor.py @@ -41,7 +41,9 @@ def i_see_five_settings_with_values(step): @step('I can modify the display name') def i_can_modify_the_display_name(step): - world.get_setting_entry(DISPLAY_NAME).find_by_css('.setting-input')[0].fill('modified') + # Verifying that the display name can be a string containing a floating point value + # (to confirm that we don't throw an error because it is of the wrong type). + world.get_setting_entry(DISPLAY_NAME).find_by_css('.setting-input')[0].fill('3.4') verify_modified_display_name() @@ -172,7 +174,7 @@ def verify_modified_randomization(): def verify_modified_display_name(): - world.verify_setting_entry(world.get_setting_entry(DISPLAY_NAME), DISPLAY_NAME, 'modified', True) + world.verify_setting_entry(world.get_setting_entry(DISPLAY_NAME), DISPLAY_NAME, '3.4', True) def verify_modified_display_name_with_special_chars(): diff --git a/cms/djangoapps/contentstore/features/section.feature b/cms/djangoapps/contentstore/features/section.feature index 236cf501fc..80ccb6cc7a 100644 --- a/cms/djangoapps/contentstore/features/section.feature +++ b/cms/djangoapps/contentstore/features/section.feature @@ -26,11 +26,9 @@ Feature: Create Section And I save a new section release date Then the section release date is updated - # Skipped because Ubuntu ChromeDriver hangs on alert - @skip Scenario: Delete section Given I have opened a new course in Studio And I have added a new section - When I press the "section" delete icon - And I confirm the alert + When I will confirm all alerts + And I press the "section" delete icon Then the section does not exist diff --git a/cms/djangoapps/contentstore/features/section.py b/cms/djangoapps/contentstore/features/section.py index 9a896d8ebe..9d63fa73c8 100644 --- a/cms/djangoapps/contentstore/features/section.py +++ b/cms/djangoapps/contentstore/features/section.py @@ -9,34 +9,34 @@ from nose.tools import assert_equal @step('I click the new section link$') -def i_click_new_section_link(step): +def i_click_new_section_link(_step): link_css = 'a.new-courseware-section-button' world.css_click(link_css) @step('I enter the section name and click save$') -def i_save_section_name(step): +def i_save_section_name(_step): save_section_name('My Section') @step('I enter a section name with a quote and click save$') -def i_save_section_name_with_quote(step): +def i_save_section_name_with_quote(_step): save_section_name('Section with "Quote"') @step('I have added a new section$') -def i_have_added_new_section(step): +def i_have_added_new_section(_step): add_section() @step('I click the Edit link for the release date$') -def i_click_the_edit_link_for_the_release_date(step): +def i_click_the_edit_link_for_the_release_date(_step): button_css = 'div.section-published-date a.edit-button' world.css_click(button_css) @step('I save a new section release date$') -def i_save_a_new_section_release_date(step): +def i_save_a_new_section_release_date(_step): set_date_and_time('input.start-date.date.hasDatepicker', '12/25/2013', 'input.start-time.time.ui-timepicker-input', '00:00') world.browser.click_link_by_text('Save') @@ -46,35 +46,35 @@ def i_save_a_new_section_release_date(step): @step('I see my section on the Courseware page$') -def i_see_my_section_on_the_courseware_page(step): +def i_see_my_section_on_the_courseware_page(_step): see_my_section_on_the_courseware_page('My Section') @step('I see my section name with a quote on the Courseware page$') -def i_see_my_section_name_with_quote_on_the_courseware_page(step): +def i_see_my_section_name_with_quote_on_the_courseware_page(_step): see_my_section_on_the_courseware_page('Section with "Quote"') @step('I click to edit the section name$') -def i_click_to_edit_section_name(step): +def i_click_to_edit_section_name(_step): world.css_click('span.section-name-span') @step('I see the complete section name with a quote in the editor$') -def i_see_complete_section_name_with_quote_in_editor(step): +def i_see_complete_section_name_with_quote_in_editor(_step): css = '.section-name-edit input[type=text]' assert world.is_css_present(css) assert_equal(world.browser.find_by_css(css).value, 'Section with "Quote"') @step('the section does not exist$') -def section_does_not_exist(step): - css = 'span.section-name-span' - assert world.browser.is_element_not_present_by_css(css) +def section_does_not_exist(_step): + css = 'h3[data-name="My Section"]' + assert world.is_css_not_present(css) @step('I see a release date for my section$') -def i_see_a_release_date_for_my_section(step): +def i_see_a_release_date_for_my_section(_step): import re css = 'span.published-status' @@ -83,26 +83,32 @@ def i_see_a_release_date_for_my_section(step): # e.g. 11/06/2012 at 16:25 msg = 'Will Release:' - date_regex = '[01][0-9]\/[0-3][0-9]\/[12][0-9][0-9][0-9]' - time_regex = '[0-2][0-9]:[0-5][0-9]' - match_string = '%s %s at %s' % (msg, date_regex, time_regex) + date_regex = r'(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) \d\d?, \d{4}' + if not re.search(date_regex, status_text): + print status_text, date_regex + time_regex = r'[0-2]\d:[0-5]\d( \w{3})?' + if not re.search(time_regex, status_text): + print status_text, time_regex + match_string = r'%s\s+%s at %s' % (msg, date_regex, time_regex) + if not re.match(match_string, status_text): + print status_text, match_string assert re.match(match_string, status_text) @step('I see a link to create a new subsection$') -def i_see_a_link_to_create_a_new_subsection(step): +def i_see_a_link_to_create_a_new_subsection(_step): css = 'a.new-subsection-item' assert world.is_css_present(css) @step('the section release date picker is not visible$') -def the_section_release_date_picker_not_visible(step): +def the_section_release_date_picker_not_visible(_step): css = 'div.edit-subsection-publish-settings' assert not world.css_visible(css) @step('the section release date is updated$') -def the_section_release_date_is_updated(step): +def the_section_release_date_is_updated(_step): css = 'span.published-status' status_text = world.css_text(css) assert_equal(status_text, 'Will Release: 12/25/2013 at 00:00 UTC') diff --git a/cms/djangoapps/contentstore/features/studio-overview-togglesection.feature b/cms/djangoapps/contentstore/features/studio-overview-togglesection.feature index c9f5b43dfb..e746f3629a 100644 --- a/cms/djangoapps/contentstore/features/studio-overview-togglesection.feature +++ b/cms/djangoapps/contentstore/features/studio-overview-togglesection.feature @@ -1,61 +1,59 @@ Feature: Overview Toggle Section - In order to quickly view the details of a course's section or to scan the inventory of sections + In order to quickly view the details of a course's section or to scan the inventory of sections As a course author I want to toggle the visibility of each section's subsection details in the overview listing - Scenario: The default layout for the overview page is to show sections in expanded view - Given I have a course with multiple sections - When I navigate to the course overview page - Then I see the "Collapse All Sections" link - And all sections are expanded + Scenario: The default layout for the overview page is to show sections in expanded view + Given I have a course with multiple sections + When I navigate to the course overview page + Then I see the "Collapse All Sections" link + And all sections are expanded - Scenario: Expand /collapse for a course with no sections - Given I have a course with no sections - When I navigate to the course overview page - Then I do not see the "Collapse All Sections" link + Scenario: Expand /collapse for a course with no sections + Given I have a course with no sections + When I navigate to the course overview page + Then I do not see the "Collapse All Sections" link - Scenario: Collapse link appears after creating first section of a course - Given I have a course with no sections - When I navigate to the course overview page - And I add a section - Then I see the "Collapse All Sections" link - And all sections are expanded + Scenario: Collapse link appears after creating first section of a course + Given I have a course with no sections + When I navigate to the course overview page + And I add a section + Then I see the "Collapse All Sections" link + And all sections are expanded - # Skipped because Ubuntu ChromeDriver hangs on alert - @skip - Scenario: Collapse link is not removed after last section of a course is deleted - Given I have a course with 1 section - And I navigate to the course overview page - When I press the "section" delete icon - And I confirm the alert - Then I see the "Collapse All Sections" link + Scenario: Collapse link is not removed after last section of a course is deleted + Given I have a course with 1 section + And I navigate to the course overview page + When I will confirm all alerts + And I press the "section" delete icon + Then I see the "Collapse All Sections" link - Scenario: Collapsing all sections when all sections are expanded - Given I navigate to the courseware page of a course with multiple sections - And all sections are expanded - When I click the "Collapse All Sections" link - Then I see the "Expand All Sections" link - And all sections are collapsed + Scenario: Collapsing all sections when all sections are expanded + Given I navigate to the courseware page of a course with multiple sections + And all sections are expanded + When I click the "Collapse All Sections" link + Then I see the "Expand All Sections" link + And all sections are collapsed - Scenario: Collapsing all sections when 1 or more sections are already collapsed - Given I navigate to the courseware page of a course with multiple sections - And all sections are expanded - When I collapse the first section - And I click the "Collapse All Sections" link - Then I see the "Expand All Sections" link - And all sections are collapsed + Scenario: Collapsing all sections when 1 or more sections are already collapsed + Given I navigate to the courseware page of a course with multiple sections + And all sections are expanded + When I collapse the first section + And I click the "Collapse All Sections" link + Then I see the "Expand All Sections" link + And all sections are collapsed - Scenario: Expanding all sections when all sections are collapsed - Given I navigate to the courseware page of a course with multiple sections - And I click the "Collapse All Sections" link - When I click the "Expand All Sections" link - Then I see the "Collapse All Sections" link - And all sections are expanded + Scenario: Expanding all sections when all sections are collapsed + Given I navigate to the courseware page of a course with multiple sections + And I click the "Collapse All Sections" link + When I click the "Expand All Sections" link + Then I see the "Collapse All Sections" link + And all sections are expanded - Scenario: Expanding all sections when 1 or more sections are already expanded - Given I navigate to the courseware page of a course with multiple sections - And I click the "Collapse All Sections" link - When I expand the first section - And I click the "Expand All Sections" link - Then I see the "Collapse All Sections" link - And all sections are expanded + Scenario: Expanding all sections when 1 or more sections are already expanded + Given I navigate to the courseware page of a course with multiple sections + And I click the "Collapse All Sections" link + When I expand the first section + And I click the "Expand All Sections" link + Then I see the "Collapse All Sections" link + And all sections are expanded diff --git a/cms/djangoapps/contentstore/features/subsection.feature b/cms/djangoapps/contentstore/features/subsection.feature index 8bb12467ff..a11467e3f9 100644 --- a/cms/djangoapps/contentstore/features/subsection.feature +++ b/cms/djangoapps/contentstore/features/subsection.feature @@ -32,12 +32,10 @@ Feature: Create Subsection And I reload the page Then I see the correct dates - # Skipped because Ubuntu ChromeDriver hangs on alert - @skip Scenario: Delete a subsection Given I have opened a new course section in Studio And I have added a new subsection And I see my subsection on the Courseware page - When I press the "subsection" delete icon - And I confirm the alert + When I will confirm all alerts + And I press the "subsection" delete icon Then the subsection does not exist diff --git a/cms/djangoapps/contentstore/features/video.feature b/cms/djangoapps/contentstore/features/video.feature index 07771c9d61..0129732d30 100644 --- a/cms/djangoapps/contentstore/features/video.feature +++ b/cms/djangoapps/contentstore/features/video.feature @@ -8,3 +8,8 @@ Feature: Video Component Scenario: Creating a video takes a single click Given I have clicked the new unit button Then creating a video takes a single click + + Scenario: Captions are shown correctly + Given I have created a Video component + And I have hidden captions + Then when I view the video it does not show the captions diff --git a/cms/djangoapps/contentstore/features/video.py b/cms/djangoapps/contentstore/features/video.py index 7cbe8a2258..fd8624999e 100644 --- a/cms/djangoapps/contentstore/features/video.py +++ b/cms/djangoapps/contentstore/features/video.py @@ -16,3 +16,13 @@ def video_takes_a_single_click(step): assert(not world.is_css_present('.xmodule_VideoModule')) world.css_click("a[data-location='i4x://edx/templates/video/default']") assert(world.is_css_present('.xmodule_VideoModule')) + + +@step('I have hidden captions') +def set_show_captions_false(step): + world.css_click('a.hide-subtitles') + + +@step('when I view the video it does not show the captions') +def does_not_show_captions(step): + assert world.css_find('.video')[0].has_class('closed') diff --git a/cms/djangoapps/contentstore/tests/test_checklists.py b/cms/djangoapps/contentstore/tests/test_checklists.py index f0889b0861..54bc726092 100644 --- a/cms/djangoapps/contentstore/tests/test_checklists.py +++ b/cms/djangoapps/contentstore/tests/test_checklists.py @@ -19,6 +19,24 @@ class ChecklistTestCase(CourseTestCase): modulestore = get_modulestore(self.course.location) return modulestore.get_item(self.course.location).checklists + + def compare_checklists(self, persisted, request): + """ + Handles url expansion as possible difference and descends into guts + :param persisted: + :param request: + """ + self.assertEqual(persisted['short_description'], request['short_description']) + compare_urls = (persisted.get('action_urls_expanded') == request.get('action_urls_expanded')) + for pers, req in zip(persisted['items'], request['items']): + self.assertEqual(pers['short_description'], req['short_description']) + self.assertEqual(pers['long_description'], req['long_description']) + self.assertEqual(pers['is_checked'], req['is_checked']) + if compare_urls: + self.assertEqual(pers['action_url'], req['action_url']) + self.assertEqual(pers['action_text'], req['action_text']) + self.assertEqual(pers['action_external'], req['action_external']) + def test_get_checklists(self): """ Tests the get checklists method. """ checklists_url = get_url_reverse('Checklists', self.course) @@ -31,9 +49,9 @@ class ChecklistTestCase(CourseTestCase): self.course.checklists = None modulestore = get_modulestore(self.course.location) modulestore.update_metadata(self.course.location, own_metadata(self.course)) - self.assertEquals(self.get_persisted_checklists(), None) + self.assertEqual(self.get_persisted_checklists(), None) response = self.client.get(checklists_url) - self.assertEquals(payload, response.content) + self.assertEqual(payload, response.content) def test_update_checklists_no_index(self): """ No checklist index, should return all of them. """ @@ -43,7 +61,8 @@ class ChecklistTestCase(CourseTestCase): 'name': self.course.location.name}) returned_checklists = json.loads(self.client.get(update_url).content) - self.assertListEqual(self.get_persisted_checklists(), returned_checklists) + for pay, resp in zip(self.get_persisted_checklists(), returned_checklists): + self.compare_checklists(pay, resp) def test_update_checklists_index_ignored_on_get(self): """ Checklist index ignored on get. """ @@ -53,7 +72,8 @@ class ChecklistTestCase(CourseTestCase): 'checklist_index': 1}) returned_checklists = json.loads(self.client.get(update_url).content) - self.assertListEqual(self.get_persisted_checklists(), returned_checklists) + for pay, resp in zip(self.get_persisted_checklists(), returned_checklists): + self.compare_checklists(pay, resp) def test_update_checklists_post_no_index(self): """ No checklist index, will error on post. """ @@ -78,13 +98,18 @@ class ChecklistTestCase(CourseTestCase): 'course': self.course.location.course, 'name': self.course.location.name, 'checklist_index': 2}) + + def get_first_item(checklist): + return checklist['items'][0] + payload = self.course.checklists[2] - self.assertFalse(payload.get('is_checked')) - payload['is_checked'] = True + self.assertFalse(get_first_item(payload).get('is_checked')) + get_first_item(payload)['is_checked'] = True returned_checklist = json.loads(self.client.post(update_url, json.dumps(payload), "application/json").content) - self.assertTrue(returned_checklist.get('is_checked')) - self.assertEqual(self.get_persisted_checklists()[2], returned_checklist) + self.assertTrue(get_first_item(returned_checklist).get('is_checked')) + pers = self.get_persisted_checklists() + self.compare_checklists(pers[2], returned_checklist) def test_update_checklists_delete_unsupported(self): """ Delete operation is not supported. """ @@ -93,4 +118,4 @@ class ChecklistTestCase(CourseTestCase): 'name': self.course.location.name, 'checklist_index': 100}) response = self.client.delete(update_url) - self.assertContains(response, 'Unsupported request', status_code=400) \ No newline at end of file + self.assertContains(response, 'Unsupported request', status_code=400) diff --git a/cms/djangoapps/contentstore/tests/test_contentstore.py b/cms/djangoapps/contentstore/tests/test_contentstore.py index 771871e9bc..03449fc22f 100644 --- a/cms/djangoapps/contentstore/tests/test_contentstore.py +++ b/cms/djangoapps/contentstore/tests/test_contentstore.py @@ -37,6 +37,9 @@ from xmodule.modulestore.exceptions import ItemNotFoundError from contentstore.views.component import ADVANCED_COMPONENT_TYPES from django_comment_common.utils import are_permissions_roles_seeded +from xmodule.exceptions import InvalidVersionError +import datetime +from pytz import UTC TEST_DATA_MODULESTORE = copy.deepcopy(settings.MODULESTORE) TEST_DATA_MODULESTORE['default']['OPTIONS']['fs_root'] = path('common/test/data') @@ -120,6 +123,17 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): def test_advanced_components_require_two_clicks(self): self.check_components_on_page(['videoalpha'], ['Video Alpha']) + def test_malformed_edit_unit_request(self): + store = modulestore('direct') + import_from_xml(store, 'common/test/data/', ['simple']) + + # just pick one vertical + descriptor = store.get_items(Location('i4x', 'edX', 'simple', 'vertical', None, None))[0] + location = descriptor.location._replace(name='.' + descriptor.location.name) + + resp = self.client.get(reverse('edit_unit', kwargs={'location': location.url()})) + self.assertEqual(resp.status_code, 400) + def check_edit_unit(self, test_course_name): import_from_xml(modulestore('direct'), 'common/test/data/', [test_course_name]) @@ -257,7 +271,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): ) self.assertTrue(getattr(draft_problem, 'is_draft', False)) - #now requery with depth + # now requery with depth course = modulestore('draft').get_item( Location(['i4x', 'edX', 'simple', 'course', '2012_Fall', None]), depth=None @@ -404,6 +418,32 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): resp = self.client.get(reverse('edit_unit', kwargs={'location': new_loc.url()})) self.assertEqual(resp.status_code, 200) + def test_illegal_draft_crud_ops(self): + draft_store = modulestore('draft') + direct_store = modulestore('direct') + + CourseFactory.create(org='MITx', course='999', display_name='Robot Super Course') + + location = Location('i4x://MITx/999/chapter/neuvo') + self.assertRaises(InvalidVersionError, draft_store.clone_item, 'i4x://edx/templates/chapter/Empty', + location) + direct_store.clone_item('i4x://edx/templates/chapter/Empty', location) + self.assertRaises(InvalidVersionError, draft_store.clone_item, location, + location) + + self.assertRaises(InvalidVersionError, draft_store.update_item, location, + 'chapter data') + + # taking advantage of update_children and other functions never checking that the ids are valid + self.assertRaises(InvalidVersionError, draft_store.update_children, location, + ['i4x://MITx/999/problem/doesntexist']) + + self.assertRaises(InvalidVersionError, draft_store.update_metadata, location, + {'due': datetime.datetime.now(UTC)}) + + self.assertRaises(InvalidVersionError, draft_store.unpublish, location) + + def test_bad_contentstore_request(self): resp = self.client.get('http://localhost:8001/c4x/CDX/123123/asset/&images_circuits_Lab7Solution2.png') self.assertEqual(resp.status_code, 400) @@ -499,7 +539,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): on_disk = loads(grading_policy.read()) self.assertEqual(on_disk, course.grading_policy) - #check for policy.json + # check for policy.json self.assertTrue(filesystem.exists('policy.json')) # compare what's on disk to what we have in the course module diff --git a/cms/djangoapps/contentstore/tests/test_course_settings.py b/cms/djangoapps/contentstore/tests/test_course_settings.py index 2a4ff46038..8c15b1ae95 100644 --- a/cms/djangoapps/contentstore/tests/test_course_settings.py +++ b/cms/djangoapps/contentstore/tests/test_course_settings.py @@ -54,6 +54,7 @@ class CourseDetailsTestCase(CourseTestCase): def test_virgin_fetch(self): details = CourseDetails.fetch(self.course_location) self.assertEqual(details.course_location, self.course_location, "Location not copied into") + self.assertIsNotNone(details.start_date.tzinfo) self.assertIsNone(details.end_date, "end date somehow initialized " + str(details.end_date)) self.assertIsNone(details.enrollment_start, "enrollment_start date somehow initialized " + str(details.enrollment_start)) self.assertIsNone(details.enrollment_end, "enrollment_end date somehow initialized " + str(details.enrollment_end)) @@ -67,7 +68,6 @@ class CourseDetailsTestCase(CourseTestCase): jsondetails = json.dumps(details, cls=CourseSettingsEncoder) jsondetails = json.loads(jsondetails) self.assertTupleEqual(Location(jsondetails['course_location']), self.course_location, "Location !=") - # Note, start_date is being initialized someplace. I'm not sure why b/c the default will make no sense. self.assertIsNone(jsondetails['end_date'], "end date somehow initialized ") self.assertIsNone(jsondetails['enrollment_start'], "enrollment_start date somehow initialized ") self.assertIsNone(jsondetails['enrollment_end'], "enrollment_end date somehow initialized ") @@ -76,6 +76,23 @@ class CourseDetailsTestCase(CourseTestCase): self.assertIsNone(jsondetails['intro_video'], "intro_video somehow initialized") self.assertIsNone(jsondetails['effort'], "effort somehow initialized") + def test_ooc_encoder(self): + """ + Test the encoder out of its original constrained purpose to see if it functions for general use + """ + details = {'location': Location(['tag', 'org', 'course', 'category', 'name']), + 'number': 1, + 'string': 'string', + 'datetime': datetime.datetime.now(UTC())} + jsondetails = json.dumps(details, cls=CourseSettingsEncoder) + jsondetails = json.loads(jsondetails) + + self.assertIn('location', jsondetails) + self.assertIn('org', jsondetails['location']) + self.assertEquals('org', jsondetails['location'][1]) + self.assertEquals(1, jsondetails['number']) + self.assertEqual(jsondetails['string'], 'string') + def test_update_and_fetch(self): # # NOTE: I couldn't figure out how to validly test time setting w/ all the conversions jsondetails = CourseDetails.fetch(self.course_location) @@ -116,11 +133,8 @@ class CourseDetailsViewTest(CourseTestCase): self.compare_details_with_encoding(json.loads(resp.content), details.__dict__, field + str(val)) @staticmethod - def convert_datetime_to_iso(datetime): - if datetime is not None: - return datetime.isoformat("T") - else: - return None + def convert_datetime_to_iso(dt): + return Date().to_json(dt) def test_update_and_fetch(self): details = CourseDetails.fetch(self.course_location) @@ -151,22 +165,12 @@ class CourseDetailsViewTest(CourseTestCase): self.assertEqual(details['intro_video'], encoded.get('intro_video', None), context + " intro_video not ==") self.assertEqual(details['effort'], encoded['effort'], context + " efforts not ==") - @staticmethod - def struct_to_datetime(struct_time): - return datetime.datetime(*struct_time[:6], tzinfo=UTC()) - def compare_date_fields(self, details, encoded, context, field): if details[field] is not None: date = Date() if field in encoded and encoded[field] is not None: - encoded_encoded = date.from_json(encoded[field]) - dt1 = CourseDetailsViewTest.struct_to_datetime(encoded_encoded) - - if isinstance(details[field], datetime.datetime): - dt2 = details[field] - else: - details_encoded = date.from_json(details[field]) - dt2 = CourseDetailsViewTest.struct_to_datetime(details_encoded) + dt1 = date.from_json(encoded[field]) + dt2 = details[field] expected_delta = datetime.timedelta(0) self.assertEqual(dt1 - dt2, expected_delta, str(dt1) + "!=" + str(dt2) + " at " + context) diff --git a/cms/djangoapps/contentstore/tests/test_utils.py b/cms/djangoapps/contentstore/tests/test_utils.py index c4b0f4bb51..fec82db1bb 100644 --- a/cms/djangoapps/contentstore/tests/test_utils.py +++ b/cms/djangoapps/contentstore/tests/test_utils.py @@ -24,6 +24,30 @@ class LMSLinksTestCase(TestCase): with mock.patch.dict('django.conf.settings.MITX_FEATURES', {'ENABLE_MKTG_SITE': False}): self.assertEquals(self.get_about_page_link(), "//localhost:8000/courses/mitX/101/test/about") + @override_settings(MKTG_URLS={'ROOT': 'http://www.dummy'}) + def about_page_marketing_site_remove_http_test(self): + """ Get URL for about page, marketing root present, remove http://. """ + with mock.patch.dict('django.conf.settings.MITX_FEATURES', {'ENABLE_MKTG_SITE': True}): + self.assertEquals(self.get_about_page_link(), "//www.dummy/courses/mitX/101/test/about") + + @override_settings(MKTG_URLS={'ROOT': 'https://www.dummy'}) + def about_page_marketing_site_remove_https_test(self): + """ Get URL for about page, marketing root present, remove https://. """ + with mock.patch.dict('django.conf.settings.MITX_FEATURES', {'ENABLE_MKTG_SITE': True}): + self.assertEquals(self.get_about_page_link(), "//www.dummy/courses/mitX/101/test/about") + + @override_settings(MKTG_URLS={'ROOT': 'www.dummyhttps://x'}) + def about_page_marketing_site_https__edge_test(self): + """ Get URL for about page, only remove https:// at the beginning of the string. """ + with mock.patch.dict('django.conf.settings.MITX_FEATURES', {'ENABLE_MKTG_SITE': True}): + self.assertEquals(self.get_about_page_link(), "//www.dummyhttps://x/courses/mitX/101/test/about") + + @override_settings(MKTG_URLS={}) + def about_page_marketing_urls_not_set_test(self): + """ Error case. ENABLE_MKTG_SITE is True, but there is either no MKTG_URLS, or no MKTG_URLS Root property. """ + with mock.patch.dict('django.conf.settings.MITX_FEATURES', {'ENABLE_MKTG_SITE': True}): + self.assertEquals(self.get_about_page_link(), None) + @override_settings(LMS_BASE=None) def about_page_no_lms_base_test(self): """ No LMS_BASE, nor is ENABLE_MKTG_SITE True """ diff --git a/cms/djangoapps/contentstore/utils.py b/cms/djangoapps/contentstore/utils.py index 703e9a266a..6f766ff7f5 100644 --- a/cms/djangoapps/contentstore/utils.py +++ b/cms/djangoapps/contentstore/utils.py @@ -4,8 +4,11 @@ from xmodule.modulestore.django import modulestore from xmodule.modulestore.exceptions import ItemNotFoundError from django.core.urlresolvers import reverse import copy +import logging +import re +from xmodule.modulestore.draft import DIRECT_ONLY_CATEGORIES -DIRECT_ONLY_CATEGORIES = ['course', 'chapter', 'sequential', 'about', 'static_tab', 'course_info'] +log = logging.getLogger(__name__) #In order to instantiate an open ended tab automatically, need to have this data OPEN_ENDED_PANEL = {"name": "Open Ended Panel", "type": "open_ended"} @@ -108,9 +111,20 @@ def get_lms_link_for_about_page(location): Returns the url to the course about page from the location tuple. """ if settings.MITX_FEATURES.get('ENABLE_MKTG_SITE', False): - # Root will be "www.edx.org". The complete URL will still not be exactly correct, - # but redirects exist from www.edx.org to get to the drupal course about page URL. - about_base = settings.MKTG_URLS.get('ROOT') + if not hasattr(settings, 'MKTG_URLS'): + log.exception("ENABLE_MKTG_SITE is True, but MKTG_URLS is not defined.") + about_base = None + else: + marketing_urls = settings.MKTG_URLS + if marketing_urls.get('ROOT', None) is None: + log.exception('There is no ROOT defined in MKTG_URLS') + about_base = None + else: + # Root will be "https://www.edx.org". The complete URL will still not be exactly correct, + # but redirects exist from www.edx.org to get to the Drupal course about page URL. + about_base = marketing_urls.get('ROOT') + # Strip off https:// (or http://) to be consistent with the formatting of LMS_BASE. + about_base = re.sub(r"^https?://", "", about_base) elif settings.LMS_BASE is not None: about_base = settings.LMS_BASE else: @@ -214,7 +228,7 @@ def add_extra_panel_tab(tab_type, course): course_tabs = copy.copy(course.tabs) changed = False #Check to see if open ended panel is defined in the course - + tab_panel = EXTRA_TAB_PANELS.get(tab_type) if tab_panel not in course_tabs: #Add panel to the tabs if it is not defined diff --git a/cms/djangoapps/contentstore/views/assets.py b/cms/djangoapps/contentstore/views/assets.py index b5041d3e9f..229788f24d 100644 --- a/cms/djangoapps/contentstore/views/assets.py +++ b/cms/djangoapps/contentstore/views/assets.py @@ -62,7 +62,7 @@ def asset_index(request, org, course, name): asset_id = asset['_id'] display_info = {} display_info['displayname'] = asset['displayname'] - display_info['uploadDate'] = get_default_time_display(asset['uploadDate'].timetuple()) + display_info['uploadDate'] = get_default_time_display(asset['uploadDate']) asset_location = StaticContent.compute_location(asset_id['org'], asset_id['course'], asset_id['name']) display_info['url'] = StaticContent.get_url_path_from_location(asset_location) @@ -103,6 +103,9 @@ def upload_asset(request, org, course, coursename): logging.error('Could not find course' + location) return HttpResponseBadRequest() + if 'file' not in request.FILES: + return HttpResponseBadRequest() + # compute a 'filename' which is similar to the location formatting, we're using the 'filename' # nomenclature since we're using a FileSystem paradigm here. We're just imposing # the Location string formatting expectations to keep things a bit more consistent @@ -131,7 +134,7 @@ def upload_asset(request, org, course, coursename): readback = contentstore().find(content.location) response_payload = {'displayname': content.name, - 'uploadDate': get_default_time_display(readback.last_modified_at.timetuple()), + 'uploadDate': get_default_time_display(readback.last_modified_at), 'url': StaticContent.get_url_path_from_location(content.location), 'thumb_url': StaticContent.get_url_path_from_location(thumbnail_location) if thumbnail_content is not None else None, 'msg': 'Upload completed' @@ -227,11 +230,9 @@ def generate_export_course(request, org, course, name): root_dir = path(mkdtemp()) # export out to a tempdir - logging.debug('root = {0}'.format(root_dir)) export_to_xml(modulestore('direct'), contentstore(), loc, root_dir, name, modulestore()) - #filename = root_dir / name + '.tar.gz' logging.debug('tar file being generated at {0}'.format(export_file.name)) tar_file = tarfile.open(name=export_file.name, mode='w:gz') diff --git a/cms/djangoapps/contentstore/views/component.py b/cms/djangoapps/contentstore/views/component.py index 8120e08107..039deb2740 100644 --- a/cms/djangoapps/contentstore/views/component.py +++ b/cms/djangoapps/contentstore/views/component.py @@ -7,7 +7,7 @@ from django.contrib.auth.decorators import login_required from django.core.exceptions import PermissionDenied from django_future.csrf import ensure_csrf_cookie from django.conf import settings - +from xmodule.modulestore.exceptions import ItemNotFoundError, InvalidLocationError from mitxmako.shortcuts import render_to_response from xmodule.modulestore import Location @@ -50,11 +50,18 @@ ADVANCED_COMPONENT_POLICY_KEY = 'advanced_modules' @login_required def edit_subsection(request, location): # check that we have permissions to edit this item - course = get_course_for_item(location) + try: + course = get_course_for_item(location) + except InvalidLocationError: + return HttpResponseBadRequest() + if not has_access(request.user, course.location): raise PermissionDenied() - item = modulestore().get_item(location, depth=1) + try: + item = modulestore().get_item(location, depth=1) + except ItemNotFoundError: + return HttpResponseBadRequest() lms_link = get_lms_link_for_item(location, course_id=course.location.course_id) preview_link = get_lms_link_for_item(location, course_id=course.location.course_id, preview=True) @@ -113,11 +120,18 @@ def edit_unit(request, location): id: A Location URL """ - course = get_course_for_item(location) + try: + course = get_course_for_item(location) + except InvalidLocationError: + return HttpResponseBadRequest() + if not has_access(request.user, course.location): raise PermissionDenied() - item = modulestore().get_item(location, depth=1) + try: + item = modulestore().get_item(location, depth=1) + except ItemNotFoundError: + return HttpResponseBadRequest() lms_link = get_lms_link_for_item(item.location, course_id=course.location.course_id) diff --git a/cms/djangoapps/contentstore/views/course.py b/cms/djangoapps/contentstore/views/course.py index 07f6b9669c..8762eb3a2a 100644 --- a/cms/djangoapps/contentstore/views/course.py +++ b/cms/djangoapps/contentstore/views/course.py @@ -2,7 +2,6 @@ Views related to operations on course objects """ import json -import time from django.contrib.auth.decorators import login_required from django_future.csrf import ensure_csrf_cookie @@ -32,6 +31,8 @@ from .component import OPEN_ENDED_COMPONENT_TYPES, \ NOTE_COMPONENT_TYPES, ADVANCED_COMPONENT_POLICY_KEY from django_comment_common.utils import seed_permissions_roles +import datetime +from django.utils.timezone import UTC # TODO: should explicitly enumerate exports with __all__ @@ -130,7 +131,7 @@ def create_new_course(request): new_course.display_name = display_name # set a default start date to now - new_course.start = time.gmtime() + new_course.start = datetime.datetime.now(UTC()) initialize_course_tabs(new_course) @@ -357,52 +358,55 @@ def course_advanced_updates(request, org, course, name): # Whether or not to filter the tabs key out of the settings metadata filter_tabs = True - #Check to see if the user instantiated any advanced components. This is a hack - #that does the following : - # 1) adds/removes the open ended panel tab to a course automatically if the user + # Check to see if the user instantiated any advanced components. This is a hack + # that does the following : + # 1) adds/removes the open ended panel tab to a course automatically if the user # has indicated that they want to edit the combinedopendended or peergrading module # 2) adds/removes the notes panel tab to a course automatically if the user has # indicated that they want the notes module enabled in their course # TODO refactor the above into distinct advanced policy settings if ADVANCED_COMPONENT_POLICY_KEY in request_body: - #Get the course so that we can scrape current tabs + # Get the course so that we can scrape current tabs course_module = modulestore().get_item(location) - #Maps tab types to components + # Maps tab types to components tab_component_map = { - 'open_ended': OPEN_ENDED_COMPONENT_TYPES, + 'open_ended': OPEN_ENDED_COMPONENT_TYPES, 'notes': NOTE_COMPONENT_TYPES, } - #Check to see if the user instantiated any notes or open ended components + # Check to see if the user instantiated any notes or open ended components for tab_type in tab_component_map.keys(): component_types = tab_component_map.get(tab_type) found_ac_type = False for ac_type in component_types: if ac_type in request_body[ADVANCED_COMPONENT_POLICY_KEY]: - #Add tab to the course if needed + # Add tab to the course if needed changed, new_tabs = add_extra_panel_tab(tab_type, course_module) - #If a tab has been added to the course, then send the metadata along to CourseMetadata.update_from_json + # If a tab has been added to the course, then send the metadata along to CourseMetadata.update_from_json if changed: course_module.tabs = new_tabs request_body.update({'tabs': new_tabs}) - #Indicate that tabs should not be filtered out of the metadata + # Indicate that tabs should not be filtered out of the metadata filter_tabs = False - #Set this flag to avoid the tab removal code below. + # Set this flag to avoid the tab removal code below. found_ac_type = True break - #If we did not find a module type in the advanced settings, + # If we did not find a module type in the advanced settings, # we may need to remove the tab from the course. if not found_ac_type: - #Remove tab from the course if needed + # Remove tab from the course if needed changed, new_tabs = remove_extra_panel_tab(tab_type, course_module) if changed: course_module.tabs = new_tabs request_body.update({'tabs': new_tabs}) - #Indicate that tabs should *not* be filtered out of the metadata + # Indicate that tabs should *not* be filtered out of the metadata filter_tabs = False - - response_json = json.dumps(CourseMetadata.update_from_json(location, + try: + response_json = json.dumps(CourseMetadata.update_from_json(location, request_body, filter_tabs=filter_tabs)) + except (TypeError, ValueError), e: + return HttpResponseBadRequest("Incorrect setting format. " + str(e), content_type="text/plain") + return HttpResponse(response_json, mimetype="application/json") diff --git a/cms/djangoapps/models/settings/course_details.py b/cms/djangoapps/models/settings/course_details.py index 0dbb47b31b..07eb4bc309 100644 --- a/cms/djangoapps/models/settings/course_details.py +++ b/cms/djangoapps/models/settings/course_details.py @@ -3,26 +3,26 @@ from xmodule.modulestore.exceptions import ItemNotFoundError from xmodule.modulestore.inheritance import own_metadata import json from json.encoder import JSONEncoder -import time from contentstore.utils import get_modulestore from models.settings import course_grading from contentstore.utils import update_item from xmodule.fields import Date import re import logging +import datetime class CourseDetails(object): def __init__(self, location): - self.course_location = location # a Location obj + self.course_location = location # a Location obj self.start_date = None # 'start' - self.end_date = None # 'end' + self.end_date = None # 'end' self.enrollment_start = None self.enrollment_end = None - self.syllabus = None # a pdf file asset - self.overview = "" # html to render as the overview - self.intro_video = None # a video pointer - self.effort = None # int hours/week + self.syllabus = None # a pdf file asset + self.overview = "" # html to render as the overview + self.intro_video = None # a video pointer + self.effort = None # int hours/week @classmethod def fetch(cls, course_location): @@ -73,9 +73,9 @@ class CourseDetails(object): """ Decode the json into CourseDetails and save any changed attrs to the db """ - ## TODO make it an error for this to be undefined & for it to not be retrievable from modulestore + # TODO make it an error for this to be undefined & for it to not be retrievable from modulestore course_location = jsondict['course_location'] - ## Will probably want to cache the inflight courses because every blur generates an update + # Will probably want to cache the inflight courses because every blur generates an update descriptor = get_modulestore(course_location).get_item(course_location) dirty = False @@ -181,7 +181,7 @@ class CourseSettingsEncoder(json.JSONEncoder): return obj.__dict__ elif isinstance(obj, Location): return obj.dict() - elif isinstance(obj, time.struct_time): + elif isinstance(obj, datetime.datetime): return Date().to_json(obj) else: return JSONEncoder.default(self, obj) diff --git a/cms/envs/acceptance.py b/cms/envs/acceptance.py index 36616ab257..6293219f43 100644 --- a/cms/envs/acceptance.py +++ b/cms/envs/acceptance.py @@ -23,7 +23,7 @@ MODULESTORE_OPTIONS = { 'db': 'test_xmodule', 'collection': 'acceptance_modulestore', 'fs_root': TEST_ROOT / "data", - 'render_template': 'mitxmako.shortcuts.render_to_string', + 'render_template': 'mitxmako.shortcuts.render_to_string' } MODULESTORE = { diff --git a/cms/envs/aws.py b/cms/envs/aws.py index af5f39631d..35b15fe6ba 100644 --- a/cms/envs/aws.py +++ b/cms/envs/aws.py @@ -103,6 +103,7 @@ DEFAULT_FROM_EMAIL = ENV_TOKENS.get('DEFAULT_FROM_EMAIL', DEFAULT_FROM_EMAIL) DEFAULT_FEEDBACK_EMAIL = ENV_TOKENS.get('DEFAULT_FEEDBACK_EMAIL', DEFAULT_FEEDBACK_EMAIL) ADMINS = ENV_TOKENS.get('ADMINS', ADMINS) SERVER_EMAIL = ENV_TOKENS.get('SERVER_EMAIL', SERVER_EMAIL) +MKTG_URLS = ENV_TOKENS.get('MKTG_URLS', MKTG_URLS) #Timezone overrides TIME_ZONE = ENV_TOKENS.get('TIME_ZONE', TIME_ZONE) diff --git a/cms/envs/common.py b/cms/envs/common.py index 04d5888750..5962cec340 100644 --- a/cms/envs/common.py +++ b/cms/envs/common.py @@ -25,6 +25,7 @@ Longer TODO: import sys import lms.envs.common +from lms.envs.common import USE_TZ from path import path ############################ FEATURE CONFIGURATION ############################# @@ -34,8 +35,8 @@ MITX_FEATURES = { 'GITHUB_PUSH': False, 'ENABLE_DISCUSSION_SERVICE': False, 'AUTH_USE_MIT_CERTIFICATES': False, - 'STUB_VIDEO_FOR_TESTING': False, # do not display video when running automated acceptance tests - 'STAFF_EMAIL': '', # email address for staff (eg to request course creation) + 'STUB_VIDEO_FOR_TESTING': False, # do not display video when running automated acceptance tests + 'STAFF_EMAIL': '', # email address for staff (eg to request course creation) 'STUDIO_NPS_SURVEY': True, 'SEGMENT_IO': True, @@ -183,7 +184,7 @@ STATICFILES_DIRS = [ # Locale/Internationalization TIME_ZONE = 'America/New_York' # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name -LANGUAGE_CODE = 'en' # http://www.i18nguy.com/unicode/language-identifiers.html +LANGUAGE_CODE = 'en' # http://www.i18nguy.com/unicode/language-identifiers.html USE_I18N = True USE_L10N = True @@ -335,3 +336,14 @@ INSTALLED_APPS = ( ################# EDX MARKETING SITE ################################## EDXMKTG_COOKIE_NAME = 'edxloggedin' +MKTG_URLS = {} +MKTG_URL_LINK_MAP = { + 'ABOUT': 'about_edx', + 'CONTACT': 'contact', + 'FAQ': 'help_edx', + 'COURSES': 'courses', + 'ROOT': 'root', + 'TOS': 'tos', + 'HONOR': 'honor', + 'PRIVACY': 'privacy_edx', +} diff --git a/cms/envs/dev.py b/cms/envs/dev.py index e63968d338..cbe47a1fe1 100644 --- a/cms/envs/dev.py +++ b/cms/envs/dev.py @@ -22,7 +22,7 @@ modulestore_options = { 'db': 'xmodule', 'collection': 'modulestore', 'fs_root': GITHUB_REPO_ROOT, - 'render_template': 'mitxmako.shortcuts.render_to_string', + 'render_template': 'mitxmako.shortcuts.render_to_string' } MODULESTORE = { @@ -64,7 +64,7 @@ REPOS = { }, 'content-mit-6002x': { 'branch': 'master', - #'origin': 'git@github.com:MITx/6002x-fall-2012.git', + # 'origin': 'git@github.com:MITx/6002x-fall-2012.git', 'origin': 'git@github.com:MITx/content-mit-6002x.git', }, '6.00x': { @@ -165,3 +165,11 @@ MITX_FEATURES['ENABLE_SERVICE_STATUS'] = True # segment-io key for dev SEGMENT_IO_KEY = 'mty8edrrsg' + + +##################################################################### +# Lastly, see if the developer has any local overrides. +try: + from .private import * +except ImportError: + pass diff --git a/cms/envs/test.py b/cms/envs/test.py index 8a3f9ba158..1569d0a42d 100644 --- a/cms/envs/test.py +++ b/cms/envs/test.py @@ -48,7 +48,7 @@ MODULESTORE_OPTIONS = { 'db': 'test_xmodule', 'collection': 'test_modulestore', 'fs_root': TEST_ROOT / "data", - 'render_template': 'mitxmako.shortcuts.render_to_string', + 'render_template': 'mitxmako.shortcuts.render_to_string' } MODULESTORE = { @@ -121,7 +121,7 @@ CELERY_RESULT_BACKEND = 'cache' BROKER_TRANSPORT = 'memory' ################### Make tests faster -#http://slacy.com/blog/2012/04/make-your-tests-faster-in-django-1-4/ +# http://slacy.com/blog/2012/04/make-your-tests-faster-in-django-1-4/ PASSWORD_HASHERS = ( 'django.contrib.auth.hashers.SHA1PasswordHasher', 'django.contrib.auth.hashers.MD5PasswordHasher', diff --git a/cms/pydev_manage.py b/cms/pydev_manage.py new file mode 100644 index 0000000000..22c38d89eb --- /dev/null +++ b/cms/pydev_manage.py @@ -0,0 +1,11 @@ +''' +Used for pydev eclipse. Should be innocuous for everyone else. +Created on May 8, 2013 + +@author: dmitchell +''' +#!/home//mitx_all/python/bin/python +from django.core import management + +if __name__ == '__main__': + management.execute_from_command_line() diff --git a/cms/static/js/base.js b/cms/static/js/base.js index fe60d80239..c626fa1b3f 100644 --- a/cms/static/js/base.js +++ b/cms/static/js/base.js @@ -411,8 +411,12 @@ function showFileSelectionMenu(e) { } function startUpload(e) { + var files = $('.file-input').get(0).files; + if (files.length === 0) + return; + $('.upload-modal h1').html(gettext('Uploading…')); - $('.upload-modal .file-name').html($('.file-input').val().replace('C:\\fakepath\\', '')); + $('.upload-modal .file-name').html(files[0].name); $('.upload-modal .file-chooser').ajaxSubmit({ beforeSend: resetUploadBar, uploadProgress: showUploadFeedback, diff --git a/cms/static/sass/_base.scss b/cms/static/sass/_base.scss index e0e7543b8e..3713f83ae3 100644 --- a/cms/static/sass/_base.scss +++ b/cms/static/sass/_base.scss @@ -14,7 +14,7 @@ body { color: $gray-d2; } -body, input { +body, input, button { font-family: 'Open Sans', sans-serif; } diff --git a/cms/templates/edit_subsection.html b/cms/templates/edit_subsection.html index 9bb9b3a506..cbce91ab44 100644 --- a/cms/templates/edit_subsection.html +++ b/cms/templates/edit_subsection.html @@ -1,7 +1,7 @@ <%inherit file="base.html" /> <%! import logging - from xmodule.util.date_utils import get_time_struct_display + from xmodule.util.date_utils import get_default_time_display %> <%! from django.core.urlresolvers import reverse %> @@ -36,11 +36,15 @@
- +
- +
% if subsection.lms.start != parent_item.lms.start and subsection.lms.start: @@ -48,7 +52,7 @@

The date above differs from the release date of ${parent_item.display_name_with_default}, which is unset. % else:

The date above differs from the release date of ${parent_item.display_name_with_default} – - ${get_time_struct_display(parent_item.lms.start, '%m/%d/%Y at %H:%M UTC')}. + ${get_default_time_display(parent_item.lms.start)}. % endif

% endif @@ -65,11 +69,15 @@
- +
- +
Remove due date
diff --git a/cms/templates/overview.html b/cms/templates/overview.html index d327c8b324..43d0afc263 100644 --- a/cms/templates/overview.html +++ b/cms/templates/overview.html @@ -1,7 +1,7 @@ <%inherit file="base.html" /> <%! import logging - from xmodule.util.date_utils import get_time_struct_display + from xmodule.util import date_utils %> <%! from django.core.urlresolvers import reverse %> <%block name="title">Course Outline @@ -154,14 +154,19 @@

<% - start_date_str = get_time_struct_display(section.lms.start, '%m/%d/%Y') - start_time_str = get_time_struct_display(section.lms.start, '%H:%M') + if section.lms.start is not None: + start_date_str = section.lms.start.strftime('%m/%d/%Y') + start_time_str = section.lms.start.strftime('%H:%M') + else: + start_date_str = '' + start_time_str = '' %> %if section.lms.start is None: This section has not been released. Schedule %else: - Will Release: ${get_time_struct_display(section.lms.start, '%m/%d/%Y at %H:%M UTC')} + Will Release: + ${date_utils.get_default_time_display(section.lms.start)} Edit %endif
diff --git a/cms/xmodule_namespace.py b/cms/xmodule_namespace.py index 4857fe68ca..eef4b41f37 100644 --- a/cms/xmodule_namespace.py +++ b/cms/xmodule_namespace.py @@ -5,7 +5,6 @@ Namespace defining common fields used by Studio for all blocks import datetime from xblock.core import Namespace, Scope, ModelType, String -from xmodule.fields import StringyBoolean class DateTuple(ModelType): @@ -28,4 +27,3 @@ class CmsNamespace(Namespace): """ published_date = DateTuple(help="Date when the module was published", scope=Scope.settings) published_by = String(help="Id of the user who published this module", scope=Scope.settings) - diff --git a/common/djangoapps/contentserver/middleware.py b/common/djangoapps/contentserver/middleware.py index 8e9e70046d..7deb0901aa 100644 --- a/common/djangoapps/contentserver/middleware.py +++ b/common/djangoapps/contentserver/middleware.py @@ -1,7 +1,4 @@ -import logging -import time - -from django.http import HttpResponse, Http404, HttpResponseNotModified +from django.http import HttpResponse, HttpResponseNotModified from xmodule.contentstore.django import contentstore from xmodule.contentstore.content import StaticContent, XASSET_LOCATION_TAG @@ -20,7 +17,7 @@ class StaticContentServer(object): # return a 'Bad Request' to browser as we have a malformed Location response = HttpResponse() response.status_code = 400 - return response + return response # first look in our cache so we don't have to round-trip to the DB content = get_cached_content(loc) diff --git a/common/djangoapps/mitxmako/tests.py b/common/djangoapps/mitxmako/tests.py index 21866eb9b5..e7e56a9472 100644 --- a/common/djangoapps/mitxmako/tests.py +++ b/common/djangoapps/mitxmako/tests.py @@ -1,16 +1,15 @@ from django.test import TestCase from django.test.utils import override_settings from django.core.urlresolvers import reverse -from django.conf import settings from mitxmako.shortcuts import marketing_link from mock import patch +from util.testing import UrlResetMixin -class ShortcutsTests(TestCase): +class ShortcutsTests(UrlResetMixin, TestCase): """ Test the mitxmako shortcuts file """ - @override_settings(MKTG_URLS={'ROOT': 'dummy-root', 'ABOUT': '/about-us'}) @override_settings(MKTG_URL_LINK_MAP={'ABOUT': 'login'}) def test_marketing_link(self): diff --git a/common/djangoapps/student/management/commands/assigngroups.py b/common/djangoapps/student/management/commands/assigngroups.py index fb7bfc85cd..5269c8690e 100644 --- a/common/djangoapps/student/management/commands/assigngroups.py +++ b/common/djangoapps/student/management/commands/assigngroups.py @@ -14,6 +14,7 @@ import sys import datetime import json +from pytz import UTC middleware.MakoMiddleware() @@ -32,7 +33,7 @@ def group_from_value(groups, v): class Command(BaseCommand): - help = \ + help = \ ''' Assign users to test groups. Takes a list of groups: a:0.3,b:0.4,c:0.3 file.txt "Testing something" @@ -75,7 +76,7 @@ Will log what happened to file.txt. utg = UserTestGroup() utg.name = group utg.description = json.dumps({"description": args[2]}, - {"time": datetime.datetime.utcnow().isoformat()}) + {"time": datetime.datetime.now(UTC).isoformat()}) group_objects[group] = utg group_objects[group].save() diff --git a/common/djangoapps/student/management/commands/pearson_export_cdd.py b/common/djangoapps/student/management/commands/pearson_export_cdd.py index bad98b9d25..efb4a55387 100644 --- a/common/djangoapps/student/management/commands/pearson_export_cdd.py +++ b/common/djangoapps/student/management/commands/pearson_export_cdd.py @@ -8,6 +8,7 @@ from django.conf import settings from django.core.management.base import BaseCommand, CommandError from student.models import TestCenterUser +from pytz import UTC class Command(BaseCommand): @@ -58,7 +59,7 @@ class Command(BaseCommand): def handle(self, **options): # update time should use UTC in order to be comparable to the user_updated_at # field - uploaded_at = datetime.utcnow() + uploaded_at = datetime.now(UTC) # if specified destination is an existing directory, then # create a filename for it automatically. If it doesn't exist, @@ -100,7 +101,7 @@ class Command(BaseCommand): extrasaction='ignore') writer.writeheader() for tcu in TestCenterUser.objects.order_by('id'): - if tcu.needs_uploading: # or dump_all + if tcu.needs_uploading: # or dump_all record = dict((csv_field, ensure_encoding(getattr(tcu, model_field))) for csv_field, model_field in Command.CSV_TO_MODEL_FIELDS.items()) diff --git a/common/djangoapps/student/management/commands/pearson_export_ead.py b/common/djangoapps/student/management/commands/pearson_export_ead.py index 03dbce0024..ec10ab1599 100644 --- a/common/djangoapps/student/management/commands/pearson_export_ead.py +++ b/common/djangoapps/student/management/commands/pearson_export_ead.py @@ -8,6 +8,7 @@ from django.conf import settings from django.core.management.base import BaseCommand, CommandError from student.models import TestCenterRegistration, ACCOMMODATION_REJECTED_CODE +from pytz import UTC class Command(BaseCommand): @@ -51,7 +52,7 @@ class Command(BaseCommand): def handle(self, **options): # update time should use UTC in order to be comparable to the user_updated_at # field - uploaded_at = datetime.utcnow() + uploaded_at = datetime.now(UTC) # if specified destination is an existing directory, then # create a filename for it automatically. If it doesn't exist, diff --git a/common/djangoapps/student/management/commands/pearson_import_conf_zip.py b/common/djangoapps/student/management/commands/pearson_import_conf_zip.py index d0b2938df0..2339383719 100644 --- a/common/djangoapps/student/management/commands/pearson_import_conf_zip.py +++ b/common/djangoapps/student/management/commands/pearson_import_conf_zip.py @@ -13,6 +13,7 @@ from django.core.management.base import BaseCommand, CommandError from django.conf import settings from student.models import TestCenterUser, TestCenterRegistration +from pytz import UTC class Command(BaseCommand): @@ -68,7 +69,7 @@ class Command(BaseCommand): Command.datadog_error("Found authorization record for user {}".format(registration.testcenter_user.user.username), eacfile.name) # now update the record: registration.upload_status = row['Status'] - registration.upload_error_message = row['Message'] + registration.upload_error_message = row['Message'] try: registration.processed_at = strftime('%Y-%m-%d %H:%M:%S', strptime(row['Date'], '%Y/%m/%d %H:%M:%S')) except ValueError as ve: @@ -80,7 +81,7 @@ class Command(BaseCommand): except ValueError as ve: Command.datadog_error("Bad AuthorizationID value found for {}: message {}".format(client_authorization_id, ve), eacfile.name) - registration.confirmed_at = datetime.utcnow() + registration.confirmed_at = datetime.now(UTC) registration.save() except TestCenterRegistration.DoesNotExist: Command.datadog_error("Failed to find record for client_auth_id {}".format(client_authorization_id), eacfile.name) diff --git a/common/djangoapps/student/management/commands/pearson_make_tc_registration.py b/common/djangoapps/student/management/commands/pearson_make_tc_registration.py index b10cf143a0..50e56bb4be 100644 --- a/common/djangoapps/student/management/commands/pearson_make_tc_registration.py +++ b/common/djangoapps/student/management/commands/pearson_make_tc_registration.py @@ -1,5 +1,4 @@ from optparse import make_option -from time import strftime from django.contrib.auth.models import User from django.core.management.base import BaseCommand, CommandError @@ -128,8 +127,8 @@ class Command(BaseCommand): exam = CourseDescriptor.TestCenterExam(course_id, exam_name, exam_info) # update option values for date_first and date_last to use YYYY-MM-DD format # instead of YYYY-MM-DDTHH:MM - our_options['eligibility_appointment_date_first'] = strftime("%Y-%m-%d", exam.first_eligible_appointment_date) - our_options['eligibility_appointment_date_last'] = strftime("%Y-%m-%d", exam.last_eligible_appointment_date) + our_options['eligibility_appointment_date_first'] = exam.first_eligible_appointment_date.strftime("%Y-%m-%d") + our_options['eligibility_appointment_date_last'] = exam.last_eligible_appointment_date.strftime("%Y-%m-%d") if exam is None: raise CommandError("Exam for course_id {} does not exist".format(course_id)) diff --git a/common/djangoapps/student/migrations/0025_auto__add_field_courseenrollmentallowed_auto_enroll.py b/common/djangoapps/student/migrations/0025_auto__add_field_courseenrollmentallowed_auto_enroll.py new file mode 100644 index 0000000000..8ce1d0cda1 --- /dev/null +++ b/common/djangoapps/student/migrations/0025_auto__add_field_courseenrollmentallowed_auto_enroll.py @@ -0,0 +1,173 @@ +# -*- coding: utf-8 -*- +import datetime +from south.db import db +from south.v2 import SchemaMigration +from django.db import models + + +class Migration(SchemaMigration): + + def forwards(self, orm): + # Adding field 'CourseEnrollmentAllowed.auto_enroll' + db.add_column('student_courseenrollmentallowed', 'auto_enroll', + self.gf('django.db.models.fields.BooleanField')(default=False), + keep_default=False) + + + def backwards(self, orm): + # Deleting field 'CourseEnrollmentAllowed.auto_enroll' + db.delete_column('student_courseenrollmentallowed', 'auto_enroll') + + + models = { + 'auth.group': { + 'Meta': {'object_name': 'Group'}, + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}), + 'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}) + }, + 'auth.permission': { + 'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'}, + 'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'}) + }, + 'auth.user': { + 'Meta': {'object_name': 'User'}, + 'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}), + 'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}), + 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}), + 'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}), + 'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}), + 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}), + 'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}), + 'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}), + 'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'}) + }, + 'contenttypes.contenttype': { + 'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"}, + 'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}) + }, + 'student.courseenrollment': { + 'Meta': {'unique_together': "(('user', 'course_id'),)", 'object_name': 'CourseEnrollment'}, + 'course_id': ('django.db.models.fields.CharField', [], {'max_length': '255', 'db_index': 'True'}), + 'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'null': 'True', 'db_index': 'True', 'blank': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"}) + }, + 'student.courseenrollmentallowed': { + 'Meta': {'unique_together': "(('email', 'course_id'),)", 'object_name': 'CourseEnrollmentAllowed'}, + 'auto_enroll': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'course_id': ('django.db.models.fields.CharField', [], {'max_length': '255', 'db_index': 'True'}), + 'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'null': 'True', 'db_index': 'True', 'blank': 'True'}), + 'email': ('django.db.models.fields.CharField', [], {'max_length': '255', 'db_index': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}) + }, + 'student.pendingemailchange': { + 'Meta': {'object_name': 'PendingEmailChange'}, + 'activation_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '32', 'db_index': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'new_email': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '255', 'blank': 'True'}), + 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}) + }, + 'student.pendingnamechange': { + 'Meta': {'object_name': 'PendingNameChange'}, + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'new_name': ('django.db.models.fields.CharField', [], {'max_length': '255', 'blank': 'True'}), + 'rationale': ('django.db.models.fields.CharField', [], {'max_length': '1024', 'blank': 'True'}), + 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}) + }, + 'student.registration': { + 'Meta': {'object_name': 'Registration', 'db_table': "'auth_registration'"}, + 'activation_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '32', 'db_index': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']", 'unique': 'True'}) + }, + 'student.testcenterregistration': { + 'Meta': {'object_name': 'TestCenterRegistration'}, + 'accommodation_code': ('django.db.models.fields.CharField', [], {'max_length': '64', 'blank': 'True'}), + 'accommodation_request': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '1024', 'blank': 'True'}), + 'authorization_id': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'db_index': 'True'}), + 'client_authorization_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '20', 'db_index': 'True'}), + 'confirmed_at': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'db_index': 'True'}), + 'course_id': ('django.db.models.fields.CharField', [], {'max_length': '128', 'db_index': 'True'}), + 'created_at': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'db_index': 'True', 'blank': 'True'}), + 'eligibility_appointment_date_first': ('django.db.models.fields.DateField', [], {'db_index': 'True'}), + 'eligibility_appointment_date_last': ('django.db.models.fields.DateField', [], {'db_index': 'True'}), + 'exam_series_code': ('django.db.models.fields.CharField', [], {'max_length': '15', 'db_index': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'processed_at': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'db_index': 'True'}), + 'testcenter_user': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['student.TestCenterUser']"}), + 'updated_at': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'db_index': 'True', 'blank': 'True'}), + 'upload_error_message': ('django.db.models.fields.CharField', [], {'max_length': '512', 'blank': 'True'}), + 'upload_status': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '20', 'blank': 'True'}), + 'uploaded_at': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'db_index': 'True'}), + 'user_updated_at': ('django.db.models.fields.DateTimeField', [], {'db_index': 'True'}) + }, + 'student.testcenteruser': { + 'Meta': {'object_name': 'TestCenterUser'}, + 'address_1': ('django.db.models.fields.CharField', [], {'max_length': '40'}), + 'address_2': ('django.db.models.fields.CharField', [], {'max_length': '40', 'blank': 'True'}), + 'address_3': ('django.db.models.fields.CharField', [], {'max_length': '40', 'blank': 'True'}), + 'candidate_id': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'db_index': 'True'}), + 'city': ('django.db.models.fields.CharField', [], {'max_length': '32', 'db_index': 'True'}), + 'client_candidate_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '50', 'db_index': 'True'}), + 'company_name': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '50', 'blank': 'True'}), + 'confirmed_at': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'db_index': 'True'}), + 'country': ('django.db.models.fields.CharField', [], {'max_length': '3', 'db_index': 'True'}), + 'created_at': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'db_index': 'True', 'blank': 'True'}), + 'extension': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '8', 'blank': 'True'}), + 'fax': ('django.db.models.fields.CharField', [], {'max_length': '35', 'blank': 'True'}), + 'fax_country_code': ('django.db.models.fields.CharField', [], {'max_length': '3', 'blank': 'True'}), + 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'db_index': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '50', 'db_index': 'True'}), + 'middle_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}), + 'phone': ('django.db.models.fields.CharField', [], {'max_length': '35'}), + 'phone_country_code': ('django.db.models.fields.CharField', [], {'max_length': '3', 'db_index': 'True'}), + 'postal_code': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '16', 'blank': 'True'}), + 'processed_at': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'db_index': 'True'}), + 'salutation': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'}), + 'state': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '20', 'blank': 'True'}), + 'suffix': ('django.db.models.fields.CharField', [], {'max_length': '255', 'blank': 'True'}), + 'updated_at': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'db_index': 'True', 'blank': 'True'}), + 'upload_error_message': ('django.db.models.fields.CharField', [], {'max_length': '512', 'blank': 'True'}), + 'upload_status': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '20', 'blank': 'True'}), + 'uploaded_at': ('django.db.models.fields.DateTimeField', [], {'db_index': 'True', 'null': 'True', 'blank': 'True'}), + 'user': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['auth.User']", 'unique': 'True'}), + 'user_updated_at': ('django.db.models.fields.DateTimeField', [], {'db_index': 'True'}) + }, + 'student.userprofile': { + 'Meta': {'object_name': 'UserProfile', 'db_table': "'auth_userprofile'"}, + 'allow_certificate': ('django.db.models.fields.BooleanField', [], {'default': 'True'}), + 'courseware': ('django.db.models.fields.CharField', [], {'default': "'course.xml'", 'max_length': '255', 'blank': 'True'}), + 'gender': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '6', 'null': 'True', 'blank': 'True'}), + 'goals': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'language': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '255', 'blank': 'True'}), + 'level_of_education': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '6', 'null': 'True', 'blank': 'True'}), + 'location': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '255', 'blank': 'True'}), + 'mailing_address': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}), + 'meta': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '255', 'blank': 'True'}), + 'user': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'profile'", 'unique': 'True', 'to': "orm['auth.User']"}), + 'year_of_birth': ('django.db.models.fields.IntegerField', [], {'db_index': 'True', 'null': 'True', 'blank': 'True'}) + }, + 'student.usertestgroup': { + 'Meta': {'object_name': 'UserTestGroup'}, + 'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '32', 'db_index': 'True'}), + 'users': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.User']", 'db_index': 'True', 'symmetrical': 'False'}) + } + } + + complete_apps = ['student'] \ No newline at end of file diff --git a/common/djangoapps/student/models.py b/common/djangoapps/student/models.py index 56b1293c2d..af93c34317 100644 --- a/common/djangoapps/student/models.py +++ b/common/djangoapps/student/models.py @@ -16,7 +16,6 @@ import json import logging import uuid from random import randint -from time import strftime from django.conf import settings @@ -27,6 +26,7 @@ from django.dispatch import receiver from django.forms import ModelForm, forms import comment_client as cc +from pytz import UTC log = logging.getLogger(__name__) @@ -54,7 +54,7 @@ class UserProfile(models.Model): class Meta: db_table = "auth_userprofile" - ## CRITICAL TODO/SECURITY + # CRITICAL TODO/SECURITY # Sanitize all fields. # This is not visible to other users, but could introduce holes later user = models.OneToOneField(User, unique=True, db_index=True, related_name='profile') @@ -254,7 +254,7 @@ class TestCenterUserForm(ModelForm): def update_and_save(self): new_user = self.save(commit=False) # create additional values here: - new_user.user_updated_at = datetime.utcnow() + new_user.user_updated_at = datetime.now(UTC) new_user.upload_status = '' new_user.save() log.info("Updated demographic information for user's test center exam registration: username \"{}\" ".format(new_user.user.username)) @@ -429,8 +429,8 @@ class TestCenterRegistration(models.Model): registration.course_id = exam.course_id registration.accommodation_request = accommodation_request.strip() registration.exam_series_code = exam.exam_series_code - registration.eligibility_appointment_date_first = strftime("%Y-%m-%d", exam.first_eligible_appointment_date) - registration.eligibility_appointment_date_last = strftime("%Y-%m-%d", exam.last_eligible_appointment_date) + registration.eligibility_appointment_date_first = exam.first_eligible_appointment_date.strftime("%Y-%m-%d") + registration.eligibility_appointment_date_last = exam.last_eligible_appointment_date.strftime("%Y-%m-%d") registration.client_authorization_id = cls._create_client_authorization_id() # accommodation_code remains blank for now, along with Pearson confirmation information return registration @@ -556,7 +556,7 @@ class TestCenterRegistrationForm(ModelForm): def update_and_save(self): registration = self.save(commit=False) # create additional values here: - registration.user_updated_at = datetime.utcnow() + registration.user_updated_at = datetime.now(UTC) registration.upload_status = '' registration.save() log.info("Updated registration information for user's test center exam registration: username \"{}\" course \"{}\", examcode \"{}\"".format(registration.testcenter_user.user.username, registration.course_id, registration.exam_series_code)) @@ -598,7 +598,7 @@ def unique_id_for_user(user): return h.hexdigest() -## TODO: Should be renamed to generic UserGroup, and possibly +# TODO: Should be renamed to generic UserGroup, and possibly # Given an optional field for type of group class UserTestGroup(models.Model): users = models.ManyToManyField(User, db_index=True) @@ -626,7 +626,6 @@ class Registration(models.Model): def activate(self): self.user.is_active = True self.user.save() - #self.delete() class PendingNameChange(models.Model): @@ -648,7 +647,7 @@ class CourseEnrollment(models.Model): created = models.DateTimeField(auto_now_add=True, null=True, db_index=True) class Meta: - unique_together = (('user', 'course_id'), ) + unique_together = (('user', 'course_id'),) def __unicode__(self): return "[CourseEnrollment] %s: %s (%s)" % (self.user, self.course_id, self.created) @@ -662,16 +661,17 @@ class CourseEnrollmentAllowed(models.Model): """ email = models.CharField(max_length=255, db_index=True) course_id = models.CharField(max_length=255, db_index=True) + auto_enroll = models.BooleanField(default=0) created = models.DateTimeField(auto_now_add=True, null=True, db_index=True) class Meta: - unique_together = (('email', 'course_id'), ) + unique_together = (('email', 'course_id'),) def __unicode__(self): return "[CourseEnrollmentAllowed] %s: %s (%s)" % (self.email, self.course_id, self.created) -#cache_relation(User.profile) +# cache_relation(User.profile) #### Helper methods for use from python manage.py shell and other classes. diff --git a/common/djangoapps/student/tests/factories.py b/common/djangoapps/student/tests/factories.py index d73bb6f01d..49864fcbd4 100644 --- a/common/djangoapps/student/tests/factories.py +++ b/common/djangoapps/student/tests/factories.py @@ -5,6 +5,7 @@ from django.contrib.auth.models import Group from datetime import datetime from factory import DjangoModelFactory, SubFactory, PostGenerationMethodCall, post_generation, Sequence from uuid import uuid4 +from pytz import UTC # Factories don't have __init__ methods, and are self documenting # pylint: disable=W0232 @@ -46,8 +47,8 @@ class UserFactory(DjangoModelFactory): is_staff = False is_active = True is_superuser = False - last_login = datetime(2012, 1, 1) - date_joined = datetime(2011, 1, 1) + last_login = datetime(2012, 1, 1, tzinfo=UTC) + date_joined = datetime(2011, 1, 1, tzinfo=UTC) @post_generation def profile(obj, create, extracted, **kwargs): diff --git a/common/djangoapps/student/views.py b/common/djangoapps/student/views.py index 4ab5d95833..f936a17d40 100644 --- a/common/djangoapps/student/views.py +++ b/common/djangoapps/student/views.py @@ -32,7 +32,7 @@ from student.models import (Registration, UserProfile, TestCenterUser, TestCente TestCenterRegistration, TestCenterRegistrationForm, PendingNameChange, PendingEmailChange, CourseEnrollment, unique_id_for_user, - get_testcenter_registration) + get_testcenter_registration, CourseEnrollmentAllowed) from certificates.models import CertificateStatuses, certificate_status_for_student @@ -49,6 +49,7 @@ from courseware.views import get_module_for_descriptor, jump_to from courseware.model_data import ModelDataCache from statsd import statsd +from pytz import UTC log = logging.getLogger("mitx.student") Article = namedtuple('Article', 'title url author image deck publication publish_date') @@ -77,7 +78,7 @@ def index(request, extra_context={}, user=None): ''' # The course selection work is done in courseware.courses. - domain = settings.MITX_FEATURES.get('FORCE_UNIVERSITY_DOMAIN') # normally False + domain = settings.MITX_FEATURES.get('FORCE_UNIVERSITY_DOMAIN') # normally False # do explicit check, because domain=None is valid if domain == False: domain = request.META.get('HTTP_HOST') @@ -264,7 +265,6 @@ def dashboard(request): if not user.is_active: message = render_to_string('registration/activate_account_notice.html', {'email': user.email}) - # Global staff can see what courses errored on their dashboard staff_access = False errored_courses = {} @@ -356,16 +356,16 @@ def change_enrollment(request): course = course_from_id(course_id) except ItemNotFoundError: log.warning("User {0} tried to enroll in non-existent course {1}" - .format(user.username, course_id)) + .format(user.username, course_id)) return HttpResponseBadRequest("Course id is invalid") if not has_access(user, course, 'enroll'): return HttpResponseBadRequest("Enrollment is closed") org, course_num, run = course_id.split("/") statsd.increment("common.student.enrollment", - tags=["org:{0}".format(org), - "course:{0}".format(course_num), - "run:{0}".format(run)]) + tags=["org:{0}".format(org), + "course:{0}".format(course_num), + "run:{0}".format(run)]) try: enrollment, created = CourseEnrollment.objects.get_or_create(user=user, course_id=course.id) @@ -382,9 +382,9 @@ def change_enrollment(request): org, course_num, run = course_id.split("/") statsd.increment("common.student.unenrollment", - tags=["org:{0}".format(org), - "course:{0}".format(course_num), - "run:{0}".format(run)]) + tags=["org:{0}".format(org), + "course:{0}".format(course_num), + "run:{0}".format(run)]) return HttpResponse() except CourseEnrollment.DoesNotExist: @@ -454,7 +454,6 @@ def login_user(request, error=""): expires_time = time.time() + max_age expires = cookie_date(expires_time) - response.set_cookie(settings.EDXMKTG_COOKIE_NAME, 'true', max_age=max_age, expires=expires, domain=settings.SESSION_COOKIE_DOMAIN, @@ -515,8 +514,8 @@ def _do_create_account(post_vars): Note: this function is also used for creating test users. """ user = User(username=post_vars['username'], - email=post_vars['email'], - is_active=False) + email=post_vars['email'], + is_active=False) user.set_password(post_vars['password']) registration = Registration() # TODO: Rearrange so that if part of the process fails, the whole process fails. @@ -632,7 +631,7 @@ def create_account(request, post_override=None): # Ok, looks like everything is legit. Create the account. ret = _do_create_account(post_vars) - if isinstance(ret, HttpResponse): # if there was an error then return that + if isinstance(ret, HttpResponse): # if there was an error then return that return ret (user, profile, registration) = ret @@ -670,7 +669,7 @@ def create_account(request, post_override=None): if DoExternalAuth: eamap.user = login_user - eamap.dtsignup = datetime.datetime.now() + eamap.dtsignup = datetime.datetime.now(UTC) eamap.save() log.debug('Updated ExternalAuthMap for %s to be %s' % (post_vars['username'], eamap)) @@ -698,7 +697,6 @@ def create_account(request, post_override=None): expires_time = time.time() + max_age expires = cookie_date(expires_time) - response.set_cookie(settings.EDXMKTG_COOKIE_NAME, 'true', max_age=max_age, expires=expires, domain=settings.SESSION_COOKIE_DOMAIN, @@ -708,7 +706,6 @@ def create_account(request, post_override=None): return response - def exam_registration_info(user, course): """ Returns a Registration object if the user is currently registered for a current exam of the course. Returns None if the user is not registered, or if there is no @@ -849,7 +846,6 @@ def create_exam_registration(request, post_override=None): response_data['non_field_errors'] = form.non_field_errors() return HttpResponse(json.dumps(response_data), mimetype="application/json") - # only do the following if there is accommodation text to send, # and a destination to which to send it. # TODO: still need to create the accommodation email templates @@ -872,7 +868,6 @@ def create_exam_registration(request, post_override=None): # response_data['non_field_errors'] = [ 'Could not send accommodation e-mail.', ] # return HttpResponse(json.dumps(response_data), mimetype="application/json") - js = {'success': True} return HttpResponse(json.dumps(js), mimetype="application/json") @@ -916,6 +911,16 @@ def activate_account(request, key): if not r[0].user.is_active: r[0].activate() already_active = False + + #Enroll student in any pending courses he/she may have if auto_enroll flag is set + student = User.objects.filter(id=r[0].user_id) + if student: + ceas = CourseEnrollmentAllowed.objects.filter(email=student[0].email) + for cea in ceas: + if cea.auto_enroll: + course_id = cea.course_id + enrollment, created = CourseEnrollment.objects.get_or_create(user_id=student[0].id, course_id=course_id) + resp = render_to_response("registration/activation_complete.html", {'user_logged_in': user_logged_in, 'already_active': already_active}) return resp if len(r) == 0: diff --git a/common/djangoapps/terrain/steps.py b/common/djangoapps/terrain/steps.py index 1d9e59cd72..6e512982b7 100644 --- a/common/djangoapps/terrain/steps.py +++ b/common/djangoapps/terrain/steps.py @@ -159,3 +159,33 @@ def registered_edx_user(step, uname): @step(u'All dialogs should be closed$') def dialogs_are_closed(step): assert world.dialogs_closed() + + +@step('I will confirm all alerts') +def i_confirm_all_alerts(step): + """ + Please note: This method must be called RIGHT BEFORE an expected alert + Window variables are page local and thus all changes are removed upon navigating to a new page + In addition, this method changes the functionality of ONLY future alerts + """ + world.browser.execute_script('window.confirm = function(){return true;} ; window.alert = function(){return;}') + + +@step('I will cancel all alerts') +def i_cancel_all_alerts(step): + """ + Please note: This method must be called RIGHT BEFORE an expected alert + Window variables are page local and thus all changes are removed upon navigating to a new page + In addition, this method changes the functionality of ONLY future alerts + """ + world.browser.execute_script('window.confirm = function(){return false;} ; window.alert = function(){return;}') + + +@step('I will answer all prompts with "([^"]*)"') +def i_answer_prompts_with(step, prompt): + """ + Please note: This method must be called RIGHT BEFORE an expected alert + Window variables are page local and thus all changes are removed upon navigating to a new page + In addition, this method changes the functionality of ONLY future alerts + """ + world.browser.execute_script('window.prompt = function(){return %s;}') % prompt diff --git a/common/djangoapps/tests.py b/common/djangoapps/tests.py new file mode 100644 index 0000000000..8e78ee7f37 --- /dev/null +++ b/common/djangoapps/tests.py @@ -0,0 +1,49 @@ +''' +Created on Jun 6, 2013 + +@author: dmitchell +''' +from xmodule.modulestore.tests.factories import CourseFactory, ItemFactory +from student.tests.factories import AdminFactory +from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase +import xmodule_modifiers +import datetime +from pytz import UTC +from xmodule.modulestore.tests import factories + +class TestXmoduleModfiers(ModuleStoreTestCase): + + # FIXME disabled b/c start date inheritance is not occuring and render_... in get_html is failing due + # to middleware.lookup['main'] not being defined + def _test_add_histogram(self): + instructor = AdminFactory.create() + self.client.login(username=instructor.username, password='test') + + course = CourseFactory.create(org='test', + number='313', display_name='histogram test') + section = ItemFactory.create( + parent_location=course.location, display_name='chapter hist', + template='i4x://edx/templates/chapter/Empty') + problem = ItemFactory.create( + parent_location=section.location, display_name='problem hist 1', + template='i4x://edx/templates/problem/Blank_Common_Problem') + problem.has_score = False # don't trip trying to retrieve db data + + late_problem = ItemFactory.create( + parent_location=section.location, display_name='problem hist 2', + template='i4x://edx/templates/problem/Blank_Common_Problem') + late_problem.lms.start = datetime.datetime.now(UTC) + datetime.timedelta(days=32) + late_problem.has_score = False + + + problem_module = factories.get_test_xmodule_for_descriptor(problem) + problem_module.get_html = xmodule_modifiers.add_histogram(lambda:'', problem_module, instructor) + + self.assertRegexpMatches( + problem_module.get_html(), r'.*Not yet.*') + + problem_module = factories.get_test_xmodule_for_descriptor(late_problem) + problem_module.get_html = xmodule_modifiers.add_histogram(lambda: '', problem_module, instructor) + + self.assertRegexpMatches( + problem_module.get_html(), r'.*Yes!.*') diff --git a/common/djangoapps/track/views.py b/common/djangoapps/track/views.py index ae3a1dcb3e..b2935a6a89 100644 --- a/common/djangoapps/track/views.py +++ b/common/djangoapps/track/views.py @@ -14,6 +14,7 @@ from mitxmako.shortcuts import render_to_response from django_future.csrf import ensure_csrf_cookie from track.models import TrackingLog +from pytz import UTC log = logging.getLogger("tracking") @@ -59,7 +60,7 @@ def user_track(request): "event": request.GET['event'], "agent": agent, "page": request.GET['page'], - "time": datetime.datetime.utcnow().isoformat(), + "time": datetime.datetime.now(UTC).isoformat(), "host": request.META['SERVER_NAME'], } log_event(event) @@ -85,11 +86,11 @@ def server_track(request, event_type, event, page=None): "event": event, "agent": agent, "page": page, - "time": datetime.datetime.utcnow().isoformat(), + "time": datetime.datetime.now(UTC).isoformat(), "host": request.META['SERVER_NAME'], } - if event_type.startswith("/event_logs") and request.user.is_staff: # don't log + if event_type.startswith("/event_logs") and request.user.is_staff: # don't log return log_event(event) diff --git a/common/djangoapps/util/testing.py b/common/djangoapps/util/testing.py new file mode 100644 index 0000000000..d33f1c8f8b --- /dev/null +++ b/common/djangoapps/util/testing.py @@ -0,0 +1,34 @@ +import sys + +from django.conf import settings +from django.core.urlresolvers import clear_url_caches + + +class UrlResetMixin(object): + """Mixin to reset urls.py before and after a test + + Django memoizes the function that reads the urls module (whatever module + urlconf names). The module itself is also stored by python in sys.modules. + To fully reload it, we need to reload the python module, and also clear django's + cache of the parsed urls. + + However, the order in which we do this doesn't matter, because neither one will + get reloaded until the next request + + Doing this is expensive, so it should only be added to tests that modify settings + that affect the contents of urls.py + """ + + def _reset_urls(self, urlconf=None): + if urlconf is None: + urlconf = settings.ROOT_URLCONF + + if urlconf in sys.modules: + reload(sys.modules[urlconf]) + clear_url_caches() + + def setUp(self): + """Reset django default urlconf before tests and after tests""" + super(UrlResetMixin, self).setUp() + self._reset_urls() + self.addCleanup(self._reset_urls) diff --git a/common/djangoapps/util/tests/test_zendesk.py b/common/djangoapps/util/tests/test_submit_feedback.py similarity index 69% rename from common/djangoapps/util/tests/test_zendesk.py rename to common/djangoapps/util/tests/test_submit_feedback.py index 51d06a92ed..b66d3d642b 100644 --- a/common/djangoapps/util/tests/test_zendesk.py +++ b/common/djangoapps/util/tests/test_submit_feedback.py @@ -15,8 +15,9 @@ import mock @mock.patch.dict("django.conf.settings.MITX_FEATURES", {"ENABLE_FEEDBACK_SUBMISSION": True}) @override_settings(ZENDESK_URL="dummy", ZENDESK_USER="dummy", ZENDESK_API_KEY="dummy") +@mock.patch("util.views.dog_stats_api") @mock.patch("util.views._ZendeskApi", autospec=True) -class SubmitFeedbackViaZendeskTest(TestCase): +class SubmitFeedbackTest(TestCase): def setUp(self): """Set up data for the test case""" self._request_factory = RequestFactory() @@ -26,18 +27,19 @@ class SubmitFeedbackViaZendeskTest(TestCase): username="test", profile__name="Test User" ) - # This contains a tag to ensure that tags are submitted correctly + # This contains issue_type and course_id to ensure that tags are submitted correctly self._anon_fields = { "email": "test@edx.org", "name": "Test User", "subject": "a subject", "details": "some details", - "tag": "a tag" + "issue_type": "test_issue", + "course_id": "test_course" } - # This does not contain a tag to ensure that tag is optional + # This does not contain issue_type nor course_id to ensure that they are optional self._auth_fields = {"subject": "a subject", "details": "some details"} - def _test_request(self, user, fields): + def _build_and_run_request(self, user, fields): """ Generate a request and invoke the view, returning the response. @@ -48,12 +50,14 @@ class SubmitFeedbackViaZendeskTest(TestCase): "/submit_feedback", data=fields, HTTP_REFERER="test_referer", - HTTP_USER_AGENT="test_user_agent" + HTTP_USER_AGENT="test_user_agent", + REMOTE_ADDR="1.2.3.4", + SERVER_NAME="test_server" ) req.user = user - return views.submit_feedback_via_zendesk(req) + return views.submit_feedback(req) - def _assert_bad_request(self, response, field, zendesk_mock_class): + def _assert_bad_request(self, response, field, zendesk_mock_class, datadog_mock): """ Assert that the given `response` contains correct failure data. @@ -67,8 +71,9 @@ class SubmitFeedbackViaZendeskTest(TestCase): self.assertTrue("error" in resp_json) # There should be absolutely no interaction with Zendesk self.assertFalse(zendesk_mock_class.return_value.mock_calls) + self.assertFalse(datadog_mock.mock_calls) - def _test_bad_request_omit_field(self, user, fields, omit_field, zendesk_mock_class): + def _test_bad_request_omit_field(self, user, fields, omit_field, zendesk_mock_class, datadog_mock): """ Invoke the view with a request missing a field and assert correctness. @@ -79,10 +84,10 @@ class SubmitFeedbackViaZendeskTest(TestCase): have been invoked. """ filtered_fields = {k: v for (k, v) in fields.items() if k != omit_field} - resp = self._test_request(user, filtered_fields) - self._assert_bad_request(resp, omit_field, zendesk_mock_class) + resp = self._build_and_run_request(user, filtered_fields) + self._assert_bad_request(resp, omit_field, zendesk_mock_class, datadog_mock) - def _test_bad_request_empty_field(self, user, fields, empty_field, zendesk_mock_class): + def _test_bad_request_empty_field(self, user, fields, empty_field, zendesk_mock_class, datadog_mock): """ Invoke the view with an empty field and assert correctness. @@ -94,8 +99,8 @@ class SubmitFeedbackViaZendeskTest(TestCase): """ altered_fields = fields.copy() altered_fields[empty_field] = "" - resp = self._test_request(user, altered_fields) - self._assert_bad_request(resp, empty_field, zendesk_mock_class) + resp = self._build_and_run_request(user, altered_fields) + self._assert_bad_request(resp, empty_field, zendesk_mock_class, datadog_mock) def _test_success(self, user, fields): """ @@ -105,30 +110,46 @@ class SubmitFeedbackViaZendeskTest(TestCase): `fields` in the POST body. The response should have a 200 (success) status code. """ - resp = self._test_request(user, fields) + resp = self._build_and_run_request(user, fields) self.assertEqual(resp.status_code, 200) - def test_bad_request_anon_user_no_name(self, zendesk_mock_class): + def _assert_datadog_called(self, datadog_mock, with_tags): + expected_datadog_calls = [ + mock.call.increment( + views.DATADOG_FEEDBACK_METRIC, + tags=(["course_id:test_course", "issue_type:test_issue"] if with_tags else []) + ) + ] + self.assertEqual(datadog_mock.mock_calls, expected_datadog_calls) + + def test_bad_request_anon_user_no_name(self, zendesk_mock_class, datadog_mock): """Test a request from an anonymous user not specifying `name`.""" - self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "name", zendesk_mock_class) - self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "name", zendesk_mock_class) + self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "name", zendesk_mock_class, datadog_mock) + self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "name", zendesk_mock_class, datadog_mock) - def test_bad_request_anon_user_no_email(self, zendesk_mock_class): + def test_bad_request_anon_user_no_email(self, zendesk_mock_class, datadog_mock): """Test a request from an anonymous user not specifying `email`.""" - self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "email", zendesk_mock_class) - self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "email", zendesk_mock_class) + self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "email", zendesk_mock_class, datadog_mock) + self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "email", zendesk_mock_class, datadog_mock) - def test_bad_request_anon_user_no_subject(self, zendesk_mock_class): + def test_bad_request_anon_user_invalid_email(self, zendesk_mock_class, datadog_mock): + """Test a request from an anonymous user specifying an invalid `email`.""" + fields = self._anon_fields.copy() + fields["email"] = "This is not a valid email address!" + resp = self._build_and_run_request(self._anon_user, fields) + self._assert_bad_request(resp, "email", zendesk_mock_class, datadog_mock) + + def test_bad_request_anon_user_no_subject(self, zendesk_mock_class, datadog_mock): """Test a request from an anonymous user not specifying `subject`.""" - self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "subject", zendesk_mock_class) - self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "subject", zendesk_mock_class) + self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "subject", zendesk_mock_class, datadog_mock) + self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "subject", zendesk_mock_class, datadog_mock) - def test_bad_request_anon_user_no_details(self, zendesk_mock_class): + def test_bad_request_anon_user_no_details(self, zendesk_mock_class, datadog_mock): """Test a request from an anonymous user not specifying `details`.""" - self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "details", zendesk_mock_class) - self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "details", zendesk_mock_class) + self._test_bad_request_omit_field(self._anon_user, self._anon_fields, "details", zendesk_mock_class, datadog_mock) + self._test_bad_request_empty_field(self._anon_user, self._anon_fields, "details", zendesk_mock_class, datadog_mock) - def test_valid_request_anon_user(self, zendesk_mock_class): + def test_valid_request_anon_user(self, zendesk_mock_class, datadog_mock): """ Test a valid request from an anonymous user. @@ -138,14 +159,14 @@ class SubmitFeedbackViaZendeskTest(TestCase): zendesk_mock_instance = zendesk_mock_class.return_value zendesk_mock_instance.create_ticket.return_value = 42 self._test_success(self._anon_user, self._anon_fields) - expected_calls = [ + expected_zendesk_calls = [ mock.call.create_ticket( { "ticket": { "requester": {"name": "Test User", "email": "test@edx.org"}, "subject": "a subject", "comment": {"body": "some details"}, - "tags": ["a tag"] + "tags": ["test_course", "test_issue", "LMS"] } } ), @@ -157,26 +178,29 @@ class SubmitFeedbackViaZendeskTest(TestCase): "public": False, "body": "Additional information:\n\n" - "HTTP_USER_AGENT: test_user_agent\n" - "HTTP_REFERER: test_referer" + "Client IP: 1.2.3.4\n" + "Host: test_server\n" + "Page: test_referer\n" + "Browser: test_user_agent" } } } ) ] - self.assertEqual(zendesk_mock_instance.mock_calls, expected_calls) + self.assertEqual(zendesk_mock_instance.mock_calls, expected_zendesk_calls) + self._assert_datadog_called(datadog_mock, with_tags=True) - def test_bad_request_auth_user_no_subject(self, zendesk_mock_class): + def test_bad_request_auth_user_no_subject(self, zendesk_mock_class, datadog_mock): """Test a request from an authenticated user not specifying `subject`.""" - self._test_bad_request_omit_field(self._auth_user, self._auth_fields, "subject", zendesk_mock_class) - self._test_bad_request_empty_field(self._auth_user, self._auth_fields, "subject", zendesk_mock_class) + self._test_bad_request_omit_field(self._auth_user, self._auth_fields, "subject", zendesk_mock_class, datadog_mock) + self._test_bad_request_empty_field(self._auth_user, self._auth_fields, "subject", zendesk_mock_class, datadog_mock) - def test_bad_request_auth_user_no_details(self, zendesk_mock_class): + def test_bad_request_auth_user_no_details(self, zendesk_mock_class, datadog_mock): """Test a request from an authenticated user not specifying `details`.""" - self._test_bad_request_omit_field(self._auth_user, self._auth_fields, "details", zendesk_mock_class) - self._test_bad_request_empty_field(self._auth_user, self._auth_fields, "details", zendesk_mock_class) + self._test_bad_request_omit_field(self._auth_user, self._auth_fields, "details", zendesk_mock_class, datadog_mock) + self._test_bad_request_empty_field(self._auth_user, self._auth_fields, "details", zendesk_mock_class, datadog_mock) - def test_valid_request_auth_user(self, zendesk_mock_class): + def test_valid_request_auth_user(self, zendesk_mock_class, datadog_mock): """ Test a valid request from an authenticated user. @@ -186,14 +210,14 @@ class SubmitFeedbackViaZendeskTest(TestCase): zendesk_mock_instance = zendesk_mock_class.return_value zendesk_mock_instance.create_ticket.return_value = 42 self._test_success(self._auth_user, self._auth_fields) - expected_calls = [ + expected_zendesk_calls = [ mock.call.create_ticket( { "ticket": { "requester": {"name": "Test User", "email": "test@edx.org"}, "subject": "a subject", "comment": {"body": "some details"}, - "tags": [] + "tags": ["LMS"] } } ), @@ -206,27 +230,31 @@ class SubmitFeedbackViaZendeskTest(TestCase): "body": "Additional information:\n\n" "username: test\n" - "HTTP_USER_AGENT: test_user_agent\n" - "HTTP_REFERER: test_referer" + "Client IP: 1.2.3.4\n" + "Host: test_server\n" + "Page: test_referer\n" + "Browser: test_user_agent" } } } ) ] - self.assertEqual(zendesk_mock_instance.mock_calls, expected_calls) + self.assertEqual(zendesk_mock_instance.mock_calls, expected_zendesk_calls) + self._assert_datadog_called(datadog_mock, with_tags=False) - def test_get_request(self, zendesk_mock_class): + def test_get_request(self, zendesk_mock_class, datadog_mock): """Test that a GET results in a 405 even with all required fields""" req = self._request_factory.get("/submit_feedback", data=self._anon_fields) req.user = self._anon_user - resp = views.submit_feedback_via_zendesk(req) + resp = views.submit_feedback(req) self.assertEqual(resp.status_code, 405) self.assertIn("Allow", resp) self.assertEqual(resp["Allow"], "POST") # There should be absolutely no interaction with Zendesk self.assertFalse(zendesk_mock_class.mock_calls) + self.assertFalse(datadog_mock.mock_calls) - def test_zendesk_error_on_create(self, zendesk_mock_class): + def test_zendesk_error_on_create(self, zendesk_mock_class, datadog_mock): """ Test Zendesk returning an error on ticket creation. @@ -235,11 +263,12 @@ class SubmitFeedbackViaZendeskTest(TestCase): err = ZendeskError(msg="", error_code=404) zendesk_mock_instance = zendesk_mock_class.return_value zendesk_mock_instance.create_ticket.side_effect = err - resp = self._test_request(self._anon_user, self._anon_fields) + resp = self._build_and_run_request(self._anon_user, self._anon_fields) self.assertEqual(resp.status_code, 500) self.assertFalse(resp.content) + self._assert_datadog_called(datadog_mock, with_tags=True) - def test_zendesk_error_on_update(self, zendesk_mock_class): + def test_zendesk_error_on_update(self, zendesk_mock_class, datadog_mock): """ Test for Zendesk returning an error on ticket update. @@ -250,20 +279,21 @@ class SubmitFeedbackViaZendeskTest(TestCase): err = ZendeskError(msg="", error_code=500) zendesk_mock_instance = zendesk_mock_class.return_value zendesk_mock_instance.update_ticket.side_effect = err - resp = self._test_request(self._anon_user, self._anon_fields) + resp = self._build_and_run_request(self._anon_user, self._anon_fields) self.assertEqual(resp.status_code, 200) + self._assert_datadog_called(datadog_mock, with_tags=True) @mock.patch.dict("django.conf.settings.MITX_FEATURES", {"ENABLE_FEEDBACK_SUBMISSION": False}) - def test_not_enabled(self, zendesk_mock_class): + def test_not_enabled(self, zendesk_mock_class, datadog_mock): """ Test for Zendesk submission not enabled in `settings`. We should raise Http404. """ with self.assertRaises(Http404): - self._test_request(self._anon_user, self._anon_fields) + self._build_and_run_request(self._anon_user, self._anon_fields) - def test_zendesk_not_configured(self, zendesk_mock_class): + def test_zendesk_not_configured(self, zendesk_mock_class, datadog_mock): """ Test for Zendesk not fully configured in `settings`. @@ -273,7 +303,7 @@ class SubmitFeedbackViaZendeskTest(TestCase): def test_case(missing_config): with mock.patch(missing_config, None): with self.assertRaises(Exception): - self._test_request(self._anon_user, self._anon_fields) + self._build_and_run_request(self._anon_user, self._anon_fields) test_case("django.conf.settings.ZENDESK_URL") test_case("django.conf.settings.ZENDESK_USER") diff --git a/common/djangoapps/util/views.py b/common/djangoapps/util/views.py index d0aa0dc680..aa592d25e8 100644 --- a/common/djangoapps/util/views.py +++ b/common/djangoapps/util/views.py @@ -12,6 +12,7 @@ from django.core.validators import ValidationError, validate_email from django.http import Http404, HttpResponse, HttpResponseBadRequest, HttpResponseNotAllowed, HttpResponseServerError from django.shortcuts import redirect from django_future.csrf import ensure_csrf_cookie +from dogapi import dog_stats_api from mitxmako.shortcuts import render_to_response, render_to_string from urllib import urlencode import zendesk @@ -73,11 +74,64 @@ class _ZendeskApi(object): self._zendesk_instance.update_ticket(ticket_id=ticket_id, data=update) -def submit_feedback_via_zendesk(request): +def _record_feedback_in_zendesk(realname, email, subject, details, tags, additional_info): """ Create a new user-requested Zendesk ticket. - If Zendesk submission is not enabled, any request will raise `Http404`. + Once created, the ticket will be updated with a private comment containing + additional information from the browser and server, such as HTTP headers + and user state. Returns a boolean value indicating whether ticket creation + was successful, regardless of whether the private comment update succeeded. + """ + zendesk_api = _ZendeskApi() + + additional_info_string = ( + "Additional information:\n\n" + + "\n".join("%s: %s" % (key, value) for (key, value) in additional_info.items() if value is not None) + ) + + # Tag all issues with LMS to distinguish channel in Zendesk; requested by student support team + zendesk_tags = list(tags.values()) + ["LMS"] + new_ticket = { + "ticket": { + "requester": {"name": realname, "email": email}, + "subject": subject, + "comment": {"body": details}, + "tags": zendesk_tags + } + } + try: + ticket_id = zendesk_api.create_ticket(new_ticket) + except zendesk.ZendeskError as err: + log.error("Error creating Zendesk ticket: %s", str(err)) + return False + + # Additional information is provided as a private update so the information + # is not visible to the user. + ticket_update = {"ticket": {"comment": {"public": False, "body": additional_info_string}}} + try: + zendesk_api.update_ticket(ticket_id, ticket_update) + except zendesk.ZendeskError as err: + log.error("Error updating Zendesk ticket: %s", str(err)) + # The update is not strictly necessary, so do not indicate failure to the user + pass + + return True + + +DATADOG_FEEDBACK_METRIC = "lms_feedback_submissions" + + +def _record_feedback_in_datadog(tags): + datadog_tags = ["{k}:{v}".format(k=k, v=v) for k, v in tags.items()] + dog_stats_api.increment(DATADOG_FEEDBACK_METRIC, tags=datadog_tags) + + +def submit_feedback(request): + """ + Create a new user-requested ticket, currently implemented with Zendesk. + + If feedback submission is not enabled, any request will raise `Http404`. If any configuration parameter (`ZENDESK_URL`, `ZENDESK_USER`, or `ZENDESK_API_KEY`) is missing, any request will raise an `Exception`. The request must be a POST request specifying `subject` and `details`. @@ -85,12 +139,9 @@ def submit_feedback_via_zendesk(request): `email`. If the user is authenticated, the `name` and `email` will be populated from the user's information. If any required parameter is missing, a 400 error will be returned indicating which field is missing and - providing an error message. If Zendesk returns any error on ticket - creation, a 500 error will be returned with no body. Once created, the - ticket will be updated with a private comment containing additional - information from the browser and server, such as HTTP headers and user - state. Whether or not the update succeeds, if the user's ticket is - successfully created, an empty successful response (200) will be returned. + providing an error message. If Zendesk ticket creation fails, 500 error + will be returned with no body; if ticket creation succeeds, an empty + successful response (200) will be returned. """ if not settings.MITX_FEATURES.get('ENABLE_FEEDBACK_SUBMISSION', False): raise Http404() @@ -124,9 +175,9 @@ def submit_feedback_via_zendesk(request): subject = request.POST["subject"] details = request.POST["details"] - tags = [] - if "tag" in request.POST: - tags = [request.POST["tag"]] + tags = dict( + [(tag, request.POST[tag]) for tag in ["issue_type", "course_id"] if tag in request.POST] + ) if request.user.is_authenticated(): realname = request.user.profile.name @@ -140,41 +191,18 @@ def submit_feedback_via_zendesk(request): except ValidationError: return build_error_response(400, "email", required_field_errs["email"]) - for header in ["HTTP_REFERER", "HTTP_USER_AGENT"]: - additional_info[header] = request.META.get(header) + for header, pretty in [ + ("HTTP_REFERER", "Page"), + ("HTTP_USER_AGENT", "Browser"), + ("REMOTE_ADDR", "Client IP"), + ("SERVER_NAME", "Host") + ]: + additional_info[pretty] = request.META.get(header) - zendesk_api = _ZendeskApi() + success = _record_feedback_in_zendesk(realname, email, subject, details, tags, additional_info) + _record_feedback_in_datadog(tags) - additional_info_string = ( - "Additional information:\n\n" + - "\n".join("%s: %s" % (key, value) for (key, value) in additional_info.items() if value is not None) - ) - - new_ticket = { - "ticket": { - "requester": {"name": realname, "email": email}, - "subject": subject, - "comment": {"body": details}, - "tags": tags - } - } - try: - ticket_id = zendesk_api.create_ticket(new_ticket) - except zendesk.ZendeskError as err: - log.error("Error creating Zendesk ticket: %s", str(err)) - return HttpResponse(status=500) - - # Additional information is provided as a private update so the information - # is not visible to the user. - ticket_update = {"ticket": {"comment": {"public": False, "body": additional_info_string}}} - try: - zendesk_api.update_ticket(ticket_id, ticket_update) - except zendesk.ZendeskError as err: - log.error("Error updating Zendesk ticket: %s", str(err)) - # The update is not strictly necessary, so do not indicate failure to the user - pass - - return HttpResponse() + return HttpResponse(status=(200 if success else 500)) def info(request): diff --git a/common/djangoapps/xmodule_modifiers.py b/common/djangoapps/xmodule_modifiers.py index 45691cd854..570b38c942 100644 --- a/common/djangoapps/xmodule_modifiers.py +++ b/common/djangoapps/xmodule_modifiers.py @@ -1,7 +1,6 @@ import re import json import logging -import time import static_replace from django.conf import settings @@ -9,6 +8,8 @@ from functools import wraps from mitxmako.shortcuts import render_to_string from xmodule.seq_module import SequenceModule from xmodule.vertical_module import VerticalModule +import datetime +from django.utils.timezone import UTC log = logging.getLogger("mitx.xmodule_modifiers") @@ -83,7 +84,7 @@ def grade_histogram(module_id): cursor.execute(q, [module_id]) grades = list(cursor.fetchall()) - grades.sort(key=lambda x: x[0]) # Add ORDER BY to sql query? + grades.sort(key=lambda x: x[0]) # Add ORDER BY to sql query? if len(grades) >= 1 and grades[0][0] is None: return [] return grades @@ -101,7 +102,7 @@ def add_histogram(get_html, module, user): @wraps(get_html) def _get_html(): - if type(module) in [SequenceModule, VerticalModule]: # TODO: make this more general, eg use an XModule attribute instead + if type(module) in [SequenceModule, VerticalModule]: # TODO: make this more general, eg use an XModule attribute instead return get_html() module_id = module.id @@ -132,7 +133,7 @@ def add_histogram(get_html, module, user): # useful to indicate to staff if problem has been released or not # TODO (ichuang): use _has_access_descriptor.can_load in lms.courseware.access, instead of now>mstart comparison here - now = time.gmtime() + now = datetime.datetime.now(UTC()) is_released = "unknown" mstart = module.descriptor.lms.start diff --git a/common/lib/calc/calc.py b/common/lib/calc/calc.py index 2ee82e2fb4..f0934a9ed5 100644 --- a/common/lib/calc/calc.py +++ b/common/lib/calc/calc.py @@ -1,34 +1,63 @@ +""" +Parser and evaluator for FormulaResponse and NumericalResponse + +Uses pyparsing to parse. Main function as of now is evaluator(). +""" + import copy -import logging import math import operator import re import numpy -import numbers import scipy.constants +import calcfunctions -from pyparsing import Word, alphas, nums, oneOf, Literal -from pyparsing import ZeroOrMore, OneOrMore, StringStart -from pyparsing import StringEnd, Optional, Forward -from pyparsing import CaselessLiteral, Group, StringEnd -from pyparsing import NoMatch, stringEnd, alphanums +# have numpy raise errors on functions outside its domain +# See http://docs.scipy.org/doc/numpy/reference/generated/numpy.seterr.html +numpy.seterr(all='ignore') # Also: 'ignore', 'warn' (default), 'raise' -default_functions = {'sin': numpy.sin, +from pyparsing import (Word, nums, Literal, + ZeroOrMore, MatchFirst, + Optional, Forward, + CaselessLiteral, + stringEnd, Suppress, Combine) + +DEFAULT_FUNCTIONS = {'sin': numpy.sin, 'cos': numpy.cos, 'tan': numpy.tan, + 'sec': calcfunctions.sec, + 'csc': calcfunctions.csc, + 'cot': calcfunctions.cot, 'sqrt': numpy.sqrt, 'log10': numpy.log10, 'log2': numpy.log2, 'ln': numpy.log, + 'exp': numpy.exp, 'arccos': numpy.arccos, 'arcsin': numpy.arcsin, 'arctan': numpy.arctan, + 'arcsec': calcfunctions.arcsec, + 'arccsc': calcfunctions.arccsc, + 'arccot': calcfunctions.arccot, 'abs': numpy.abs, 'fact': math.factorial, - 'factorial': math.factorial + 'factorial': math.factorial, + 'sinh': numpy.sinh, + 'cosh': numpy.cosh, + 'tanh': numpy.tanh, + 'sech': calcfunctions.sech, + 'csch': calcfunctions.csch, + 'coth': calcfunctions.coth, + 'arcsinh': numpy.arcsinh, + 'arccosh': numpy.arccosh, + 'arctanh': numpy.arctanh, + 'arcsech': calcfunctions.arcsech, + 'arccsch': calcfunctions.arccsch, + 'arccoth': calcfunctions.arccoth } -default_variables = {'j': numpy.complex(0, 1), +DEFAULT_VARIABLES = {'i': numpy.complex(0, 1), + 'j': numpy.complex(0, 1), 'e': numpy.e, 'pi': numpy.pi, 'k': scipy.constants.k, @@ -37,65 +66,166 @@ default_variables = {'j': numpy.complex(0, 1), 'q': scipy.constants.e } -log = logging.getLogger("mitx.courseware.capa") +# We eliminated the following extreme suffixes: +# P (1e15), E (1e18), Z (1e21), Y (1e24), +# f (1e-15), a (1e-18), z (1e-21), y (1e-24) +# since they're rarely used, and potentially +# confusing. They may also conflict with variables if we ever allow e.g. +# 5R instead of 5*R +SUFFIXES = {'%': 0.01, 'k': 1e3, 'M': 1e6, 'G': 1e9, 'T': 1e12, + 'c': 1e-2, 'm': 1e-3, 'u': 1e-6, 'n': 1e-9, 'p': 1e-12} class UndefinedVariable(Exception): - def raiseself(self): - ''' Helper so we can use inside of a lambda ''' - raise self - - -general_whitespace = re.compile('[^\w]+') + """ + Used to indicate the student input of a variable, which was unused by the + instructor. + """ + pass def check_variables(string, variables): - '''Confirm the only variables in string are defined. + """ + Confirm the only variables in string are defined. - Pyparsing uses a left-to-right parser, which makes the more + Otherwise, raise an UndefinedVariable containing all bad variables. + + Pyparsing uses a left-to-right parser, which makes a more elegant approach pretty hopeless. - - achar = reduce(lambda a,b:a|b ,map(Literal,alphas)) # Any alphabetic character - undefined_variable = achar + Word(alphanums) - undefined_variable.setParseAction(lambda x:UndefinedVariable("".join(x)).raiseself()) - varnames = varnames | undefined_variable - ''' - possible_variables = re.split(general_whitespace, string) # List of all alnums in string - bad_variables = list() - for v in possible_variables: - if len(v) == 0: + """ + general_whitespace = re.compile('[^\\w]+') + # List of all alnums in string + possible_variables = re.split(general_whitespace, string) + bad_variables = [] + for var in possible_variables: + if len(var) == 0: continue - if v[0] <= '9' and '0' <= 'v': # Skip things that begin with numbers + if var[0].isdigit(): # Skip things that begin with numbers continue - if v not in variables: - bad_variables.append(v) + if var not in variables: + bad_variables.append(var) if len(bad_variables) > 0: raise UndefinedVariable(' '.join(bad_variables)) +def lower_dict(input_dict): + """ + takes each key in the dict and makes it lowercase, still mapping to the + same value. + + keep in mind that it is possible (but not useful?) to define different + variables that have the same lowercase representation. It would be hard to + tell which is used in the final dict and which isn't. + """ + return {k.lower(): v for k, v in input_dict.iteritems()} + + +# The following few functions define parse actions, which are run on lists of +# results from each parse component. They convert the strings and (previously +# calculated) numbers into the number that component represents. + +def super_float(text): + """ + Like float, but with si extensions. 1k goes to 1000 + """ + if text[-1] in SUFFIXES: + return float(text[:-1]) * SUFFIXES[text[-1]] + else: + return float(text) + + +def number_parse_action(parse_result): + """ + Create a float out of its string parts + + e.g. [ '7', '.', '13' ] -> [ 7.13 ] + Calls super_float above + """ + return super_float("".join(parse_result)) + + +def exp_parse_action(parse_result): + """ + Take a list of numbers and exponentiate them, right to left + + e.g. [ 3, 2, 3 ] (which is 3^2^3 = 3^(2^3)) -> 6561 + """ + # pyparsing.ParseResults doesn't play well with reverse() + parse_result = reversed(parse_result) + # the result of an exponentiation is called a power + power = reduce(lambda a, b: b ** a, parse_result) + return power + + +def parallel(parse_result): + """ + Compute numbers according to the parallel resistors operator + + BTW it is commutative. Its formula is given by + out = 1 / (1/in1 + 1/in2 + ...) + e.g. [ 1, 2 ] => 2/3 + + Return NaN if there is a zero among the inputs + """ + # convert from pyparsing.ParseResults, which doesn't support '0 in parse_result' + parse_result = parse_result.asList() + if len(parse_result) == 1: + return parse_result[0] + if 0 in parse_result: + return float('nan') + reciprocals = [1. / e for e in parse_result] + return 1. / sum(reciprocals) + + +def sum_parse_action(parse_result): + """ + Add the inputs + + [ 1, '+', 2, '-', 3 ] -> 0 + + Allow a leading + or - + """ + total = 0.0 + current_op = operator.add + for token in parse_result: + if token is '+': + current_op = operator.add + elif token is '-': + current_op = operator.sub + else: + total = current_op(total, token) + return total + + +def prod_parse_action(parse_result): + """ + Multiply the inputs + + [ 1, '*', 2, '/', 3 ] => 0.66 + """ + prod = 1.0 + current_op = operator.mul + for token in parse_result: + if token is '*': + current_op = operator.mul + elif token is '/': + current_op = operator.truediv + else: + prod = current_op(prod, token) + return prod + + def evaluator(variables, functions, string, cs=False): - ''' + """ Evaluate an expression. Variables are passed as a dictionary from string to value. Unary functions are passed as a dictionary from string to function. Variables must be floats. cs: Case sensitive - TODO: Fix it so we can pass integers and complex numbers in variables dict - ''' - # log.debug("variables: {0}".format(variables)) - # log.debug("functions: {0}".format(functions)) - # log.debug("string: {0}".format(string)) - - def lower_dict(d): - return dict([(k.lower(), d[k]) for k in d]) - - all_variables = copy.copy(default_variables) - all_functions = copy.copy(default_functions) - - if not cs: - all_variables = lower_dict(all_variables) - all_functions = lower_dict(all_functions) + """ + all_variables = copy.copy(DEFAULT_VARIABLES) + all_functions = copy.copy(DEFAULT_FUNCTIONS) all_variables.update(variables) all_functions.update(functions) @@ -113,122 +243,59 @@ def evaluator(variables, functions, string, cs=False): if string.strip() == "": return float('nan') - ops = {"^": operator.pow, - "*": operator.mul, - "/": operator.truediv, - "+": operator.add, - "-": operator.sub, - } - # We eliminated extreme ones, since they're rarely used, and potentially - # confusing. They may also conflict with variables if we ever allow e.g. - # 5R instead of 5*R - suffixes = {'%': 0.01, 'k': 1e3, 'M': 1e6, 'G': 1e9, - 'T': 1e12, # 'P':1e15,'E':1e18,'Z':1e21,'Y':1e24, - 'c': 1e-2, 'm': 1e-3, 'u': 1e-6, - 'n': 1e-9, 'p': 1e-12} # ,'f':1e-15,'a':1e-18,'z':1e-21,'y':1e-24} - - def super_float(text): - ''' Like float, but with si extensions. 1k goes to 1000''' - if text[-1] in suffixes: - return float(text[:-1]) * suffixes[text[-1]] - else: - return float(text) - - def number_parse_action(x): # [ '7' ] -> [ 7 ] - return [super_float("".join(x))] - - def exp_parse_action(x): # [ 2 ^ 3 ^ 2 ] -> 512 - x = [e for e in x if isinstance(e, numbers.Number)] # Ignore ^ - x.reverse() - x = reduce(lambda a, b: b ** a, x) - return x - - def parallel(x): # Parallel resistors [ 1 2 ] => 2/3 - # convert from pyparsing.ParseResults, which doesn't support '0 in x' - x = list(x) - if len(x) == 1: - return x[0] - if 0 in x: - return float('nan') - x = [1. / e for e in x if isinstance(e, numbers.Number)] # Ignore || - return 1. / sum(x) - - def sum_parse_action(x): # [ 1 + 2 - 3 ] -> 0 - total = 0.0 - op = ops['+'] - for e in x: - if e in set('+-'): - op = ops[e] - else: - total = op(total, e) - return total - - def prod_parse_action(x): # [ 1 * 2 / 3 ] => 0.66 - prod = 1.0 - op = ops['*'] - for e in x: - if e in set('*/'): - op = ops[e] - else: - prod = op(prod, e) - return prod - - def func_parse_action(x): - return [all_functions[x[0]](x[1])] - # SI suffixes and percent - number_suffix = reduce(lambda a, b: a | b, map(Literal, suffixes.keys()), NoMatch()) - (dot, minus, plus, times, div, lpar, rpar, exp) = map(Literal, ".-+*/()^") + number_suffix = MatchFirst([Literal(k) for k in SUFFIXES.keys()]) + plus_minus = Literal('+') | Literal('-') + times_div = Literal('*') | Literal('/') number_part = Word(nums) # 0.33 or 7 or .34 or 16. inner_number = (number_part + Optional("." + Optional(number_part))) | ("." + number_part) + # by default pyparsing allows spaces between tokens--Combine prevents that + inner_number = Combine(inner_number) # 0.33k or -17 - number = (Optional(minus | plus) + inner_number - + Optional(CaselessLiteral("E") + Optional((plus | minus)) + number_part) + number = (inner_number + + Optional(CaselessLiteral("E") + Optional(plus_minus) + number_part) + Optional(number_suffix)) - number = number.setParseAction(number_parse_action) # Convert to number + number.setParseAction(number_parse_action) # Convert to number # Predefine recursive variables expr = Forward() - factor = Forward() - def sreduce(f, l): - ''' Same as reduce, but handle len 1 and len 0 lists sensibly ''' - if len(l) == 0: - return NoMatch() - if len(l) == 1: - return l[0] - return reduce(f, l) + # Handle variables passed in. + # E.g. if we have {'R':0.5}, we make the substitution. + # We sort the list so that var names (like "e2") match before + # mathematical constants (like "e"). This is kind of a hack. + all_variables_keys = sorted(all_variables.keys(), key=len, reverse=True) + varnames = MatchFirst([CasedLiteral(k) for k in all_variables_keys]) + varnames.setParseAction( + lambda x: [all_variables[k] for k in x] + ) - # Handle variables passed in. E.g. if we have {'R':0.5}, we make the substitution. - # Special case for no variables because of how we understand PyParsing is put together - if len(all_variables) > 0: - # We sort the list so that var names (like "e2") match before - # mathematical constants (like "e"). This is kind of a hack. - all_variables_keys = sorted(all_variables.keys(), key=len, reverse=True) - varnames = sreduce(lambda x, y: x | y, map(lambda x: CasedLiteral(x), all_variables_keys)) - varnames.setParseAction(lambda x: map(lambda y: all_variables[y], x)) - else: - varnames = NoMatch() + # if all_variables were empty, then pyparsing wants + # varnames = NoMatch() + # this is not the case, as all_variables contains the defaults # Same thing for functions. - if len(all_functions) > 0: - funcnames = sreduce(lambda x, y: x | y, - map(lambda x: CasedLiteral(x), all_functions.keys())) - function = funcnames + lpar.suppress() + expr + rpar.suppress() - function.setParseAction(func_parse_action) - else: - function = NoMatch() + all_functions_keys = sorted(all_functions.keys(), key=len, reverse=True) + funcnames = MatchFirst([CasedLiteral(k) for k in all_functions_keys]) + function = funcnames + Suppress("(") + expr + Suppress(")") + function.setParseAction( + lambda x: [all_functions[x[0]](x[1])] + ) - atom = number | function | varnames | lpar + expr + rpar - factor << (atom + ZeroOrMore(exp + atom)).setParseAction(exp_parse_action) # 7^6 - paritem = factor + ZeroOrMore(Literal('||') + factor) # 5k || 4k - paritem = paritem.setParseAction(parallel) - term = paritem + ZeroOrMore((times | div) + paritem) # 7 * 5 / 4 - 3 - term = term.setParseAction(prod_parse_action) - expr << Optional((plus | minus)) + term + ZeroOrMore((plus | minus) + term) # -5 + 4 - 3 - expr = expr.setParseAction(sum_parse_action) + atom = number | function | varnames | Suppress("(") + expr + Suppress(")") + + # Do the following in the correct order to preserve order of operation + pow_term = atom + ZeroOrMore(Suppress("^") + atom) + pow_term.setParseAction(exp_parse_action) # 7^6 + par_term = pow_term + ZeroOrMore(Suppress('||') + pow_term) # 5k || 4k + par_term.setParseAction(parallel) + prod_term = par_term + ZeroOrMore(times_div + par_term) # 7 * 5 / 4 - 3 + prod_term.setParseAction(prod_parse_action) + sum_term = Optional(plus_minus) + prod_term + ZeroOrMore(plus_minus + prod_term) # -5 + 4 - 3 + sum_term.setParseAction(sum_parse_action) + expr << sum_term # finish the recursion return (expr + stringEnd).parseString(string)[0] diff --git a/common/lib/calc/calcfunctions.py b/common/lib/calc/calcfunctions.py new file mode 100644 index 0000000000..d0ac4f7a3d --- /dev/null +++ b/common/lib/calc/calcfunctions.py @@ -0,0 +1,99 @@ +""" +Provide the mathematical functions that numpy doesn't. + +Specifically, the secant/cosecant/cotangents and their inverses and +hyperbolic counterparts +""" +import numpy + + +# Normal Trig +def sec(arg): + """ + Secant + """ + return 1 / numpy.cos(arg) + + +def csc(arg): + """ + Cosecant + """ + return 1 / numpy.sin(arg) + + +def cot(arg): + """ + Cotangent + """ + return 1 / numpy.tan(arg) + + +# Inverse Trig +# http://en.wikipedia.org/wiki/Inverse_trigonometric_functions#Relationships_among_the_inverse_trigonometric_functions +def arcsec(val): + """ + Inverse secant + """ + return numpy.arccos(1. / val) + + +def arccsc(val): + """ + Inverse cosecant + """ + return numpy.arcsin(1. / val) + + +def arccot(val): + """ + Inverse cotangent + """ + if numpy.real(val) < 0: + return -numpy.pi / 2 - numpy.arctan(val) + else: + return numpy.pi / 2 - numpy.arctan(val) + + +# Hyperbolic Trig +def sech(arg): + """ + Hyperbolic secant + """ + return 1 / numpy.cosh(arg) + + +def csch(arg): + """ + Hyperbolic cosecant + """ + return 1 / numpy.sinh(arg) + + +def coth(arg): + """ + Hyperbolic cotangent + """ + return 1 / numpy.tanh(arg) + + +# And their inverses +def arcsech(val): + """ + Inverse hyperbolic secant + """ + return numpy.arccosh(1. / val) + + +def arccsch(val): + """ + Inverse hyperbolic cosecant + """ + return numpy.arcsinh(1. / val) + + +def arccoth(val): + """ + Inverse hyperbolic cotangent + """ + return numpy.arctanh(1. / val) diff --git a/common/lib/calc/tests/test_calc.py b/common/lib/calc/tests/test_calc.py index cfa1b7525d..13cd9e9471 100644 --- a/common/lib/calc/tests/test_calc.py +++ b/common/lib/calc/tests/test_calc.py @@ -194,6 +194,105 @@ class EvaluatorTest(unittest.TestCase): arctan_angles = arcsin_angles self.assert_function_values('arctan', arctan_inputs, arctan_angles) + def test_reciprocal_trig_functions(self): + """ + Test the reciprocal trig functions provided in calc.py + + which are: sec, csc, cot, arcsec, arccsc, arccot + """ + angles = ['-pi/4', 'pi/6', 'pi/5', '5*pi/4', '9*pi/4', '1 + j'] + sec_values = [1.414, 1.155, 1.236, -1.414, 1.414, 0.498 + 0.591j] + csc_values = [-1.414, 2, 1.701, -1.414, 1.414, 0.622 - 0.304j] + cot_values = [-1, 1.732, 1.376, 1, 1, 0.218 - 0.868j] + + self.assert_function_values('sec', angles, sec_values) + self.assert_function_values('csc', angles, csc_values) + self.assert_function_values('cot', angles, cot_values) + + arcsec_inputs = ['1.1547', '1.2361', '2', '-2', '-1.4142', '0.4983+0.5911*j'] + arcsec_angles = [0.524, 0.628, 1.047, 2.094, 2.356, 1 + 1j] + self.assert_function_values('arcsec', arcsec_inputs, arcsec_angles) + + arccsc_inputs = ['-1.1547', '-1.4142', '2', '1.7013', '1.1547', '0.6215-0.3039*j'] + arccsc_angles = [-1.047, -0.785, 0.524, 0.628, 1.047, 1 + 1j] + self.assert_function_values('arccsc', arccsc_inputs, arccsc_angles) + + # Has the same range as arccsc + arccot_inputs = ['-0.5774', '-1', '1.7321', '1.3764', '0.5774', '(0.2176-0.868*j)'] + arccot_angles = arccsc_angles + self.assert_function_values('arccot', arccot_inputs, arccot_angles) + + def test_hyperbolic_functions(self): + """ + Test the hyperbolic functions + + which are: sinh, cosh, tanh, sech, csch, coth + """ + inputs = ['0', '0.5', '1', '2', '1+j'] + neg_inputs = ['0', '-0.5', '-1', '-2', '-1-j'] + negate = lambda x: [-k for k in x] + + # sinh is odd + sinh_vals = [0, 0.521, 1.175, 3.627, 0.635 + 1.298j] + self.assert_function_values('sinh', inputs, sinh_vals) + self.assert_function_values('sinh', neg_inputs, negate(sinh_vals)) + + # cosh is even - do not negate + cosh_vals = [1, 1.128, 1.543, 3.762, 0.834 + 0.989j] + self.assert_function_values('cosh', inputs, cosh_vals) + self.assert_function_values('cosh', neg_inputs, cosh_vals) + + # tanh is odd + tanh_vals = [0, 0.462, 0.762, 0.964, 1.084 + 0.272j] + self.assert_function_values('tanh', inputs, tanh_vals) + self.assert_function_values('tanh', neg_inputs, negate(tanh_vals)) + + # sech is even - do not negate + sech_vals = [1, 0.887, 0.648, 0.266, 0.498 - 0.591j] + self.assert_function_values('sech', inputs, sech_vals) + self.assert_function_values('sech', neg_inputs, sech_vals) + + # the following functions do not have 0 in their domain + inputs = inputs[1:] + neg_inputs = neg_inputs[1:] + + # csch is odd + csch_vals = [1.919, 0.851, 0.276, 0.304 - 0.622j] + self.assert_function_values('csch', inputs, csch_vals) + self.assert_function_values('csch', neg_inputs, negate(csch_vals)) + + # coth is odd + coth_vals = [2.164, 1.313, 1.037, 0.868 - 0.218j] + self.assert_function_values('coth', inputs, coth_vals) + self.assert_function_values('coth', neg_inputs, negate(coth_vals)) + + def test_hyperbolic_inverses(self): + """ + Test the inverse hyperbolic functions + + which are of the form arc[X]h + """ + results = [0, 0.5, 1, 2, 1 + 1j] + + sinh_vals = ['0', '0.5211', '1.1752', '3.6269', '0.635+1.2985*j'] + self.assert_function_values('arcsinh', sinh_vals, results) + + cosh_vals = ['1', '1.1276', '1.5431', '3.7622', '0.8337+0.9889*j'] + self.assert_function_values('arccosh', cosh_vals, results) + + tanh_vals = ['0', '0.4621', '0.7616', '0.964', '1.0839+0.2718*j'] + self.assert_function_values('arctanh', tanh_vals, results) + + sech_vals = ['1.0', '0.8868', '0.6481', '0.2658', '0.4983-0.5911*j'] + self.assert_function_values('arcsech', sech_vals, results) + + results = results[1:] + csch_vals = ['1.919', '0.8509', '0.2757', '0.3039-0.6215*j'] + self.assert_function_values('arccsch', csch_vals, results) + + coth_vals = ['2.164', '1.313', '1.0373', '0.868-0.2176*j'] + self.assert_function_values('arccoth', coth_vals, results) + def test_other_functions(self): """ Test the non-trig functions provided in calc.py diff --git a/common/lib/capa/capa/capa_problem.py b/common/lib/capa/capa/capa_problem.py index 150b3b3c9b..7dcd7b925e 100644 --- a/common/lib/capa/capa/capa_problem.py +++ b/common/lib/capa/capa/capa_problem.py @@ -470,6 +470,7 @@ class LoncapaProblem(object): python_path=python_path, cache=self.system.cache, slug=self.problem_id, + unsafely=self.system.can_execute_unsafe_code(), ) except Exception as err: log.exception("Error while execing script code: " + all_code) diff --git a/common/lib/capa/capa/inputtypes.py b/common/lib/capa/capa/inputtypes.py index 65280d6d29..446b832dd7 100644 --- a/common/lib/capa/capa/inputtypes.py +++ b/common/lib/capa/capa/inputtypes.py @@ -144,11 +144,11 @@ class InputTypeBase(object): self.tag = xml.tag self.system = system - ## NOTE: ID should only come from one place. If it comes from multiple, - ## we use state first, XML second (in case the xml changed, but we have - ## existing state with an old id). Since we don't make this guarantee, - ## we can swap this around in the future if there's a more logical - ## order. + # NOTE: ID should only come from one place. If it comes from multiple, + # we use state first, XML second (in case the xml changed, but we have + # existing state with an old id). Since we don't make this guarantee, + # we can swap this around in the future if there's a more logical + # order. self.input_id = state.get('id', xml.get('id')) if self.input_id is None: @@ -769,7 +769,7 @@ class MatlabInput(CodeInput): # construct xqueue headers qinterface = self.system.xqueue['interface'] - qtime = datetime.strftime(datetime.utcnow(), xqueue_interface.dateformat) + qtime = datetime.utcnow().strftime(xqueue_interface.dateformat) callback_url = self.system.xqueue['construct_callback']('ungraded_response') anonymous_student_id = self.system.anonymous_student_id queuekey = xqueue_interface.make_hashkey(str(self.system.seed) + qtime + diff --git a/common/lib/capa/capa/responsetypes.py b/common/lib/capa/capa/responsetypes.py index 0fa50079de..80227490da 100644 --- a/common/lib/capa/capa/responsetypes.py +++ b/common/lib/capa/capa/responsetypes.py @@ -288,7 +288,14 @@ class LoncapaResponse(object): } try: - safe_exec.safe_exec(code, globals_dict, python_path=self.context['python_path'], slug=self.id) + safe_exec.safe_exec( + code, + globals_dict, + python_path=self.context['python_path'], + slug=self.id, + random_seed=self.context['seed'], + unsafely=self.system.can_execute_unsafe_code(), + ) except Exception as err: msg = 'Error %s in evaluating hint function %s' % (err, hintfn) msg += "\nSee XML source line %s" % getattr( @@ -973,7 +980,14 @@ class CustomResponse(LoncapaResponse): 'ans': ans, } globals_dict.update(kwargs) - safe_exec.safe_exec(code, globals_dict, python_path=self.context['python_path'], slug=self.id) + safe_exec.safe_exec( + code, + globals_dict, + python_path=self.context['python_path'], + slug=self.id, + random_seed=self.context['seed'], + unsafely=self.system.can_execute_unsafe_code(), + ) return globals_dict['cfn_return'] return check_function @@ -1090,7 +1104,14 @@ class CustomResponse(LoncapaResponse): # exec the check function if isinstance(self.code, basestring): try: - safe_exec.safe_exec(self.code, self.context, cache=self.system.cache, slug=self.id) + safe_exec.safe_exec( + self.code, + self.context, + cache=self.system.cache, + slug=self.id, + random_seed=self.context['seed'], + unsafely=self.system.can_execute_unsafe_code(), + ) except Exception as err: self._handle_exec_exception(err) @@ -1717,6 +1738,7 @@ class FormulaResponse(LoncapaResponse): student_variables = dict() # ranges give numerical ranges for testing for var in ranges: + # TODO: allow specified ranges (i.e. integers and complex numbers) for random variables value = random.uniform(*ranges[var]) instructor_variables[str(var)] = value student_variables[str(var)] = value @@ -1814,7 +1836,14 @@ class SchematicResponse(LoncapaResponse): ] self.context.update({'submission': submission}) try: - safe_exec.safe_exec(self.code, self.context, cache=self.system.cache, slug=self.id) + safe_exec.safe_exec( + self.code, + self.context, + cache=self.system.cache, + slug=self.id, + random_seed=self.context['seed'], + unsafely=self.system.can_execute_unsafe_code(), + ) except Exception as err: msg = 'Error %s in evaluating SchematicResponse' % err raise ResponseError(msg) diff --git a/common/lib/capa/capa/safe_exec/safe_exec.py b/common/lib/capa/capa/safe_exec/safe_exec.py index 67e93be46f..3ab8f0bf9e 100644 --- a/common/lib/capa/capa/safe_exec/safe_exec.py +++ b/common/lib/capa/capa/safe_exec/safe_exec.py @@ -1,6 +1,7 @@ """Capa's specialized use of codejail.safe_exec.""" from codejail.safe_exec import safe_exec as codejail_safe_exec +from codejail.safe_exec import not_safe_exec as codejail_not_safe_exec from codejail.safe_exec import json_safe, SafeExecException from . import lazymod from statsd import statsd @@ -71,7 +72,7 @@ def update_hash(hasher, obj): @statsd.timed('capa.safe_exec.time') -def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None, slug=None): +def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None, slug=None, unsafely=False): """ Execute python code safely. @@ -90,6 +91,8 @@ def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None `slug` is an arbitrary string, a description that's meaningful to the caller, that will be used in log messages. + If `unsafely` is true, then the code will actually be executed without sandboxing. + """ # Check the cache for a previous result. if cache: @@ -111,9 +114,15 @@ def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None # Create the complete code we'll run. code_prolog = CODE_PROLOG % random_seed + # Decide which code executor to use. + if unsafely: + exec_fn = codejail_not_safe_exec + else: + exec_fn = codejail_safe_exec + # Run the code! Results are side effects in globals_dict. try: - codejail_safe_exec( + exec_fn( code_prolog + LAZY_IMPORTS + code, globals_dict, python_path=python_path, slug=slug, ) diff --git a/common/lib/capa/capa/safe_exec/tests/test_safe_exec.py b/common/lib/capa/capa/safe_exec/tests/test_safe_exec.py index 4592af8305..f8a8a32297 100644 --- a/common/lib/capa/capa/safe_exec/tests/test_safe_exec.py +++ b/common/lib/capa/capa/safe_exec/tests/test_safe_exec.py @@ -1,13 +1,17 @@ """Test safe_exec.py""" import hashlib +import os import os.path import random import textwrap import unittest +from nose.plugins.skip import SkipTest + from capa.safe_exec import safe_exec, update_hash from codejail.safe_exec import SafeExecException +from codejail.jail_code import is_configured class TestSafeExec(unittest.TestCase): @@ -68,6 +72,24 @@ class TestSafeExec(unittest.TestCase): self.assertIn("ZeroDivisionError", cm.exception.message) +class TestSafeOrNot(unittest.TestCase): + def test_cant_do_something_forbidden(self): + # Can't test for forbiddenness if CodeJail isn't configured for python. + if not is_configured("python"): + raise SkipTest + + g = {} + with self.assertRaises(SafeExecException) as cm: + safe_exec("import os; files = os.listdir('/')", g) + self.assertIn("OSError", cm.exception.message) + self.assertIn("Permission denied", cm.exception.message) + + def test_can_do_something_forbidden_if_run_unsafely(self): + g = {} + safe_exec("import os; files = os.listdir('/')", g, unsafely=True) + self.assertEqual(g['files'], os.listdir('/')) + + class DictCache(object): """A cache implementation over a simple dict, for testing.""" diff --git a/common/lib/capa/capa/templates/annotationinput.html b/common/lib/capa/capa/templates/annotationinput.html index e0172bb13b..145a7c2cad 100644 --- a/common/lib/capa/capa/templates/annotationinput.html +++ b/common/lib/capa/capa/templates/annotationinput.html @@ -14,7 +14,7 @@
${comment}
${comment_prompt}
- +
${tag_prompt}
    @@ -22,11 +22,11 @@
  • % if has_options_value: % if all([c == 'correct' for c in option['choice'], status]): - + Status: Correct % elif all([c == 'partially-correct' for c in option['choice'], status]): - + Status: Partially Correct % elif all([c == 'incorrect' for c in option['choice'], status]): - + Status: Incorrect % endif % endif @@ -53,11 +53,11 @@ % endif % if status == 'unsubmitted': - + Status: Unanswered % elif status == 'incomplete': - + Status: Incorrect % elif status == 'incorrect' and not has_options_value: - + Status: Incorrect % endif

    diff --git a/common/lib/capa/capa/templates/chemicalequationinput.html b/common/lib/capa/capa/templates/chemicalequationinput.html index 17c84114e5..34709c3e5e 100644 --- a/common/lib/capa/capa/templates/chemicalequationinput.html +++ b/common/lib/capa/capa/templates/chemicalequationinput.html @@ -11,13 +11,13 @@
    % endif - -

    +

    % if status == 'unsubmitted': unanswered % elif status == 'correct': diff --git a/common/lib/capa/capa/templates/choicegroup.html b/common/lib/capa/capa/templates/choicegroup.html index c9cc3fd28d..17f7efcec4 100644 --- a/common/lib/capa/capa/templates/choicegroup.html +++ b/common/lib/capa/capa/templates/choicegroup.html @@ -3,12 +3,12 @@ % if input_type == 'checkbox' or not value: % if status == 'unsubmitted' or show_correctness == 'never': - % elif status == 'correct': - + % elif status == 'correct': + Status: correct % elif status == 'incorrect': - + Status: incorrect % elif status == 'incomplete': - + Status: incomplete % endif % endif

    @@ -18,7 +18,7 @@ % for choice_id, choice_description in choices: + /> ${choice_description} + + % if input_type == 'radio' and ( (isinstance(value, basestring) and (choice_id == value)) or (not isinstance(value, basestring) and choice_id in value) ): + <% + if status == 'correct': + correctness = 'correct' + elif status == 'incorrect': + correctness = 'incorrect' + else: + correctness = None + %> + % if correctness and not show_correctness=='never': + Status: ${correctness} + % endif + % endif + % endfor diff --git a/common/lib/capa/capa/templates/codeinput.html b/common/lib/capa/capa/templates/codeinput.html index eb8cad0d70..08ad4ff062 100644 --- a/common/lib/capa/capa/templates/codeinput.html +++ b/common/lib/capa/capa/templates/codeinput.html @@ -1,5 +1,5 @@
    - +

    Enroll or un-enroll one or many students: enter emails, separated by new lines or commas;

    + +

    + Auto-enroll students when they activate +

    + %endif @@ -539,17 +541,17 @@ function goto( mode)

    -

    ${datatable['title']}

    +

    ${datatable['title'] | h}

    %for hname in datatable['header']: - + %endfor %for row in datatable['data']: %for value in row: - + %endfor %endfor diff --git a/lms/templates/courseware/progress.html b/lms/templates/courseware/progress.html index fcd4348f96..bf1414cc07 100644 --- a/lms/templates/courseware/progress.html +++ b/lms/templates/courseware/progress.html @@ -35,7 +35,7 @@ ${progress_graph.body(grade_summary, course.grade_cutoffs, "grade-detail-graph", %if not course.disable_progress_graph: -
    + %endif
      @@ -54,7 +54,12 @@ ${progress_graph.body(grade_summary, course.grade_cutoffs, "grade-detail-graph", %>

      - ${ section['display_name'] } + ${ section['display_name'] } + %if total > 0 or earned > 0: + + ${"{0:.3n} of {1:.3n} possible points".format( float(earned), float(total) )} + + %endif %if total > 0 or earned > 0: ${"({0:.3n}/{1:.3n}) {2}".format( float(earned), float(total), percentageString )} %endif diff --git a/lms/templates/help_modal.html b/lms/templates/help_modal.html index 350f858334..deb2db3610 100644 --- a/lms/templates/help_modal.html +++ b/lms/templates/help_modal.html @@ -21,9 +21,12 @@ discussion_link = get_discussion_link(course) if course else None %> +% if discussion_link:

      For questions on course lectures, homework, tools, or materials for this course, post in the course discussion forum.

      +% endif +

      Have general questions about edX? You can find lots of helpful information in the edX FAQ.

      @@ -61,7 +64,10 @@ discussion_link = get_discussion_link(course) if course else None - + +% if course: + +% endif
      @@ -111,21 +117,41 @@ discussion_link = get_discussion_link(course) if course else None $("#feedback_success_wrapper").css("display", "none"); $("#help_wrapper").css("display", "block"); }); - showFeedback = function(e, tag, title) { + showFeedback = function(event, issue_type, title, subject_label, details_label) { $("#help_wrapper").css("display", "none"); - $("#feedback_form input[name='tag']").val(tag); + $("#feedback_form input[name='issue_type']").val(issue_type); $("#feedback_form_wrapper").css("display", "block"); $("#feedback_form_wrapper header").html("

      " + title + "

      "); - e.preventDefault(); + $("#feedback_form_wrapper label[data-field='subject']").html(subject_label); + $("#feedback_form_wrapper label[data-field='details']").html(details_label); + event.preventDefault(); }; - $("#feedback_link_problem").click(function(e) { - showFeedback(e, "problem", "Report a Problem"); + $("#feedback_link_problem").click(function(event) { + showFeedback( + event, + "problem", + "Report a Problem", + "Brief description of the problem*", + "Details of the problem you are encountering* Include error messages, steps which lead to the issue, etc." + ); }); - $("#feedback_link_suggestion").click(function(e) { - showFeedback(e, "suggestion", "Make a Suggestion"); + $("#feedback_link_suggestion").click(function(event) { + showFeedback( + event, + "suggestion", + "Make a Suggestion", + "Brief description of your suggestion*", + "Details*" + ); }); - $("#feedback_link_question").click(function(e) { - showFeedback(e, "question", "Ask a Question"); + $("#feedback_link_question").click(function(event) { + showFeedback( + event, + "question", + "Ask a Question", + "Brief summary of your question*", + "Details*" + ); }); $("#feedback_form").submit(function() { $("input[type='submit']", this).attr("disabled", "disabled"); diff --git a/lms/templates/instructor/staff_grading.html b/lms/templates/instructor/staff_grading.html index 1c5f7364ad..0a28a2b026 100644 --- a/lms/templates/instructor/staff_grading.html +++ b/lms/templates/instructor/staff_grading.html @@ -29,7 +29,7 @@

      Instructions

      -

      This is the list of problems that current need to be graded in order to train the machine learning models. Each problem needs to be trained separately, and we have indicated the number of student submissions that need to be graded in order for a model to be generated. You can grade more than the minimum required number of submissions--this will improve the accuracy of machine learning, though with diminishing returns. You can see the current accuracy of machine learning while grading.

      +

      This is the list of problems that currently need to be graded in order to train the machine learning models. Each problem needs to be trained separately, and we have indicated the number of student submissions that need to be graded in order for a model to be generated. You can grade more than the minimum required number of submissions--this will improve the accuracy of machine learning, though with diminishing returns. You can see the current accuracy of machine learning while grading.

      Problem List

      diff --git a/lms/templates/login.html b/lms/templates/login.html index b185b02c84..fbd2f6d07c 100644 --- a/lms/templates/login.html +++ b/lms/templates/login.html @@ -86,11 +86,7 @@
    ${hname}${hname | h}
    ${value}${value | h}