diff --git a/cms/djangoapps/contentstore/features/advanced-settings.feature b/cms/djangoapps/contentstore/features/advanced-settings.feature index a11a6cb869..b2941ac7a5 100644 --- a/cms/djangoapps/contentstore/features/advanced-settings.feature +++ b/cms/djangoapps/contentstore/features/advanced-settings.feature @@ -2,6 +2,7 @@ Feature: Advanced (manual) course policy In order to specify course policy settings for which no custom user interface exists I want to be able to manually enter JSON key /value pairs + Scenario: A course author sees default advanced settings Given I have opened a new course in Studio When I select the Advanced Settings @@ -11,6 +12,8 @@ Feature: Advanced (manual) course policy Given I am on the Advanced Course Settings page in Studio Then the settings are alphabetized + # Sauce labs does not play nicely with CodeMirror + @skip_sauce Scenario: Test cancel editing key value Given I am on the Advanced Course Settings page in Studio When I edit the value of a policy key @@ -19,6 +22,8 @@ Feature: Advanced (manual) course policy And I reload the page Then the policy key value is unchanged + # Sauce labs does not play nicely with CodeMirror + @skip_sauce Scenario: Test editing key value Given I am on the Advanced Course Settings page in Studio When I edit the value of a policy key and save @@ -26,6 +31,8 @@ Feature: Advanced (manual) course policy And I reload the page Then the policy key value is changed + # Sauce labs does not play nicely with CodeMirror + @skip_sauce Scenario: Test how multi-line input appears Given I am on the Advanced Course Settings page in Studio When I create a JSON object as a value for "discussion_topics" @@ -33,6 +40,8 @@ Feature: Advanced (manual) course policy And I reload the page Then it is displayed as formatted + # Sauce labs does not play nicely with CodeMirror + @skip_sauce Scenario: Test error if value supplied is of the wrong type Given I am on the Advanced Course Settings page in Studio When I create a JSON object as a value for "display_name" @@ -41,6 +50,8 @@ Feature: Advanced (manual) course policy Then the policy key value is unchanged # This feature will work in Firefox only when Firefox is the active window + # Sauce labs does not play nicely with CodeMirror + @skip_sauce Scenario: Test automatic quoting of non-JSON values Given I am on the Advanced Course Settings page in Studio When I create a non-JSON value not in quotes @@ -48,6 +59,8 @@ Feature: Advanced (manual) course policy And I reload the page Then it is displayed as a string + # Sauce labs does not play nicely with CodeMirror + @skip_sauce Scenario: Confirmation is shown on save Given I am on the Advanced Course Settings page in Studio When I edit the value of a policy key diff --git a/cms/djangoapps/contentstore/features/checklists.feature b/cms/djangoapps/contentstore/features/checklists.feature index f13ce53fc2..6289df9cfc 100644 --- a/cms/djangoapps/contentstore/features/checklists.feature +++ b/cms/djangoapps/contentstore/features/checklists.feature @@ -10,7 +10,10 @@ Feature: Course checklists Then I can check and uncheck tasks in a checklist And They are correctly selected after reloading the page - # CHROME ONLY, due to issues getting link to be active in firefox + # There are issues getting link to be active in browsers other than chrome + @skip_firefox + @skip_internetexplorer + @skip_safari Scenario: A task can link to a location within Studio Given I have opened Checklists When I select a link to the course outline @@ -18,7 +21,10 @@ Feature: Course checklists And I press the browser back button Then I am brought back to the course outline in the correct state - # CHROME ONLY, due to issues getting link to be active in firefox + # There are issues getting link to be active in browsers other than chrome + @skip_firefox + @skip_internetexplorer + @skip_safari Scenario: A task can link to a location outside Studio Given I have opened Checklists When I select a link to help page diff --git a/cms/djangoapps/contentstore/features/course-overview.feature b/cms/djangoapps/contentstore/features/course-overview.feature index a9aed5d982..2cbb22ddd7 100644 --- a/cms/djangoapps/contentstore/features/course-overview.feature +++ b/cms/djangoapps/contentstore/features/course-overview.feature @@ -64,6 +64,10 @@ Feature: Course Overview And I change an assignment's grading status Then I am shown a notification + # Notification is not shown on reorder for IE + # Safari does not have moveMouseTo implemented + @skip_internetexplorer + @skip_safari Scenario: Notification is shown on subsection reorder Given I have opened a new course section in Studio And I have added a new subsection diff --git a/cms/djangoapps/contentstore/features/course-settings.feature b/cms/djangoapps/contentstore/features/course-settings.feature index 8f00452efe..9976179b68 100644 --- a/cms/djangoapps/contentstore/features/course-settings.feature +++ b/cms/djangoapps/contentstore/features/course-settings.feature @@ -1,6 +1,8 @@ Feature: Course Settings As a course author, I want to be able to configure my course settings. + # Safari has trouble keeps dates on refresh + @skip_safari Scenario: User can set course dates Given I have opened a new course in Studio When I select Schedule and Details @@ -8,12 +10,16 @@ Feature: Course Settings And I press the "Save" notification button Then I see the set dates on refresh + # IE has trouble with saving information + @skip_internetexplorer Scenario: User can clear previously set course dates (except start date) Given I have set course dates And I clear all the dates except start And I press the "Save" notification button Then I see cleared dates on refresh + # IE has trouble with saving information + @skip_internetexplorer Scenario: User cannot clear the course start date Given I have set course dates And I press the "Save" notification button @@ -21,6 +27,10 @@ Feature: Course Settings Then I receive a warning about course start date And The previously set start date is shown on refresh + # IE has trouble with saving information + # Safari gets CSRF token errors + @skip_internetexplorer + @skip_safari Scenario: User can correct the course start date warning Given I have tried to clear the course start And I have entered a new course start date @@ -28,12 +38,16 @@ Feature: Course Settings Then The warning about course start date goes away And My new course start date is shown on refresh + # Safari does not save + refresh properly through sauce labs + @skip_safari Scenario: Settings are only persisted when saved Given I have set course dates And I press the "Save" notification button When I change fields Then I do not see the new changes persisted on refresh + # Safari does not save + refresh properly through sauce labs + @skip_safari Scenario: Settings are reset on cancel Given I have set course dates And I press the "Save" notification button @@ -41,6 +55,8 @@ Feature: Course Settings And I press the "Cancel" notification button Then I do not see the changes + # Safari gets CSRF token errors + @skip_safari Scenario: Confirmation is shown on save Given I have opened a new course in Studio When I select Schedule and Details diff --git a/cms/djangoapps/contentstore/features/course-team.py b/cms/djangoapps/contentstore/features/course-team.py index db7b4d81f9..8b31d325e5 100644 --- a/cms/djangoapps/contentstore/features/course-team.py +++ b/cms/djangoapps/contentstore/features/course-team.py @@ -91,7 +91,7 @@ def remove_course_team_admin(_step, outer_capture, name): @step(u'"([^"]*)" logs in$') def other_user_login(_step, name): - world.browser.cookies.delete() + world.visit('logout') world.visit('/') signin_css = 'a.action-signin' diff --git a/cms/djangoapps/contentstore/features/course-updates.feature b/cms/djangoapps/contentstore/features/course-updates.feature index fb18e51f2d..41ee785db5 100644 --- a/cms/djangoapps/contentstore/features/course-updates.feature +++ b/cms/djangoapps/contentstore/features/course-updates.feature @@ -1,6 +1,8 @@ Feature: Course updates As a course author, I want to be able to provide updates to my students + # Internet explorer can't select all so the update appears weirdly + @skip_internetexplorer Scenario: Users can add updates Given I have opened a new course in Studio And I go to the course updates page @@ -8,6 +10,8 @@ Feature: Course updates Then I should see the update "Hello" And I see a "saving" notification + # Internet explorer can't select all so the update appears weirdly + @skip_internetexplorer Scenario: Users can edit updates Given I have opened a new course in Studio And I go to the course updates page @@ -33,6 +37,8 @@ Feature: Course updates Then I should see the date "June 1, 2013" And I see a "saving" notification + # Internet explorer can't select all so the update appears weirdly + @skip_internetexplorer Scenario: Users can change handouts Given I have opened a new course in Studio And I go to the course updates page diff --git a/cms/djangoapps/contentstore/features/discussion-editor.feature b/cms/djangoapps/contentstore/features/discussion-editor.feature index 8fb14c3205..e4b1f5450b 100644 --- a/cms/djangoapps/contentstore/features/discussion-editor.feature +++ b/cms/djangoapps/contentstore/features/discussion-editor.feature @@ -6,6 +6,8 @@ Feature: Discussion Component Editor And I edit and select Settings Then I see three alphabetized settings and their expected values + # Safari doesn't save the name properly + @skip_safari Scenario: User can modify display name Given I have created a Discussion Tag And I edit and select Settings diff --git a/cms/djangoapps/contentstore/features/grading.feature b/cms/djangoapps/contentstore/features/grading.feature index 5cf5a72866..0b34feb7aa 100644 --- a/cms/djangoapps/contentstore/features/grading.feature +++ b/cms/djangoapps/contentstore/features/grading.feature @@ -13,7 +13,7 @@ Feature: Course Grading When I add "6" new grades Then I see I now have "5" grades - #Cannot reliably make the delete button appear so using javascript instead + # Cannot reliably make the delete button appear so using javascript instead Scenario: Users can delete grading ranges Given I have opened a new course in Studio And I am viewing the grading settings @@ -21,6 +21,9 @@ Feature: Course Grading And I delete a grade Then I see I now have "2" grades + # IE and Safari cannot reliably drag and drop through selenium + @skip_internetexplorer + @skip_safari Scenario: Users can move grading ranges Given I have opened a new course in Studio And I am viewing the grading settings @@ -85,6 +88,9 @@ Feature: Course Grading When I change assignment type "Homework" to "" Then the save button is disabled + # IE and Safari cannot type in grade range name + @skip_internetexplorer + @skip_safari Scenario: User can edit grading range names Given I have opened a new course in Studio And I have populated the course diff --git a/cms/djangoapps/contentstore/features/grading.py b/cms/djangoapps/contentstore/features/grading.py index 719b3f7f7c..93e44b3893 100644 --- a/cms/djangoapps/contentstore/features/grading.py +++ b/cms/djangoapps/contentstore/features/grading.py @@ -112,10 +112,10 @@ def changes_not_persisted(step): @step(u'I see the assignment type "(.*)"$') def i_see_the_assignment_type(_step, name): - assignment_css = '#course-grading-assignment-name' - assignments = world.css_find(assignment_css) - types = [ele['value'] for ele in assignments] - assert name in types + assignment_css = '#course-grading-assignment-name' + assignments = world.css_find(assignment_css) + types = [ele['value'] for ele in assignments] + assert name in types @step(u'I change the highest grade range to "(.*)"$') @@ -144,6 +144,7 @@ def cannot_edit_fail(_step): pass # We should get this exception on failing to edit the element + @step(u'I change the grace period to "(.*)"$') def i_change_grace_period(_step, grace_period): grace_period_css = '#course-grading-graceperiod' diff --git a/cms/djangoapps/contentstore/features/html-editor.feature b/cms/djangoapps/contentstore/features/html-editor.feature index 4cd5e1c1b9..4419d6018b 100644 --- a/cms/djangoapps/contentstore/features/html-editor.feature +++ b/cms/djangoapps/contentstore/features/html-editor.feature @@ -6,6 +6,8 @@ Feature: HTML Editor And I edit and select Settings Then I see only the HTML display name setting + # Safari doesn't save the name properly + @skip_safari Scenario: User can modify display name Given I have created a Blank HTML Page And I edit and select Settings diff --git a/cms/djangoapps/contentstore/features/problem-editor.feature b/cms/djangoapps/contentstore/features/problem-editor.feature index 50c49a1896..1296acec1c 100644 --- a/cms/djangoapps/contentstore/features/problem-editor.feature +++ b/cms/djangoapps/contentstore/features/problem-editor.feature @@ -7,12 +7,16 @@ Feature: Problem Editor Then I see five alphabetized settings and their expected values And Edit High Level Source is not visible + # Safari is having trouble saving the values on sauce + @skip_safari Scenario: User can modify String values Given I have created a Blank Common Problem When I edit and select Settings Then I can modify the display name And my display name change is persisted on save + # Safari is having trouble saving the values on sauce + @skip_safari Scenario: User can specify special characters in String values Given I have created a Blank Common Problem When I edit and select Settings @@ -25,6 +29,8 @@ Feature: Problem Editor Then I can revert the display name to unset And my display name is unset on save + # IE will not click the revert button properly + @skip_internetexplorer Scenario: User can select values in a Select Given I have created a Blank Common Problem When I edit and select Settings @@ -32,6 +38,8 @@ Feature: Problem Editor And my change to randomization is persisted And I can revert to the default value for randomization + # Safari will input it as 35. + @skip_safari Scenario: User can modify float input values Given I have created a Blank Common Problem When I edit and select Settings @@ -44,16 +52,22 @@ Feature: Problem Editor When I edit and select Settings Then if I set the weight to "abc", it remains unset + # Safari will input it as 234. + @skip_safari Scenario: User cannot type decimal values integer number field Given I have created a Blank Common Problem When I edit and select Settings Then if I set the max attempts to "2.34", it will persist as a valid integer + # Safari will input it incorrectly + @skip_safari Scenario: User cannot type out of range values in an integer number field Given I have created a Blank Common Problem When I edit and select Settings Then if I set the max attempts to "-3", it will persist as a valid integer + # Safari will input it as 35. + @skip_safari Scenario: Settings changes are not saved on Cancel Given I have created a Blank Common Problem When I edit and select Settings @@ -67,6 +81,8 @@ Feature: Problem Editor Then Edit High Level Source is visible # This feature will work in Firefox only when Firefox is the active window + # IE will not interact with the high level source in sauce labs + @skip_internetexplorer Scenario: High Level source is persisted for LaTeX problem (bug STUD-280) Given I have created a LaTeX Problem When I edit and compile the High Level Source diff --git a/cms/djangoapps/contentstore/features/static-pages.feature b/cms/djangoapps/contentstore/features/static-pages.feature index 9997df69f0..c1a8ec91fc 100644 --- a/cms/djangoapps/contentstore/features/static-pages.feature +++ b/cms/djangoapps/contentstore/features/static-pages.feature @@ -15,6 +15,8 @@ Feature: Static Pages And I "delete" the "Empty" page Then I should not see a "Empty" static page + # Safari won't update the name properly + @skip_safari Scenario: Users can edit static pages Given I have opened a new course in Studio And I go to the static pages page diff --git a/cms/djangoapps/contentstore/features/subsection.feature b/cms/djangoapps/contentstore/features/subsection.feature index 84755b3644..6703c60c3b 100644 --- a/cms/djangoapps/contentstore/features/subsection.feature +++ b/cms/djangoapps/contentstore/features/subsection.feature @@ -25,6 +25,8 @@ Feature: Create Subsection And I reload the page Then I see it marked as Homework + # Safari has trouble saving the date in Sauce + @skip_safari Scenario: Set a due date in a different year (bug #256) Given I have opened a new subsection in Studio And I set the subsection release date to 12/25/2011 03:00 diff --git a/cms/djangoapps/contentstore/features/textbooks.feature b/cms/djangoapps/contentstore/features/textbooks.feature index 0758a0b57b..36de10daa1 100644 --- a/cms/djangoapps/contentstore/features/textbooks.feature +++ b/cms/djangoapps/contentstore/features/textbooks.feature @@ -5,6 +5,9 @@ Feature: Textbooks When I go to the textbooks page Then I should see a message telling me to create a new textbook + # IE and Safari on sauce labs will not upload the textbook correctly resulting in an error + @skip_internetexplorer + @skip_safari Scenario: Create a textbook Given I have opened a new course in Studio And I go to the textbooks page diff --git a/cms/djangoapps/contentstore/features/upload.feature b/cms/djangoapps/contentstore/features/upload.feature index 8d40163685..441de597ea 100644 --- a/cms/djangoapps/contentstore/features/upload.feature +++ b/cms/djangoapps/contentstore/features/upload.feature @@ -1,6 +1,8 @@ Feature: Upload Files As a course author, I want to be able to upload files for my students + # Uploading isn't working on safari with sauce labs + @skip_safari Scenario: Users can upload files Given I have opened a new course in Studio And I go to the files and uploads page @@ -8,6 +10,8 @@ Feature: Upload Files Then I should see the file "test" was uploaded And The url for the file "test" is valid + # Uploading isn't working on safari with sauce labs + @skip_safari Scenario: Users can update files Given I have opened a new course in studio And I go to the files and uploads page @@ -15,6 +19,8 @@ Feature: Upload Files And I upload the file "test" Then I should see only one "test" + # Uploading isn't working on safari with sauce labs + @skip_safari Scenario: Users can delete uploaded files Given I have opened a new course in studio And I go to the files and uploads page @@ -23,12 +29,16 @@ Feature: Upload Files Then I should not see the file "test" was uploaded And I see a confirmation that the file was deleted + # Uploading isn't working on safari with sauce labs + @skip_safari Scenario: Users can download files Given I have opened a new course in studio And I go to the files and uploads page When I upload the file "test" Then I can download the correct "test" file + # Uploading isn't working on safari with sauce labs + @skip_safari Scenario: Users can download updated files Given I have opened a new course in studio And I go to the files and uploads page diff --git a/cms/djangoapps/contentstore/features/upload.py b/cms/djangoapps/contentstore/features/upload.py index a989d6c07f..882b36e6b2 100644 --- a/cms/djangoapps/contentstore/features/upload.py +++ b/cms/djangoapps/contentstore/features/upload.py @@ -10,6 +10,7 @@ import os TEST_ROOT = settings.COMMON_TEST_DATA_ROOT + @step(u'I go to the files and uploads page') def go_to_uploads(_step): menu_css = 'li.nav-course-courseware' @@ -106,8 +107,8 @@ def get_index(file_name): def get_file(file_name): index = get_index(file_name) assert index != -1 - url_css = 'a.filename' + def get_url(): return world.css_find(url_css)[index]._element.get_attribute('href') url = world.retry_on_exception(get_url) diff --git a/cms/djangoapps/contentstore/features/video-editor.feature b/cms/djangoapps/contentstore/features/video-editor.feature index a53183e37c..d238a7e523 100644 --- a/cms/djangoapps/contentstore/features/video-editor.feature +++ b/cms/djangoapps/contentstore/features/video-editor.feature @@ -6,17 +6,23 @@ Feature: Video Component Editor And I edit the component Then I see the correct video settings and default values + # Safari has trouble saving values on Sauce + @skip_safari Scenario: User can modify Video display name Given I have created a Video component And I edit the component Then I can modify the display name And my video display name change is persisted on save + # Sauce Labs cannot delete cookies + @skip_sauce Scenario: Captions are hidden when "show captions" is false Given I have created a Video component And I have set "show captions" to False Then when I view the video it does not show the captions + # Sauce Labs cannot delete cookies + @skip_sauce Scenario: Captions are shown when "show captions" is true Given I have created a Video component And I have set "show captions" to True diff --git a/cms/djangoapps/contentstore/features/video.feature b/cms/djangoapps/contentstore/features/video.feature index 50c06fde63..d2f9915f55 100644 --- a/cms/djangoapps/contentstore/features/video.feature +++ b/cms/djangoapps/contentstore/features/video.feature @@ -10,15 +10,21 @@ Feature: Video Component Given I have clicked the new unit button Then creating a video takes a single click + # Sauce Labs cannot delete cookies + @skip_sauce Scenario: Captions are hidden correctly Given I have created a Video component And I have hidden captions Then when I view the video it does not show the captions + # Sauce Labs cannot delete cookies + @skip_sauce Scenario: Captions are shown correctly Given I have created a Video component Then when I view the video it does show the captions + # Sauce Labs cannot delete cookies + @skip_sauce Scenario: Captions are toggled correctly Given I have created a Video component And I have toggled captions diff --git a/cms/djangoapps/contentstore/management/commands/import.py b/cms/djangoapps/contentstore/management/commands/import.py index 46f439b055..e0d58b32f0 100644 --- a/cms/djangoapps/contentstore/management/commands/import.py +++ b/cms/djangoapps/contentstore/management/commands/import.py @@ -2,7 +2,7 @@ Script for importing courseware from XML format """ -from django.core.management.base import BaseCommand, CommandError +from django.core.management.base import BaseCommand, CommandError, make_option from xmodule.modulestore.xml_importer import import_from_xml from xmodule.modulestore.django import modulestore from xmodule.contentstore.django import contentstore @@ -14,18 +14,26 @@ class Command(BaseCommand): """ help = 'Import the specified data directory into the default ModuleStore' + option_list = BaseCommand.option_list + ( + make_option('--nostatic', + action='store_true', + help='Skip import of static content'), + ) + def handle(self, *args, **options): "Execute the command" if len(args) == 0: - raise CommandError("import requires at least one argument: [...]") + raise CommandError("import requires at least one argument: [--nostatic] [...]") data_dir = args[0] + do_import_static = not (options.get('nostatic', False)) if len(args) > 1: course_dirs = args[1:] else: course_dirs = None print("Importing. Data_dir={data}, course_dirs={courses}".format( data=data_dir, - courses=course_dirs)) + courses=course_dirs, + dis=do_import_static)) import_from_xml(modulestore('direct'), data_dir, course_dirs, load_error_modules=False, - static_content_store=contentstore(), verbose=True) + static_content_store=contentstore(), verbose=True, do_import_static=do_import_static) diff --git a/cms/djangoapps/contentstore/tests/test_contentstore.py b/cms/djangoapps/contentstore/tests/test_contentstore.py index 7491e5ab4a..f03ee3b81a 100644 --- a/cms/djangoapps/contentstore/tests/test_contentstore.py +++ b/cms/djangoapps/contentstore/tests/test_contentstore.py @@ -476,7 +476,7 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): content_store = contentstore() module_store = modulestore('direct') - import_from_xml(module_store, 'common/test/data/', ['toy'], static_content_store=content_store) + import_from_xml(module_store, 'common/test/data/', ['toy'], static_content_store=content_store, verbose=True) course_location = CourseDescriptor.id_to_location('edX/toy/2012_Fall') course = module_store.get_item(course_location) @@ -945,8 +945,17 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): 'vertical', 'vertical_test', None]), depth=1) self.assertTrue(getattr(vertical, 'is_draft', False)) + self.assertNotIn('index_in_children_list', child.xml_attributes) + self.assertNotIn('parent_sequential_url', vertical.xml_attributes) + for child in vertical.get_children(): self.assertTrue(getattr(child, 'is_draft', False)) + self.assertNotIn('index_in_children_list', child.xml_attributes) + if hasattr(child, 'data'): + self.assertNotIn('index_in_children_list', child.data) + self.assertNotIn('parent_sequential_url', child.xml_attributes) + if hasattr(child, 'data'): + self.assertNotIn('parent_sequential_url', child.data) # make sure that we don't have a sequential that is in draft mode sequential = draft_store.get_item(Location(['i4x', 'edX', 'toy', @@ -1057,6 +1066,38 @@ class ContentStoreToyCourseTest(ModuleStoreTestCase): # It should now contain empty data self.assertEquals(imported_word_cloud.data, '') + def test_html_export_roundtrip(self): + """ + Test that a course which has HTML that has style formatting is preserved in export/import + """ + module_store = modulestore('direct') + content_store = contentstore() + + import_from_xml(module_store, 'common/test/data/', ['toy']) + + location = CourseDescriptor.id_to_location('edX/toy/2012_Fall') + + # Export the course + root_dir = path(mkdtemp_clean()) + export_to_xml(module_store, content_store, location, root_dir, 'test_roundtrip') + + # Reimport and get the video back + import_from_xml(module_store, root_dir) + + # get the sample HTML with styling information + html_module = module_store.get_instance( + 'edX/toy/2012_Fall', + Location(['i4x', 'edX', 'toy', 'html', 'with_styling']) + ) + self.assertIn('

', html_module.data) + + # get the sample HTML with just a simple tag information + html_module = module_store.get_instance( + 'edX/toy/2012_Fall', + Location(['i4x', 'edX', 'toy', 'html', 'just_img']) + ) + self.assertIn('', html_module.data) + def test_course_handouts_rewrites(self): module_store = modulestore('direct') diff --git a/cms/djangoapps/contentstore/tests/test_import_nostatic.py b/cms/djangoapps/contentstore/tests/test_import_nostatic.py new file mode 100644 index 0000000000..aad6ffbfe4 --- /dev/null +++ b/cms/djangoapps/contentstore/tests/test_import_nostatic.py @@ -0,0 +1,123 @@ +#pylint: disable=E1101 +''' +Tests for importing with no static +''' + +from django.test.client import Client +from django.test.utils import override_settings +from django.conf import settings +from path import path +import copy + +from django.contrib.auth.models import User + +from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase + +from xmodule.modulestore import Location +from xmodule.modulestore.django import modulestore +from xmodule.contentstore.django import contentstore +from xmodule.modulestore.xml_importer import import_from_xml +from xmodule.contentstore.content import StaticContent + +from xmodule.course_module import CourseDescriptor + +from xmodule.exceptions import NotFoundError +from uuid import uuid4 + + +TEST_DATA_CONTENTSTORE = copy.deepcopy(settings.CONTENTSTORE) +TEST_DATA_CONTENTSTORE['OPTIONS']['db'] = 'test_xcontent_%s' % uuid4().hex + + +@override_settings(CONTENTSTORE=TEST_DATA_CONTENTSTORE) +class ContentStoreImportNoStaticTest(ModuleStoreTestCase): + """ + Tests that rely on the toy and test_import_course courses. + NOTE: refactor using CourseFactory so they do not. + """ + def setUp(self): + + settings.MODULESTORE['default']['OPTIONS']['fs_root'] = path('common/test/data') + settings.MODULESTORE['direct']['OPTIONS']['fs_root'] = path('common/test/data') + uname = 'testuser' + email = 'test+courses@edx.org' + password = 'foo' + + # Create the use so we can log them in. + self.user = User.objects.create_user(uname, email, password) + + # Note that we do not actually need to do anything + # for registration if we directly mark them active. + self.user.is_active = True + # Staff has access to view all courses + self.user.is_staff = True + + # Save the data that we've just changed to the db. + self.user.save() + + self.client = Client() + self.client.login(username=uname, password=password) + + def load_test_import_course(self): + ''' + Load the standard course used to test imports (for do_import_static=False behavior). + ''' + content_store = contentstore() + module_store = modulestore('direct') + import_from_xml(module_store, 'common/test/data/', ['test_import_course'], static_content_store=content_store, do_import_static=False, verbose=True) + course_location = CourseDescriptor.id_to_location('edX/test_import_course/2012_Fall') + course = module_store.get_item(course_location) + self.assertIsNotNone(course) + + return module_store, content_store, course, course_location + + def test_static_import(self): + ''' + Stuff in static_import should always be imported into contentstore + ''' + _, content_store, course, course_location = self.load_test_import_course() + + # make sure we have ONE asset in our contentstore ("should_be_imported.html") + all_assets = content_store.get_all_content_for_course(course_location) + print "len(all_assets)=%d" % len(all_assets) + self.assertEqual(len(all_assets), 1) + + content = None + try: + location = StaticContent.get_location_from_path('/c4x/edX/test_import_course/asset/should_be_imported.html') + content = content_store.find(location) + except NotFoundError: + pass + + self.assertIsNotNone(content) + + # make sure course.lms.static_asset_path is correct + print "static_asset_path = {0}".format(course.lms.static_asset_path) + self.assertEqual(course.lms.static_asset_path, 'test_import_course') + + def test_asset_import_nostatic(self): + ''' + This test validates that an image asset is NOT imported when do_import_static=False + ''' + content_store = contentstore() + + module_store = modulestore('direct') + import_from_xml(module_store, 'common/test/data/', ['toy'], static_content_store=content_store, do_import_static=False, verbose=True) + + course_location = CourseDescriptor.id_to_location('edX/toy/2012_Fall') + module_store.get_item(course_location) + + # make sure we have NO assets in our contentstore + all_assets = content_store.get_all_content_for_course(course_location) + print "len(all_assets)=%d" % len(all_assets) + self.assertEqual(len(all_assets), 0) + + def test_no_static_link_rewrites_on_import(self): + module_store = modulestore('direct') + import_from_xml(module_store, 'common/test/data/', ['toy'], do_import_static=False, verbose=True) + + handouts = module_store.get_item(Location(['i4x', 'edX', 'toy', 'course_info', 'handouts', None])) + self.assertIn('/static/', handouts.data) + + handouts = module_store.get_item(Location(['i4x', 'edX', 'toy', 'html', 'toyhtml', None])) + self.assertIn('/static/', handouts.data) diff --git a/cms/envs/acceptance.py b/cms/envs/acceptance.py index 7debfe18d1..3b89e2e988 100644 --- a/cms/envs/acceptance.py +++ b/cms/envs/acceptance.py @@ -8,6 +8,7 @@ so that we can run the lettuce acceptance tests. # pylint: disable=W0401, W0614 from .test import * +from lms.envs.sauce import * # You need to start the server in debug mode, # otherwise the browser will not render the pages correctly @@ -17,7 +18,7 @@ DEBUG = True import logging logging.disable(logging.ERROR) import os -import random +from random import choice, randint def seed(): @@ -75,7 +76,6 @@ DATABASES = { # Use the auto_auth workflow for creating users and logging them in MITX_FEATURES['AUTOMATIC_AUTH_FOR_TESTING'] = True - # HACK # Setting this flag to false causes imports to not load correctly in the lettuce python files # We do not yet understand why this occurs. Setting this to true is a stopgap measure @@ -84,5 +84,5 @@ USE_I18N = True # Include the lettuce app for acceptance testing, including the 'harvest' django-admin command INSTALLED_APPS += ('lettuce.django',) LETTUCE_APPS = ('contentstore',) -LETTUCE_SERVER_PORT = random.randint(1024, 65535) -LETTUCE_BROWSER = 'chrome' +LETTUCE_SERVER_PORT = choice(PORTS) if SAUCE.get('SAUCE_ENABLED') else randint(1024, 65535) +LETTUCE_BROWSER = os.environ.get('LETTUCE_BROWSER', 'chrome') diff --git a/common/djangoapps/static_replace/__init__.py b/common/djangoapps/static_replace/__init__.py index d7f2df8322..712664bf39 100644 --- a/common/djangoapps/static_replace/__init__.py +++ b/common/djangoapps/static_replace/__init__.py @@ -90,7 +90,7 @@ def replace_course_urls(text, course_id): return re.sub(_url_replace_regex('/course/'), replace_course_url, text) -def replace_static_urls(text, data_directory, course_id=None): +def replace_static_urls(text, data_directory, course_id=None, static_asset_path=''): """ Replace /static/$stuff urls either with their correct url as generated by collectstatic, (/static/$md5_hashed_stuff) or by the course-specific content static url @@ -100,6 +100,7 @@ def replace_static_urls(text, data_directory, course_id=None): text: The source text to do the substitution in data_directory: The directory in which course data is stored course_id: The course identifier used to distinguish static content for this course in studio + static_asset_path: Path for static assets, which overrides data_directory and course_namespace, if nonempty """ def replace_static_url(match): @@ -116,7 +117,7 @@ def replace_static_urls(text, data_directory, course_id=None): if settings.DEBUG and finders.find(rest, True): return original # if we're running with a MongoBacked store course_namespace is not None, then use studio style urls - elif course_id and modulestore().get_modulestore_type(course_id) != XML_MODULESTORE_TYPE: + elif (not static_asset_path) and course_id and modulestore().get_modulestore_type(course_id) != XML_MODULESTORE_TYPE: # first look in the static file pipeline and see if we are trying to reference # a piece of static content which is in the mitx repo (e.g. JS associated with an xmodule) @@ -135,7 +136,7 @@ def replace_static_urls(text, data_directory, course_id=None): url = StaticContent.convert_legacy_static_url_with_course_id(rest, course_id) # Otherwise, look the file up in staticfiles_storage, and append the data directory if needed else: - course_path = "/".join((data_directory, rest)) + course_path = "/".join((static_asset_path or data_directory, rest)) try: if staticfiles_storage.exists(rest): @@ -152,7 +153,7 @@ def replace_static_urls(text, data_directory, course_id=None): return re.sub( - _url_replace_regex('/static/(?!{data_dir})'.format(data_dir=data_directory)), + _url_replace_regex('/static/(?!{data_dir})'.format(data_dir=static_asset_path or data_directory)), replace_static_url, text ) diff --git a/common/djangoapps/terrain/browser.py b/common/djangoapps/terrain/browser.py index c2bf2bbbf3..cf53aa4f69 100644 --- a/common/djangoapps/terrain/browser.py +++ b/common/djangoapps/terrain/browser.py @@ -11,6 +11,10 @@ from logging import getLogger from django.core.management import call_command from django.conf import settings from selenium.common.exceptions import WebDriverException +from selenium.webdriver.common.desired_capabilities import DesiredCapabilities +from requests import put +from base64 import encodestring +from json import dumps # Let the LMS and CMS do their one-time setup # For example, setting up mongo caches @@ -42,43 +46,93 @@ LOGGER.info("Loading the lettuce acceptance testing terrain file...") MAX_VALID_BROWSER_ATTEMPTS = 20 +def get_username_and_key(): + """ + Returns the Sauce Labs username and access ID as set by environment variables + """ + return {"username": settings.SAUCE.get('USERNAME'), "access-key": settings.SAUCE.get('ACCESS_ID')} + + +def set_job_status(jobid, passed=True): + """ + Sets the job status on sauce labs + """ + body_content = dumps({"passed": passed}) + config = get_username_and_key() + base64string = encodestring('{}:{}'.format(config['username'], config['access-key']))[:-1] + result = put('http://saucelabs.com/rest/v1/{}/jobs/{}'.format(config['username'], world.jobid), + data=body_content, + headers={"Authorization": "Basic {}".format(base64string)}) + return result.status_code == 200 + + +def make_desired_capabilities(): + """ + Returns a DesiredCapabilities object corresponding to the environment sauce parameters + """ + desired_capabilities = settings.SAUCE.get('BROWSER', DesiredCapabilities.CHROME) + desired_capabilities['platform'] = settings.SAUCE.get('PLATFORM') + desired_capabilities['version'] = settings.SAUCE.get('VERSION') + desired_capabilities['device-type'] = settings.SAUCE.get('DEVICE') + desired_capabilities['name'] = settings.SAUCE.get('SESSION') + desired_capabilities['build'] = settings.SAUCE.get('BUILD') + desired_capabilities['video-upload-on-pass'] = False + desired_capabilities['sauce-advisor'] = False + desired_capabilities['record-screenshots'] = False + desired_capabilities['selenium-version'] = "2.34.0" + desired_capabilities['max-duration'] = 3600 + desired_capabilities['public'] = 'public restricted' + return desired_capabilities + + @before.harvest def initial_setup(server): """ Launch the browser once before executing the tests. """ - browser_driver = getattr(settings, 'LETTUCE_BROWSER', 'chrome') + world.absorb(settings.SAUCE.get('SAUCE_ENABLED'), 'SAUCE_ENABLED') - # There is an issue with ChromeDriver2 r195627 on Ubuntu - # in which we sometimes get an invalid browser session. - # This is a work-around to ensure that we get a valid session. - success = False - num_attempts = 0 - while (not success) and num_attempts < MAX_VALID_BROWSER_ATTEMPTS: + if not world.SAUCE_ENABLED: + browser_driver = getattr(settings, 'LETTUCE_BROWSER', 'chrome') - # Get a browser session - world.browser = Browser(browser_driver) + # There is an issue with ChromeDriver2 r195627 on Ubuntu + # in which we sometimes get an invalid browser session. + # This is a work-around to ensure that we get a valid session. + success = False + num_attempts = 0 + while (not success) and num_attempts < MAX_VALID_BROWSER_ATTEMPTS: + world.browser = Browser(browser_driver) - # Try to visit the main page - # If the browser session is invalid, this will - # raise a WebDriverException - try: - world.visit('/') + # Try to visit the main page + # If the browser session is invalid, this will + # raise a WebDriverException + try: + world.visit('/') - except WebDriverException: - world.browser.quit() - num_attempts += 1 + except WebDriverException: + world.browser.quit() + num_attempts += 1 - else: - success = True + else: + success = True - # If we were unable to get a valid session within the limit of attempts, - # then we cannot run the tests. - if not success: - raise IOError("Could not acquire valid {driver} browser session.".format(driver=browser_driver)) + # If we were unable to get a valid session within the limit of attempts, + # then we cannot run the tests. + if not success: + raise IOError("Could not acquire valid {driver} browser session.".format(driver=browser_driver)) - # Set the browser size to 1280x1024 - world.browser.driver.set_window_size(1280, 1024) + world.browser.driver.set_window_size(1280, 1024) + + else: + config = get_username_and_key() + world.browser = Browser( + 'remote', + url="http://{}:{}@ondemand.saucelabs.com:80/wd/hub".format(config['username'], config['access-key']), + **make_desired_capabilities() + ) + world.browser.driver.implicitly_wait(30) + + world.absorb(world.browser.driver.session_id, 'jobid') @before.each_scenario @@ -97,7 +151,6 @@ def clear_data(scenario): world.spew('scenario_dict') - @after.each_scenario def reset_databases(scenario): ''' @@ -128,4 +181,6 @@ def teardown_browser(total): """ Quit the browser after executing the tests. """ + if world.SAUCE_ENABLED: + set_job_status(world.jobid, total.scenarios_ran == total.scenarios_passed) world.browser.quit() diff --git a/common/djangoapps/terrain/steps.py b/common/djangoapps/terrain/steps.py index 9cf2aeda49..f13b3ff932 100644 --- a/common/djangoapps/terrain/steps.py +++ b/common/djangoapps/terrain/steps.py @@ -99,7 +99,7 @@ def i_am_logged_in_user(step): @step('I am not logged in$') def i_am_not_logged_in(step): - world.browser.cookies.delete() + world.visit('logout') @step('I am staff for course "([^"]*)"$') @@ -138,10 +138,13 @@ def should_have_link_with_path_and_text(step, path, text): @step(r'should( not)? see "(.*)" (?:somewhere|anywhere) (?:in|on) (?:the|this) page') def should_see_in_the_page(step, doesnt_appear, text): + multiplier = 1 + if world.SAUCE_ENABLED: + multiplier = 2 if doesnt_appear: - assert world.browser.is_text_not_present(text, wait_time=5) + assert world.browser.is_text_not_present(text, wait_time=5*multiplier) else: - assert world.browser.is_text_present(text, wait_time=5) + assert world.browser.is_text_present(text, wait_time=5*multiplier) @step('I am logged in$') @@ -150,7 +153,7 @@ def i_am_logged_in(step): world.log_in(username='robot', password='test') world.browser.visit(django_url('/')) # You should not see the login link - assert_equals(world.browser.find_by_css('a#login'), []) + assert world.is_css_not_present('a#login') @step(u'I am an edX user$') diff --git a/common/djangoapps/xmodule_modifiers.py b/common/djangoapps/xmodule_modifiers.py index 6b73395599..16fe12371b 100644 --- a/common/djangoapps/xmodule_modifiers.py +++ b/common/djangoapps/xmodule_modifiers.py @@ -76,7 +76,7 @@ def replace_course_urls(get_html, course_id): return _get_html -def replace_static_urls(get_html, data_dir, course_id=None): +def replace_static_urls(get_html, data_dir, course_id=None, static_asset_path=''): """ Updates the supplied module with a new get_html function that wraps the old get_html function and substitutes urls of the form /static/... @@ -85,7 +85,7 @@ def replace_static_urls(get_html, data_dir, course_id=None): @wraps(get_html) def _get_html(): - return static_replace.replace_static_urls(get_html(), data_dir, course_id) + return static_replace.replace_static_urls(get_html(), data_dir, course_id, static_asset_path=static_asset_path) return _get_html diff --git a/common/lib/xmodule/xmodule/html_module.py b/common/lib/xmodule/xmodule/html_module.py index 567f5c7eef..7a68c42ac9 100644 --- a/common/lib/xmodule/xmodule/html_module.py +++ b/common/lib/xmodule/xmodule/html_module.py @@ -33,11 +33,13 @@ class HtmlFields(object): class HtmlModule(HtmlFields, XModule): - js = {'coffee': [resource_string(__name__, 'js/src/javascript_loader.coffee'), - resource_string(__name__, 'js/src/collapsible.coffee'), - resource_string(__name__, 'js/src/html/display.coffee') - ] - } + js = { + 'coffee': [ + resource_string(__name__, 'js/src/javascript_loader.coffee'), + resource_string(__name__, 'js/src/collapsible.coffee'), + resource_string(__name__, 'js/src/html/display.coffee') + ] + } js_module_name = "HTMLModule" css = {'scss': [resource_string(__name__, 'css/html/display.scss')]} @@ -118,8 +120,10 @@ class HtmlDescriptor(HtmlFields, XmlDescriptor, EditingDescriptor): # from .html # 'filename' in html pointers is a relative path # (not same as 'html/blah.html' when the pointer is in a directory itself) - pointer_path = "{category}/{url_path}".format(category='html', - url_path=name_to_pathname(location.name)) + pointer_path = "{category}/{url_path}".format( + category='html', + url_path=name_to_pathname(location.name) + ) base = path(pointer_path).dirname() # log.debug("base = {0}, base.dirname={1}, filename={2}".format(base, base.dirname(), filename)) filepath = "{base}/{name}.html".format(base=base, name=filename) @@ -164,19 +168,16 @@ class HtmlDescriptor(HtmlFields, XmlDescriptor, EditingDescriptor): # TODO (vshnayder): make export put things in the right places. def definition_to_xml(self, resource_fs): - '''If the contents are valid xml, write them to filename.xml. Otherwise, - write just to filename.xml, and the html + ''' Write to filename.xml, and the html string to filename.html. ''' - try: - return etree.fromstring(self.data) - except etree.XMLSyntaxError: - pass - # Not proper format. Write html to file, return an empty tag + # Write html to file, return an empty tag pathname = name_to_pathname(self.url_name) - filepath = u'{category}/{pathname}.html'.format(category=self.category, - pathname=pathname) + filepath = u'{category}/{pathname}.html'.format( + category=self.category, + pathname=pathname + ) resource_fs.makedir(os.path.dirname(filepath), recursive=True, allow_recreate=True) with resource_fs.open(filepath, 'w') as filestream: @@ -190,6 +191,7 @@ class HtmlDescriptor(HtmlFields, XmlDescriptor, EditingDescriptor): elt.set("filename", relname) return elt + class AboutFields(object): display_name = String( help="Display name for this module", @@ -202,12 +204,14 @@ class AboutFields(object): scope=Scope.content ) + class AboutModule(AboutFields, HtmlModule): """ Overriding defaults but otherwise treated as HtmlModule. """ pass + class AboutDescriptor(AboutFields, HtmlDescriptor): """ These pieces of course content are treated as HtmlModules but we need to overload where the templates are located @@ -216,6 +220,7 @@ class AboutDescriptor(AboutFields, HtmlDescriptor): template_dir_name = "about" module_class = AboutModule + class StaticTabFields(object): """ The overrides for Static Tabs @@ -241,6 +246,7 @@ class StaticTabModule(StaticTabFields, HtmlModule): """ pass + class StaticTabDescriptor(StaticTabFields, HtmlDescriptor): """ These pieces of course content are treated as HtmlModules but we need to overload where the templates are located diff --git a/common/lib/xmodule/xmodule/js/spec/helper.coffee b/common/lib/xmodule/xmodule/js/spec/helper.coffee index 1dfccdf521..f3cecf71cb 100644 --- a/common/lib/xmodule/xmodule/js/spec/helper.coffee +++ b/common/lib/xmodule/xmodule/js/spec/helper.coffee @@ -90,7 +90,13 @@ jasmine.stubbedHtml5Speeds = ['0.75', '1.0', '1.25', '1.50'] jasmine.stubRequests = -> spyOn($, 'ajax').andCallFake (settings) -> if match = settings.url.match /youtube\.com\/.+\/videos\/(.+)\?v=2&alt=jsonc/ - settings.success data: jasmine.stubbedMetadata[match[1]] + if settings.success + # match[1] - it's video ID + settings.success data: jasmine.stubbedMetadata[match[1]] + else { + always: (callback) -> + callback.call(window, {}, 'success'); + } else if match = settings.url.match /static(\/.*)?\/subs\/(.+)\.srt\.sjson/ settings.success jasmine.stubbedCaption else if settings.url.match /.+\/problem_get$/ diff --git a/common/lib/xmodule/xmodule/js/spec/video/general_spec.js b/common/lib/xmodule/xmodule/js/spec/video/general_spec.js index accfba0dbe..9194106fff 100644 --- a/common/lib/xmodule/xmodule/js/spec/video/general_spec.js +++ b/common/lib/xmodule/xmodule/js/spec/video/general_spec.js @@ -4,8 +4,6 @@ beforeEach(function () { jasmine.stubRequests(); - oldOTBD = window.onTouchBasedDevice; - window.onTouchBasedDevice = jasmine.createSpy('onTouchBasedDevice').andReturn(false); this.videosDefinition = '0.75:7tqY6eQzVhE,1.0:cogebirgzzM'; this['7tqY6eQzVhE'] = '7tqY6eQzVhE'; this['cogebirgzzM'] = 'cogebirgzzM'; @@ -16,7 +14,6 @@ window.onYouTubePlayerAPIReady = undefined; window.onHTML5PlayerAPIReady = undefined; $('source').remove(); - window.onTouchBasedDevice = oldOTBD; }); describe('constructor', function () { @@ -58,6 +55,46 @@ expect(this.state.speed).toEqual('0.75'); }); }); + + describe('Check Youtube link existence', function () { + var statusList = { + error: 'html5', + timeout: 'html5', + abort: 'html5', + parsererror: 'html5', + success: 'youtube', + notmodified: 'youtube' + }; + + function stubDeffered(data, status) { + return { + always: function(callback) { + callback.call(window, data, status); + } + } + } + + function checkPlayer(videoType, data, status) { + this.state = new window.Video('#example'); + spyOn(this.state , 'getVideoMetadata') + .andReturn(stubDeffered(data, status)); + this.state.initialize('#example'); + + expect(this.state.videoType).toEqual(videoType); + } + + it('if video id is incorrect', function () { + checkPlayer('html5', { error: {} }, 'success'); + }); + + $.each(statusList, function(status, mode){ + it('Status:' + status + ', mode:' + mode, function () { + checkPlayer(mode, {}, status); + }); + }); + + }); + }); describe('HTML5', function () { diff --git a/common/lib/xmodule/xmodule/js/spec/video/video_player_spec.js b/common/lib/xmodule/xmodule/js/spec/video/video_player_spec.js index e92f251f70..0873426aa9 100644 --- a/common/lib/xmodule/xmodule/js/spec/video/video_player_spec.js +++ b/common/lib/xmodule/xmodule/js/spec/video/video_player_spec.js @@ -79,6 +79,8 @@ it('create Youtube player', function() { var oldYT = window.YT; + jasmine.stubRequests(); + window.YT = { Player: function () { }, PlayerState: oldYT.PlayerState diff --git a/common/lib/xmodule/xmodule/js/src/video/01_initialize.js b/common/lib/xmodule/xmodule/js/src/video/01_initialize.js index 9a6e20421d..79bc16dbda 100644 --- a/common/lib/xmodule/xmodule/js/src/video/01_initialize.js +++ b/common/lib/xmodule/xmodule/js/src/video/01_initialize.js @@ -30,8 +30,7 @@ function (VideoPlayer) { */ return function (state, element) { _makeFunctionsPublic(state); - _initialize(state, element); - _renderElements(state); + state.initialize(element); }; // *************************************************************** @@ -56,59 +55,12 @@ function (VideoPlayer) { // Old private functions. Now also public so that can be // tested by Jasmine. + state.initialize = _.bind(initialize, state); state.parseSpeed = _.bind(parseSpeed, state); state.fetchMetadata = _.bind(fetchMetadata, state); state.parseYoutubeStreams = _.bind(parseYoutubeStreams, state); state.parseVideoSources = _.bind(parseVideoSources, state); - } - - // function _initialize(element) - // The function set initial configuration and preparation. - - function _initialize(state, element) { - // This is used in places where we instead would have to check if an element has a CSS class 'fullscreen'. - state.isFullScreen = false; - - // The parent element of the video, and the ID. - state.el = $(element).find('.video'); - state.id = state.el.attr('id').replace(/video_/, ''); - - // We store all settings passed to us by the server in one place. These are "read only", so don't - // modify them. All variable content lives in 'state' object. - state.config = { - element: element, - - start: state.el.data('start'), - end: state.el.data('end'), - - caption_data_dir: state.el.data('caption-data-dir'), - caption_asset_path: state.el.data('caption-asset-path'), - show_captions: (state.el.data('show-captions').toString().toLowerCase() === 'true'), - youtubeStreams: state.el.data('streams'), - - sub: state.el.data('sub'), - mp4Source: state.el.data('mp4-source'), - webmSource: state.el.data('webm-source'), - oggSource: state.el.data('ogg-source'), - - fadeOutTimeout: 1400, - - availableQualities: ['hd720', 'hd1080', 'highres'] - }; - - if (!(_parseYouTubeIDs(state))) { - // If we do not have YouTube ID's, try parsing HTML5 video sources. - _prepareHTML5Video(state); - } - - _configureCaptions(state); - _setPlayerMode(state); - - // Possible value are: 'visible', 'hiding', and 'invisible'. - state.controlState = 'visible'; - state.controlHideTimeout = null; - state.captionState = 'visible'; - state.captionHideTimeout = null; + state.getVideoMetadata = _.bind(getVideoMetadata, state); } // function _renderElements(state) @@ -228,12 +180,83 @@ function (VideoPlayer) { state.setSpeed($.cookie('video_speed')); } + function _setConfigurations(state) { + _configureCaptions(state); + _setPlayerMode(state); + + // Possible value are: 'visible', 'hiding', and 'invisible'. + state.controlState = 'visible'; + state.controlHideTimeout = null; + state.captionState = 'visible'; + state.captionHideTimeout = null; + } + // *************************************************************** // Public functions start here. // These are available via the 'state' object. Their context ('this' keyword) is the 'state' object. // The magic private function that makes them available and sets up their context is makeFunctionsPublic(). // *************************************************************** + // function initialize(element) + // The function set initial configuration and preparation. + + function initialize(element) { + var _this = this; + // This is used in places where we instead would have to check if an element has a CSS class 'fullscreen'. + this.isFullScreen = false; + + // The parent element of the video, and the ID. + this.el = $(element).find('.video'); + this.id = this.el.attr('id').replace(/video_/, ''); + + // We store all settings passed to us by the server in one place. These are "read only", so don't + // modify them. All variable content lives in 'state' object. + this.config = { + element: element, + + start: this.el.data('start'), + end: this.el.data('end'), + + caption_data_dir: this.el.data('caption-data-dir'), + caption_asset_path: this.el.data('caption-asset-path'), + show_captions: (this.el.data('show-captions').toString().toLowerCase() === 'true'), + youtubeStreams: this.el.data('streams'), + + sub: this.el.data('sub'), + mp4Source: this.el.data('mp4-source'), + webmSource: this.el.data('webm-source'), + oggSource: this.el.data('ogg-source'), + + fadeOutTimeout: 1400, + + availableQualities: ['hd720', 'hd1080', 'highres'] + }; + + if (!(_parseYouTubeIDs(this))) { + // If we do not have YouTube ID's, try parsing HTML5 video sources. + _prepareHTML5Video(this); + _setConfigurations(this); + _renderElements(this); + } else { + this.getVideoMetadata() + .always(function(json, status) { + var err = $.isPlainObject(json.error) || + (status !== "success" && status !== "notmodified"); + + if (err){ + // When the youtube link doesn't work for any reason + // (for example, the great firewall in china) any + // alternate sources should automatically play. + _prepareHTML5Video(_this); + _this.el.find('a.quality_control').hide(); + } + + _setConfigurations(_this); + _renderElements(_this); + }); + } + } + // function parseYoutubeStreams(state, youtubeStreams) // // Take a string in the form: @@ -297,9 +320,9 @@ function (VideoPlayer) { this.metadata = {}; $.each(this.videos, function (speed, url) { - $.get('https://gdata.youtube.com/feeds/api/videos/' + url + '?v=2&alt=jsonc', (function(data) { + _this.getVideoMetadata(url, function(data) { _this.metadata[data.data.id] = data.data; - }), 'jsonp'); + }); }); } @@ -329,6 +352,24 @@ function (VideoPlayer) { } } + function getVideoMetadata(url, callback) { + var successHandler, xhr; + + if (typeof url !== 'string') { + url = this.videos['1.0'] || ''; + } + + successHandler = ($.isFunction(callback)) ? callback : null; + xhr = $.ajax({ + url: 'https://gdata.youtube.com/feeds/api/videos/' + url + '?v=2&alt=jsonc', + timeout: 500, + dataType: 'jsonp', + success: successHandler + }); + + return xhr; + } + function stopBuffering() { var video; diff --git a/common/lib/xmodule/xmodule/modulestore/inheritance.py b/common/lib/xmodule/xmodule/modulestore/inheritance.py index 1314c72094..aeec53cc29 100644 --- a/common/lib/xmodule/xmodule/modulestore/inheritance.py +++ b/common/lib/xmodule/xmodule/modulestore/inheritance.py @@ -9,7 +9,8 @@ INHERITABLE_METADATA = ( # intended to be set per-course, but can be overridden in for specific # elements. Can be a float. 'days_early_for_beta', - 'giturl' # for git edit link + 'giturl', # for git edit link + 'static_asset_path', # for static assets placed outside xcontent contentstore ) diff --git a/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py b/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py index 17036a16bf..40b3c6fd83 100644 --- a/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py +++ b/common/lib/xmodule/xmodule/modulestore/tests/test_mongo.py @@ -10,7 +10,9 @@ from xblock.runtime import KeyValueStore, InvalidScopeError from xmodule.tests import DATA_DIR from xmodule.modulestore import Location from xmodule.modulestore.mongo import MongoModuleStore, MongoKeyValueStore -from xmodule.modulestore.xml_importer import import_from_xml +from xmodule.modulestore.draft import DraftModuleStore +from xmodule.modulestore.xml_importer import import_from_xml, perform_xlint +from xmodule.contentstore.mongo import MongoContentStore from xmodule.modulestore.tests.test_modulestore import check_path_to_location @@ -35,7 +37,7 @@ class TestMongoModuleStore(object): # is ok only as long as none of the tests modify the db. # If (when!) that changes, need to either reload the db, or load # once and copy over to a tmp db for each test. - cls.store = cls.initdb() + cls.store, cls.content_store, cls.draft_store = cls.initdb() @classmethod def teardownClass(cls): @@ -46,10 +48,28 @@ class TestMongoModuleStore(object): def initdb(): # connect to the db store = MongoModuleStore(HOST, DB, COLLECTION, FS_ROOT, RENDER_TEMPLATE, default_class=DEFAULT_CLASS) + # since MongoModuleStore and MongoContentStore are basically assumed to be together, create this class + # as well + content_store = MongoContentStore(HOST, DB) + # + # Also test draft store imports + # + draft_store = DraftModuleStore(HOST, DB, COLLECTION, FS_ROOT, RENDER_TEMPLATE, default_class=DEFAULT_CLASS) # Explicitly list the courses to load (don't want the big one) - courses = ['toy', 'simple'] - import_from_xml(store, DATA_DIR, courses) - return store + courses = ['toy', 'simple', 'simple_with_draft'] + import_from_xml(store, DATA_DIR, courses, draft_store=draft_store, static_content_store=content_store) + + # also test a course with no importing of static content + import_from_xml( + store, + DATA_DIR, + ['test_import_course'], + static_content_store=content_store, + do_import_static=False, + verbose=True + ) + + return store, content_store, draft_store @staticmethod def destroy_db(connection): @@ -77,10 +97,12 @@ class TestMongoModuleStore(object): def test_get_courses(self): '''Make sure the course objects loaded properly''' courses = self.store.get_courses() - assert_equals(len(courses), 2) + assert_equals(len(courses), 4) courses.sort(key=lambda c: c.id) assert_equals(courses[0].id, 'edX/simple/2012_Fall') - assert_equals(courses[1].id, 'edX/toy/2012_Fall') + assert_equals(courses[1].id, 'edX/simple_with_draft/2012_Fall') + assert_equals(courses[2].id, 'edX/test_import_course/2012_Fall') + assert_equals(courses[3].id, 'edX/toy/2012_Fall') def test_loads(self): assert_not_equals( @@ -112,6 +134,13 @@ class TestMongoModuleStore(object): '''Make sure that path_to_location works''' check_path_to_location(self.store) + def test_xlinter(self): + ''' + Run through the xlinter, we know the 'toy' course has violations, but the + number will continue to grow over time, so just check > 0 + ''' + assert_not_equals(perform_xlint(DATA_DIR, ['toy']), 0) + def test_get_courses_has_no_templates(self): courses = self.store.get_courses() for course in courses: @@ -129,7 +158,7 @@ class TestMongoModuleStore(object): Assumes the information is desired for courses[1] ('toy' course). """ - return courses[1].tabs[index]['name'] + return courses[2].tabs[index]['name'] # There was a bug where model.save was not getting called after the static tab name # was set set for tabs that have a URL slug. 'Syllabus' and 'Resources' fall into that diff --git a/common/lib/xmodule/xmodule/modulestore/xml_importer.py b/common/lib/xmodule/xmodule/modulestore/xml_importer.py index dac0aea273..c793365060 100644 --- a/common/lib/xmodule/xmodule/modulestore/xml_importer.py +++ b/common/lib/xmodule/xmodule/modulestore/xml_importer.py @@ -51,7 +51,10 @@ def import_static_content(modules, course_loc, course_data_path, static_content_ content.thumbnail_location = thumbnail_location #then commit the content - static_content_store.save(content) + try: + static_content_store.save(content) + except Exception as err: + log.exception('Error importing {0}, error={1}'.format(fullname_with_subpath, err)) #store the remapping information which will be needed to subsitute in the module data remap_dict[fullname_with_subpath] = content_loc.name @@ -64,7 +67,8 @@ def import_static_content(modules, course_loc, course_data_path, static_content_ def import_from_xml(store, data_dir, course_dirs=None, default_class='xmodule.raw_module.RawDescriptor', load_error_modules=True, static_content_store=None, target_location_namespace=None, - verbose=False, draft_store=None): + verbose=False, draft_store=None, + do_import_static=True): """ Import the specified xml data_dir into the "store" modulestore, using org and course as the location org and course. @@ -77,6 +81,10 @@ def import_from_xml(store, data_dir, course_dirs=None, expects a 'url_name' as an identifier to where things are on disk e.g. ../policies//policy.json as well as metadata keys in the policy.json. so we need to keep the original url_name during import + do_import_static: if False, then static files are not imported into the static content store. This can be employed for courses which + have substantial unchanging static content, which is to inefficient to import every time the course is loaded. + Static content for some courses may also be served directly by nginx, instead of going through django. + """ xml_module_store = XMLModuleStore( @@ -116,8 +124,17 @@ def import_from_xml(store, data_dir, course_dirs=None, course_data_path = path(data_dir) / module.data_dir course_location = module.location + log.debug('======> IMPORTING course to location {0}'.format(course_location)) + module = remap_namespace(module, target_location_namespace) + if not do_import_static: + module.lms.static_asset_path = module.data_dir # for old-style xblock where this was actually linked to kvs + module._model_data['static_asset_path'] = module.data_dir + log.debug('course static_asset_path={0}'.format(module.lms.static_asset_path)) + + log.debug('course data_dir={0}'.format(module.data_dir)) + # cdodge: more hacks (what else). Seems like we have a problem when importing a course (like 6.002) which # does not have any tabs defined in the policy file. The import goes fine and then displays fine in LMS, # but if someone tries to add a new tab in the CMS, then the LMS barfs because it expects that - @@ -129,18 +146,35 @@ def import_from_xml(store, data_dir, course_dirs=None, {"type": "wiki", "name": "Wiki"}] # note, add 'progress' when we can support it on Edge import_module(module, store, course_data_path, static_content_store, course_location, - target_location_namespace or course_location) + target_location_namespace or course_location, do_import_static=do_import_static) course_items.append(module) # then import all the static content - if static_content_store is not None: + if static_content_store is not None and do_import_static: _namespace_rename = target_location_namespace if target_location_namespace is not None else course_location # first pass to find everything in /static/ import_static_content(xml_module_store.modules[course_id], course_location, course_data_path, static_content_store, _namespace_rename, subpath='static', verbose=verbose) + elif verbose and not do_import_static: + log.debug('Skipping import of static content, since do_import_static={0}'.format(do_import_static)) + + # no matter what do_import_static is, import "static_import" directory + + # This is needed because the "about" pages (eg "overview") are loaded via load_extra_content, and + # do not inherit the lms metadata from the course module, and thus do not get "static_content_store" + # properly defined. Static content referenced in those extra pages thus need to come through the + # c4x:// contentstore, unfortunately. Tell users to copy that content into the "static_import" subdir. + + simport = 'static_import' + if os.path.exists(course_data_path / simport): + _namespace_rename = target_location_namespace if target_location_namespace is not None else course_location + + import_static_content(xml_module_store.modules[course_id], course_location, course_data_path, static_content_store, + _namespace_rename, subpath=simport, verbose=verbose) + # finally loop through all the modules for module in xml_module_store.modules[course_id].itervalues(): if module.category == 'course': @@ -156,7 +190,8 @@ def import_from_xml(store, data_dir, course_dirs=None, log.debug('importing module location {0}'.format(module.location)) import_module(module, store, course_data_path, static_content_store, course_location, - target_location_namespace if target_location_namespace else course_location) + target_location_namespace if target_location_namespace else course_location, + do_import_static=do_import_static) # now import any 'draft' items if draft_store is not None: @@ -176,7 +211,8 @@ def import_from_xml(store, data_dir, course_dirs=None, def import_module(module, store, course_data_path, static_content_store, - source_course_location, dest_course_location, allow_not_found=False): + source_course_location, dest_course_location, allow_not_found=False, + do_import_static=True): logging.debug('processing import of module {0}...'.format(module.location.url())) @@ -196,7 +232,7 @@ def import_module(module, store, course_data_path, static_content_store, else: module_data = content - if isinstance(module_data, basestring): + if isinstance(module_data, basestring) and do_import_static: # we want to convert all 'non-portable' links in the module_data (if it is a string) to # portable strings (e.g. /static/) module_data = rewrite_nonportable_content_links( @@ -212,6 +248,15 @@ def import_module(module, store, course_data_path, static_content_store, # NOTE: It's important to use own_metadata here to avoid writing # inherited metadata everywhere. + + # remove any export/import only xml_attributes which are used to wire together draft imports + if 'parent_sequential_url' in module.xml_attributes: + del module.xml_attributes['parent_sequential_url'] + + if 'index_in_children_list' in module.xml_attributes: + del module.xml_attributes['index_in_children_list'] + module.save() + store.update_metadata(module.location, dict(own_metadata(module))) @@ -281,7 +326,7 @@ def import_course_draft(xml_module_store, store, draft_store, course_data_path, # this is to make sure private only verticals show up in the list of children since # they would have been filtered out from the non-draft store export if module.location.category == 'vertical': - module.location = module.location._replace(revision=None) + non_draft_location = module.location._replace(revision=None) sequential_url = module.xml_attributes['parent_sequential_url'] index = int(module.xml_attributes['index_in_children_list']) @@ -291,15 +336,12 @@ def import_course_draft(xml_module_store, store, draft_store, course_data_path, seq_location = seq_location._replace(org=target_location_namespace.org, course=target_location_namespace.course ) - sequential = store.get_item(seq_location) + sequential = store.get_item(seq_location, depth=0) - if module.location.url() not in sequential.children: - sequential.children.insert(index, module.location.url()) + if non_draft_location.url() not in sequential.children: + sequential.children.insert(index, non_draft_location.url()) store.update_children(sequential.location, sequential.children) - del module.xml_attributes['parent_sequential_url'] - del module.xml_attributes['index_in_children_list'] - import_module(module, draft_store, course_data_path, static_content_store, source_location_namespace, target_location_namespace, allow_not_found=True) for child in module.get_children(): diff --git a/common/lib/xmodule/xmodule/tests/test_export.py b/common/lib/xmodule/xmodule/tests/test_export.py index d9b80422e9..5c5d8307af 100644 --- a/common/lib/xmodule/xmodule/tests/test_export.py +++ b/common/lib/xmodule/xmodule/tests/test_export.py @@ -89,12 +89,6 @@ class RoundTripTestCase(unittest.TestCase): print("Checking module equality") for location in initial_import.modules[course_id].keys(): print("Checking", location) - if location.category == 'html': - print( - "Skipping html modules--they can't import in" - " final form without writing files..." - ) - continue self.assertEquals(initial_import.modules[course_id][location], second_import.modules[course_id][location]) diff --git a/common/test/data/simple_with_draft/README.md b/common/test/data/simple_with_draft/README.md new file mode 100644 index 0000000000..69ff6b4ed0 --- /dev/null +++ b/common/test/data/simple_with_draft/README.md @@ -0,0 +1,2 @@ +This is a simple, but non-trivial, course using multiple module types and some nested structure. + diff --git a/common/test/data/simple_with_draft/course.xml b/common/test/data/simple_with_draft/course.xml new file mode 100644 index 0000000000..c130686012 --- /dev/null +++ b/common/test/data/simple_with_draft/course.xml @@ -0,0 +1,31 @@ + + +

+ + +
+ + +
+ + + +
+ + + diff --git a/common/test/data/simple_with_draft/drafts/vertical/test_vertical.xml b/common/test/data/simple_with_draft/drafts/vertical/test_vertical.xml new file mode 100644 index 0000000000..4433d282a4 --- /dev/null +++ b/common/test/data/simple_with_draft/drafts/vertical/test_vertical.xml @@ -0,0 +1,5 @@ + + + Foobar - edit in draft + + \ No newline at end of file diff --git a/common/test/data/simple_with_draft/html/toylab.html b/common/test/data/simple_with_draft/html/toylab.html new file mode 100644 index 0000000000..81df84bd63 --- /dev/null +++ b/common/test/data/simple_with_draft/html/toylab.html @@ -0,0 +1,3 @@ +Lab 2A: Superposition Experiment + +

Isn't the toy course great?

diff --git a/common/test/data/simple_with_draft/problem/L1_Problem_1.xml b/common/test/data/simple_with_draft/problem/L1_Problem_1.xml new file mode 100644 index 0000000000..2ba0617904 --- /dev/null +++ b/common/test/data/simple_with_draft/problem/L1_Problem_1.xml @@ -0,0 +1,43 @@ + + +

+

Finger Exercise 1

+

+

+Here are two definitions:

+
    +
  1. +

    +Declarative knowledge refers to statements of fact.

    +
    2. +
  2. +

    +Imperative knowledge refers to 'how to' methods.

    +
    3. +
+

+Which of the following choices is correct?

+
    +
  1. +

    +Statement 1 is true, Statement 2 is false

    +
    2. +
  2. +

    +Statement 1 is false, Statement 2 is true

    +
    3. +
  3. +

    +Statement 1 and Statement 2 are both false

    +
    4. +
  4. +

    +Statement 1 and Statement 2 are both true

    +
    5. +
+

+ + + +

+ diff --git a/common/test/data/simple_with_draft/problem/ps01-simple.xml b/common/test/data/simple_with_draft/problem/ps01-simple.xml new file mode 100644 index 0000000000..e70d8f2c8d --- /dev/null +++ b/common/test/data/simple_with_draft/problem/ps01-simple.xml @@ -0,0 +1,62 @@ +