Andal.LND/edx-platform
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Bok-Choy Video Transcript Tests

Browse Source
This commit is contained in:
Muhammad Ammar
2014-08-20 09:54:06 +00:00
parent ca3bd2f83f
commit cfeb7102a6
3 changed files with 344 additions and 139 deletions
Download Patch File Download Diff File Expand all files Collapse all files

134
cms/djangoapps/contentstore/features/transcripts.feature
View File

@@ -17,60 +17,6 @@ Feature: CMS Transcripts
# one stored on YouTube
# t_not_exist - this file does not exist on YouTube; it exists locally
#1
Scenario: Check input error messages
Given I have created a Video component
And I edit the component
#User inputs html5 links with equal extension
And I enter a "123.webm" source to field number 1
And I enter a "456.webm" source to field number 2
Then I see error message "file_type"
# Currently we are working with 2nd field. It means, that if 2nd field
# contain incorrect value, 1st and 3rd fields should be disabled until
# 2nd field will be filled by correct correct value
And I expect 1, 3 inputs are disabled
When I clear fields
And I expect inputs are enabled
#User input URL with incorrect format
And I enter a "http://link.c" source to field number 1
Then I see error message "url_format"
# Currently we are working with 1st field. It means, that if 1st field
# contain incorrect value, 2nd and 3rd fields should be disabled until
# 1st field will be filled by correct correct value
And I expect 2, 3 inputs are disabled
#User input URL with incorrect format
And I enter a "http://goo.gl/pxxZrg" source to field number 1
And I enter a "http://goo.gl/pxxZrg" source to field number 2
Then I see error message "links_duplication"
And I expect 1, 3 inputs are disabled
And I clear fields
And I expect inputs are enabled
And I enter a "http://youtu.be/t_not_exist" source to field number 1
Then I do not see error message
And I expect inputs are enabled
#2
Scenario: Testing interaction with test youtube server
Given I have created a Video component with subtitles
And I edit the component
# first part of url will be substituted by mock_youtube_server address
# for t__eq_exist id server will respond with transcripts
And I enter a "http://youtu.be/t__eq_exist" source to field number 1
Then I see status message "not found on edx"
# t__eq_exist subs locally not presented at this moment
And I see button "import"
# for t_not_exist id server will respond with 404
And I enter a "http://youtu.be/t_not_exist" source to field number 1
Then I see status message "not found"
And I do not see button "import"
And I see button "disabled_download_to_edit"
#3
Scenario: Youtube id only: check "not found" and "import" states
Given I have created a Video component with subtitles
@@ -93,66 +39,6 @@ Feature: CMS Transcripts
And I see button "download_to_edit"
And I see value "t__eq_exist" in the field "Default Timed Transcript"
#4
Scenario: Youtube id only: check "Found" state
Given I have created a Video component with subtitles "t_not_exist"
And I edit the component
And I enter a "http://youtu.be/t_not_exist" source to field number 1
Then I see status message "found"
And I see value "t_not_exist" in the field "Default Timed Transcript"
#5
Scenario: Youtube id only: check "Found" state when user sets youtube_id with local and server subs and they are equal
Given I have created a Video component with subtitles "t__eq_exist"
And I edit the component
And I enter a "http://youtu.be/t__eq_exist" source to field number 1
And I see status message "found"
And I see value "t__eq_exist" in the field "Default Timed Transcript"
#6
Scenario: Youtube id only: check "Found" state when user sets youtube_id with local and server subs and they are not equal
Given I have created a Video component with subtitles "t_neq_exist"
And I edit the component
And I enter a "http://youtu.be/t_neq_exist" source to field number 1
And I see status message "replace"
And I see button "replace"
And I click transcript button "replace"
And I see status message "found"
And I see value "t_neq_exist" in the field "Default Timed Transcript"
#7
Scenario: html5 source only: check "Not Found" state
Given I have created a Video component
And I edit the component
And I enter a "t_not_exist.mp4" source to field number 1
Then I see status message "not found"
And I see value "" in the field "Default Timed Transcript"
#8
Scenario: html5 source only: check "Found" state
Given I have created a Video component with subtitles "t_not_exist"
And I edit the component
And I enter a "t_not_exist.mp4" source to field number 1
Then I see status message "found"
And I see value "t_not_exist" in the field "Default Timed Transcript"
#9
Scenario: User sets youtube_id w/o server but with local subs and one html5 link w/o subs
Given I have created a Video component with subtitles "t_not_exist"
And I edit the component
And I enter a "http://youtu.be/t_not_exist" source to field number 1
Then I see status message "found"
And I enter a "test_video_name.mp4" source to field number 2
Then I see status message "found"
And I see value "t_not_exist" in the field "Default Timed Transcript"
# Disabled 1/29/14 due to flakiness observed in master
#10
@@ -170,26 +56,6 @@ Feature: CMS Transcripts
# Then I see status message "found"
# And I see value "t__eq_exist" in the field "Default Timed Transcript"
#11
Scenario: User sets youtube_id w/o local but with server subs and one html5 link w/o transcripts w/o import action, then another one html5 link w/o transcripts
Given I have created a Video component
And I edit the component
And I enter a "http://youtu.be/t__eq_exist" source to field number 1
Then I see status message "not found on edx"
And I see button "import"
And I see button "upload_new_timed_transcripts"
And I enter a "t_not_exist.mp4" source to field number 2
Then I see status message "not found on edx"
And I see button "import"
And I see button "upload_new_timed_transcripts"
And I enter a "t_not_exist.webm" source to field number 3
Then I see status message "not found on edx"
And I see button "import"
And I see button "upload_new_timed_transcripts"
#12
Scenario: Entering youtube (no importing), and 2 html5 sources without transcripts - "Not Found"
Given I have created a Video component

116
common/test/acceptance/pages/studio/video/video.py
View File

@@ -1,7 +1,7 @@
"""
CMS Video
"""
import time
import os
import requests
from bok_choy.promise import EmptyPromise, Promise
@@ -21,6 +21,11 @@ CLASS_SELECTORS = {
'upload_dialog': '.wrapper-modal-window-assetupload',
'xblock': '.add-xblock-component',
'slider_range': '.slider-range',
'error': '.transcripts-error-message',
'url_inputs': '.videolist-settings-item input.input',
'collapse_bar': '.videolist-extra-videos',
'status': '.transcripts-message-status',
'attach_transcript': '.file-chooser > input[type="file"]',
}
BUTTON_SELECTORS = {
@@ -32,6 +37,14 @@ BUTTON_SELECTORS = {
'handout_clear': '.wrapper-comp-setting.file-uploader .setting-clear',
'translations_clear': '.metadata-video-translations .setting-clear',
'translation_add': '.wrapper-translations-settings > a',
'import': '.setting-import',
'download_to_edit': '.setting-download',
'disabled_download_to_edit': '.setting-download.is-disabled',
'upload_new_timed_transcripts': '.setting-upload',
'replace': '.setting-replace',
'choose': '.setting-choose',
'use_existing': '.setting-use-existing',
'collapse_link': '.collapse-action.collapse-setting',
}
DISPLAY_NAME = "Component Display Name"
@@ -60,6 +73,10 @@ DEFAULT_SETTINGS = [
]
# We should wait 300 ms for event handler invocation + 200ms for safety.
DELAY = 0.5
@js_defined('window.Video', 'window.RequireJS.require', 'window.jQuery', 'window.XModule', 'window.XBlock',
'window.MathJax.isReady')
class VideoComponentPage(VideoPage):
@@ -267,7 +284,7 @@ class VideoComponentPage(VideoPage):
is_verified = self._verify_setting_entry(setting,
DEFAULT_SETTINGS[counter][0],
DEFAULT_SETTINGS[counter][1])
if is_verified is False:
if not is_verified:
return is_verified
return True
@@ -350,7 +367,7 @@ class VideoComponentPage(VideoPage):
bool: If `field_name` has `field_value`
"""
setting = self._get_setting_entry(field_name)
_, setting = self._get_setting_entry(field_name)
return self._verify_setting_entry(setting, field_name, field_value)
def _get_setting_entry(self, field_name):
@@ -364,9 +381,9 @@ class VideoComponentPage(VideoPage):
setting (WebElement): Selenium WebElement
"""
for setting in self.q(css='.wrapper-comp-setting').results:
for index, setting in enumerate(self.q(css='.wrapper-comp-setting').results):
if setting.find_element_by_class_name('setting-label').get_attribute('innerHTML') == field_name:
return setting
return index, setting
def translations_count(self):
"""
@@ -472,3 +489,92 @@ class VideoComponentPage(VideoPage):
self.wait_for_captions()
selector = '.subtitles > li:nth-child({})'
return ' '.join([self.q(css=selector.format(i)).text[0] for i in range(1, 6)])
def set_url_field(self, url, field_number):
"""
Set video url field in basic settings tab.
Arguments:
url (str): video url
field_number (int): video url field number
"""
if self.q(css=CLASS_SELECTORS['collapse_bar']).visible is False:
self.click_button('collapse_link')
self.q(css=CLASS_SELECTORS['url_inputs']).nth(field_number - 1).fill(url)
time.sleep(DELAY)
self.wait_for_ajax()
def message(self, message_type):
"""
Get video url field status/error message.
Arguments:
message_type(str): type(status, error) of message
Returns:
str: status/error message
"""
if self.q(css=CLASS_SELECTORS[message_type]).visible:
return self.q(css=CLASS_SELECTORS[message_type]).text[0]
else:
return ''
def url_field_status(self, *field_numbers):
"""
Get video url field status(enable/disable).
Arguments:
url (str): video url
field_numbers (tuple or None): field numbers to check status for, None means get status for all.
tuple items will be integers and must start from 1
Returns:
dict: field numbers as keys and field status(bool) as values, False means a field is disabled
"""
if field_numbers:
index_list = [number - 1 for number in field_numbers]
else:
index_list = range(3) # maximum three fields
statuses = {}
for index in index_list:
status = 'is-disabled' not in self.q(css=CLASS_SELECTORS['url_inputs']).nth(index).attrs('class')[0]
statuses[index + 1] = status
return statuses
def clear_fields(self):
"""
Clear video url fields.
"""
script = """
$('{selector}')
.prop('disabled', false)
.removeClass('is-disabled')
.val('')
.trigger('input');
""".format(selector=CLASS_SELECTORS['url_inputs'])
self.browser.execute_script(script)
time.sleep(DELAY)
self.wait_for_ajax()
def is_transcript_button_visible(self, button_name):
"""
Check if a transcript related button is visible.
Arguments:
button_name (str): name of button
field_numbers (tuple or None): field numbers to check status for, None means get status for all.
tuple items will be integers and must start from 1
Returns:
bool: is button visible
"""
return self.q(css=BUTTON_SELECTORS[button_name]).visible

233
common/test/acceptance/tests/video/test_studio_video_transcript.py Normal file
View File

@@ -0,0 +1,233 @@
# -*- coding: utf-8 -*-
"""
Acceptance tests for CMS Video Transcripts.
For transcripts acceptance tests there are 3 available caption
files. They can be used to test various transcripts features. Two of
them can be imported from YouTube.
The length of each file name is 11 characters. This is because the
YouTube's ID length is 11 characters. If file name is not of length 11,
front-end validation will not pass.
t__eq_exist - this file exists on YouTube, and can be imported
via the transcripts menu; after import, this file will
be equal to the one stored locally
t_neq_exist - same as above, except local file will differ from the
one stored on YouTube
t_not_exist - this file does not exist on YouTube; it exists locally
"""
from .test_studio_video_module import CMSVideoBaseTest
class VideoTranscriptTest(CMSVideoBaseTest):
"""
CMS Video Transcript Test Class
"""
def setUp(self):
super(VideoTranscriptTest, self).setUp()
def _create_video_component(self, subtitles=False, subtitle_id='OEoXaMPEzfM'):
"""
Create a video component and navigate to unit page
Arguments:
subtitles (bool): Upload subtitles or not
subtitle_id (str): subtitle file id
"""
if subtitles:
self.assets.append('subs_{}.srt.sjson'.format(subtitle_id))
self.navigate_to_course_unit()
def test_input_validation(self):
"""
Scenario: Check input error messages
Given I have created a Video component
Entering "123.webm" and "456.webm" source to field number 1 and 2 respectively should disable field 1 and 3
Then I see error message "Link types should be unique."
When I clear fields, input fields should be enabled
And I enter a "http://link.c" source to field number 1 should disable fields 2 and 3
Then I see error message "Incorrect url format."
Entering "http://goo.gl/pxxZrg" source to both field number 1 and 2 should disable fields 1 and 3
Then I see error message "Links should be unique."
When I clear fields, input fields should be enabled
And I enter a "http://youtu.be/t_not_exist" source to field number 1
Then I do not see error message
And I expect inputs are enabled
"""
self._create_video_component()
self.edit_component()
#User inputs html5 links with equal extension
self.video.set_url_field('123.webm', 1)
self.video.set_url_field('456.webm', 2)
self.assertEqual(self.video.message('error'), 'Link types should be unique.')
# Currently we are working with 2nd field. It means, that if 2nd field
# contain incorrect value, 1st and 3rd fields should be disabled until
# 2nd field will be filled by correct correct value
self.assertEqual(self.video.url_field_status(1, 3).values(), [False, False])
self.video.clear_fields()
self.assertEqual(self.video.url_field_status().values(), [True, True, True])
#User input URL with incorrect format
self.video.set_url_field('http://link.c', 1)
self.assertEqual(self.video.message('error'), 'Incorrect url format.')
self.assertEqual(self.video.url_field_status(2, 3).values(), [False, False])
#User input URL with incorrect format
self.video.set_url_field('http://goo.gl/pxxZrg', 1)
self.video.set_url_field('http://goo.gl/pxxZrg', 2)
self.assertEqual(self.video.message('error'), 'Links should be unique.')
self.assertEqual(self.video.url_field_status(1, 3).values(), [False, False])
self.video.clear_fields()
self.assertEqual(self.video.url_field_status().values(), [True, True, True])
self.video.set_url_field('http://youtu.be/t_not_exist', 1)
self.assertEqual(self.video.message('error'), '')
self.assertEqual(self.video.url_field_status().values(), [True, True, True])
def test_youtube_server_interaction(self):
"""
Scenario: Testing interaction with test youtube server
Given I have created a Video component with subtitles
And I enter a "http://youtu.be/t__eq_exist" source to field number 1
Then I see status message "No EdX Timed Transcript"
And I see button "import"
And I enter a "http://youtu.be/t_not_exist" source to field number 1
Then I see status message "No Timed Transcript"
And I do not see button "import"
And I see button "disabled_download_to_edit"
"""
self._create_video_component(subtitles=True)
self.edit_component()
# first part of url will be substituted by mock_youtube_server address
# for t__eq_exist id server will respond with transcripts
self.video.set_url_field('http://youtu.be/t__eq_exist', 1)
self.assertEqual(self.video.message('status'), 'No EdX Timed Transcript')
self.assertTrue(self.video.is_transcript_button_visible('import'))
self.video.set_url_field('http://youtu.be/t_not_exist', 1)
self.assertEqual(self.video.message('status'), 'No Timed Transcript')
self.assertFalse(self.video.is_transcript_button_visible('import'))
self.assertTrue(self.video.is_transcript_button_visible('disabled_download_to_edit'))
def test_youtube_id_w_found_state(self):
"""
Scenario: Youtube id only: check "Found" state
Given I have created a Video component with subtitles "t_not_exist"
And I enter a "http://youtu.be/t_not_exist" source to field number 1
Then I see status message "Timed Transcript Found"
And I see value "t_not_exist" in the field "Default Timed Transcript"
"""
self._create_video_component(subtitles=True, subtitle_id='t_not_exist')
self.edit_component()
self.video.set_url_field('http://youtu.be/t_not_exist', 1)
self.assertEqual(self.video.message('status'), 'Timed Transcript Found')
self.open_advanced_tab()
self.assertTrue(self.video.verify_field_value('Default Timed Transcript', 't_not_exist'))
def test_youtube_id_w_same_local_server_subs(self):
"""
Scenario: Youtube id only: check "Found" state when user sets youtube_id with same local and server subs
Given I have created a Video component with subtitles "t__eq_exist"
And I enter a "http://youtu.be/t__eq_exist" source to field number 1
And I see status message "Timed Transcript Found"
And I see value "t__eq_exist" in the field "Default Timed Transcript"
"""
self._create_video_component(subtitles=True, subtitle_id='t__eq_exist')
self.edit_component()
self.video.set_url_field('http://youtu.be/t__eq_exist', 1)
self.assertEqual(self.video.message('status'), 'Timed Transcript Found')
self.open_advanced_tab()
self.assertTrue(self.video.verify_field_value('Default Timed Transcript', 't__eq_exist'))
def test_youtube_id_w_different_local_server_sub(self):
"""
Scenario: Youtube id only: check "Found" state when user sets youtube_id with different local and server subs
Given I have created a Video component with subtitles "t_neq_exist"
And I enter a "http://youtu.be/t_neq_exist" source to field number 1
And I see status message "Timed Transcript Conflict"
And I see button "replace"
And I click transcript button "replace"
And I see status message "Timed Transcript Found"
And I see value "t_neq_exist" in the field "Default Timed Transcript"
"""
self._create_video_component(subtitles=True, subtitle_id='t_neq_exist')
self.edit_component()
self.video.set_url_field('http://youtu.be/t_neq_exist', 1)
self.assertEqual(self.video.message('status'), 'Timed Transcript Conflict')
self.assertTrue(self.video.is_transcript_button_visible('replace'))
self.video.click_button('replace')
self.assertEqual(self.video.message('status'), 'Timed Transcript Found')
self.open_advanced_tab()
self.assertTrue(self.video.verify_field_value('Default Timed Transcript', 't_neq_exist'))
def test_html5_source_w_not_found_state(self):
"""
Scenario: html5 source only: check "Not Found" state
Given I have created a Video component
And I enter a "t_not_exist.mp4" source to field number 1
Then I see status message "No Timed Transcript"
And I see value "" in the field "Default Timed Transcript"
"""
self._create_video_component()
self.edit_component()
self.video.set_url_field('t_not_exist.mp4', 1)
self.assertEqual(self.video.message('status'), 'No Timed Transcript')
self.open_advanced_tab()
self.assertTrue(self.video.verify_field_value('Default Timed Transcript', ''))
def test_html5_source_w_found_state(self):
"""
Scenario: html5 source only: check "Found" state
Given I have created a Video component with subtitles "t_not_exist"
And I enter a "t_not_exist.mp4" source to field number 1
Then I see status message "Timed Transcript Found"
And I see value "t_not_exist" in the field "Default Timed Transcript"
"""
self._create_video_component(subtitles=True, subtitle_id='t_not_exist')
self.edit_component()
self.video.set_url_field('t_not_exist.mp4', 1)
self.assertEqual(self.video.message('status'), 'Timed Transcript Found')
self.open_advanced_tab()
self.assertTrue(self.video.verify_field_value('Default Timed Transcript', 't_not_exist'))
def test_set_youtube_id_wo_server(self):
"""
Scenario: User sets youtube_id w/o server but with local subs and one html5 link w/o subs
Given I have created a Video component with subtitles "t_not_exist"
urls = ['http://youtu.be/t_not_exist', 'test_video_name.mp4']
for each url in urls do the following
Enter `url` to field number n
Status message "Timed Transcript Found" is shown
And I see value "t_not_exist" in the field "Default Timed Transcript"
"""
self._create_video_component(subtitles=True, subtitle_id='t_not_exist')
self.edit_component()
urls = ['http://youtu.be/t_not_exist', 'test_video_name.mp4']
for index, url in enumerate(urls, 1):
self.video.set_url_field(url, index)
self.assertEqual(self.video.message('status'), 'Timed Transcript Found')
self.open_advanced_tab()
self.assertTrue(self.video.verify_field_value('Default Timed Transcript', 't_not_exist'))
def test_set_youtube_id_wo_local(self):
"""
        Scenario: User sets youtube_id w/o local but with server subs and one html5 link w/o
                  transcripts w/o import action, then another one html5 link w/o transcripts
        Given I have created a Video component
        urls = ['http://youtu.be/t__eq_exist', 't_not_exist.mp4', 't_not_exist.webm']
        for each url in urls do the following
            Enter `url` to field number `n`
            Status message `No EdX Timed Transcript` is shown
            `import` and `upload_new_timed_transcripts` are shown
        """
self._create_video_component()
self.edit_component()
urls = ['http://youtu.be/t__eq_exist', 't_not_exist.mp4', 't_not_exist.webm']
for index, url in enumerate(urls, 1):
self.video.set_url_field(url, index)
self.assertEqual(self.video.message('status'), 'No EdX Timed Transcript')
self.assertTrue(self.video.is_transcript_button_visible('import'))
self.assertTrue(self.video.is_transcript_button_visible('upload_new_timed_transcripts'))