diff --git a/docs/course_authors/source/ora_students.rst b/docs/course_authors/source/ora_students.rst index fad8138966..09b12132f9 100644 --- a/docs/course_authors/source/ora_students.rst +++ b/docs/course_authors/source/ora_students.rst @@ -1,32 +1,28 @@ .. _ORA for Students: Open Response Assessments for Students -====================================== +====================================== .. _ORA Introduction: Introduction to Open Response Assessments ----------------------------------------- -.. list-table:: - :widths: 80 - :header-rows: 1 +.. warning:: - * - To instructors: - - * - Modify this section according to your course. For example, you - can delete sentences such as "For more information, see :ref:`ORA Peer Assessment`" - and "For more information, see :ref:`ORA AI Assessment`" if your ORA problem doesn't - contain peer assessments or AI assessments and you want to delete these sections from - this document. - - -Open response assessments allow you to submit a short written answer, -an essay, or a file such as an image or computer code file. + Modify this section according to your course. For example, you + can delete sentences such as "For more information, see :ref:`ORA Peer Assessment`" + and "For more information, see :ref:`ORA AI Assessment`" if your ORA problem doesn't + contain peer assessments or AI assessments and you want to delete these sections from + this document. + + +Open response assessments allow you to submit a short written answer, +an essay, or a file such as an image or computer code file. When you come to an open response assessment problem, you see the name of the problem, the assessment types, the text of the question, the field where you'll -enter your response, and the **Save** and **Submit** buttons. +enter your response, and the **Save** and **Submit** buttons. .. image:: /Images/ExampleORA.gif @@ -36,17 +32,17 @@ that you'll click to upload your file. .. image:: /Images/ExampleORA_File.gif The *assessment types* can include *self assessment*, *peer assessment*, and *artificial intelligence (AI) assessment*. The -assessment types run in the order in which they appear in the problem. +assessment types run in the order in which they appear in the problem. - In a self assessment, you assess your response according a rubric that the instructor has created. For more information, see :ref:`ORA Self Assessment`. - + - In a peer assessment, you grade responses that your peers have submitted while several of your peers - grade your response. For more information, see + grade your response. For more information, see :ref:`ORA Peer Assessment`. - -- In an AI assessment, a computer algorithm grades your response. For more information, + +- In an AI assessment, a computer algorithm grades your response. For more information, see :ref:`ORA AI Assessment`. An open response assessment problem doesn't have to use all assessment types. For example, one problem @@ -55,8 +51,8 @@ and peer assessment, and another problem may use only peer assessment. You'll answer open response assessment problems in much the same way that you answer other problems. For more information about how to submit responses, see :ref:`ORA Submit a Response`. - -When you submit a response to an open response assessment, the next step + +When you submit a response to an open response assessment, the next step depends on the type of assessment that the problem uses. For more information, see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, and :ref:`ORA AI Assessment`. @@ -66,12 +62,12 @@ see :ref:`ORA Access Scores`. If you want to experiment with open response assessments, you can try out the open assessment problems in the `EdX Demo `_ -course. To get started, go -to the `Self-Assessed Essay `_ +course. To get started, go +to the `Self-Assessed Essay `_ unit, and then enter a response in the **Response** field under the -question. You can enter your own response, or you can use one of the sample +question. You can enter your own response, or you can use one of the sample responses in the `Sample Answers `_ -unit. +unit. .. _ORA Submit a Response: @@ -83,19 +79,19 @@ or uploading a file. #. Enter the response that you want to submit. - - If you're submitting a written response, type your response in the + - If you're submitting a written response, type your response in the **Response** field. - - - If you're uploading a file, click **Choose File** under the **Response** - field. In the dialog box that opens, select the file that you want to upload, + + - If you're uploading a file, click **Choose File** under the **Response** + field. In the dialog box that opens, select the file that you want to upload, and then click **Open**. #. Click **Submit**, and then click **OK** in the dialog box to continue. - .. note:: If you want to save your response and work on it again later, click **Save**. - An "Answer saved, but not yet submitted" message appears directly under the **Save** and + .. note:: If you want to save your response and work on it again later, click **Save**. + An "Answer saved, but not yet submitted" message appears directly under the **Save** and **Submit** buttons. - + After you submit your response, the assessment types start running in the order in which they appear in the problem. For more information, see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Assessment`. @@ -105,15 +101,11 @@ see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Asse Self Assessment --------------- -.. list-table:: - :widths: 80 - :header-rows: 1 +.. warning:: + + You can delete this section if your ORA problem doesn't use self assessments. + - * - To instructors: - - * - You can delete this section if your ORA problem doesn't use self assessments. - - In a self assessment, the rubric for the problem appears below your response immediately after you submit the response. You then assess your response based on the rubric. @@ -122,7 +114,7 @@ Perform a Self Assessment #. Submit a response to a self-assessed ORA problem. -#. When the rubric appears, compare your response with the rubric, and select the +#. When the rubric appears, compare your response with the rubric, and select the option that you think is appropriate for each category. .. image:: /Images/Rubric1.gif @@ -133,25 +125,21 @@ Perform a Self Assessment yourself. .. _ORA Peer Assessment: - + Peer Assessment --------------- -.. list-table:: - :widths: 80 - :header-rows: 1 +.. warning:: + + You can delete this section if your ORA problem doesn't use peer assessments. + - * - To instructors: - - * - You can delete this section if your ORA problem doesn't use peer assessments. - - In a peer assessment, several students in the course grade your response while you grade -other students' responses. You have to grade a number of your peers' responses before -you receive your score. (After you grade the minimum number of responses required to +other students' responses. You have to grade a number of your peers' responses before +you receive your score. (After you grade the minimum number of responses required to receive your score, you can grade as many additional responses as you want.) -After you submit your response for grading, the following +After you submit your response for grading, the following message appears under your response. **Your response has been submitted. Please check back later for your grade.** @@ -185,53 +173,49 @@ below. Step 1: Access responses from other students ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. list-table:: - :widths: 80 - :header-rows: 1 +.. warning:: - * - To instructors: - - * - Modify the content in this section according to - your course. For example, if your students can only grade by using the **Open - Ended Console** page, change the introductory sentence below, and delete the - second and third bullets. + Modify the content in this section according to + your course. For example, if your students can only grade by using the **Open + Ended Console** page, change the introductory sentence below, and delete the + second and third bullets. **Note** *You can only grade a response if you've submitted a response to the question, an instructor has already graded at least 20 responses, and -there are more essays from other students left to grade. If you haven't submitted +there are more essays from other students left to grade. If you haven't submitted a response or no responses are available for grading, you see a yellow message in the interface.* .. image:: /Images/PAStudent_NoSubmissions.gif - + There are several ways to access other students' responses, depending on -the way that the course is set up. +the way that the course is set up. - Through the **Open Ended Console** page. This option is always - available for every course. To access the **Open Ended Console** page, + available for every course. To access the **Open Ended Console** page, click the **Open Ended Panel** tab at the top of any page in the course. When you see the list of problems that have responses available to grade, - click the name of the problem that you want to open it. + click the name of the problem that you want to open it. .. image:: /Images/PGI_FromOEC_2Problems.gif - Through the courseware, in a specific unit. This option is only available if the instructor has included a peer grading interface for the problem in the body of - the course. To access responses in the courseware, go to the unit that contains - the open response assessment problem. Scroll down past the response that you + the course. To access responses in the courseware, go to the unit that contains + the open response assessment problem. Scroll down past the response that you submitted until you see the peer grading interface that appears below the problem. - + .. image:: /Images/PGI_InUnitComposite.gif - + - Through the courseware, in a separate section. This option may not be available - for your course. If it is, you'll see the section for peer grading in the - course accordion on the left side of your screen. For example, MIT's 6.00x: - Introduction to Computer Science and Programming course has a separate section - that holds all the course peer grading interfaces. To access peer grading for + for your course. If it is, you'll see the section for peer grading in the + course accordion on the left side of your screen. For example, MIT's 6.00x: + Introduction to Computer Science and Programming course has a separate section + that holds all the course peer grading interfaces. To access peer grading for a problem, you click the problem name. - + .. image:: /Images/PGI_Multiple-600x.gif @@ -242,9 +226,9 @@ Step 2: Learn to grade Before you grade your peers' responses, you must learn to grade the same way that an instructor would. In this process, called -*calibration*, you'll grade several responses that an instructor has already -graded. If your grading is similar to the instructor's, you can begin grading -other students' responses to the question. +*calibration*, you'll grade several responses that an instructor has already +graded. If your grading is similar to the instructor's, you can begin grading +other students' responses to the question. #. Click the name of the problem. When the **Learning to grade** page opens, click **Start learning to grade**. @@ -274,22 +258,22 @@ Step 3: Grade responses ^^^^^^^^^^^^^^^^^^^^^^^ When you grade a peer assessment response, you can not only select -options in the rubric, but also provide additional feedback for the +options in the rubric, but also provide additional feedback for the student who submitted the response. #. When the response opens, select the options in the rubric that you feel best apply to the response, as you did in the calibration process. If you have concerns about the response, you can select other - options to flag the response for instructor review. You don't have to fill + options to flag the response for instructor review. You don't have to fill out the rubric before you select these options. - If you aren't sure how to grade the response, select the **I am unsure about - the scores I have given above** check box. - - If the response is offensive, or if you suspect that it contains plagiarized + the scores I have given above** check box. + - If the response is offensive, or if you suspect that it contains plagiarized material, select the **This submission has explicit, offensive, or (I suspect) plagiarized content** check box. - + #. Under **Written Feedback**, write a comment about the score that you gave the response. @@ -310,50 +294,46 @@ to the **Peer Grading** page and click the name of the problem that you want to continue grading. .. note:: When a response opens for you to grade, it leaves the current "grading pool" - that other instructors or students are grading from, which prevents other - instructors or students from + that other instructors or students are grading from, which prevents other + instructors or students from grading the response while you are working on it. If you do not submit a score - for this response within 30 minutes, the response returns to the grading pool + for this response within 30 minutes, the response returns to the grading pool (so that it again becomes available for others to grade), even if you still have the response open on your screen. - - If the response returns to the grading pool (because the 30 minutes have passed), - but the response is still open on your screen, you can still submit feedback for - that response. If another instructor or student grades the response after it returns to the + + If the response returns to the grading pool (because the 30 minutes have passed), + but the response is still open on your screen, you can still submit feedback for + that response. If another instructor or student grades the response after it returns to the grading pool but before you submit your feedback, the response receives two grades. - - If you click your browser's **Back** button to return to the problem list before you - click **Submit** to submit your feedback for a response, the response stays outside - the grading pool until 30 minutes have passed. When the response returns to the - grading pool, you can grade it. + + If you click your browser's **Back** button to return to the problem list before you + click **Submit** to submit your feedback for a response, the response stays outside + the grading pool until 30 minutes have passed. When the response returns to the + grading pool, you can grade it. .. _ORA AI Assessment: Artificial Intelligence (AI) Assessment --------------------------------------- -.. list-table:: - :widths: 80 - :header-rows: 1 +.. warning:: - * - To instructors: - - * - You can delete this section if your ORA problem doesn't use AI assessments. - + You can delete this section if your ORA problem doesn't use AI assessments. -In an AI assessment, an instructor grades a sample set of student responses to the -open response assessment problem. A machine learning algorithm then creates a model + +In an AI assessment, an instructor grades a sample set of student responses to the +open response assessment problem. A machine learning algorithm then creates a model based on the instructor's scores and grades the remaining students' responses. After you submit your response to an AI assessment, the following message appears under your response. **Your response has been submitted. Please check back later for your grade.** - + Depending on the time that it takes for the instructor to grade a sample set of responses, you may receive your grade within minutes, or you may have to wait a few days. You won't receive a notification when your score is ready, so keep -checking back. +checking back. For more information about accessing your scores, see :ref:`ORA Access Scores`. @@ -362,17 +342,13 @@ For more information about accessing your scores, see :ref:`ORA Access Scores`. Access Scores and Feedback -------------------------- -.. list-table:: - :widths: 80 - :header-rows: 1 +.. warning:: + + Modify the text in this section to apply to your course. - * - To instructors: - - * - Modify the text in this section to apply to your course. - For *self assessments*, the score that you give yourself appears as soon as you submit -the score. +the score. For *peer assessments* and *AI assessments*, you'll access your scores through the **Open Ended Console** page. @@ -386,12 +362,12 @@ For *peer assessments* and *AI assessments*, you'll access your scores through t see whether your responses have been graded. The status for each problem is either **Waiting to be Graded** or **Finished**. -#. If **Finished** appears in the **Status** column for the problem you want, +#. If **Finished** appears in the **Status** column for the problem you want, click the name of the problem to see your score for that problem. When you click the name of the problem, the problem opens in the courseware. For both AI and peer assessments, the score appears below your response -in an abbreviated version of the rubric. +in an abbreviated version of the rubric. .. image:: /Images/AIScoredResponse.gif @@ -415,14 +391,10 @@ For more information about grading peer assessments, see :ref:`ORA Peer Assessme Resubmitting a Response ----------------------- -.. list-table:: - :widths: 80 - :header-rows: 1 +.. warning:: + + You can delete this section if you don't allow students to submit multiple responses. - * - To instructors: - - * - You can delete this section if you don't allow students to submit multiple responses. - Some open response assessments allow multiple attempts. For these problems, a **New Submission** button appears below your original response. @@ -430,5 +402,3 @@ response. If you want to answer the question again, click **New Submission** to clear your former response, and click **OK** in the dialog box that appears. You can then enter a new response for the problem. - - \ No newline at end of file