diff --git a/docs/en_us/course_authors/source/Images/Grade_Report.png b/docs/en_us/course_authors/source/Images/Grade_Report.png index e370f83686..609a5a6522 100644 Binary files a/docs/en_us/course_authors/source/Images/Grade_Report.png and b/docs/en_us/course_authors/source/Images/Grade_Report.png differ diff --git a/docs/en_us/course_authors/source/Images/Grade_Report_example.png b/docs/en_us/course_authors/source/Images/Grade_Report_example.png index 577d66475c..f3d77241c3 100644 Binary files a/docs/en_us/course_authors/source/Images/Grade_Report_example.png and b/docs/en_us/course_authors/source/Images/Grade_Report_example.png differ diff --git a/docs/en_us/course_authors/source/Images/Grading_Configuration.png b/docs/en_us/course_authors/source/Images/Grading_Configuration.png index 682e6aa23b..a035073f5b 100644 Binary files a/docs/en_us/course_authors/source/Images/Grading_Configuration.png and b/docs/en_us/course_authors/source/Images/Grading_Configuration.png differ diff --git a/docs/en_us/course_authors/source/Images/Grading_Configuration_Studio.png b/docs/en_us/course_authors/source/Images/Grading_Configuration_Studio.png index 82363e3564..41068662fa 100644 Binary files a/docs/en_us/course_authors/source/Images/Grading_Configuration_Studio.png and b/docs/en_us/course_authors/source/Images/Grading_Configuration_Studio.png differ diff --git a/docs/en_us/course_authors/source/Images/Problem_URL.png b/docs/en_us/course_authors/source/Images/Problem_URL.png index eaa44f5aba..8c694f1269 100644 Binary files a/docs/en_us/course_authors/source/Images/Problem_URL.png and b/docs/en_us/course_authors/source/Images/Problem_URL.png differ diff --git a/docs/en_us/course_authors/source/Images/SAD_Answer_Review.png b/docs/en_us/course_authors/source/Images/SAD_Answer_Review.png index 74fa33f0c1..a4ea6cae22 100644 Binary files a/docs/en_us/course_authors/source/Images/SAD_Answer_Review.png and b/docs/en_us/course_authors/source/Images/SAD_Answer_Review.png differ diff --git a/docs/en_us/course_authors/source/Images/SAD_Column_Chart.png b/docs/en_us/course_authors/source/Images/SAD_Column_Chart.png index 87f452cfd8..4f8966a86d 100644 Binary files a/docs/en_us/course_authors/source/Images/SAD_Column_Chart.png and b/docs/en_us/course_authors/source/Images/SAD_Column_Chart.png differ diff --git a/docs/en_us/course_authors/source/Images/image245.png b/docs/en_us/course_authors/source/Images/StudentView_GradeCutoffs.png similarity index 100% rename from docs/en_us/course_authors/source/Images/image245.png rename to docs/en_us/course_authors/source/Images/StudentView_GradeCutoffs.png diff --git a/docs/en_us/course_authors/source/Images/image247.png b/docs/en_us/course_authors/source/Images/StudentView_Problems.png similarity index 100% rename from docs/en_us/course_authors/source/Images/image247.png rename to docs/en_us/course_authors/source/Images/StudentView_Problems.png diff --git a/docs/en_us/course_authors/source/Images/Student_Progress.png b/docs/en_us/course_authors/source/Images/Student_Progress.png index cbde3791bc..4aba567875 100644 Binary files a/docs/en_us/course_authors/source/Images/Student_Progress.png and b/docs/en_us/course_authors/source/Images/Student_Progress.png differ diff --git a/docs/en_us/course_authors/source/Images/Student_Progress_list.png b/docs/en_us/course_authors/source/Images/Student_Progress_list.png index 4319489a6a..60cdb12a81 100644 Binary files a/docs/en_us/course_authors/source/Images/Student_Progress_list.png and b/docs/en_us/course_authors/source/Images/Student_Progress_list.png differ diff --git a/docs/en_us/course_authors/source/Images/Student_Progress_mouseover.png b/docs/en_us/course_authors/source/Images/Student_Progress_mouseover.png index bfcb255e20..5e9e5eb9dd 100644 Binary files a/docs/en_us/course_authors/source/Images/Student_Progress_mouseover.png and b/docs/en_us/course_authors/source/Images/Student_Progress_mouseover.png differ diff --git a/docs/en_us/course_authors/source/Images/score_histogram.png b/docs/en_us/course_authors/source/Images/score_histogram.png index f552bfbdb3..dcc61c4b7c 100644 Binary files a/docs/en_us/course_authors/source/Images/score_histogram.png and b/docs/en_us/course_authors/source/Images/score_histogram.png differ diff --git a/docs/en_us/course_authors/source/Images/score_histogram_callouts.png b/docs/en_us/course_authors/source/Images/score_histogram_callouts.png index 97c4cb816a..17c5323633 100644 Binary files a/docs/en_us/course_authors/source/Images/score_histogram_callouts.png and b/docs/en_us/course_authors/source/Images/score_histogram_callouts.png differ diff --git a/docs/en_us/course_authors/source/Images/student_answer_calc.png b/docs/en_us/course_authors/source/Images/student_answer_calc.png index 12b84f2050..661d6f972f 100644 Binary files a/docs/en_us/course_authors/source/Images/student_answer_calc.png and b/docs/en_us/course_authors/source/Images/student_answer_calc.png differ diff --git a/docs/en_us/course_authors/source/Images/student_answer_excel.png b/docs/en_us/course_authors/source/Images/student_answer_excel.png index dd32823599..a807987ca8 100644 Binary files a/docs/en_us/course_authors/source/Images/student_answer_excel.png and b/docs/en_us/course_authors/source/Images/student_answer_excel.png differ diff --git a/docs/en_us/course_authors/source/change_log.rst b/docs/en_us/course_authors/source/change_log.rst index e0e20e19b8..a6a4e0413b 100644 --- a/docs/en_us/course_authors/source/change_log.rst +++ b/docs/en_us/course_authors/source/change_log.rst @@ -12,6 +12,17 @@ May, 2014 * - Date - Change + * - 05/22/14 + - Reorganized, updated, and streamlined these sections in the :ref:`Running + Your Course Index` chapter: + + * Divided *Grade and Answer Data* into separate sections for + :ref:`Grades` and :ref:`Review_Answers`. + * Moved information about the student **Progress** page from the + *Checking Student Progress and Issuing Certificates* section to + :ref:`Grades`. + * Renamed the *Checking Student Progress and Issuing Certificates* + section to :ref:`Checking Student Progress and Issuing Certificates`. * - 05/21/14 - Added warning that version 1 of the :ref:`Open Response Assessments` suite has been deprecated and added a link to the `new ORA information `_. diff --git a/docs/en_us/course_authors/source/creating_content/create_problem.rst b/docs/en_us/course_authors/source/creating_content/create_problem.rst index e2a9911688..055dbe96fd 100644 --- a/docs/en_us/course_authors/source/creating_content/create_problem.rst +++ b/docs/en_us/course_authors/source/creating_content/create_problem.rst @@ -242,6 +242,8 @@ Maximum Attempts This setting specifies the number of times a student can try to answer the problem. By default, a student has an unlimited number of attempts. +.. note:: Only questions that have a **Maximum Attempts** setting of 1 or higher are included on the Student Answer Distribution report that you can download during your course. + .. _Problem Weight: ============================== diff --git a/docs/en_us/course_authors/source/running_course/checking_student_progress.rst b/docs/en_us/course_authors/source/running_course/checking_student_progress.rst index 066c6e2bdb..f40c7306c5 100644 --- a/docs/en_us/course_authors/source/running_course/checking_student_progress.rst +++ b/docs/en_us/course_authors/source/running_course/checking_student_progress.rst @@ -1,106 +1,19 @@ .. _Checking Student Progress and Issuing Certificates: ################################################### -Checking Student Progress and Issuing Certificates -################################################### - - -The grading policy and stored problem scores are used to record progress through the course, determine -final grades, and issue certificates at the end. This unit will give you -some advance information about how the grading policy will be visible to the -students during the run of the course and what you will need to do at the -end of the course to give out grades. - -.. _A Student's View: - -****************************** -A Student's View -****************************** - -Students can check their progress by clicking on -the **Progress** tab in the course. The -student's progress through the graded part of the course is displayed at the -top of this page, above the subsection scores, as a chart with entries for -all the assignments, total percentage earned in the course so far and -percent needed for each grade cutoff. Here is an example of a student's -progress through edX101. - - -.. image:: ../Images/image245.png - :width: 800 - :alt: Image of a student's Course Progress page with the grade cutoffs legend highlighted - - -The student can see from this page that edX101 was graded as a Pass/Fail course with a cutoff -of 34% and that the grading rubric contained one assignment type, called -Learning Sequence, consisting of 11 assignments total. Furthermore, this particular student has only -submitted correct responses to two assignments, and that her current total percent grade in -the course is 6%. By hovering over each progress bar, the student can -get further statistics of how much each assignment was counted as. - - -As was mentioned in the unit on Viewing Scores, further down on the Progress -page is a list of all the subsections in the course, with the scores -recorded for the student for all problems in the course. Here is a scrolled -down view of the example Progress page for the student in the example above: - - -.. image:: ../Images/image247.png - :width: 800 - :alt: Image of a student's Course Progress page with problems highlighted - -Again, note that point scores from graded sections are called "Problem -Scores", while point scores from ungraded sections are called "Practice -Scores". - -.. _Check Progress of Students as an Instructor: - -********************************************** -Check Progress of Students as an Instructor -********************************************** - -To check the progress of the student, go to the -Instructor Dashboard of your course click -the Grades page. The Instructor Dashboard for courses sometimes changes as -more course-specific tools get added. Here is the current view of the top of -the Grades page of the Instructor dashboard for edX101: - - -.. image:: ../Images/image249.png - :width: 800 - :alt: Image of the Grades tab of the Instructor Dashboard - -Here you see several options for viewing or downloading student grades, -viewing individual progress through a course or resetting problem attempts. - -.. note:: - - Only the top part of the Grades page is shown. The page continues - with some more information about course statistics. The link to the - student's progress page should give you a view of exactly what the student - would see, including scores for graded and ungraded assignments. - - -.. note:: - - The stored scores visible to you on the Instructor tab and to - the students from the Progress tab in the course are a snapshot of the - current state of the problem score database. They may be slightly out of - sync with actual problem scores. (Asynchronicities may happen if, for example, - the weight of a live problem was changed during an assignment, and not - everyone has yet resubmitted their answers for that problem.) Scores and grades - are usually recomputed at the end of the semester before determining final - grades and issuing Certificates. - -.. _Assign Final Grades and Issuing Certificates: - -*********************************************** Assign Final Grades and Issue Certificates -*********************************************** +################################################### +.. This chapter will be renamed and expanded to include course wrap-up activities and best practices. -The final grades of a student in the course and the grading -rubric you have set are used to determine whether the student has earned a -Certificate of Mastery for the course. The process for issuing Certificates -has to be started manually by you or by the edX support team at the end of the -course. +To assign a final grade to each student enrolled in a course, you generate +grades after the **Course End Date** and **Time** have passed. See +:ref:`Access_grades`. +The final grades of a student in the course and the grading configuration you +set in Studio are used to determine whether the student has earned a Certificate +of Mastery for the course. + +The process of issuing certificates is completed at edX: work with your edX +program manager to schedule a date to issue certificates and complete this +process. + \ No newline at end of file diff --git a/docs/en_us/course_authors/source/running_course/course_answers.rst b/docs/en_us/course_authors/source/running_course/course_answers.rst new file mode 100644 index 0000000000..3055522481 --- /dev/null +++ b/docs/en_us/course_authors/source/running_course/course_answers.rst @@ -0,0 +1,321 @@ +.. _Review_Answers: + +############################ +Answer Data +############################ + +To review student answers to the problems in your course, you can review the +answer submitted by a selected student for a specific problem, download a +course-wide report of answer data, or review a histogram of student answers for +a selected problem. + +* :ref:`Student_Answer_Submission` + +* :ref:`Student_Answer_Distribution` + +* :ref:`score_histogram` + +.. _Student_Answer_Submission: + +************************************************************ +Check a Student's Answer Submission and Submission History +************************************************************ + +For a single student and problem, you can review the exact response submitted, +the number of attempts made, and the date and time of the submission. You +identify the student by supplying a username. + +To review a response submitted by a student: + +#. View the live version of your course. + +#. Click **Courseware** and navigate to the unit that contains the problem + you want to review. + +#. Display the problem and then click **Submission History** below the problem. + +#. Enter the username for the student whose work you want to review, then click + **View History** at the end of the page. + + Information about the response or responses provided by the student displays. + +To close the Submission History Viewer, click on the browser page outside of the +viewer. + +.. _Student_Answer_Distribution: + +**************************************** +Student Answer Distribution +**************************************** + +For certain types of problems in your course, you can download a CSV file with +data about the distribution of student answers. Student answer distribution data +is included in the file for problems of these types: + +* Checkboxes (````) +* Dropdown (````) +* Multiple choice (````) +* Numerical input (````) +* Text input (````) +* Math expression input (````) + +The file includes a row for each problem-answer combination selected by your +students. For example, for a problem that has a total of five possible answers +the file includes up to five rows, one for each answer selected by at least one +student. For problems with **Randomization** enabled in Studio (sometimes +called rerandomization), there is one row for each problem-variant-answer +combination selected by your students. See :ref:`Problem Settings`. + +The CSV file contains the following columns: + +.. list-table:: + :widths: 20 60 + :header-rows: 1 + + * - Column + - Description + * - ModuleID + - The internal identifier for the Problem component. + * - PartID + - For a Problem component that contains multiple problems, the internal + identifier for each individual problem. For a Problem component that + contains a single problem, the internal identifier of that problem. + * - Correct Answer + - 0 if this **AnswerValue** is incorrect, or 1 if this **AnswerValue** is + correct. + * - Count + - The number of times that students entered or selected this answer as + their most recent submission for the problem or problem variant. For + problems with the number of **Attempts** set to a value greater than 1, + this means that each student contributes a maximum of 1 to this count, + even if the same answer is provided in multiple attempts. + * - ValueID + - The internal identifier of the answer choice for checkboxes and multiple + choice problems. Blank for dropdown, numerical input, text input, and + math expression input problems. + * - AnswerValue + - The text label of the answer choice for checkboxes, dropdown, and + multiple choice problems. The value entered by the student for numerical + input, text input, and math expression input problems. + * - Variant + - For problems that use the **Randomization** setting in Studio, contains + the unique identifier for a variant of the problem. Blank for problems + that do not use the **Randomization** setting, or that use the **Never** + option for this setting. + * - Problem Display Name + - The **Display Name** defined for the problem. + * - Question + - The label for accessibility that appears above the answer choices or the + text entry field for the problem. In Studio's Simple Editor, this text is + surrounded by two pairs of angle brackets (>>Question<<). Blank for + questions that do not have an accessibility label defined. + +Entries are sorted by the value in each column, starting with the ModuleID on +the left and continuing through the columns to the right. + +.. _Download_Answer_Distributions: + +=================================================== +Download the Student Answer Distribution Report +=================================================== + +An automated process runs periodically on the edX servers to update the CSV file +of student answer data. A link to the most recently updated version of the CSV +file is available on the Instructor Dashboard. + +To download the most recent file of student answer data: + +#. View the live version of your course. + +#. Click **Instructor**, then click **Data Download**. + +#. At the bottom of the page, click the ``{course_id}_answer_distribution.csv`` + file name. You may have to scroll down to find this file. + +========================================================================== +Frequently Asked Questions about the Student Answer Distribution Report +========================================================================== + +Answers to questions about the student answer distribution report follow. + +**My course doesn't have a student answer distribution report. How can I +generate it?** + +Student answer distribution reports are generated automatically, and refreshed +several times each day. The ``{course_id}_answer_distribution.csv`` file +displays after all of the ``{course_id}_grade_report_{date}.csv`` files. Be sure +to scroll to the end of the list of available reports. + +**Why are some problems missing from this report? The ones that are missing do +have the problem types listed under** :ref:`Review_Answers`. + +This report includes only problems that at least one student has answered since +early March 2014. For those problems, this report only includes activity that +occurred after October 2013. + +**Why don't I see an AnswerValue for some of my problems?** + +For checkboxes and multiple choice problems, the answer choices actually +selected by a student after early March 2014 display as described in the +previous answer. Answer choices selected by at least one student after October +2013, but not selected since early March 2014, are included on the report but do +not include an **AnswerValue**. The **ValueID** does display the internal +identifiers, such as choice_1 and choice_2, for those answers. + +**Why don't I see a Question for some of my problems?** + +The value in the **Question** column is the accessibility label for the problem. +For more information about how to set up labels for problems, see :ref:`Create +Exercises`. + +Also, for problems that use the **Randomization** setting in Studio, if a +particular answer has not been selected since early March 2014, the **Question** +is blank for that answer. + +**My students are saying that they answered a question, but it isn't showing up +in the report. How can that be?** + +Only questions that have a **Maximum Attempts** setting of 1 or higher are +included on the report. + +**I made a correction to a problem after it was released. How can I tell which +students tried to answer it before I made the change?** + +Problem **Count** values reflect the entire problem history. If you change a +problem after it is released, it may not be possible for you to determine which +answers were given before and after you made the change. + +**Why is the same answer showing up in two different rows when I view the report +in a spreadsheet?** + +Some spreadsheet applications can alter the data in the CSV report for display +purposes. For example, for different student answers of "0.5" and ".5" Excel +correctly includes the two different lines from the CSV, but displays the +**AnswerValue** on both of them as "0.5". If you notice answers that appear to +be the same on separate lines with separate counts, you can review the actual, +unaltered data by opening the CSV file in a text editor. + +**Why are there strange characters in the report when I view it in a +spreadsheet?** + +The CSV file is UTF-8 encoded, but not all spreadsheet applications interpret +and render UTF-8 encoded characters correctly. For example, a student answer +distribution report with answer values in French displays differently in +Microsoft Excel for Mac than in OpenOffice Calc. + + Answer Values in Microsoft Excel for Mac: + + .. image:: ../Images/student_answer_excel.png + :alt: A spreadsheet that replaces accented French characters with underscores + + Answer Values in OpenOffice Calc: + + .. image:: ../Images/student_answer_calc.png + :alt: A spreadsheet that displays accented French characters correctly + +If you notice characters that do not display as expected in a spreadsheet, try a +different spreadsheet application such as LibreOffice or Apache OpenOffice to +open the CSV file. + +========================================================================== +Interpret the Student Answer Distribution Report +========================================================================== + +You can use the Student Answer Distribution report to review student responses +to assignments, which can then help you evaluate the structure and completeness +of your courseware and problem components. + +As an example, you define a text input question in Studio to have a single +correct answer, "Warfarin". When you produce the Student Answer Distribution +report, you verify that this answer was in fact marked correct: there is a 1 in +the **Correct Answer** column for this **AnswerValue**. + +.. image:: ../Images/SAD_Answer_Review.png + :alt: In Excel, 5 rows show 5 answer values, 4 of which show comprehension of the question, but only 1 answer is marked as correct + +However, as you view the report you notice other student answers that you did +not set up to be marked as correct in Studio, but that you might (or might not) +also consider to be correct, such as "Warfarin or Coumadin". The **Correct +Answer** column shows that the other answers were marked incorrect (0), but for +future iterations of your course you may want to revise the question or update +the problem to evaluate additional variations of the answer as correct. + +Many spreadsheet applications offer data visualization options, such as charts +or graphs. Charts can help make your students' common misconceptions easier to +identify. + +.. image:: ../Images/SAD_Column_Chart.png + :alt: In Excel, AnswerValue and Count columns next to each other, values for 4 rows selected, and a column chart of the count for the 4 answers + +In this example, the Student Answer Distribution report is open in Microsoft +Excel. To create a chart that shows how many of your students chose various +answers to a multiple choice question, you move the **AnswerValue** and +**Count** columns next to each other. After you click and drag to select the +report cells that contain the data you want to chart, you click the Charts +toolbar and then click the type of chart you want. + +.. note:: Refer to the help for the spreadsheet application that you use for information on using these options. You may have to make changes to your spreadsheet, such as reordering columns. Save a copy of the file you originally downloaded as a backup before you begin. + +You can adjust your course content based on common student mistakes. While most +students in this example selected the correct answer, the number of incorrect +answer(s) can guide future changes to the courseware. + +.. _score_histogram: + +************************************************** +View a Histogram of Scores for a Single Problem +************************************************** + +You can view a chart of the score distribution for a specified problem. + +.. note:: To view the score distribution for a problem, you need its unique identifier. You can display a histogram for problems that have the ``/problem/`` prefix in the unique identifier. See :ref:`find_URL`. + +To display the distribution of scores for a problem: + +#. View the live version of your course. + +#. Click **Instructor**, then click **Analytics**. + +#. In the Score Distribution section, select a problem by using its unique + identifier. + + The **Analytics** page updates to display a histogram of scores for that + problem. + + .. image:: ../Images/score_histogram.png + :alt: Graph of the number of students who received different scores for a + selected problem + +.. **Question**: (sent to Olga 31 Jan 14) this is a tough UI to use: how do they correlate the codes in this drop-down with actual constructed problems? the copy-and-paste UI on the Student Admin page actually works a little better imo. LMS-2522 + +=================================================== +Interpret a Score Histogram +=================================================== + +The histogram of scores for a selected problem provides the following +information. + + .. image:: ../Images/score_histogram_callouts.png + :alt: Histogram with indicators for the number of students scored value and + the x-axis numbers that indicate plotted scores + +* The x-axis indicates the number of points that students can receive for the + problem, unweighted. Although decimal values are marked on the x-axis, each + score is a whole number. The whole number value at the left of a plotted bar + is the score that applies to it. + +* The y-axis indicates the number of students who have answered and checked the + problem. + +* Above the graph, a number of **students scored** displays. This number + indicates the total number of database records that exist for the problem: it + includes not only students who have answered and checked the problem to + receive a score, but also students who have opened the problem but not yet + completed it. + +* An automated process runs approximately weekly on the edX servers to update + score histograms. This process runs less frequently than the process that + updates the student answer distribution report: at certain times during a + course the number of students shown in a score histogram can be quite + different than the **Count** for the same problem in a student answer + distribution report. diff --git a/docs/en_us/course_authors/source/running_course/course_grades.rst b/docs/en_us/course_authors/source/running_course/course_grades.rst index c0e32f351e..cef084419d 100644 --- a/docs/en_us/course_authors/source/running_course/course_grades.rst +++ b/docs/en_us/course_authors/source/running_course/course_grades.rst @@ -1,7 +1,7 @@ .. _Grades: ############################ -Grade and Answer Data +Student Grades and Grading ############################ You can review information about how grading is configured for your course, and @@ -14,9 +14,13 @@ changes you can make, see the following topics: * :ref:`Access_grades` +* :ref:`check_student_progress` + * :ref:`Adjust_grades` -To review student answers to the problems in your course, you can download data for course problems or review a graph for a selected problem. See :ref:`Review_Answers`. +To review student answers to the problems in your course, you can check the answer submitted by a specified student for a selected problem, download course-wide answer data, or review a graph of all answer data for a selected problem. See :ref:`Review_Answers`. + +.. all answer-related info moved into new course_answers.rst For information about how you establish a grading policy and work with the Problem components in your course, see :ref:`Establish a Grading Policy` or :ref:`Working with Problem Components`. @@ -51,14 +55,8 @@ You establish a grading policy for your course when you create it in Studio. Whi .. _Access_grades: ******************************************************** -Access Student Grades -******************************************************** - -You can generate and review your students' grades at any time during your course. You can generate grades for all currently enrolled students, or check the progress of a single student (who can be enrolled or unenrolled). - -========================================================= Generate Grades for Enrolled Students -========================================================= +******************************************************** When you initiate calculations to grade student work, a process starts on the edX servers. The complexity of your grading configuration and the number of students enrolled in your course affect how long this process takes. You can download the calculated grades in a CSV (comma-separated values) file when the grading process is complete. You cannot view student grades on the Instructor Dashboard. @@ -107,25 +105,27 @@ You can open ``{course_id}_grade_report_{date}.csv`` files in a spreadsheet appl The CSV file contains one row of data for each student, and columns that provide the following information. -* Student identifiers, including an internal ``id``, ``email`` address, and ``username``. +* Student identifiers, including an internal **id**, **email** address, and **username**. -* ``grade``, with the total score a student has currently attained in the course. This value is expressed as a decimal: a student with a grade of 0.65 has earned 65% of the credit in the course, and a student with a grade of 1 has earned 100%. +* **grade**, with the total score a student has currently attained in the course. This value is expressed as a decimal: a student with a grade of 0.65 has earned 65% of the credit in the course, and a student with a grade of 1 has earned 100%. -* Each ``{assignment type} {number}`` defined in your grading configuration, with the score a student attained for that specific assignment. For example, column HW 03 shows the scores for the third homework assignment. +* Each **{assignment type} {number}** defined in your grading configuration, with the score a student attained for that specific assignment. For example, column HW 03 shows the scores for the third homework assignment. -* An ``{assignment type} Avg`` with each student's current average score for that assignment type: for example, HW Avg. +* An **{assignment type} Avg** with each student's current average score for that assignment type: for example, HW Avg. .. note:: The grade reports do not include information about individual questions within the assignments, or include student answer distributions. -========================================================= +.. _check_student_progress: + +**************************************** Check the Progress of a Single Student -========================================================= +**************************************** To check a single student's progress, you can locate the specific row on the grade report or review the student's **Progress** page. The **Progress** page includes a chart that plots the score the student has earned for every graded assignment and the total grade as of the current date. Below the chart, each assignment and the score attained are listed. To review a student's **Progress** page, you supply an email address or username. You can check the progress for students who are either enrolled in, or who have unenrolled from, the course. -Students can view a similar chart and assignment list (of their own progress only) when they are logged in to the course. +Students can view a similar chart and assignment list (of their own progress only) when they are logged in to the course. See :ref:`A Students View`. To view the **Progress** page for a student: @@ -176,25 +176,45 @@ On the student's **Progress** page, you see the same information graphically rep The chart on the **Progress** page includes y-axis labels for the grade ranges defined for the course. In this example, Pass is set to 60%, so at the end of the course students with a grade of 0.60 or higher can receive certificates. -=========================================================== -Check a Student's Answer Submission and Submission History -=========================================================== +.. note:: Student scores on the **Progress** page are a snapshot of the current state of the problem score database. They can, at times, be out of sync with actual problem scores. For example, asynchronicities can occur if the weight of a live problem was changed during an assignment, and not all students have resubmitted their answers for that problem. +.. note above moved from checking_student_progress.rst -For a single student and problem, you can review the exact response submitted, the number of attempts made, and the date and time of the submission. You identify the student by supplying a username. +.. _A Students View: -To review a response submitted by a student: +============================================= +A Student's View of Course Progress +============================================= +.. this section moved from checking_student_progress.rst -#. View the live version of your course. - -#. Click **Courseware** and navigate to the component that contains the problem you want to review. - -#. Display the problem and then click **Submission History** below the problem. - -#. Enter the username for the student whose work you want to review, then click **View History** at the end of the page. - - Information about the response or responses provided by the student displays. - -To close the Submission History Viewer, click on the browser page outside of the viewer. +Students can check their progress by clicking **Progress** in the course +navigation bar. The student's progress through the graded part of the course +displays at the top of this page, above the subsection scores. Progress is +visualized as a chart with entries for all the assignments, total percentage +earned in the course so far, and percent needed for each grade cutoff. Here is +an example of a student's progress through edX101. + +.. image:: ../Images/StudentView_GradeCutoffs.png + :width: 800 + :alt: Image of a student's Course Progress page with the grade cutoffs legend highlighted + +The student can see from this page that edX101 was graded as a Pass/Fail course +with a cutoff of 34% and that the grading rubric contained one assignment type, +called Learning Sequence, consisting of 11 assignments total. Furthermore, this +particular student has only submitted correct responses to two assignments, and +that her current total percent grade in the course is 6%. By hovering over each +progress bar, the student can get further statistics of how much each assignment +was counted as. + +Further down on the **Progress** page is a list of all the subsections in the +course, with the scores recorded for the student for all problems in the course. +Here is the **Progress** page for the student in the example above: + +.. image:: ../Images/StudentView_Problems.png + :width: 800 + :alt: Image of a student's Course Progress page with problems highlighted + +Note that point scores from graded sections are called "Problem Scores", +while point scores from ungraded sections are called "Practice Scores". .. _Adjust_grades: @@ -243,8 +263,8 @@ To find the unique identifier in the URL for a problem: Information about the problem displays, including its location or URL. This URL ends with the type of module, which is typically "problem", and the unique identifier. -.. image:: ../Images/Problem_URL.png - :alt: The Staff Debug view of a problem with the unique identifier indicated at the end of a URL address + .. image:: ../Images/Problem_URL.png + :alt: The Staff Debug view of a problem with the unique identifier indicated at the end of a URL address 4. To copy the identifier for the problem, select it, right click, and choose **Copy**. @@ -411,198 +431,3 @@ See :ref:`find_URL`. #. In the **Student-Specific Grade Adjustment** section of the page, enter both the student’s email address or username and the unique problem identifier, and then click **Delete Student State for Problem**. - -.. _Review_Answers: - -**************************************** -Student Answer Distribution -**************************************** - -For certain types of problems in your course, you can download a CSV file with data about the distribution of student answers. Student answer distribution data is included in the file for problems of these types: - -* Checkboxes (````) -* Dropdown (````) -* Multiple choice (````) -* Numerical input (````) -* Text input (````) -* Math expression input (````) - -The file includes a row for each problem-answer combination selected by your students. For example, for a problem that has a total of five possible answers the file includes up to five rows, one for each answer selected by at least one student. For problems that use rerandomization (the **Randomization** setting in Studio), there is one row for each problem-variant-answer combination selected by your students. - -The CSV file contains the following columns: - -.. list-table:: - :widths: 20 60 - :header-rows: 1 - - * - Column - - Description - * - ModuleID - - The internal identifier for the Problem component. - * - PartID - - For a Problem component that contains multiple problems, the internal identifier for each individual problem. For a Problem component that contains a single problem, the internal identifier of that problem. - * - Correct Answer - - 0 if this **AnswerValue** is incorrect, or 1 if this **AnswerValue** is correct. - * - Count - - The number of times that students entered or selected this answer as their most recent submission for the problem or problem variant. For problems with the number of **Attempts** set to a value greater than 1, this means that each student contributes a maximum of 1 to this count, even if the same answer is provided in multiple attempts. - * - ValueID - - The internal identifier of the answer choice for checkboxes and multiple choice problems. Blank for dropdown, numerical input, text input, and math expression input problems. - * - AnswerValue - - The text label of the answer choice for checkboxes, dropdown, and multiple choice problems. The value entered by the student for numerical input, text input, and math expression input problems. - * - Variant - - For problems that use the **Randomization** setting in Studio, contains the unique identifier for a variant of the problem. Blank for problems that do not use the **Randomization** setting, or that use the **Never** option for this setting. - * - Problem Display Name - - The **Display Name** defined for the problem. - * - Question - - The label for accessibility that appears above the answer choices or the text entry field for the problem. In Studio's Simple Editor, this text is surrounded by two pairs of angle brackets (>>Question<<). Blank for questions that do not have an accessibility label defined. - -Entries are sorted by the value in each column, starting with the ModuleID on the left and continuing through the columns to the right. - -.. _Download_Answer_Distributions: - -=================================================== -Download the Student Answer Distribution Report -=================================================== - -An automated process runs periodically on the edX servers to update the CSV file of student answer data. A link to the most recently updated version of the CSV file is available on the Instructor Dashboard. - -To download the most recent file of student answer data: - -#. View the live version of your course. - -#. Click **Instructor**, then click **Data Download**. - -#. At the bottom of the page, click the ``{course_id}_answer_distribution.csv`` file name. You may have to scroll down to find this file. - -========================================================================== -Frequently Asked Questions about the Student Answer Distribution Report -========================================================================== - -Answers to questions about the student answer distribution report follow. - -**My course doesn't have a student answer distribution report. How can I generate it?** - -Student answer distribution reports are generated automatically, and refreshed several times each day. The ``{course_id}_answer_distribution.csv`` file displays after all of the ``{course_id}_grade_report_{date}.csv`` files. Be sure to scroll to the end of the list of available reports. - -**Why are some problems missing from this report? The ones that are missing do have the problem types listed under** :ref:`Review_Answers`. - -This report includes only problems that at least one student has answered since early March 2014. For those problems, this report only includes activity that occurred after October 2013. - -**Why don't I see an AnswerValue for some of my problems?** - -For checkboxes and multiple choice problems, the answer choices actually selected by a student after early March 2014 display as described in the previous answer. Answer choices selected by at least one student after October 2013, but not selected since early March 2014, are included on the report but do not include an **AnswerValue**. The **ValueID** does display the internal identifiers, such as choice_1 and choice_2, for those answers. - -**Why don't I see a Question for some of my problems?** - -The value in the **Question** column is the accessibility label for the problem. For more information about how to set up labels for problems, see :ref:`Create Exercises`. - -Also, for problems that use the **Randomization** setting in Studio, if a particular answer has not been selected since early March 2014, the **Question** is blank for that answer. - -**My students are saying that they answered a question, but it isn't showing up in the report. How can that be?** - -Only questions that have a **Maximum Attempts** setting of 1 or higher are included on the report. - -**I made a correction to a problem after it was released. How can I tell which students tried to answer it before I made the change?** - -Problem **Count** values reflect the entire problem history. If you change a problem after it is released, it may not be possible for you to determine which answers were given before and after you made the change. - -**Why is the same answer showing up in two different rows when I view the report in a spreadsheet?** - -Some spreadsheet applications can alter the data in the CSV report for display purposes. For example, for different student answers of "0.5" and ".5" Excel correctly includes the two different lines from the CSV, but displays the **AnswerValue** on both of them as "0.5". If you notice answers that appear to be the same on separate lines with separate counts, you can review the actual, unaltered data by opening the CSV file in a text editor. - -**Why are there strange characters in the report when I view it in a spreadsheet?** - -The CSV file is UTF-8 encoded, but not all spreadsheet applications interpret and render UTF-8 encoded characters correctly. For example, a student answer distribution report with answer values in French displays differently in Microsoft Excel for Mac than in OpenOffice Calc. - - Answer Values in Microsoft Excel for Mac: - - .. image:: ../Images/student_answer_excel.png - :alt: A spreadsheet that replaces accented French characters with underscores - - Answer Values in OpenOffice Calc: - - .. image:: ../Images/student_answer_calc.png - :alt: A spreadsheet that displays accented French characters correctly - -If you notice characters that do not display as expected in a spreadsheet, try a different spreadsheet application such as LibreOffice or Apache OpenOffice to open the CSV file. - -========================================================================== -Interpret the Student Answer Distribution Report -========================================================================== - -You can use the Student Answer Distribution report to review student responses to assignments, which can then help you evaluate the structure and completeness of your courseware and problem components. - -As an example, you define a text input question in Studio to have a single correct answer, "Warfarin". When you produce the Student Answer Distribution report, you verify that this answer was in fact marked correct: there is a 1 in the **Correct Answer** column for this **AnswerValue**. - -.. image:: ../Images/SAD_Answer_Review.png - :alt: In Excel, 5 rows show 5 answer values, 4 of which show comprehension of the question, but only 1 answer is marked as correct - -However, as you view the report you notice other student answers that you did not set up to be marked as correct in Studio, but that you might (or might not) also consider to be correct, such as "Warfarin or Coumadin". The **Correct Answer** column shows that the other answers were marked incorrect (0), but for future iterations of your course you may want to revise the question or update the problem to evaluate additional variations of the answer as correct. - -Many spreadsheet applications offer data visualization options, such as charts or graphs. Charts can help make your students' common misconceptions easier to identify. - -.. image:: ../Images/SAD_Column_Chart.png - :alt: In Excel, AnswerValue and Count columns next to each other, values for 4 rows selected, and a column chart of the count for the 4 answers - -In this example, the Student Answer Distribution report is open in Microsoft Excel. To create a chart that shows how many of your students chose various answers to a multiple choice question, you move the **AnswerValue** and **Count** columns next to each other. After you click and drag to select the report cells that contain the data you want to chart, you click the Charts toolbar and then click the type of chart you want. - -.. note:: Refer to the help for the spreadsheet application that you use for information on using these options. You may have to make changes to your spreadsheet, such as reordering columns. Save a copy of the file you originally downloaded as a backup before you begin. - -You can adjust your course content based on common student mistakes. While most students in this example selected the correct answer, the number of incorrect answer(s) can guide future changes to the courseware. - -=================================================== -View a Histogram of Scores for a Single Problem -=================================================== - -You can view a chart of the score distribution for a specified problem. - -.. note:: To view the score distribution for a problem, you need its unique identifier. You can display a histogram for problems that have the ``/problem/`` prefix in the unique identifier. See :ref:`find_URL`. - -To display the distribution of scores for a problem: - -#. View the live version of your course. - -#. Click **Instructor**, then click **Analytics**. - -#. In the Score Distribution section, select a problem by using its unique identifier. - - The **Analytics** page updates to display a histogram of scores for that - problem. - - .. image:: ../Images/score_histogram.png - :alt: Graph of the number of students who received different scores for a - selected problem - -.. **Question**: (sent to Olga 31 Jan 14) this is a tough UI to use: how do they correlate the codes in this drop-down with actual constructed problems? the copy-and-paste UI on the Student Admin page actually works a little better imo. LMS-2522 - -=================================================== -Interpret a Score Histogram -=================================================== - -The histogram of scores for a selected problem provides the following -information. - - .. image:: ../Images/score_histogram_callouts.png - :alt: Histogram with indicators for the number of students scored value and - the x-axis numbers that indicate plotted scores - -* The x-axis indicates the number of points that students can receive for the - problem, unweighted. Although decimal values are marked on the x-axis, each - score is a whole number. The whole number value at the left of a plotted bar - is the score that applies to it. - -* The y-axis indicates the number of students who have answered and checked the - problem. - -* Above the graph, a number of **students scored** displays. This number - indicates the total number of database records that exist for the problem: it - includes not only students who have answered and checked the problem to - receive a score, but also students who have opened the problem but not yet - completed it. - -* An automated process runs approximately weekly on the edX servers to update - score histograms. This process runs less frequently than the process that - updates the student answer distribution report: at certain times during a - course the number of students shown in a score histogram can be quite - different than the **Count** for the same problem in a student answer - distribution report. diff --git a/docs/en_us/course_authors/source/running_course/index.rst b/docs/en_us/course_authors/source/running_course/index.rst index bd4101e935..0e88568c79 100644 --- a/docs/en_us/course_authors/source/running_course/index.rst +++ b/docs/en_us/course_authors/source/running_course/index.rst @@ -13,5 +13,6 @@ Running Your Course discussion_guidance_moderators course_data course_student + course_answers course_grades checking_student_progress \ No newline at end of file