Spring 2020: What We Learned
Spring 2020 Course Instructor Feedback Comment Analysis (view as PDF)
Prepared by the Office of Strategic Planning and Institutional Research
In Spring 2020, additional questions were added to the Course Instructor Feedback surveys in order to better understand student perspectives on the transition to remote teaching and learning necessitated by COVID-19. The following open-ended question was posed to students: “What worked and did not work about remote teaching and learning, in this course? Please note any elements of remote instruction you would like to see incorporated in traditional instruction.”
Notre Dame Learning was interested in understanding if the responses to this question could offer insights into best practices for large classes that transitioned to remote instruction. OSPIR was asked to analyze a sample of responses to this question and provide a summary of their analysis.
OSPIR defined “large classes” as those classes enrolling 40 or more students. The initial intent was to take a sample of both highly rated and poorly rated classes. Due to an insufficient number of poorly rated classes, OSPIR elected to focus on highly rated classes. “Highly rated” was defined as courses in which at least 95% of respondents Strongly Agreed or Somewhat Agreed with the statement: The quality of instruction after spring break met or exceeded my expectations for remote teaching and learning.
From the pool of highly rated, large classes, OSPIR performed a purposive selection of 24 classes, representative of divisions, class types, and class sizes. The comments from these 24 classes were reviewed and analyzed. The themes are outlined below.
Interaction: Students crave interaction with their instructors and classmates. This was largely accomplished through synchronous class meetings, question and answer sessions, and office hours. There did not seem to be many opportunities for asynchronous interaction, but one creative example that was well-received was mandatory participation in a discussion forum.
Accountability: Students acknowledged that they are responsible for motivating themselves and holding themselves accountable. However, they also expressed appreciation for “nudges” that were built into the course to keep them engaged. Examples cited include periodic deadlines, live sessions, reflection assignments, discussion fora, and low-stakes quizzes. In courses without at least one of these features, students often commented that the lack of accountability led to them feeling unmotivated and disengaged.
Time Commitment: Students in some classes reported a significant increase in the time commitment and volume of work required following the transition to remote learning, which was unexpected and often led to negative feelings about the course. Students expected the length of mandatory viewing assignments to be approximately equal to the amount of time they would have physically been in class on campus. When this threshold was greatly exceeded, students expressed that they felt their time was being disrespected by the instructor.
Most people like opportunities for live interaction and students expressed dissatisfaction when such opportunities weren’t available. Synchronous sessions appear to have been primarily held using Zoom although some instructors instead used Panopto. These sessions took a variety of forms: lectures, lectures combined with discussions, and question and answer sessions, among others. Some students reported being burned out by the length of the lecture sessions, feeling that their attention span was shorter for online lectures than for in-person lectures.
Recordings: Recordings of synchronous sessions were extremely important for students who could not attend the live session due to time zone differences or who unexpectedly had wifi issues during the scheduled time. Even students who attended the synchronous sessions often mentioned their appreciation for being able to reference the recordings later and to revisit any concepts they found difficult.
Full Class Sessions: Some professors encouraged students to use the Zoom chat feature to ask and answer questions. Zoom chat and Zoom polling received great feedback due to their ability to simulate a normal class environment and keep students engaged. One professor used an iPad to draw organic structures during live sessions, simulating what he or she would have drawn on the board during an in-person class session. Students very much appreciated the live aspect of these iPad drawings and gave positive reviews to this practice. Some professors offered extra credit for participation during live sessions. The consensus was that offering opportunities for interaction led to more engaging and stimulating class sessions.
Breakout Sessions: Breakout sessions received overall mixed reviews although in some classes, they were very well-received. Negative feedback mostly fell into two categories. First, there were some issues with the act of assigning students to their breakout groups. In some cases, professors used pre-assigned breakout groups and, if attendance wasn’t strong, students could end up alone in a breakout room. In other cases, professors assigned the groups during the live sessions, causing downtime and creating an appearance of disorganization. Second, in some classes, the students would be sent to the breakout rooms and would, in some cases, sit in silence with no student willing to initiate or facilitate conversation. It might be useful to share a Google doc with discussion prompts and conversation goals for the breakout sessions. Additionally, it may help to assign a discussion facilitator for each group in a simple way, such as appointing the person in each group who has the next birthday or the person whose last name is alphabetically first.
Providing recorded materials (either intentionally asynchronous or recordings of synchronous sessions) was generally well-appreciated. Students often mentioned the value of being able to review recorded examples of worked problems and to revisit complex topics more than once. Downloadable slides and/or notes pages were also appreciated when they accompanied asynchronous video content. The one frustration commonly noted was that it was difficult to ask clarifying questions while watching asynchronous videos, highlighting the importance of having opportunities for live interaction and quick response times to emailed questions.
Posting Schedule: Students noted that it was helpful for materials to be posted on a regular schedule, as opposed to being posted all at once. According to students, this helped to increase their engagement, as they had to continue to return to the course material each week, rather than rushing through all of the material at once to complete the course requirements.
Quality: It was extremely important that recordings had high audio quality. It was also important that things that need to be seen (e.g., writing on a board) were large enough to be clearly visible.
Length: Students provided mixed feedback regarding the preferred length of the videos. Some students thought they were too long at around an hour, while others seemed comfortable with videos that replaced a typical in-class lecture. Students commented that the inclusion of humor and/or guest speakers helped to break up longer videos. In classes where they were provided, there was positive feedback for several shorter videos as opposed to one longer video.
Accountability: Students seemed to appreciate mechanisms put in place to keep them accountable for engaging with asynchronous content. In courses without these mechanisms, students often commented that they wished they had something to keep them more accountable. Students tied accountability to engagement. In classes with no accountability mechanism, students reported being less likely to watch the videos or pay close attention to them and an overall lower level of engagement. The following are some examples of accountability mechanisms that were detailed in the comments:
- Requiring written reflection assignments for each asynchronous video.
- Requiring participation in a discussion forum. Students were required to respond to open-ended prompts and to each other. The instructor and TAs were active in the forum discussions.
- Giving low-stakes quizzes over asynchronous content. Some students provided feedback that the quizzes helped them focus on the most important points, as defined by the instructor.
Office hours were greatly appreciated by students and were a vital interactive component of classes where content was delivered asynchronously.
Timing: It was important to ensure that there were sufficient options for people in various time zones. Some students liked when office hours were held during the regularly scheduled class time because they liked hearing other students’ questions and answers and also enjoyed the community feel.
Group Size: Students sometimes found it difficult to ask questions in office hours that had a lot of active attendees. A good solution might be to have a mix of a class office hour period and one-on-one office hours or small-group office hours. Online sign-ups seemed like an effective way to allocate office hours for one-on-one or small-group sessions. This can be done in Google Calendar using the Appointment Slots functionality.
Students reported that deadlines for assignments were not always reasonably flexible to accommodate a remote instruction setting where some students might have had unforeseen difficulties with internet connection or a shared family computer.
Time: The window during which the exam is available must be sufficient to accommodate various time zones. Some students also expressed frustration with the time allotted for taking the exam, as the time limit may need to be longer than is typical to account for the challenges of taking an exam in a new format and the associated stress.
Questions: It is extremely important that the instructor has communicated to students how to ask clarification questions during exams and that students are able to do so. Students who mentioned having difficulties with this typically seemed very dissatisfied with not knowing how to ask a question.
Grading: Some students were frustrated that, in a class that had awarded partial credit for exam problems in the first half of the semester, the final exam format allowed only for the online submission of final answers, eliminating the opportunity to earn partial credit. This might be mitigated by allowing for the submission of scanned work pages, though this does impose an additional technology burden on the student who would need access to a scanner or smartphone. Exams graded on a curve were sometimes a source of additional stress because students who are committed to the honor code were afraid other students might cheat during a non-proctored exam, negatively impacting honest students’ grades. These students mentioned that they would have appreciated an open note exam to ensure a level playing field or a decision not to grade on a curve.
Students found geographically separated group work to be challenging, both because they are used to working with one another in person and because time zone differences made coordination difficult. Additionally, some students’ technology limitations made it difficult for them to participate effectively in group work.
Students expressed appreciation for instructors who incorporated content about COVID-19 in a way that was relevant to the course.
Expectations: It is important to ensure that students have a clear understanding of the expectations for the course. It is helpful to explicitly state which videos, lectures, and homework assignments are mandatory and how they correspond to one another. In courses where the structure changed substantially following the transition to remote learning, some students indicated that they would have appreciated an updated syllabus reflecting the new expectations.
Communications: Email seemed to work well. Students responded favorably to weekly emails from instructors outlining the materials, expectations, and deadlines for the week. Similar emails corresponding with a class’ typical meeting days were also well-received. Professors should try to avoid sending multiple emails in a single day. Students placed a high value on prompt responses to their attempts to reach out to instructors.
Tone: Numerous comments expressed appreciation for their instructors’ encouragement, understanding, empathy, kindness, and flexibility. Instructors who kept a sense of humor in personal communications or during class were also met with positive feedback.
Zoom was the most frequently mentioned technology and it seemed that students were largely comfortable with it. There was also a lot of praise for Panopto videos, as students liked the ability to see multiple video feeds at a single time, use text search within the recorded audio, and dynamically type notes so that they were tied to a single moment in the video. Being able to pause, rewind, and control play speed were very important to students and if an uncommon software is used, there is likely value in spending time ensuring that students know how to properly control the video player. Students preferred videos being posted on a site, as opposed to receiving them via email and found it overwhelming when courses required interacting with a number of online resources on a variety of platforms. There seemed to be a strong preference for a central hub in which all required resources were posted. The most commonly reported central hub used was Sakai. Students largely empathised with instructors who were learning the technology as quickly as they could given the circumstances, but it was clear that instructors with better command of the technologies being used appeared more organized to the students.
The labs we reviewed took different formats, but were generally positively reviewed by their students. The following are examples:
- One engineering lab shifted all exercises to be post-processing and data analysis. The instructor would give a lecture on the theory, followed by a TA performing the experiment. Some students expressed disappointment with not being able to do the experiments themselves, but most indicated they thought the overall format was reasonable.
- A physics lab tried to convert the lab activities to allow students to do experiments at home with everyday items. In cases where the student could not access these items, they were able to devise their own experiments achieving the same learning goals.
- Some labs were simulations followed by Zoom breakout sessions for lab groups to collaborate. One piece of software used for these simulations was PHeT, which received mostly positive reviews.
- One class mailed students equipment to do the labs, which students appreciated. They particularly enjoyed the opportunity to conduct experiments of their own design with the equipment as well. Unfortunately, some of the equipment wasn’t well-calibrated, so data didn’t always match the expected results.
Collaboration: Some students provided feedback that indicated they may need more guidance on how to work effectively in a remote group setting. The most commonly mentioned settings include breakout sessions, team project work, and general homework/test study sessions. In all of these contexts, students would likely benefit from additional guidance on tools and approaches that can make these experiences more successful. Approaches to improving these interactions include explaining expectations for breakout sessions, sharing a Google doc with discussion prompts, and/or randomly assigning a group leader (perhaps by indicating the person with the most recent birthday, or whose middle name is first alphabetically should facilitate the conversation). Students might also benefit from advice on how best to leverage technology in their collaboration. For example, the instructor can advise if the best approach is for one student’s screen to be shared or if all students should contribute to one shared Google doc.
Studying: Some students noted that they are used to working and studying with other students and didn’t feel like they had a good mechanism or substitute for that. As a result, they didn’t feel that they grasped new concepts as fully as they had prior to the transition to remote learning. In courses where worked example problems were available for review, this seemed to help students. Tips on how students can continue to collaborate with one another remotely would likely be much appreciated by those students who thrive with these types of interactions.
Schedules: Some students reported difficulties adapting to the more flexibly scheduled course work after the shift to remote learning. Some reported that they wanted to maintain a similar schedule to their on-campus time by watching their required videos during their scheduled class time. However, this became challenging when classes offered synchronous sessions during regular class time in addition to videos that needed to be watched. Tips and advice from advisors or others to build an effective remote learning schedule may be beneficial for some students.