{"id":1525,"date":"2023-05-26T15:53:31","date_gmt":"2023-05-26T15:53:31","guid":{"rendered":"https:\/\/blogs.kcl.ac.uk\/activelearning\/?p=1525"},"modified":"2023-05-26T16:02:12","modified_gmt":"2023-05-26T16:02:12","slug":"collaborative-reading-before-a-flipped-class-2","status":"publish","type":"post","link":"https:\/\/blogs.kcl.ac.uk\/activelearning\/2023\/05\/26\/collaborative-reading-before-a-flipped-class-2\/","title":{"rendered":"Collaborative reading before a flipped class"},"content":{"rendered":"<p><em>This case study is from <strong><a href=\"https:\/\/www.kcl.ac.uk\/people\/francisco-rodriguez-fortuno\">Francisco (Paco) Rodr\u00edguez-Fortu\u00f1o<\/a>, Reader in Physics at King\u2019s College London<\/strong>. Paco teaches a core module on Electromagnetism to around 160 second year undergraduate Physics students. The format is flipped, which means that the session that used to be a lecture is now dedicated to resolving difficulties and misunderstandings with reading that students complete in advance.<\/em><\/p>\n<h2><a name=\"_Toc135988271\"><\/a>What is it?<\/h2>\n<p><a href=\"https:\/\/www.perusall.com\/about\">Perusall<\/a> is a platform developed at Harvard University to make students\u2019 reading collaborative. I heard about Perusall from <a href=\"https:\/\/mazur.harvard.edu\/\">Eric Mazur<\/a>, the Harvard physics professor, when he visited King\u2019s. With Perusall I divide students randomly into groups and they follow my instructions to leave comments on my notes, upvote and start discussions. I can set deadlines and students get reminders by email. Perusall can generate grades (I\u2019ll explain why this is important later). It also gives me charts of engagement data. It\u2019s free if you give students freely available material to read or watch.<\/p>\n<p>I\u2019m using it for flipping. I need students to have read the same set of detailed lecture notes before coming to class.<\/p>\n<h2><a name=\"_Toc135988272\"><\/a>Why did you initiate this?<\/h2>\n<p>I needed to get the flipping right. Students were introduced to electromagnetism in Year 1, so what I didn\u2019t want to do is re-teach those concepts and risk losing the students who had already learned them. So, I wanted to improve their engagement and preparation to make sure they were coming to class with similar knowledge.\u00a0I also hoped they would help each other, because when I was studying and doing problem exercises, I was asking my friends all the time.<\/p>\n<p>The software made it easy to try this collaborative approach. An overarching feeling I have about Perusall is that you can tell it was made by lecturers. All the \u2018little things\u2019 that you need are there. I can easily give my teaching assistant the right level of access to the groups because there\u2019s a \u2018Mark as TA\u2019 setting. I can also set a page range of a document rather than the whole thing. \u00a0Commenting is easy \u2013 students highlight text or click on a specific part of a figure, and start typing. Students can type in <a href=\"https:\/\/www.latex-project.org\/\">LaTeX<\/a> maths notation as well as natural language, so they can comment using equations. I can easily see the comments in different ways, and I can draw students\u2019 attention to valuable comments in other groups. I can set the grading assignment by assignment or for the whole class, and I can make the criteria add up to more than 100% &#8211; that means there can be several different ways to get full marks. I can see the breakdown of each student\u2019s grade.<\/p>\n<h2><a name=\"_Toc135988273\"><\/a>How is it set up?<\/h2>\n<p>In Perusall, I set up the course. That generates a unique access code which I pass to my students with a link. I did that in the first lecture, they registered, and they were in \u2013 they only have to do that once. For each assignment they are randomly allocated into groups of 20.<\/p>\n<p>Then each week I upload the lecture notes (a PDF document of around 15 pages) into the Perusall library \u2013 that\u2019s drag and drop and very easy. Then I create a Perusall assignment related to those notes, with instructions. Each student\u2019s engagement is graded, and Perusall does this automatically based on my parameters.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1526 size-full\" src=\"https:\/\/blogs.kcl.ac.uk\/activelearning\/files\/2023\/05\/paco_perusall_01.png\" alt=\"Perusall grade settings, displaying criteria and editable fields for weighting and other options\" width=\"602\" height=\"296\" srcset=\"https:\/\/blogs.kcl.ac.uk\/activelearning\/files\/2023\/05\/paco_perusall_01.png 602w, https:\/\/blogs.kcl.ac.uk\/activelearning\/files\/2023\/05\/paco_perusall_01-300x148.png 300w\" sizes=\"auto, (max-width: 602px) 100vw, 602px\" \/><\/p>\n<p>The grading criteria Perusall uses are:<\/p>\n<ul>\n<li>A quality score based on Perusall\u2019s algorithm. I asked for three high quality comments and Perusall does a good job analysing the language. To gain full marks students also need to comment on different parts of the document.<\/li>\n<li>A score for opening the assignment at least twice (once to make comments and at least once more to read others\u2019 answers).<\/li>\n<li>A score for reading to the end.<\/li>\n<li>A score for active engagement time (commenting, interacting with the comments and voting)<\/li>\n<li>A score for upvoting others\u2019 comments.<\/li>\n<li>A score for quiz questions \u2013 I didn\u2019t use this.<\/li>\n<\/ul>\n<p>Then I set a deadline and release the activity. Students sign into Perusall and easily find their assignment. They can see the notes and then in an adjacent pane they have some tools for leaving comments, responding and upvoting. When I come back to read the comments, they appear to me as highlights or markers on my notes. When I click on each, I can read the comment thread and upvote any I think are helpful. My upvotes show as \u2018Upvoted by instructor\u2019 so students know they can trust the response. When I see a particularly good comment that I want all the students to see, I copy it to other groups because there isn\u2019t a \u2018make public\u2019 option currently.<\/p>\n<p>Then we have the lecture. I know from Perusall which parts were well understood by students, so I can focus on the difficult bits. These are the parts that students have not been able to clear up on their own. If I see a lively discussion with many mistakes, that\u2019s the kind of thing I will go through line by line in the lecture.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1527 size-full\" src=\"https:\/\/blogs.kcl.ac.uk\/activelearning\/files\/2023\/05\/paco_perusall_02.png\" alt=\"The Perusall annotation interface. Text in the left pane with comments displaying as highlights. Comments expanded in the right pane.\" width=\"602\" height=\"296\" srcset=\"https:\/\/blogs.kcl.ac.uk\/activelearning\/files\/2023\/05\/paco_perusall_02.png 602w, https:\/\/blogs.kcl.ac.uk\/activelearning\/files\/2023\/05\/paco_perusall_02-300x148.png 300w\" sizes=\"auto, (max-width: 602px) 100vw, 602px\" \/><\/p>\n<h2><a name=\"_Toc135988274\"><\/a>Considerations<\/h2>\n<p>Students can only see the comments by other students in their own groups. This gives them a manageable volume of comments to read, so that nobody\u2019s comment gets lost. I thought it would also be more intimate, and I hoped that the shyer students would feel less exposed. The group size of 20 was the Perusall default. When I asked the students whether groups should be bigger or smaller they told me 20 worked well and to leave it like that. With 20 there is good coverage of the notes and they also see a wide range of other students\u2019 comments. \u00a0I may still experiment.<\/p>\n<p>I followed Eric Mazur\u2019s advice and defaults in general, since I figured they were there for good reason \u2013 probably the result of a lot of experimentation \u2013 and I didn\u2019t want the first time to be a disaster. His strong advice was to grade the work, so I did that. I was generous with the grading approach. There were 10 marks available for each pre-reading and students got more than 7 if they spent time engaging with the document, left a comment and upvoted others.\u00a0 I\u2019m still thinking about how tough the grading should be.<\/p>\n<p>An unexpected outcome is the effect the usual pre-deadline rush has on the quality of the comments. In the preceding few days the comments are good, but in the hour before the deadline they become very low quality. I think this is students chasing marks. It\u2019s a problem with a spontaneous solution \u2013 the lateness acts as a filter. I leave students as much time as possible to comment, so the timing of the deadline is quite close to the lecture. This means I often don\u2019t have time to read the late, low quality comments. So if students want the benefit of their comments being read and possibly incorporated into the lecture, that\u2019s another incentive to post early. But they need to know this. I may increase the weighting to 20% next year and perhaps also ask for more comments to see if that motivates them more to comment early and improve the quality.<\/p>\n<p>I\u2019m also still thinking about how much I should respond to comments, and how long I should leave confused comments before I step in. I enjoy seeing them learning from each other, but wouldn\u2019t want students to learn the wrong thing from a thread if they stopped reading it before the group reached the right conclusion.<\/p>\n<p>One disadvantage is that some students have their own preferred note-making software, and if they use that then they can\u2019t collaborate and Perusall can\u2019t grade them. But there are so many benefits to collaborating that I think it\u2019s worth swapping. Perusall supports video now \u2013 threaded comments at different time points \u2013 so that\u2019s one possibility. I think my students prefer video, but video takes me more time to make. These are second year students, so I think they\u2019re mature enough for notes. I\u2019ve also spoken to them, so I know that if everything is videos then there\u2019s no variety \u2013 just hours of videos to get through.<\/p>\n<h2>How did you introduce it to students?<\/h2>\n<p>I told them it was an experiment I wanted to try. I think they could tell I was excited about it and felt excited too. I asked them to create accounts and sign in during the first session, so I could demonstrate what to do. They also had information about timing, grading criteria and everything I wanted them to do.<\/p>\n<h2><a name=\"_Toc135988275\"><\/a>How did you check students&#8217; learning?<\/h2>\n<p>I was able to check the two thirds who engaged by reading the comments, and also by looking at the charts that Perusall generates. I was surprised by how accurate the automated judgements about comment quality were. Looking at the upvoting tells me how many students share the same question, so I can pay attention to that. Students respond to each other without knowing the answer but trying to figure it out. Some of the discussions are very long and reading them gives me a strong feeling that students are really learning. When my TA and I upvote comments, they appear differently from students\u2019 upvotes. This lets students know we are paying attention.<\/p>\n<h2><a name=\"_Toc135988276\"><\/a>What benefits did you see?<\/h2>\n<p>When I let students know that their interaction would be graded, the time they spent with the notes jumped from an average of 35 minutes to an average of 1 hour 41 minutes. They are also asking far more questions than when I used Padlet. They questions are more specific than Padlet too, because students can attach them to the specific parts they don\u2019t understand.<\/p>\n<p>One unexpected outcome was how much I have improved my notes. There were some explanations that many students were commenting on, and I realised they were bad explanations. They were finding mistakes and helping me make the notes better. Another was the way the questions reminded me not to make assumptions about what students know \u2013 five upvotes on a request for me to define the concept \u2018anti-parallel\u2019 told me that I couldn\u2019t assume this was common knowledge. I also became more distinct in my notes. Sometimes the questions made me question my own teaching in ways I would never have done if I hadn\u2019t used Perusall. I found this very satisfying and a real advantage.<\/p>\n<p>I really enjoyed reading the comments and seeing the learning. Some students have told me they would like to see Perusall in all their modules.<\/p>\n<h2><a name=\"_Toc135988277\"><\/a>What advice do you have for colleagues trying this?<\/h2>\n<p>From the one sample I have to go on, my takeaway is make it graded. I made the first pre-reading of the module formative, but it didn\u2019t work because around three quarters of the students weren\u2019t doing it. So the following week I told them it would count, and it made a big difference to the numbers and also the time they spent with the notes.<\/p>\n<p>If in doubt, go with the platform defaults the first time round. I figured they exist for a reason, and they did work well for me.<\/p>\n<p>I recommended taking a bit of time in your first lecture to get students to register and make sure they can access the notes.<\/p>\n<p>Giving attention to the comments is important. With a class this size I would also say that if you yourself are not able to read and respond to the comments then aim to have at least two GTAs looking at them.<\/p>\n<p>It\u2019s good for students to have all the notes and comments in one place to refer to. Since Perusall allows you to set a page range for each exercise, you can potentially upload all your notes as a single PDF. This will save students time when they\u2019re revising.<\/p>\n<h2>Examples and resources<\/h2>\n<p>I did not need guides or tutorials. Everything was self-explanatory in the <a href=\"https:\/\/www.perusall.com\/about\">Perusall website<\/a>, and easy to figure out independently.<\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"mh-excerpt\"><p>This case study is from Francisco (Paco) Rodr\u00edguez-Fortu\u00f1o, Reader in Physics at King\u2019s College London. Paco teaches a core module on Electromagnetism to around 160 second year undergraduate Physics students. The format is flipped, which <a class=\"mh-excerpt-more\" href=\"https:\/\/blogs.kcl.ac.uk\/activelearning\/2023\/05\/26\/collaborative-reading-before-a-flipped-class-2\/\" title=\"Collaborative reading before a flipped class\">[&#8230;]<\/a><\/p>\n<\/div>","protected":false},"author":374,"featured_media":1527,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[49,50,57,22,32,4,61,33,29,38,36],"tags":[],"class_list":["post-1525","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-30-60-minutes","category-break-out-groups","category-case-studies","category-discussion","category-experience-all-levels","category-nms","category-online","category-preparation-higher","category-questions-from-students","category-reflecting-on-teaching","category-skills"],"_links":{"self":[{"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/posts\/1525","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/users\/374"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/comments?post=1525"}],"version-history":[{"count":4,"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/posts\/1525\/revisions"}],"predecessor-version":[{"id":1531,"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/posts\/1525\/revisions\/1531"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/media\/1527"}],"wp:attachment":[{"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/media?parent=1525"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/categories?post=1525"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.kcl.ac.uk\/activelearning\/wp-json\/wp\/v2\/tags?post=1525"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}