Rubrics With Rox

At the end of the course, I had the pleasure of working with Rox on this piece of writing so it is written in the third person.

During the third week of the Principles of Learning class, we were introduced to Dr. Nieminen, who is an Assistant Professor at the University of Hong Kong, as well as Honorary Research Fellow at Deakin University (Nieminen, 2021). Dr Nieminen felt that the use of rubrics best addressed the needs of students for formative assessment and summative evaluation. He demonstrated his meaning with a slide presentation during which he had us question what was represented by the concept of knowledge (Nieminen, 2022). This was an integral part of his lesson because the way that knowledge is defined and assessed by an instructor is important. Under Dr. Nieminen’s tutelage, our objective in Week 3 was to begin the process of creating a rubric using specific parts (in bold) of two Learning Expectations from the course (Ruttenberg-Rozen, 2022):

Learning Outcome 5. Interrogate identity and beliefs about education, learning and knowledge. G4. Advance inquiry into authentic, ill-defined educational problems, recognizing the complexities of multiple components of a problem/solution in relation to the whole.

Learning Outcome 8. Prepare scientific communication materials that effectively address and engage a variety of audiences. C3. Prepare educational materials that effectively address and engage a variety of audiences (academic, professional, and general) for a range of purposes, including presenting and synthesizing research, informing, instructing, and persuading.

Dr. Nieminen reminded us that the rubric should focus on what was important which was not always the same as marking what was the simplest element to assess. He asked us to consider the clarity and reasonableness of the rubric and advised that co-created rubrics were often just that.

We were invited to copy a shared template that could be used as a starting point in our own rubric development. After a period of time allotted to work independently on our concepts, and reflect on our products, we were invited to show our efforts to both professors involved, Dr. Ruttenberg-Rozen and Dr. Nieminen, for feedback. We were also invited to share our final work with a group in a breakout room during a subsequent Zoom class. Later we were asked to create an infographic to summarize what we had done.       

Reflection

In Wendy’s enthusiasm to create a useful rubric, she reviewed the Learning Outcomes described in the course syllabus as being pertinent to the Artifacts project. There are four Outcomes to be used as a specific focus (Learning Outcomes 1, 2, 3 and 8), with nine sub-sections (Ruttenberg-Rozen, 2022). Wendy created a rubric that included all of these parts. When it came time for sharing with the two professors, she submitted the rubric in two formats. Positive feedback was received but no direction on how to improve and no comment was made of the number of Outcomes and sub-sections. Because of that, Wendy felt the rubric was adequate for the task.

Similarly, Rox created a rubric with all of the relevant Learning Outcomes in the syllabus as well. On the date chosen for reviewing rubrics in the breakout room on Zoom, the one designed by Rox, with colours for clarity was held up as an example of a thorough and complete piece of work. Because Rox and Wendy share everything that could make the load lighter for others, it was agreed that if anyone from the group was interested in using all or parts of Rox’s rubric, they were welcome to do so.

As time in the Principles of Learning course drew to a close, Wendy and Rox met to review assignments and work that was being submitted for final evaluation. As they looked over Dr. Nieminen’s slides and discussed the implications of rubrics for both students and teachers, they realized that they had made a significant miscalculation. There were simply too many expectations in the rubrics to be practical for evaluating the volume of assignments that each student had to hand in. If Dr. Nieminen had intended two Learning Outcomes to be assessed for each of the twelve artifacts, then there would be twenty-four separate calculations for each student. Using Rox or Wendy’s rubrics, with nine areas there would be 108 separate assessments for each student; far too many to respectfully expect a professor to manage in the marking turn-around time and excessive for use as a self-evaluation.

According to Bearman and Ajjawi (2021), rubrics are most effective when they are co-created and the student has multiple opportunities to use them. The repeated use of the rubric allows for ever-increasing familiarity and is conducive to deep reflection on the lessons learned. This contemplation is of particular import to both Wendy and Rox as they intend on moving forward with their graduate degrees and can benefit from this as an exercise.

A salient point brought up by Rox in Perusall when reading the Bearman and Ajjawi (2021) article was that co-created rubrics are an important part of our learning at the graduate level. They give us a “communal understanding” of what is important in the course (Hayward, 2022).

In a study done on the use of co-created rubrics (Fraile et al., 2017), it was found that students did not necessarily perform better when using rubrics over time but they did develop better self-regulation skills when the rubrics were utilized as part of formative assessments. That study also indicated that there was little research on co-created rubrics.

After the last class, Rox and Wendy met again to discuss the rubric assignment, the work they had completed, and the work that they had yet to do. The thought of evaluating all of these items for each of the students seemed a daunting task for even the hardiest of professors.

The rubric was redesigned. The new rubric was intended to be used to mark student growth over the twelve-week Principles of Learning course. Not every Artifact write-up should be graded using this rubric. Instead, it should be used to mark the best demonstrations of growth within the collection of Artifacts and Reflections (See Figure 1 on Page 5). Wendy and Rox had long conversations as to the effectiveness of rubrics in general. Rox felt that the majority of words on a rubric are superfluous; describing things that the learner is not doing. Co-constructed single-point rubrics are more clear and more effective as a learning tool for assessment. In a Perusall response to an article entitled, Experiential Learning Theory: A Dynamic, Holistic Approach to Management Learning, Education and Development by Kolb and Kolb (2009, Chapter 3), Rox explained learning as akin to house construction. In that analogy, one could consider a measuring tape as a tool used to achieve the construction of a completed house as the outcome (or in this case, the learner’s growth). Rox carefully chose his words saying: “When a measure becomes the goal, it stops being a useful metric” (Hayward, 2022).Wendy has a completely different point of view, not seeing a rubric as the outcome, but rather the stepping-stones to arrive at the outcome. She likes rubrics because of their clarity, especially when they include exemplars. In meetings with Grade Team Partners as part of the School Improvement Plan, she co-created rubrics with staff and piloted them to ensure that the resulting marks would be consistent across projects and Primary level classes. By working as a team, the teachers were able to coordinate samples of what a students’ work should include and look like across grades. There were three goals for this project: consistency in explaining tasks and marking, clarity of expectations for the students, and the provision of models for parents in the community who were very active in working with their children at home. When the rubric and exemplars were posted in a classroom, it had the added benefit of providing supply teachers and guests with an understanding of what was required in the specific task; the look fors, as it were. In a primary classroom, creating rubrics that were accessible to the students was key. Wendy consistently discussed the Ministry Expectations with the children. At the beginning of Grade 1, the children knew that a sentence started with a capital and ended with punctuation. It contained a message in words made up of groups of letters that made sense when put together (even if they were not spelled correctly), with spaces between words to assist the reader in making sense of the idea being expressed. The students were taught to use a check list that included all the required parts, (providing pictures and words for those who could not yet read) and imparting to their parents what they needed in preparation for helping their children with the next assignment. The students were encouraged to post their completed sentences on the bulletin board as samples of each level but were also allowed to pass, or not post if they chose. Each piece was celebrated regardless of the rubric level because it helped them and others understand the growth process.

It would be nice if tasks didn’t need to be rated but the reality is, most people want to know what they did well, what they could do to make their contribution better, and how their work fits into a hierarchy of levels. Wendy’s dog trainer circle of friends would not be satisfied if their skilled and beautifully maintained animals entered into shows were simply given a thumbs up by the judge. They want the ribbons that indicate their dog came in first, second, or third. Likewise, most artists who submit work to juried shows expect that their work will be judged on its merit so that medals and awards can be distributed accordingly. We live in a competitive world. Not knowing where we fit to the general order does not help us to set goals and move forward. In education, not having been graded does not help students achieve scholarships or the coveted supervisor for theses or dissertations. Rox continues to disagree.