Nationwide the majority of nursing instructors, whether hospital or academically based, are responsible for teaching various forms of online or hybrid courses. The American Association of Colleges of Nursing reported that more than 60% of associate to bachelor's degree nursing programs are offered at least partially online.1 There are several benefits associated with teaching in a digital learning environment (DLE) including (a) increased access for students to high-quality nursing programs especially in rural areas, (b) expanding ability to collaborate regardless of geographic area, and (c) improving technological literacy.2 With a dramatic rise in the use of DLEs, faculty are examining innovative ways to engage in cost-effective, efficient reviews of courses while meeting accrediting agencies requirements for continuous quality improvement.3 Often when revising and enhancing courses, faculty rely on student satisfaction surveys, which may be biased and unreliable.4

Many universities have adopted Quality Matters (QM), a peer-reviewed faculty-centric online course assessment with standards and associated rubric.5 The QM rubric and review process were developed by MarylandOnline Inc Consortium addressing increasing concerns over the quality of online education. The rubric is useful when assessing DLEs for quality and course structure.6-8 When applying the rubric, faculty ensure that course learning objectives, assessment and measurement techniques, instructional materials, course activities, learner interaction, and course technology are correctly aligned and integrated. The rubric contains a set of criteria with assigned point values to determine whether established standards have been met. For a course to be deemed QM certified, all standards must be met.8 In essence, the rubric becomes a guide driving course structure, navigation, and component alignment. The rubric can be used as a course quality improvement tool, and there is evidence suggesting that application of the QM rubric increases positive student outcomes including course grades.9 However, using QM as a standalone tool does not guarantee increased student retention or satisfaction with courses.10 Therefore, it is important to use other evaluation methods along with the QM rubric.

Many higher learning institutions do not use instructional designers, leading to inconsistencies in content delivery.11 Introducing the rubric to faculty during early stages of course development or when classes receive low student satisfaction scores may lead to improved quality and better student learning outcomes. Of note, there are associated costs for universities adopting QM including subscription fees and training.

The purpose of this article is to describe and recommend integration of a QM assignment for nurse educator (NE) students in a graduate assessment and evaluation course. In this unique learning experience, NE students are given an opportunity to enter a living DLE and provide faculty real-time assessment of course methods, demonstration of learning outcomes, and written feedback for course improvement. This activity benefits graduate students evaluating courses as well faculty undergoing evaluation.

QM as an Innovative Learning Activity

In an effort to increase exposure to DLEs and options for addressing quality in online learning platforms, faculty developed a QM learning activity for master's degree students enrolled in the NE specialty. Graduate students in the assessment and evaluation strategies course learn to interpret evaluation results to generate, plan, and design change for learners, courses, and programs. The specific intent of this assignment is to familiarize students with the capabilities and limitations of a DLE and application of QM standards to actual (real-time) online and hybrid courses. The focus of the assignment is on QM standards 1 to 5 as these are most closely linked to course objectives. Currently, our university subscribes to QM, with certified instructional course designers available to complete assessments. Permission to use the rubric for graduate studies was obtained from QM because the university's license is for faculty use only.

At the beginning of the course, NE students are given preparatory materials to familiarize themselves with the application and use of the QM tool. There is an open question and answer period following the review of the materials as the activity is graded. Simultaneously, faculty teaching the assessment course solicit volunteers among other instructors in the nursing program willing to allow an NE student to engage in a QM course review. Faculty volunteering courses are then randomly paired with NE students. Once paired, NE students email the course instructor requesting enrollment as an external evaluator (visiting faculty) restricting access to all sensitive, graded materials. Once in the course, NE students analyze the syllabus, course objectives, and assessment methods while applying the QM rubric (for standards 1-5). Based on their analysis, students also (a) describe any issues or concerns with course objectives and how they link to the mission/vision/values of the college and university, (b) assess and describe methodological issues including various ways students demonstrate learning outcomes associated with objectives, and (c) explore and describe any associated grading rubrics.

Finally, NE students submit with the narrative a scored QM rubric in table form. If QM scores are lower than maximum for the standard or weaknesses are noted in the course, the students are required to submit cited, evidence-based recommendations for suggested course changes linked to the specific QM standard. Ungraded assignments are returned to the faculty volunteer for course improvement purposes. The only restriction is that the course under review must have already occurred or not be associated with the NE students plan of study.

Student Learning and Faculty Feedback

Ongoing, systematic peer review processes within courses and programs are paramount to positive learning outcomes. The QM process is intended to be peer-driven, diagnostic, and collegial-not evaluative and judgmental. When this assignment was initially developed, there was some hesitancy by faculty to volunteer their courses for evaluation. However, faculty feedback of these reviews has been overwhelmingly positive. This is potentially due to the nonthreatening approach as these reviews are not associated with tenure, promotion, or merit. Anecdotally, there is now typically a waitlist to have a course reviewed by an NE student, reflecting faculty buy-in to the activity.

Faculty have incorporated suggested changes and immediately emailed to be on the waitlist again ensuring ongoing opportunities for NE students and improved course quality. Similarly, NE feedback on course evaluations is positive, with students excited about the opportunity to assess a "real" course and connect with faculty outside their program of study. Over the course of the last 4 years, NE students have reached out independently during the assessment to alert faculty to incorrect grading methods (percent/point errors in the DLE), broken embedded links, missing documents, errors in tests, and so on. Faculty response to this is overwhelmingly positive.

In this unique collaborative learning activity, NE students are given an opportunity to enter a DLE, complete an assessment, and provide evidence-based feedback for course improvement to faculty volunteers. The assignment is intended to be flexible and has changed to meet the needs of learners. Initially, students were assigned to courses only within our college. Based on feedback, students may now evaluate any hospital/community course using a DLE. Finally, there is an associated cost with the use of QM. However, graduate faculty choosing to use this activity could simply implement their own program's peer evaluation processes into the assessment course, allowing students the opportunity to practice and apply evaluation strategies while in the student role.

References