Peer review is a long-standing practice in academia, used for professional development, annual evaluations, and promotion and tenure review. Along with student evaluations, portfolios, and letters of recommendation, peer review can strengthen the evaluation process as faculty peers are qualified to evaluate aspects of teaching such as course content, course objectives, and instructional methods and materials (Courneya, Pratt, & Collins, 2007).
Iqbal (2013) and Ponte (2013) both identified difficulties associated with the faculty evaluation process including subjectivity, lack of validity, unreliability of evaluation, unprepared peer evaluators, and criteria used for evaluation. Suggested improvements to the process included the development of more objective value measures of work. This article describes the development of an evidence-based peer review instrument for online teaching.
LITERATURE REVIEW
The literature indicates that constructivist principles of learning are frequently the basis for online education design and development and, as such, should guide evaluation. Chickering and Gamson's (1987) seven principles of effective teaching served as a framework for initial efforts in the development of online education. The principles of high expectations, effective student-faculty contact, prompt feedback, cooperation among students, active learning, time on task, and respect for diverse talents and ways of learning are time-tested principles reflected to this day in online courses (Sowan & Jenkins, 2013).
Under a 2003-2006 Department of Education Fund for the Improvement of Post-Secondary Education grant, the Quality Matters (QM) Program was developed to identify standards of best practices in the design and development of online education. Another organization, Learning Resources Network (LERN), dedicated to excellence in online education, also publishes best practices. Both QM and LERN are founded on constructivist principles and the community of inquiry (COI) framework.
The COI framework was developed by Garrison, Anderson, and Archer (2000) to guide research into online learning. In the COI framework, effective online learning experiences happen through the core elements of cognitive, social, and teaching presence. The framework has been validated through research, and a 34-item COI survey that operationalizes the elements was developed (Arbaugh, 2007; Arbaugh et al., 2008).
According to Arbaugh (2007), cognitive presence allows participants to construct meaning that is transferrable and to engage in practical inquiry. It is essential to critical thinking and accomplished through sustained communication and reflection. Social presence describes the ability of participants to be seen as real people through identifying with the community, communicating purposefully, and developing relationships. Teaching presence consists of course design and organization, facilitating discourse, and direct instruction.
The literature most frequently referenced on peer review of online teaching in schools of nursing lists the QM rubric and the core competencies for nurse educators (National League for Nursing [NLN], 2005) as the basis for evaluation criteria. The QM program provides a faculty-centered peer review process designed to certify the quality of online and blended courses for institutional and individual subscribers using a continuously improved rubric. The NLN core competencies and accompanying task statements provide a way for nurse faculty to demonstrate the depth and diversity of the faculty role in the academic setting.
Online course peer review by faculty has been discussed in two reports. Little (2009) used the University of South Florida College of Public Health online course standards and the QM peer course review rubric in a pilot study to examine the use of online course standards for peer review. A team composed of a nurse educator and an instructional designer used the two instruments to evaluate standards in two online courses in an RN to BSN program. Results indicated that both instruments measured similar elements, but the QM tool was easier to use and provided references and correlation with accreditation standards. Findings also identified the need for additional professional development of faculty in online pedagogy.
Gaskamp and Kinter (2014) shared the development of their school of nursing's online peer review based on the QM and University of South Florida College of Public Health's rubrics. This evaluation rubric is a 40-item 4-point Likert-type checklist that includes a narrative section to expand upon strengths, areas for improvement, and additional comments.
INSTRUMENT DEVELOPMENT
Identification of evidence of best practices led a committee of nurse faculty to review the NLN core competencies for nurse educators (NLN, 2005), and LERN best practices. A review of the 2011-2013 online research literature by QM affirmed the use of the COI framework. As a result, the COI framework and core competencies served as the organizing basis for the peer evaluation instrument. On the basis of the three core elements of the COI framework, headings of teaching presence, social presence, and cognitive presence were used for development of the instrument.
Best practices of online education from LERN (Boettcher & Conrad, 2010; Draves, 2013) were incorporated under each applicable heading. Not all NLN core competencies (2005) were relevant to peer review of the online classroom, but selected competencies and task statements related to facilitating learning, facilitating learner development and socialization, and use of assessment and evaluation strategies were applicable. Additional behaviors specific to the school of nursing were also incorporated.
The instrument uses a rating scale of Meets Expectations, Does Not Meet Expectations, and Not Applicable for each behavior listed under the accompanying heading. The first heading, Teaching Presence, includes design of course materials and basic course-related administration. Examples of expected behaviors include: the course syllabus, learning objectives, calendar, and guidelines are clearly stated and provide clear guidelines on expectations; various teaching methods are used; interactive technology is used to diversify content and increase interpersonal interaction; students are provided with prompt feedback on assignments; and the course is set up in a such a way that creates interest and provides ease of accessibility.
The second heading, Social Presence, includes activities that help facilitate emotional connections, trusting relationships, and identification with group members. Examples of expected behaviors include: having an obvious faculty presence that is consistent throughout the course; faculty communication that is positive and supportive in tone and shows respect to learners; opportunities for student interactions with faculty and fellow classmates through discussion, assignments, and identified channels of communication; and implementation of a course welcome and wrap-up.
The third heading, Cognitive Presence, includes activities that lead students to construct meaning that is transferrable and to engage in practical inquiry. Expected behaviors include: customized and personalized core concepts; encouragement of analysis of ideas and content by faculty; discussion posts that invite responses, questions, discussion, and reflection; a variety of groups, large and small, incorporated into learning experiences; and individual learner work experiences incorporated into learning activities. The completed instrument will provide an objective peer evaluation on the expertise of the faculty member in the online classroom.
The peer review will take place as part of the promotion and tenure process by a member of the school's promotion and tenure committee. The instrument will be reviewed annually through the faculty committee structure for its effectiveness in evaluating faculty online teaching. (Supplemental Digital Content is available for this article at http://links.lww.com/NEP/A5.) Feedback will be sought from members of the promotion and tenure committee who have utilized the instrument.
DISCUSSION
Development of an evidence-based tool for peer review has implications for practice for both the faculty peer reviewer and the faculty member being reviewed. First, an objective and concise instrument based on evidence will increase faculty competence to serve as a peer reviewer. The reviewer will know what elements to look for in an online course that demonstrate effective online faculty teaching, thus eliminating some of the subjectivity that can go into a peer review.
Second, basing the peer review instrument on the COI framework and best practices for online education assures that identified expected behaviors are relevant for evaluation of faculty teaching in online courses. In addition to organizing and designing a course and learning environment, faculty teaching online courses must communicate purposefully and facilitate development of relationships in the online classroom.
The developed instrument for peer review may be utilized by others in academia who are seeking a more objective peer review process. The instrument provides an innovative tool to ensure that peer review encompasses all aspects of the online teaching role.
REFERENCES