It is essential that nursing care be held to the highest standard to maintain patient safety and achieve positive outcomes. This includes regular assessment and care of intravenous (IV) drips and IV access, vital signs monitoring and emergency equipment, invasive devices, and hemodynamic monitoring systems as well as preparation for shift change to ensure a smooth handoff. In the transplant/surgical intensive care unit (ICU) in a large academic medical center, the new graduate nurses (NGNs) are expected to master difficult medical concepts and operate multiple complex devices after 12 weeks of orientation. Based on staff feedback, preceptor evaluation, and direct observation, it became clear that recent cohorts of NGNs were not consistently practicing to the unit standards of care. For example, NGNs would often fail to change expired central line dressings and IV tubing, which would put patients at risk for serious infectious complications. This was observed to be an issue across multiple NGN cohorts, and so it was determined that there was opportunity to reexamine the unit-based NGN orientation program.
Discussion with unit leadership, preceptors, and former cohorts of NGNs revealed that although NGNs were trained how to complete the tasks necessary to practice to the unit standard of care, they were not provided with any standardized method with which to reliably remember to complete these tasks. Experienced nurses complete these tasks from memory gained through years of repetition; however, it is not reasonable to expect the same of NGNs being oriented to the ICU environment.
Rationale
According to Patricia Benner's novice to expert theory, NGNs are classified as "novices" and are by nature task-oriented and rely on rules to guide their actions. They lack the ability to prioritize tasks to focus the overall clinical picture of the patient (Benner, 1982). NGNs often become entirely focused on remembering to complete tasks and consequently forget to pay attention to more important external demands, such as subtle changes in vital signs. Their ability to think critically is extremely limited, and they are unable to develop these skills until they master routine tasks (Benner, 1982).
The use of checklists in an ICU NGN orientation program is an ideal way to provide a standardized tool for NGNs to help them learn to practice to the standards of care. The tasks that must be completed to meet these standards, though seemingly mundane, are critical to maintaining patient safety, high-quality care, and patient/family satisfaction. In a healthcare system where reimbursement is closely linked to patient outcomes and satisfaction, it is essential for NGNs to be provided with standardized tools that ensure they can practice to the standards of care by the time they have completed orientation.
Available Knowledge
Checklists have long been incorporated as common practice in the aviation industry to ensure the safety and operability of complex systems. The integration of checklists into the standard workflow in aviation reduces variability and the risk of human error by meticulously listing each action that is required to be performed for each stage of flight. It has been previously argued that the use of checklists in health care can similarly elevate quality and safety (Clay-Williams & Colligan, 2015). This is especially apparent in the ICU setting, where the extreme complexity of patient care makes it impossible for healthcare professionals to provide care safely and effectively without the use of standardized memory aids, such as checklists (Gawande, 2007).
Clay-Williams and Colligan (2015) discuss the concept of a "normal checklist," which helps to standardize performance and organize a series of tasks that are too long to be memorized reliably, especially in an environment where interruptions are common. In the airline industry, checklists are such a seamless part of the workflow that their completion does not impose additional burden upon staff. Given the unpredictable and diverse nature of caring for human beings, the use of checklists in health care have been argued as best used as memory aids for simple, standardized, and time-critical tasks (Clay-Williams & Colligan, 2015).
Checklists have already been successfully implemented in a variety of settings in health care. The implementation of a checklist was found to increase the rate of a successful and timely implementation of a therapeutic hypothermia protocol in one ICU (Avery et al., 2015). Another meta-analysis concluded that the use of a surgical safety checklist can improve patient outcomes, team communication, and safety (Lyons & Popejoy, 2014). In the multicenter Keystone ICU Project conducted in Michigan, a simple five-item checklist was integrated into standard practice during central line insertion to reduce the prevalence of central line-associated bloodstream infections (CLABSI). Results demonstrated that the use of this checklist reduced rates of CLABSIs by 66% (Pronovost et al., 2006). The results of this study were so compelling that the Agency for Healthcare Research and Quality expanded this intervention to 1,100 ICUs nationwide, demonstrating a 40% CLABSI reduction (Agency for Healthcare Research and Quality, 2012). Finally, the use of a checklist during morning rounds in an ICU increased the appropriate implementation of 14 best practice guidelines, increased throughput, and resulted in an earlier initiation of physical therapy (Byrnes et al., 2009).
There has been less research conducted on the effectiveness of checklists used in NGN orientation programs. One study conducted in a surgical ICU in 2007 utilized skills checklists to evaluate NGN readiness to progress to subsequent stages in their orientation (Chesnutt & Everhart, 2007). However, these checklists were designed to ensure that NGNs had received training and mastered certain skills, rather than focused on completion of specific daily tasks. Most of the research regarding NGN orientation program improvement initiatives involves preceptor development and training (Hickerson et al., 2016; Senyk & Staffileno, 2017). One review evaluated various strategies utilized in the literature to develop critical thinking skills in NGNs and concluded that preceptors were critical to developing the critical thinking skills of NGNs, regardless of strategy used (Schuelke & Barnason, 2017). The study also noted that preceptors have little formalized training in education and require significant support from management to be successful (Schuelke & Barnason, 2017).
Specific Aims
The aim of this study was to assess if the use of two checklists, one designed for use at the beginning of the shift and one to be used at the end of the shift, compared to the previous practice of using no checklists, demonstrated an increase in NGNs' perceived ability to consistently practice to the unit standards of care. A secondary aim was to assess if preceptors felt that the use of the checklists improved the NGNs' ability to practice to the standards of care.
METHODS
Context
Consideration of the contextual elements of the transplant/surgical ICU's unique NGN program was critical to the development and deployment of the beginning- and end-of-shift checklists. The transplant/surgical ICU NGN orientation is an approximately 12-week program that combines didactic and clinical learning. During the first several weeks of orientation, the focus is to build on basic nursing skills and learn about the unit standards of care. Half of the NGNs begin their training on day shift, whereas the other half begins on night shift. At approximately Week 6 of orientation, the two groups switch shifts to have the opportunity to experience both the day and night shift workflows. The orientation timeline and structure are tailored to the needs of each NGN as much as possible to ensure that they are safe and competent ICU nurses by the time they complete orientation.
In the initial 2-3 weeks of orientation, the NGNs are assigned one low-complexity patient who does not require multiple interventions. Based on preceptor feedback and NGN level of comfort, the NGNs are gradually exposed to more complex patients and equipment. At Weeks 6-8, the NGNs begin to take two patients at a time or "paired" patients. For the remainder of their orientation, they will focus on "pairs" and high-acuity one-to-one patient assignments. In the latter half of the orientation, the goal is for increased independence of the NGN and a shift of the role of a preceptor from that of a partner to that of an observer and resource.
The NGN is paired with an average of four preceptors over the course of their orientation, although this number can increase because of staffing needs and unexpected absences. The NGN is formally evaluated by the preceptors every shift, utilizing standardized evaluation forms. These forms are specific to orientation week and provide guidance for both the preceptor and the NGN of what skills, experiences, and level of independence the NGN should be achieving at the specific point in their orientation. These evaluations also help unit leadership to tailor patient assignments based on the NGN's needs.
Interventions
Two checklists were developed, one that was designed to be completed at the beginning of the shift and one to be completed at the end of the shift. The beginning-of-shift checklist includes important items that should be completed as part of the nurse's initial assessment of the patient (see Figure 1). This checklist was ideally to be completed before noon/midnight of each shift. Items on the beginning-of-shift checklist were grouped into themes of tasks that are generally completed at the same time. The end-of-shift checklist was designed to prompt the NGN to reassess the patient and room prior to handoff to ensure that the upcoming shift change went smoothly (see Figure 2). The items in each checklist were not required to be completed in any specific order.
The development of the checklists was a collaborative effort on behalf of transplant/surgical ICU preceptors, former NGNs, the unit clinical nurse specialist, and the unit educator. Each preceptor in the transplant/surgical ICU was e-mailed draft copies of each checklist, and their feedback was elicited. Unit preceptors provided feedback about the checklist content as well as their recommendations on how they should be integrated into practice. Former NGNs provided feedback about the specific tasks they struggled to remember during their own orientation. In addition, the transplant/surgical ICU nurse leadership group reviewed the checklists during their monthly meeting, and feedback was given regarding content, appearance, and deployment of the checklists.
Unit preceptors were essential to facilitating the integration of the checklists into the orientation program. Prior to the NGN group receiving the checklists, unit preceptors were educated about the required use of the checklists by the NGNs and the purpose of the study. This was done both in person and via e-mail, with the goal of not only informing the staff but also eliciting further feedback and generating buy-in.
The fall 2018 surgical ICU NGN cohort was composed of eight nurses. At approximately Week 4 of their orientation, the group was educated in person and via e-mail about the purpose of both checklists as well as how to utilize them properly. The NGNs then received digital copies of the checklists and were instructed to begin utilizing them in their practice. Additional copies were kept on the unit in case an NGN forgot to bring the checklists to a shift.
Each orientee was required to complete the beginning-of-shift checklist by 1200/0000 of each shift and the end-of-shift checklist by 1900/0700 of each shift. They and their preceptors were required to sign each checklist to verify that each item had been addressed and the checklists had been completed on time. It was expected that NGNs would forget to address certain items, and the hope was that their memory would be triggered upon referencing the checklist. It was encouraged that when this occurred, the NGN would make a note at the bottom of the checklist to improve for the next shift.
The checklists were not released until Week 4 of orientation, because it was believed by unit preceptors and leadership that if they were deployed at Week 1 of orientation, they would serve as more of a distraction than a tool. The NGNs receive a significant amount of paperwork during the first several weeks of orientation, and it was thought that the checklists would have been lost among the deluge. Furthermore, during the initial weeks of orientation, the NGNs are already overwhelmed by developing a working relationship with their preceptors, learning the basic unit routine, and learning how to navigate the electronic medical record.
Measures
The fall 2018 NGN group received an identical survey at Weeks 4, 8, and 12 of their orientation. The survey was composed of eight Likert-style scale questions designed to assess how frequently orientees remembered to complete items on the checklists. Answer choices included always, often, sometimes, seldom, and never. Another survey was sent to preceptors at Weeks 4 and 12 of orientation, which consisted of the 10 questions. The first eight questions contained the exact same content and were phrased almost identically to the questions in the survey sent to the NGNs. However, instead of requiring a self-assessment, the questions were designed to assess preceptors' observations of the NGNs' ability to complete certain tasks on a routine basis. The last two questions of the preceptor survey measured preceptors' level of agreement with a statement, with answer options including strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. These two questions were designed to evaluate preceptor opinion of the overall effectiveness of the checklists as a teaching tool. All responses were anonymous.
To understand if the use of the checklists alone improved the fall 2018 NGN cohort's confidence in their ability to complete certain routine tasks, a control group was established utilizing the cohort of nurses who had completed the unit orientation approximately 1 month prior and had never been exposed to the checklists during their orientation. They were asked to complete the same survey the fall 2018 NGN cohort completed at Week 12 of their orientation. This group was still quite early in their practice and struggling to manage their time while safely caring for critically ill patients. Thus, it was hypothesized that this group would have a similar perception of their own ability to consistently practice to the unit standards of care when compared to the fall 2018 NGN group.
Analysis
The data were analyzed using independent t tests to compare means between different survey responses. Independent t tests compared NGN survey results obtained at Week 4 and Week 8 as well as Week 4 and Week 12. In addition, t-test analysis was used to compare responses from the control group and fall 2018 NGN cohort survey results at Weeks 4, 8, and 12. Finally, preceptor responses from Week 4 and Week 12 were compared. To understand the magnitude of differences between the groups, we calculated effect size using Cohen's d.
Ethical Considerations
It is a priority of the transplant/surgical ICU unit leadership that every member of each NGN cohort be given every possible opportunity to succeed during their orientation. It was felt that it would be unfair to allow some members of the NGN cohort to have access to a training tool while denying others a potentially useful resource. As a result, the entire fall 2018 NGN cohort was given access to the checklists, with the hope that they would be a valuable addition to their training. This study was exempt from review and approval by the institutional review board given the complete anonymity of results reporting. Each survey participant was provided with a standardized letter notifying them of the purpose of the study, risks, and assurance that the results of the study would not identify the participants. The authors have no conflicts of interest to disclose.
RESULTS
All eight of the NGNs participating in the fall 2018 orientation responded to the Week 4 and Week 12 surveys, whereas seven responded to the Week 8 survey. The NGN cohort (n = 7) who had recently completed the transplant/surgical ICU orientation program without use of the checklists completed the same survey and served as the control group. Fourteen nurse preceptors were assigned to the new graduate cohort during the fall of 2018. Of these 14 preceptors, 7 responded to the Week 4 survey and 13 responded to the Week 12 survey. Participating preceptors had 3-10 years of clinical experience in the transplant/surgical ICU.
When comparing survey response means with independent t tests, results did not show significant differences in patterns of response. However, a statistically significant comparison was seen between the fall 2018 NGN response to the survey question assessing critical vital signs monitoring safety checks and emergency equipment setup at Week 4 versus Week 12 (p = .033). A statistically significant comparison was also seen between the fall 2018 NGN response to the question assessing end-of-shift patient readiness for shift change at Week 4 versus Week 8 (p = .045). The relative lack of statistical significance was likely because of the small sample size; however, results did demonstrate multiple medium to large effect sizes when comparing survey responses (given in Cohen's d metric), indicating that if these effect sizes were seen in a study with a larger sample size, the comparisons could potentially have been statistically significant (see Table 1).
By Week 12 of orientation, each member of the fall 2018 NGN cohort perceived that they "always" or "often" practiced to all aspects of the unit standards of care outlined in the checklists, indicating an overall improved level of confidence in their ability to practice to the unit standards of care (see Table 1). This was especially apparent in the areas of vital signs monitoring, emergency equipment, and hemodynamic monitoring (Survey Questions 3 and 8), where 100% of the NGNs indicated they "always" practiced to the standard of care in these areas by Week 12. When comparing the survey responses from these two questions at Week 4 and Week 12, the effect size was calculated to be d = 1.32 (Question 3) and d = 1.03 (Question 8).
The control group had the added advantage of completing orientation approximately 4 weeks prior to being surveyed, yet their survey responses reflected an interesting difference in their perception of compliance with standards of care as compared to the Week 12 survey responses of the fall 2018 NGN cohort. The control group demonstrated an overall lesser degree of perceived compliance with the standards of care in the areas of IV drip management, IV access, vital signs monitoring, emergency equipment, hemodynamic monitoring, and end-of-shift IV drip and drain reconciliation (see Table 2). A medium effect size (d = 0.56) was calculated when comparing responses involving vital signs monitoring and emergency equipment, and a large effect size (d = 0.86) was calculated when comparing responses involving hemodynamic monitoring. The control group did demonstrate a slightly higher average level of perceived compliance with practice standards related to feeding tube maintenance, tube feeding administration, Foley catheter care, and preparation of the patient for shift handoff. In regard to Foley catheter care, a medium to large effect size was calculated (d = 0.74) when comparing the fall 2018 NGN and control group responses. Importantly, in all standards of care directly related to critical patient safety checks, the fall 2018 NGN cohorts' perceived ability to comply with the unit standards of care was higher than the control group.
When the fall 2018 NGN cohort was surveyed at Week 12 of orientation, two additional free response questions were included, eliciting both positive and negative feedback about the checklists. Most of the feedback regarding the checklists was positive, with the checklist being regarded as useful, but best as a temporary tool to be used during training (see Table 3).
Results indicate that preceptor perception of the NGNs' ability to "always" and "often" complete the tasks addressed in the checklists differed significantly from the NGN perception of their own compliance. No statistically significant differences were seen when comparing survey responses at Week 4 and Week 12. Preceptors' perception of each NGN's ability to "always" complete the tasks addressed in the checklists did increase modestly between Weeks 4 and 12 in the areas of IV drip and IV access management, invasive devices, and shift change preparation (see Table 4). For example, preceptor survey responses indicated that between Weeks 4 and 12, the fall 2018 NGN cohort more consistently practiced to the unit standard of care in the areas of IV drip and IV access management, with responses of "always" increasing from 28.57% to 38.46% and 28.57% to 30.77%, respectively. However, by comparison, NGN survey results in these same areas demonstrated an increase in "always" responses from 50% to 75% and 37.5% to 62.5%, respectively. When comparing the preceptor survey responses between Week 4 and Week 12, only in the area of Foley catheter management was a greater than medium effect size seen (d = 0.62).
The survey administered to the preceptors at Weeks 4 and 12 included two additional questions. The results from the first question indicated a decreased level of agreement with the statement "Using the checklists helps my orientee remember critical safety checks and adhere to the eight ICU standards of care" between Week 4 and Week 12. The results from the second question indicated an increased overall level of agreement with the statement "The checklists help guide my teaching" from Week 4 to Week 12. In the survey administered at Week 12, the preceptors were also given the opportunity to provide open-ended feedback (see Table 5).
DISCUSSION
Summary
Two simple checklists helped to progressively increase NGNs' perceived level of confidence in their ability to practice to the unit standards of care during orientation. Their perceived ability to practice to these standards was better in tasks related to critical safety checks than a control group who had not utilized the checklists during orientation and were surveyed after practicing independently for 4 weeks. Interestingly, preceptors observed NGNs were not practicing to the standard of care as regularly as the NGNs seemed to believe.
Interpretation
Although overall results indicate that the NGN cohorts' perception of their ability to practice to the standards of care increased between Week 4 and Week 12, there were some areas where NGN confidence decreased between Week 8 and Week 12 of orientation. This may be a result of developmental milestones, changes in the primary preceptor, caring for progressively more complex patients, and transitioning from low acuity to much higher acuity 1:1 patients during the transplant/surgical ICU orientation. The NGNs also switch from day to night shifts (and vice versa) and must learn new shift workflows. Finally, midway through orientation, more experienced preceptors help the NGN transition from task-based nursing care to more complex critical thinking. For the NGN, this may present new challenges and may cause simple tasks to be forgotten or deprioritized. These challenges may also affect an expert preceptor's expectations of the new graduate and assessment of their competence.
The fall 2018 NGN cohort that utilized the beginning- and end-of shift checklists during their orientation perceived that they were more consistently able to practice to the unit standards compared to a control group who completed orientation without using the checklists. These results support the use of the checklist as a memory aid to reinforce practice standards. It is important to note, however, that this was not the case in the areas of invasive device management and shift change preparation. This is likely partially because the control group was surveyed approximately 4 weeks after ending their orientation, and so they had additional time to develop their knowledge and skills on the unit.
Preceptors' perception of the NGNs' ability to routinely practice to the unit standards of care did not consistently trend upward over the course of orientation. In fact, at Week 12 of orientation, significant percentages of preceptors were still selecting "sometimes" or "seldom" as answer choices across all survey questions, whereas all fall 2018 NGN responses at week 12 were "often" or "always." This indicates a certain level of disagreement between the NGN and preceptor perceptions of NGNs' ability to consistently practice to the unit standards of care.
These response differences are likely partially because preceptors change approximately halfway through orientation. Although a preceptor may orient an NGN for the full 12 weeks, it will never be the same NGN the entire time. Newer (3-5 years of experience) preceptors also tend to precept at the beginning of orientation, and more experienced preceptors (5+ years of experience) take over in the later part of the orientation. As a result, many of the preceptors who completed the Week 4 survey did not complete the Week 12 survey and vice versa, contributing to result variation. Finally, a preceptor may orient several different NGNs over several weeks, so in certain cases, when responding to the survey, a preceptor had to generalize their responses to account for multiple NGN performances.
In the latter half of orientation, there is less focus placed upon task completion and more on developing overall clinical competence. If the NGN has been able to master the skills outlined in the checklists, this should lead to an improvement in critical thinking and perceptual skills. The preceptor may also be less likely to observe the NGN perform the tasks outlined in the checklists, instead choosing to step out of the room during these times to let the NGN practice independently. Finally, more experienced preceptors tend to train the NGNs during the second half of the orientation, and it is likely that they do not prioritize task completion when orienting the NGN. These preceptors are classified as proficient or expert according to Benner's model and, as a result, are less tied to rules and guidelines to provide excellent care (Benner, 1982). It is possible that these nurses may not fully understand the difficulties the NGNs face practicing to the standard of care without a standardized memory aid.
The comments submitted in the surveys indicate that preceptors believed the checklist was useful; however, it was difficult to motivate the NGN to complete the checklist on time, if at all. In addition, survey feedback and in-person feedback indicated that although the checklist was useful for some, there were certain NGNs whose progress was impeded by checklist use. In the task-oriented state of the "advanced beginner," some of the NGNs were observed to focus too much on the checklists. This became an issue later in orientation when they were required to develop critical thinking skills and move beyond simply completing tasks successfully.
Limitations
Based on the study results, several limitations were identified. First, because of the task-focused nature of the first portion of orientation, it may have been useful to release the checklists earlier in the orientation program. Second, the small sample size of both the NGN and preceptor groups limits the significance of the results. To understand whether the implementation of the checklists truly improves the NGN orientation process, it would be important to study multiple NGN cohorts. A third limitation was that the participating preceptors varied between the Week 4 and Week 12 surveys, which made it difficult to truly understand the relationship between NGN survey results and preceptor survey results. The varying experience levels of each preceptor and different teaching styles undoubtedly impacted their perception of each NGN's progress, as well. Finally, the control group of NGNs who had recently completed orientation had been working independently for several weeks prior to completing the survey. As a result, this groups' level of confidence in their ability to practice to the standards of care is likely influenced by the extra time they had been in practice. This makes it difficult to validate the relationship between their survey responses and the responses of the intervention NGN group at Week 12.
Lessons Learned
It became clear that for implementation of the checklists to be successful, generating preceptor buy-in regarding the importance of the checklists was essential. Although preceptors generally agreed it could be a useful tool, there was no widespread agreement that it was an essential tool. Future successful implementation would require buy-in from not only preceptors but also unit leadership and unit staff. Presenting the results of this study to the transplant/surgical ICU nursing leadership, to unit practice council, and at staff meetings will be instrumental in generating discussion and interest as well as critical support from all unit stakeholders, thereby ensuring every opportunity to fully embed the checklists into the orientation program.
In addition, preceptor feedback demonstrated that often the NGN would not use the checklists on time or sometimes not at all, which indicated that they were not user-friendly. To streamline daily use of the checklists, it would be worthwhile to consider embedding the beginning- and end-of-shift checklist content within the electronic medical record. The completion of each item on the checklists could become part of the standard charting workflow within a unit, with checklist items integrated within the same task list containing pending medications, laboratories, and wound care. The NGN would not have to remember to bring the checklists to each shift and would not have to carry additional pieces of paper around with them during their shift. The checklist's efficiency and ease of use would be further increased if checklist item completion would autopopulate certain areas of charting, such as assessment of invasive devices.
Conclusions
The integration of checklists into an NGN orientation program in the ICU is a simple, easy-to-use, and low-cost intervention with a potential to greatly improve the orientation process. These checklists create a standardized training method in which to teach NGNs to integrate unit standards of care into their daily practice. The use of beginning- and end-of-shift checklists for NGN orientation can be easily customized for other contexts, such as medical surgical wards. Suggested next steps would be to deploy the checklists across multiple cohorts of NGNs to gather a larger sample size, which would allow for a more meaningful statistical analysis.
References