The Health Information for Economic and Clinical Health Act of 2009 encouraged the implementation of electronic health records (EHRs) in healthcare organizations, and by 2016, more than 75% of hospitals in the United States reported having an EHR system in place.1 The increased use of EHRs has resulted in unintended consequences that have contributed to medical errors that impair patient safety and outcomes.2 Unintended consequences are adverse events that are directly related to information technology and result from either an EHR that functions in a manner that the vendor did not intend or from inappropriate use of EHRs by healthcare professionals.2 There are no universal design standards for EHRs, and consequently, every EHR vendor requires unique training for end-users.3

Historically, EHR competency training has been achieved through didactic lectures with hands-on experience in a live classroom, and this method fails to teach learners how to be proficient with EHRs because the contextual factors that are present in real-world settings are not included.4 Contextual factors such as a hospital's workflow, sociotechnical aspects (ie, social interactions between clinicians and how individuals interact with technology), and organizational culture alter how healthcare providers actually use EHRs.5 The acquisition of EHR competencies can be enhanced by using clinical simulations that incorporate real-world sociotechnical (ie, contextual factors) with the use of an EHR.6 Failure to address the sociotechnical factors associated with EHRs may result in unintended consequences including impaired patient safety, poor documentation quality, and inefficient workflows.5 Additionally, the use of clinical simulations allows healthcare providers to train in a safe environment that incorporates contextual factors without affecting patient safety.7

The use of clinical simulations to teach healthcare professional students EHR competencies has been described in the literature.8 The authors of this article were unable to locate best practices for using clinical simulations to train healthcare providers to use EHRs. The purpose of this literature review was to summarize the current state-of-the-science on the use of clinical simulations to train healthcare professionals to use EHRs. The findings can be used to improve clinicians' acquisition of EHR competencies via clinical simulations and inform future research. A brief overview of clinical simulations appears in the following paragraphs to provide background knowledge. This literature review includes descriptions of the study findings and elements that are useful for designing clinical simulations for EHR training, which include (1) learning objectives used in EHR training; (2) measurements used for clinical simulation outcomes; (3) benefits; (4) limitations; and (5) design considerations.

REVIEW OF CLINICAL SIMULATION

The advantages of clinical simulations are well established. In particular, participants can acquire technical and nontechnical skills in a safe environment that will not compromise patient safety.9,10 Technical skills encompass the knowledge and physical ability to complete a particular task; and nontechnical skills (ie, human factors) include the cognitive and social processes needed to accomplish the technical skills.11 Using human patient simulators or standardized patients (human actors who have been trained to act out specific patient scenarios with consistent interactions with clinical simulation participants)12 in conjunction with clinical simulations assist participants in the acquisition of technical and nontechnical skills used to manage complex events.13-15 There is evidence that clinical skill training with well-designed clinical simulations can translate into improved clinical performance.14 Clinical simulation may also foster participant autonomy and decision-making skills.10

Clinical simulations have three distinct phases that include prebriefing, clinical simulation, and debriefing.15 During the prebrief phase, a description of what the clinical simulation will involve is provided to the participant and includes an overview of relevant information that is necessary to complete the scenario.15 After the clinical simulation is completed, debriefing involves providing scenario feedback for the purpose of facilitating participant learning.10,16,17 The format of debriefing (eg, instructor led, multimedia based, or self led) is not as important as the structure and content.14

Fidelity refers to the degree that the clinical simulation imitates real-life situations.18 High-fidelity clinical simulations attempt to replicate real-life situations including most of the contextual factors.19 Low-fidelity simulations involving EHRs may exclude all contextual factors and consist of just the keyboard, mouse, and computer display.18 For example, low-fidelity simulations are useful for the initial training of clinicians in the basic functionality of the EHR. The degree of fidelity is determined by the purpose of the clinical simulation.20 For the purpose of this article, the term clinical simulation will refer to high-fidelity clinical simulations.

METHODS

In March 2019, the databases PubMed, CINAHL, and Scopus were searched for publications using the following pairs of keywords without quotations: (1) electronic health record and simulation and (2) electronic medical record and simulation. No time limit was set because of the limited amount of literature related to using clinical simulation with EHRs. Inclusion criteria were literature that mentions using clinical simulations to train healthcare providers to use EHRs. Exclusion criteria were non-English articles, not peer reviewed, and articles that focused on training students.

A total of 1832 articles were initially identified, with 523 articles remaining after duplicates were removed. The articles were initially evaluated by both authors of this literature review according to the title and abstract (see Figure 1). All of the articles that met the initial search criteria (n = 22) were electronically retrieved. Of these, 10 articles met all inclusion criteria. The additional articles that were excluded involved the use of clinical simulation to design and evaluate EHRs. The authors of this article appraised the quality of the retrieved articles using the Johns Hopkins Nursing Evidence-Based Appraisal Tool,22 with the evidence ranking as medium (n = 5) to low (n = 5) quality (see Figure 1 for grading). The thematic analysis of this literature was organized and conducted using NVivo qualitative data analysis software (version 12, 2018; QSR International Pty Ltd, Doncaster, Victoria, Australia) to organize the information for comparison and contrast.

FIGURE 1. Literature search flowchart.

SYNTHESIS OF STUDY FINDINGS

The articles in this review included a wide range of healthcare provider types, clinical settings, and study designs (see Table 1 for a summary of the literature). The healthcare provider groups included interdisciplinary teams (physicians, nurses, and pharmacists),6,24,31 physician residents,23,25-28 registered nurses,29 and anesthesia providers (nurse anesthetists and anesthesiologists).30 The authors of the reviewed articles self-identified their clinical settings as intensive care units (medical and surgical),23,28 family medicine clinics,26,27 an inpatient pediatric floor,25 an acute care facility,29 an outpatient clinic,31 and an anesthesia department.30 The most common study designs included preintervention/postintervention studies23,25,27 and case studies.24,30,31 Other approaches included randomized controlled trials,26,29 a 2 x 2 crossover design,28 and an expert consensus statement.6 See Table 2 for a summary of the identified best practices for using clinical simulations during EHR training, and a summary of the benefits and limitations are discussed in the following paragraphs.

Table 1 Summary of Literature

Table 2 Summary of Best Practices for Using Clinical Simulations for Electronic Health Record Training

Findings of the Studies in This Literature Review

The significant findings of the studies in this literature review involved the impact of clinical simulation-based EHR training on improved recognition of patient safety issues,6,24,28 EHR competency,27,29,31 confidence related to EHR use,27,29 and improving central line-associated bloodstream infection rates.23 Compared to individual healthcare professionals, interdisciplinary teams are more effective in identifying patient safety issues.6,24,28 Improved recognition of patient safety issues is related to the fact that each individual healthcare discipline focuses on specific information in the EHR, and this differing focus is complementary in interdisciplinary teams.24

Important findings were found for competency and confidence related to using clinical simulations for EHR training. The study by Steward et al29 found no difference between paper-based training and clinical simulation on competency or confidence of registered nurses using an EHR. Steward et al29 reported that a sample size of 240 participants was needed to achieve adequate power, but only 87 participants were recruited into the study. These findings may not be dependable because underpowered studies are less likely to detect real differences.32 Other studies reported large effect size differences in competency (Cohen's d = 0.6327 and 0.8431) and confidence (Cohen's d = 0.4427 and 1.2427) when clinical simulation was used for EHR training. There is some evidence that novice users of EHRs may benefit more from clinical simulations that are intended to improve competency and confidence when compared with more experienced users.27 The use of clinical simulations to teach EHR competencies has been shown to result in persistent behavior changes related to identifying patient safety issues.25,26

A study on EHR training with a clinical simulation involving central line insertions found a decrease in central line-associated bloodstream infections.23 The authors reported that the infection rate decreased by 85% in the medical intensive care unit that received the training intervention but the infection rates remained unchanged in the surgical intensive care unit that did not receive the training.23 Documentation of compliance with hand hygiene, barrier precautions, and use of chlorhexidine for central line insertion site preparation went from 65% to 100%.23 The cost savings to the hospital was estimated to be $1 669 000 because of the reduction in central line-associated bloodstream infections.23 This is the only study in this literature review that demonstrated a direct relationship of EHR training to either patient outcomes or financial impact.

Learning Objectives for Clinical Simulations Used in Electronic Health Record Training

Clinical simulation training to teach healthcare professionals EHR competencies is often combined with other learning objectives. Learning objectives are used to inform the design and content of the educational curriculum.33 The primary learning objective was to teach clinicians how to use the EHR, and all the studies included a secondary learning objective. Secondary learning objectives included improving real-world recognition of patient safety issues via an EHR,24,25,28 improving competence and confidence associated with using EHRs,27,29 improving self-efficacy with EHRs,31 improving central line-associated bloodstream infections by enhancing documentation quality and adherence to evidence-based guidelines,23 and improving clinician-patient-computer communication competencies.26 The secondary objective in all of the studies in this literature review was used to determine the content of the clinical simulation.

Measurements Used for Clinical Simulation Outcomes

The most used methods to measure outcomes associated with the clinical simulations included observer evaluations using checklists of expected behaviors24,26,28,29 and self-reported survey measures (attitudes,26,27 EHR competency,27 and confidence in using EHRs29,31). One study used the rate of central line-associated bloodstream infections that occurred within 48 hours combined with a retrospective chart review to ascertain that all elements were documented for sterile technique during central line insertion (ie, use of chlorhexidine prep, hand hygiene, and barrier precautions).23 A summary and critique of these measurement methods are presented below.

Checklists of Expected Behaviors

The most frequently used method of measuring outcomes was checklists. The checklists of expected behaviors were determined by the learning objectives and included a list of clinical scenario specific events that needed to be identified.24,26,28,29 For example, the use of checklists to identify patient safety issues was the basis for an evaluation of how interdisciplinary team communication contributed to patient outcomes, and the findings were used to improve communication skills.24,26,27 The patient safety issues included inappropriate medication administration (unnecessary medications, inappropriate dosages, and multiple medication orders for opiates), failure to provide deep vein thrombosis prophylaxis, failure to identify abnormal laboratory values (positive blood cultures, acute kidney injury, and hypercalcemia), and not recognizing trending changes in physiological parameters (tachycardia, hypoxemia, and respiratory distress).24 Another study used a checklist of observable behaviors necessary for effective clinician-patient-technology communication skills that included the clinician sharing EHR information with the patient, maintaining eye contact with the patient, displaying empathy for the patient, and using jargon-free dialogue.26

Psychometric Instruments

Reis et al26 used an unnamed psychometric instrument (measuring communication, computer skills, EHR information sharing, and overall performance) that reported high Cronbach's [alpha] (internal consistency) scores ranging from .86 to .91. A psychometric instrument is a survey that has questions that use Likert-based scales, and the scores for each individual question are summated for a total score that represents some theoretical concept (eg, confidence in using an EHR).34 In a previous study35 that described the development of the instrument used by Reis et al,26 the content validity was not established and the reported internal consistency reliability was low (Cronbach's [alpha] = 0.52). Cronbach's [alpha] quantifies how well the individual questions within a psychometric instrument measure the same theoretical concept.32,34 A Cronbach's [alpha] of 0 reflects no internal consistency and 1 indicates perfect internal consistency, with a score greater than .7 considered acceptable.36

Steward et al29 adapted the Confidence Scale (C-Scale) to measure an individual's confidence during the generation of electronic documentation. No assessment of the reliability or validity of the C-Scale was reported.29 The original content validity and reliability were reported as acceptable (Cronbach's [alpha] = .85) in the original development of the C-Scale.37 The original C-Scale was designed to measure nurse confidence in physical assessment skills and not intended to be used as a measure of confidence in electronic documentation.37 It is essential to use psychometric instruments that have established validity and reliability so that the results are dependable, and when questions in an instrument are altered, it must be reevaluated.34 The findings of Reis et al26 and Steward et al29 may not be dependable or generalizable since the psychometric properties of the instruments were not adequately assessed for validity and reliability.

Benefits of Using Clinical Simulations for Electronic Health Record Training

The benefits of using clinical simulation for EHR training include evaluating interdisciplinary team communication,6,24 incorporating the sociotechnical factors that alter the way EHRs are used,6,24,28,31 improving the recognition of patient safety issues in real-world settings,6,24 attainment of EHR competencies that can translate into clinical practice,6,25 improving clinician-patient-technology interaction competencies to maximize interactions with the patient,6,26 and providing training in an environment that does not affect patient safety.6,30,31 The use of clinical simulations has been shown to result in sustained behavioral changes associated with the use of an EHR.25 Clinical simulations provide an active learning environment that enhances knowledge acquisition by increasing learner engagement.25 Additionally, clinical simulation training for EHRs may improve end-user acceptance of new system implementations while reducing unintended consequences.30,31

The role of communication among the interdisciplinary team members and with the patient is important.6 The use of interdisciplinary teams in clinical simulations for EHR training improves information sharing and fosters shared decision-making.6 High-quality interdisciplinary team communication improves the recognition of patient safety issues and prevention of medical errors.24,25 Including a standardized patient as part of the clinical simulation increases the interactivity, engagement, and satisfaction of simulation participants.31 Each healthcare professional discipline has unique patient safety issues that they may fail to recognize, and training interdisciplinary teams to use EHRs has been shown to improve the recognition of these patient safety issues.24 For example, registered nurses were most likely to recognize the lack of nutrition and the presence of multiple opiates prescribed to a patient, and these patient safety issues can be relayed to the attending physician.24

Healthcare providers can improve clinician-patient-technology competencies during clinical simulations by practicing documentation generation and information retrieval while communicating with the patient.27 If the healthcare providers' attention is focused on using the EHR, then the social interactions with the patient are impaired.27 The use of clinical simulations to teach EHR skills can help increase the social interactions with the patient by promoting eye contact, maintaining rapport, and facilitating psychological/emotional communication.27

Limitations of Using Clinical Simulations for Electronic Health Record Training

The limitations of using clinical simulations for EHR training include (1) difficulty in acquiring all possible interdisciplinary team members because of availability or financial cost24; (2) current EHR training environments have limited datasets that do not replicate real-world environments because they cannot include real patients6; (3) the real-time cascade of events is not similar to clinical practice (eg, laboratory values and physiological vital signs are often not released in real time)6; (4) the use of clinical simulations requires more human, financial, and physical resources when compared with standard didactic approaches6,25; and (5) the impact of clinical simulation on patient outcomes has not been definitively proven.25 The EHR training environment used with clinical simulations rarely captures the complex nature of real-world patient care events because of the limited amount of data within the EHR and the inability to provide a chronological release of relevant information.6 The most substantial limitation to using clinical simulations is the need for experienced simulation personnel and the associated cost of implementation.8 Additionally, clinical simulation training may not be beneficial for improving outcomes of rare events because the low sample size makes it difficult to assess the impact of any intervention.23

Design Elements and Considerations When Developing Clinical Simulations

Current EHR training focuses exclusively on the structure and content of the graphical user-interface without incorporation of the contextual factors that alter how healthcare professionals interact with information technology.6 Integrating the contextual factors into the clinical simulation helps to create a realistic scenario that includes the mental effort that the healthcare professional will expend when using the EHR in clinical settings.6 Clinical simulations need to focus on EHR functionality within the context of real-world patient care events that have increased mental effort.6 Electronic health record training that has artificially reduced mental effort may not catch potential decision-making errors that could result in medical errors.6,26

In general, designing clinical simulations often includes a literature review to find relevant content followed by assessment and approval by content experts.38,39 Many clinical simulations are developed according to course learning objectives and the unique knowledge, skills, and attitudes associated with each specific discipline participating in the clinical simulation.40 All of the literature in this review designed clinical simulations based on unique EHR training goals, which supported improved use of EHRs in clinical settings. Other design elements and considerations include clinical scenario content, EHR content, temporal considerations, and debriefing for participant evaluations.

Clinical Scenario Content

The clinical scenario content needs to be realistic, have an appropriate level of fidelity, and contain standardized content.6,28 The specific scenario used in the clinical simulation will be determined by the educational objectives and the specific disciplines represented in the interdisciplinary team.41 A representative from each discipline should be involved during the design of the clinical simulation to ensure that the final product is relevant to their unique needs.41 The clinical simulation needs to have a fidelity high enough to mimic a real-world environment closely.28 The realism provided by a high-fidelity environment will help engage the participants and enhance the acquisition of technical and nontechnical skills.6 Using realistic scenarios will improve the ability of clinicians to translate learned skills into real-world applications.6 Additionally, the clinical simulation needs a standardized structure and format because inconsistent implementation of the scenario will have a negative impact on the effectiveness of the training.6

Electronic Health Record Content

Electronic health record training has specific requirements for the content in the EHR database. Training for EHRs needs to occur in an environment that is separate from the production EHR used in clinical practice not only because of privacy concerns and the potential to for negative impacts on patient safety, but also because of inadvertent alteration of documentation in patients' medical records.6 The EHR training environment must replicate the production EHR that includes all user-specific customizations, macros, clinical decision support tools, and order sets.6,28,31 The number of test patients and the content included for each need to be similar to real-world settings (eg, radiology reports, laboratory values, medications, and previous documentation from interdisciplinary team members).24 An EHR training environment that includes a limited number of patients or minimal documentation content for each patient will artificially reduce the mental effort required to use the EHR and will reduce the usefulness of the training because of the failure to include important contextual factors.24 Also, the content of the training EHR needs to support the clinical workflows present in the clinical simulation to achieve the training objectives effectively.6

Temporal Considerations

There are temporal considerations for the design of the clinical simulation and the content of the EHR. The clinical simulation needs a realistic sequence of events that occur in real-time because participants will be better able to translate the acquired technical and nontechnical skills into clinical practice.6,31 Patient information in the EHR needs to be released over time and coordinated with events in the clinical simulation.6 For example, a clinical simulation about a patient in acute respiratory failure needs to have laboratory values for arterial blood gas results populated in the EHR shortly after the blood sample is obtained. Additionally, the realism of the clinical simulation is improved if previous patient data are present that allows participants to evaluate trend changes over time.6,31

Debriefing Considerations

The articles in this review evaluated the learners in the clinical simulations for EHR training using debriefing techniques combined with other technologies (eg, video/audio recordings and eye-tracking). Debriefing facilitates experiential learning and occurs after the clinical simulation is concluded.42 The purpose of debriefing is to promote critical reflection on the clinical simulation experiences.42 An assumption of debriefing practices is that learning does not occur primarily through experiences but on the deliberate reflection on that experience.42

Debriefings should be interactive, contain bidirectional communication between the instructor and participant, and contain reflective discussions.42 The instructor conducting the debriefing needs to present statements and questions in a positive manner that focuses on the actions performed during the clinical simulation without implying that the participants made an error (eg, refer to errors as "opportunities for improvement").42 Implying that participants performed poorly will impair learning because negative emotional responses have been proven to prevent effective learning.43 There are multiple approaches to debriefing that are summarized in the literature.42,43 A commonly used debriefing framework that is easy to use is Debriefing With Good Judgment.42,43

The Debriefing With Good Judgment framework includes three phases named reaction, analysis, and summary. The reaction phase is focused on participants' emotional feelings and presenting the facts about the clinical simulation.43 The first step of the reaction phase is to ask a question similar to "How did you feel?" and then followed up by the instructor summarizing what the clinical simulation was about. The purpose of the reaction phase is to allow the participants to reduce negative emotional responses that may impair effective learning.43 The analysis phase includes a reflective discussion to explore why the participants did specific actions during the clinical simulation with a focus on exploring the mental frame (ie, reasoning process) that led to mistakes.43 Identifying why a mistake was performed allows the participants to adjust their mental frame and prevent similar mistakes in the future, whereas only identifying specific errors results in an unchanged mental frame that may allow the error to occur again.43 The summary phase involves reinforcing the lessons learned and important issues discovered in the analysis phase.42,43

The evaluation of clinical simulations during debriefings can also incorporate video or audio recordings,24 assessment of the final documentation,23,24 and eye-tracking hardware.24 Video-based debriefing has been shown to be equivalent in effectiveness to the other debriefing approaches and allows participants to evaluate their own performance visually.14 Video-based debriefing also allows participants to analyze interdisciplinary team dynamics that include physical interactions and communication.24 Analyzing the documentation that was generated during the clinical simulation can help participants identify areas for self-improvement related to documentation quality.24

Eye-tracking technology can measure mental workload using pupil dilation changes that reliably reflect alterations in mental effort,44 capture video of the graphical-user-interface or patient interactions,24 and identify specific locations in the graphical-user-interface on which the participant was visually focused.44 An assumption of eye-tracking is that the location on which individual fixates (ie, is visually focused) represents where information is being retrieved and mentally processed.44 An eye-tracker can generate a video that identifies where an individual physically gathered information to inform clinical decision making.44 For example, eye-tracking can reveal that a nursing student did not evaluate a patient's drug allergies prior to administration of a medication (as evidenced by no video recording of a visual fixation on allergies in the EHR or verbalization by the student for oral verification of drug allergies from the patient). The current cost and expertise needed to use eye-tracking technology can limit the usefulness of this approach (per eye-tracker cost currently ranges from a minimum of $5000 for monitor mounted and $20 000 for eye goggles).44

DISCUSSION

As previously discussed, most articles in this literature review combined clinical simulations for EHR training with secondary goals such as recognition of patient safety events, improving documentation quality, and minimizing infection rates. The authors' descriptions of how the clinical simulations were designed and the specific methods of measuring outcomes were thorough, with checklists of expected behaviors as the primary approach. The previous discussions described how to establish learning objectives, select outcome measures, and design and implement clinical simulations. However, important issues that were not addressed by the articles in this literature review included pretraining for the EHR system prior to clinical simulation and considerations specific to the resources needed for the clinical simulation laboratory.

Pretraining in Electronic Health Records Prior to Clinical Simulation

If clinical simulation participants have no foundational knowledge of how to use the EHR, then most of the participants' mental effort will be on learning the EHR at the expense of the clinical simulation.8 Conventional classroom training for EHRs is insufficient to teach new users to be proficient and a significant amount of additional on-the-job training will be required to obtain proficiency.4 The traditional didactic approach to EHR training results in clinicians becoming distracted from direct patient care because of the mental effort required to learn the EHR.4 Combining conventional EHR classroom training that occurs before the simulation-based training is a more efficacious approach if the participants are novice users of the EHR, but more experienced participants may forgo this additional training.4

Required Resources

Simulation laboratories need specific human, equipment, environmental, and financial resources.45 Some of the human resources include personnel trained in simulation techniques, information technology support personnel, and adequate staff for the daily management of the simulation laboratory.45 Personnel who support and train the instructors who will conduct the clinical simulations are crucial for successful achievement of the learner objectives.8 Developing clinical simulations and the process used for successful debriefing require instructor training to achieve proficiency and these roles are often filled by specialists with advanced educational preparation.45

The financial impact of the required resources may make a simulation laboratory cost-prohibitive for smaller hospitals. Some of the required equipment and environmental needs may include patient simulators, a control room, simulated patient care areas, conference room, wall-mounted suction equipment, oxygen outlets, air compressors, conference room(s), wall-mounted video cameras, and desktop computers.45 The initial acquisition cost of a simulation laboratory can range from $100 000 to several million dollars.46 Combining resources with other organizations (eg, other hospitals or universities) in the same geographical region may facilitate the creation of a new simulation laboratory.8

RESEARCH IMPLICATIONS

Multiple implications for research were identified in this literature review. Future research needs to include psychometric instruments that have demonstrated validity and reliability in the respective study population to improve dependability and generalizability of the findings. It would be beneficial to compare the effectiveness of clinical simulations to train novice versus experienced EHR users because this will have a direct impact on designing the EHR training. Additionally, the findings in this review need to be replicated with larger sample sizes.

LIMITATIONS

There were several important limitations to this literature review. It is possible that the use of different search keywords may have identified other relevant articles. The majority of articles in this literature review focused on EHR training for clinicians who have already been using the EHR in clinical practice, and more research is needed on the use of clinical simulation for the initial EHR training after new system implementations. The articles in this literature review are medium to low quality. Additionally, the studies in this literature review used psychometric instruments without fully describing their validity and reliability, so their findings may not be dependable.

SUMMARY

Healthcare professionals can acquire EHR competencies through well-designed clinical simulations. The benefits of using simulation-based EHR training that incorporates the organization's sociotechnical factors include improvement of interdisciplinary team communication, clinical performance, clinician-patient-technology communication skills, and recognition of patient safety issues. Some of the significant limitations include the difficulty in coordinating all of the interdisciplinary team members required to design and implement the clinical simulation and the availability of the financial resources needed to operate the simulation laboratory. What this article adds to the literature is a summary of evidence-based guidelines to design and evaluate the use of clinical simulation to train healthcare professionals to use EHRs. Design considerations for EHR training using clinical simulations involve establishing learner objectives, identifying outcome measures, establishing content requirements of both the clinical simulation and EHR, and providing adequate debriefing.

References