The presentation of new and existing infectious diseases within the United States presents significant preparedness challenges. These infectious diseases are often not on the forefront of the clinician's mind during the initial encounter, particularly in rapidly evolving situations. Ill-prepared electronic health record (EHR) systems have shown vulnerability and, at the same time, opportunity to fully capitalize on technology to trigger (alert) clinicians to the correct clinical guideline. This was particularly important during the 2014 Ebola incident in Texas when a patient who had recently returned from central Africa presented to a hospital in Dallas with symptoms of Ebola. Documentation was entered into the EHR. The documentation indicated recent travel that should have alerted the clinician to possible exposure to Ebola based on travel location and timeframe. Clinicians failed to diagnose and manage this patient, and the case had subsequent adverse outcomes for the patient and some staff. The Dallas, TX, Ebola example allowed for documentation within the EHR that included symptoms and travel history but failed to notify appropriate personnel with suggested Centers for Disease Control and Prevention (CDC) guidelines.1
Subsequently, the Zika virus presented challenges for managing yet another infectious disease presenting unique complexity to capture data on both the pregnant mother and her sexual partner, including relevant travel history of both. The Zika virus can be transfered from mother to fetus and infection during pregnancy can lead to certain birth defects.2 Now, the emergence of a pandemic with novel coronavirus (COVID-19) presents another major public health challenge that EHRs can help address with clinical decision support (CDS) alerts designed to trigger the right care at the right time for the right patient at the right time through the right route of technology solutions. These "rights" are defined as the five rights of CDS that constitute best practice in use of the EHR.3
The presentation of a patient with an infectious disease, whether it be Ebola, Zika, coronavirus, or more common conditions, such as tuberculosis (TB), shares similar challenges to capture clinical documentation upon presentation within the clinical workflow. Characteristics of the patient, symptoms, and possible exposure including travel history-types of data captured in the EHR-provide sufficient information to trigger the correct actions per the CDC guidelines. However, complex configurations within the EHR are often required to deploy the infectious disease modules with appropriate CDS. The modules need to follow guidelines suggested by the CDC and do so within the clinical workflow of the clinicians to promptly alert clinicians with suggestions for appropriate testing and treatment. Equally important with infectious diseases is that isolation protocols are put into place to protect staff, clinicians, and others from exposure. Some of these challenges are rapidly evolving situations that present additional issues for keeping up with the most up-to-date information as a volatile situation shifts and changes. This was the case with Ebola and Zika and with the COVID-19 challenges.
The strategy to implement effective CDS within the EHR is workforce intensive, and as such, more effective and efficient methods need to be developed to address a rapid response to infectious diseases. This project outlines a quality improvement (QI) strategy to enable health information technology (HIT), using CDS, to address infectious diseases within a large hospital supporting a population of more than a million people in West Texas and Eastern New Mexico.4
During the Ebola preparedness efforts, many lessons were learned about how to optimize the EHR to identify infectious diseases for notification of possible presence. These lessons learned with Ebola twere transferred to the preparation strategy for Zika. In an institution in West Texas, these challenges prompted collaboration between hospital clinicians and academic institution clinicians, including specialty physicians and clinical information technology (IT) professionals, to design an institutional Zika alert module. The Zika CDS alerts the clinician when criteria such as self-reported pregnancy status and, if applicable, travel history, sexual partner travel history, and location are recorded on the intake documentation. Overcoming technical challenges related to capturing these critical data points for Zika was successful. Both the Ebola and Zika requirements for complex CDS module configuration created a need for more effective methods to enable HIT for QI. While the technical builds within the EHR were successful within the institution, these HIT builds presented a manpower-intensive process to create and maintain, particularly when the situation nationally and/or globally is rapidly changing.
PURPOSE
The purpose of this project was to develop a modular approach to integrate rapid deployment of CDS for infectious diseases into the clinical workflow and evaluate the usability of the design. The project utilized the HIT-enabled QI model outlined by Osheroff et al3 to guide the process withan evaluation method deploying the Task, User, Representation, and Function (TURF) unified framework of EHR usability designed by Zhang and Walji.5
The TURF framework takes into account the usability of the design based on human factors science. This approach is the result of classic research funded by the Office of the National Coordinator for HIT along with the passage of the HITECH Act of 2009. This research was developed using federal funding and resulted in the National Center for Cognitive Informatics & Decision Making in Healthcare that maintains the TURF model and the subsequent software program, named 'turf' (National Center for Cognitive Informatics and Decision Making in Healthcare, Houston, TX). This framework and software are excellent for evaluating end-user experience and usability design of EHRs.
Health Information Technology-Enabled Quality Improvement
Osheroff6 designed a systems-based approach under contract to the Health Resources and Services Administration that includes a model for strategic design of CDS to improve care through HIT that enables QI. This approach recommends the use of HIT to reinforce the improvement of care, population health, and safety of populations. These methods incorporate fundamentals of QI and suggest HIT tools such as workflow mapping and CDS as an intervention to improve care. Furthermore, Osheroff6 provides a toolkit for design strategies to optimize technology for QI. This project will deploy these methods including (1) check and reinforce foundations; (2) understand HIT-enabled QI; (3) select targets and initiate a QI project; (4) document, analyze flows, and identify improvements; (5) implement and evaluate changes; and (6) harvest/spread results.
Clinical Decision Support
The purpose of CDS is to lessen the clinician's cognitive burden and to enhance, not replace, critical thinking.7 Mann et al8 explain and describe user feedback highlighting alert fatigue and the potential to add to an already demanding workload and further discuss urgency to create "smarter" alerts through various methods such as machine learning, personalization by user, and potential use of advanced technologies outside the EHR. Issues illuminated previously were discussed with a governance committee. The governance committee voted unanimously to reduce alert fatigue, to restrict the alert to only one time per patient visit per clinician entry into the chart.
As described by Miller et al,9 CDS has enabled clinicians to use evidence-based information; however, there continues to be a lack of evidence-based guidelines for CDS development, often contributing to usability issues. Miller et al9 further suggest a proactive approach that considers design and usability heuristics during the development of CDS. Moving from design to configuration and testing, Lopez et al10 recommend testing the CDS system, instead of individual parts. This approach is described as time and resource intensive, creating barriers to CDS development and implementation.
Human factors design principles were utilized to address these issues. Human factors design is defined as "designing and arranging things people use so that the people and things interact most efficiently and safely."11 A useful system design accomplishes the task the user intends to complete, independent of system implementation. Ease of learning, use, and error tolerance to reduce mental effort are desirable traits of a "usable system."5 Zhang and Walji5 align with Sheehan and Lucero12 to reinforce the concept of usability as the ease of learning a system as well as ease of recovery from an error.
Many facilities have implemented CDS within EHRs but struggle with maintenance and monitoring as well as usability evaluation. The Health Information and Management Systems Society promotes CDS as a technology to aid in clinical decision making, guiding the end user through complex systems to achieve a targeted outcome.3
Alert overrides are one of the most common issues contributing to end-user dissatisfaction, which is largely due to poor design within the clinical workflow and lack of evidence-based integration. These issues are also associated with medical errors.13 Alert overrides happen for a variety of reasons, and often, the provider will override the alert despite reserving disruptive alerts for life-threatening conditions.14 Examples of these types of alert overrides are a laboratory test in the critical range, a drug-drug interaction, a drug-food interaction, or a drug-allergy interaction. Providing instructions to end users on how to implement suggested interventions while addressing organizational challenges in specific patient settings shows the most promise.15
Setting
The project site is a large hospital with a total of 500 beds, which serves more than 2.7 million people in West Texas and Eastern New Mexico, and the primary service area covers 130 000 square miles of West Texas and small areas of Eastern New Mexico.4,16 The hospital partners with two ambulatory clinic practices including a large academic health sciences center. The hospital is recognized for the region's only level 1 trauma center and burn center serving significant areas of rural areas of the United States. The hospital also functions as the teaching hospital for the health sciences center and trains more than 400 physicians and nurses annually.4 The hospital hosts a fully integrated EHR for both the hospital and ambulatory settings.
Design Using the Task, User, Representation, and Function Framework
This project was a QI design involving CDS build within an EHR as the technology intervention strategy for improvement. As noted earlier, the TURF unified framework of EHR usability was used for evaluation of the usability of the design; TURF stands for the four key components of usability: Task, User, Representation, and Function. It was used as a framework for (1) describing, explaining, and predicting usability differences in terms of the representation effect; (2) defining, evaluating, and measuring usability objectively; (3) designing effective usability; and (4) developing EHR usability guidelines and standards.5
Task, User, Representation, and Function Toolkit
User analysis is the first step of the turf to software process, capturing data on the type of users that includes developmental stages of professional career as well as specialty. User characteristics, such as educational background, cognitive capacities, and knowledge of the EHR, are collected during the user analysis phase.
Functional analysis evaluates the work that is performed including the complexities of the work environment. Functional analysis evaluates whether the system does what it says it will do. If the work is not supported, the system will fail.
Representational analysis evaluates the appropriateness of interactions between the user and the task within the system. Disparate representations can generate different efficiencies, task difficulties, and behavioral outcomes. This stage compares similar structures across the EHR to determine whether it is efficient for the task and the user.5
Task analysis is the evaluation of identified processes to carry out steps (mental or physical) to completion. Comparing user performance with different interfaces for time on task, the number of steps, and mental effort are all metrics of efficiency for usability.5Figure 1 reflects the visual representation of the TURF model. The model was very effective for evaluating the development of the modular approach to integrate rapid deployment of CDS for infectious diseases into the clinical workflow, with an emphasis on usability of the design.
Human Subjects
Institutional approval for this QI project was obtained from the Chief Information Officer with a letter of support. The project was also approved for a QI project by the institutional QI review board. All data used for evaluation were de-identified utilizing both the expert determinations and Safe Harbor guidelines for appropriate de-identification methods.17
Description of the Challenge and the Approach
As noted, during the previous Ebola and Zika alert module configuration, institutions across the nation learned a great deal as to how to effectively utilize EHRs to support proper patient handling of a potentially infectious disease. Most notably, the need for support of a governance committee including subject matter experts (SMEs) from diverse backgrounds was needed to inform design strategies. After initial discussions with SMEs within the healthcare system in West Texas, a process for maintenance and monitoring was developed. The infectious disease workgroup is the interprofessional governance committee of infectious disease experts including providers, pharmacists, infection prevention specialists, (IPs), and clinical informaticists. The governance committee developed guidelines, and evaluated and approved local practice patterns as the criteria CDC guidelines were updated and changed. The CDC guidelines for Ebola and Zika were evolving recommendations. Governance structure involving clinical experts in clinical informatics, infectious disease, and IT is needed to respond rapidly to these types of infectious disease outbreaks, particularly in rapidly evolving crises.
For this project, TB was selected as the use case to develop, test, and evaluate the approach, but with an eye on the past as well as any future challenges with infectious disease preparedness. Tuberculosis continues to be a significant infectious disease worldwide, and the emergence of drug-resistant strains compounds the problem. Given the significance of this infectious disease and the stability of the guidelines for TB, the governance committee elected to focus on TB as the use case for the QI project.
In assessing the current process for TB, it was noted that exposure and treatment data collection occurs at intake. However, there was not a process in place to alert or notify appropriate staff of potential TB. Therefore, this infectious disease has been identified by the organization as an opportunity to enable HIT for QI of the process and to improve overall outcomes for the West Texas and Eastern New Mexico regions. Texas requires reporting of TB cases18 to the Department of State Health Services (DSHS) within 1 working day. The project will assist with the closure of a previously known gap in timely reporting to the DSHS.
To begin the assessment process, a current state workflow mapping was performed, including the process for identifying, documenting, and treating patients suspected of having an infectious disease. This process occurred with the direction and guidance from the infectious disease SMEs. While TB was the use case that informed the development of EHR code, similarities and differences with other infectious diseases were considered to allow the system to accommodate future disease challenges. A priority score was assigned for each infectious disease process alert module. Priority scores were developed to identify which commonalities would be used for additional alert creation as well as which alerts would present when the alerting criteria were similar. While the priority scores do not affect the overall alert function, they are utilized to make it easier to track criteria that must be present for each disease process, thus allowing for more rapid module creation.
A base module was created once criteria and priorities were evaluated. The overall intent for this module is to catch infectious disease characteristics that do not fit within more specific modules. This module is also used as the base module for more specific module creation as the need arises.
Priority scores were used in all the current modules and changes based on expert guidance and include the following examples. If a patient traveled to a CDC-identified area of concern within the identified timeframe, the base module would alert the clinician to allow for further clinical decision making. In this example, both travel location and travel timeframe would be assigned a lower priority score.
A more complex example related to disease processes such as measles and yellow fever included self-reported vaccination status. Prioritization for measles includes self-reported vaccination status. If the vaccination status has been confirmed, the alert will not fire. On the other hand, if the vaccination status is not addressed or addressed as "no", then the algorithm moves to the symptoms and travel location including timeframe. Measles and yellow fever would have similar symptoms as well as similar travel locations. While the mode of transmission differs between measles and yellow fever, the presence of the same criteria would allow for alerting the clinician; however, the priority would be higher for measles due to the ease of transmission and potential public health impact. These factors warrant a higher priority.
Priority scores were developed based on the severity of disease as well as the potential impact to public health. The priority scores were developed with the guidance and input of the infectious disease SMEs to specifically target clinician satisfaction and to help decrease alert fatigue.
When the criteria are met for each of these modules, an alert will fire, which is represented by Figure 2 and will include dynamic information such as exposure, travel location, travel timeframe, and symptoms. Table 1 includes a visual representation of inclusion criteria including priority scores.
Multiple end users were consulted during the design phase of this project, including primary care physicians, advanced practice providers, and nurses in both the ambulatory and acute care settings. Based on end-user feedback, a base infectious disease module was created capturing data for travel criteria and symptomology within the clinical workflow. The TB module was then created including known exposure based on patient reports of exposure without treatment. After completion of intake documentation, an alert would present to the clinician with an option to go directly to the order page to allow for placing orders, hyperlinks to go to the CDC and Infectious Disease Society of America (IDSA) TB guidelines page, and notification to the institutional infection prevention department for tracking and possible intervention. As a final step to help combat alert fatigue, the alert will only present once per clinician per patient visit.
The TB build process utilized a base model initially designed for addressing Ebola and Zika and has been identified nationally as a best practice.19 The goal for this module was to build it once and use it many times so when other infectious diseases arise, which require CDS alerts or new guideline implementation, this module can be rapidly updated and deployed. The infectious disease alert modules use Boolean logic to determine inclusion criteria for the alert and could be used in various EHRs. See Figure 3 for alert inclusion criteria. It is important to note that, in this institution, alerts are triggered based on the nursing documentation; therefore, steps to create nursing documentation and the EHR build considerations were included in the design phase. This is not always the case in other institutions; as a result, it is critical to consider the institution's specific clinical workflow and where relevant clinical assessment data are captured related to the infectious disease. Previously mentioned priority scores refer to which forms of nursing documentation are needed such as location of travel and timeframe since last travel. The test site utilizes an integrated EHR, and information is dynamic when practical. For example, when a patient is seen in the ambulatory care setting, the alert will display the ambulatory care phone number of the infection prevention department. The same is true for the acute care setting; the alert will display the hospital infection prevention department phone number. Providing relevant information at the time the alert presents to the clinician is critical to allow for accurate and timely decision making, including collaboration.
EVALUATION METHODS
De-identified TB clinical documentation data were collected as a baseline to evaluate overall improvement to the process. Documented TB criteria of "yes" (the criteria were present) compared to "no" (the criteria were not present) were collected from the EHR database. Orders for an isolation cart, diagnostic tests such as TB skin test or chest x-ray, consultation, or referral to an infectious disease specialist were also collected for the same visit and timeframe to determine appropriate diagnosis and intervention.
A structured query language developer (Oracle, Redwood Shores, CA) was used to extract 340 716 records, which met the above criteria. After initial data collection, Microsoft Excel (Microsoft, Redmond, WA) and Microsoft Access (Microsoft, Redmond, WA) were employed to further refine data for import into IBM SPSS Statistics (IBM, Armonk, NY) for statistical analysis.
Additionally, data were collected on the documentation of self-reported positive exposure to TB and the trigger of a pop-up alert to the next clinician who enters the chart. This pop-up alert informs the clinician of possible TB exposure based on intake documentation and can be tracked within the EHR to determine when, how, and who received the alert. The pop-up alert provided guidance based on the CDC or IDSA recommendations for an isolation cart, medication, referral, and consultation to specialists. The same data points were collected before and after implementation and compared. Control charts were used to determine the effect of the HIT-enabled QI intervention on the process with a special cause variation post-intervention as the desirable outcome. The measure used to examine the overall process was total hours from documentation to order time. Therefore, the strategy was to evaluate not only the appropriateness of action but also the efficiency of the actions taken.
Outcomes-based CDS should not rely on historical patterns of care but should be driven by those historical patterns of care.20 Hence, the use of control chart measurement is an effective method for informing HIT process control and to inform improvement strategies. As such, monthly data collection to examine process control was presented to the governance committee for monitoring and feedback, as displayed in Figure 4.
Additional measures and strategies evaluated the usability and alert to the clinician. The goal was ease of use of the EHR leading to more satisfied providers with a decreased cognitive workload and safer clinical decisions. Usability of the design was evaluated with the turf software. The turf software has the capability to measure effectiveness and usability of the EHR design. The turf software includes the System Usability Scale (SUS) satisfaction evaluation tool to evaluate end-user satisfaction. The SUS is a 10-question satisfaction tool scoring on a 5-point Likert scale ranging from strongly disagree to strongly agree, with a score above 68 considered satisfied.
The final evaluation was completed using the turf software to evaluate navigation of the EHR build, including heat maps and other measurable options to detect challenges with the build when comparing various end users' navigation.5 Screenshots were captured during testing scenarios with focus group members.
Methods included a test case scenario for a TB patient that 12 clinicians undertook with the EHR build to mimic clinical workflow. Data were captured within the software to examine factors that reflect usability on items such as time on task and number of steps to complete the testing scenario. Time on task was a measurement of amount of time (seconds) it took each participant to complete the scenario.5 Start time was upon opening the chart, and finish time was a verbal report of completion. The number of steps to complete the task was measured from the start of the scenario to the end of the scenario. Steps to complete the task involve both mental and physical steps to complete, for example, the user time to recall a test name and then where to navigate within the EHR to place the order.5 Additionally, where and how the end user navigated the system and whether alerts triggered as they should were also captured within the software. Heat maps compare the end-user variability of mouse events such as location concentration frequency, time on location, and the number of clicks.
RESULTS
Results of the evaluation included comparisons between pre-implementation and post-implementation based on the intake documentation time to order time in hours when the patient self-reported current exposure to TB. Mean order times between 0 and 8 hours were selected for evaluation, and these mean order times contained 85% of all records considered. Self-reported TB cases pre-alert and post-alert had mean order times of 0.78 hours (SD, 0.97) and 0.57 hours (SD, 1.13), respectively. A Mann-Whitney U test revealed no significant differences in mean documentation to order time in hours of pre-alert (Md = 1, N = 9) and post-alert (Md = 0, N = 7), U = 24.5, z = .41, r = 0.1.
A final focus group of 12 participants, six nurses and six providers, was selected by invitation based on governance committee recommendations as well as a sample of convenience. Table 2 shows the characteristics of the participants. The participants worked in both the ambulatory and acute settings. The mean age of the participants was 38.25 (SD, 12.30) years, and the mean years of experience in their role was 9.73 (SD, 9.83). Approximately 50% of all participants held advanced degrees (master's level), with 100% of participants self-reporting product experience with the EHR and overall computer competency as intermediate and advanced.
System Usability Scale mean satisfaction scores were evaluated between nurses and providers scoring 91.25 (SD, 10.7) and 80.83 (SD, 14.38), respectively. An overall mean satisfaction score of 86.04 (SD, 13.25) was reported across all participants. A Mann-Whitney U test revealed no significant differences in SUS satisfaction scores between nurses (Md = 95, N = 6) and providers (Md = 87.5, N = 6), U = 35, z = -1.62, r = -0.47. Two additional measures resulted in significant findings, namely, the mean time on task and number of steps to task completion. Both tests were significantly higher for the provider group. Mean time on task was evaluated between nurses and providers scoring 156.1 (SD, 43.7) and 390.58 (SD, 213.88), respectively. An overall mean time on task score of 273.33 (SD, 191.46) was reported across all participants. A Mann-Whitney U test revealed significant differences in time on task between nurses (Md = 149.93, N = 6) and providers (Md = 307.53, N = 6), U = 36, z = 2.88, r = 0.83.
Mean number of steps to task completion was evaluated between nurses and providers scoring 233.5 (SD, 76.24) and 579.67 (SD, 158.94), respectively. An overall mean number of steps to task completion of 406.58 (SD, 216.35) was reported across all participants. A Mann-Whitney U test revealed significant differences in the number of steps to task completion for nurses (Md = 251.5, N = 6) and providers (Md = 632.5, N = 6), U = 35, z = 2.72, r = 0.79.
DISCUSSION
Results from this improvement project suggest that both providers and nurses were very satisfied with the implementation of an infectious disease alert module. The decreased documentation to diagnostic order time will benefit both patients and staff. The increased mean time on task as well as the mean number of steps to complete the testing scenario for providers is not surprising at the test site. The provider group took steps to investigate previous diagnostic images, evaluate laboratory data, and read previous documentation from other providers. The provider group also documented progress notes before placing diagnostic orders for the simulation, which increased the time on task and number of steps to complete. The nursing group filled out the required fields and moved on to the next task.
The heat maps support extended times and steps for providers by showing darker colors as areas of greater concentration and mouse clicks. The darker the color on the heat map, the greater the focus on the area within the document and location in the EHR. The heat maps were clinically significant for evaluation of workflow processes and arranging on-screen documentation in a logical order much like the heat maps used to track emerging and ongoing infectious diseases such as COVID-19.21
The governance committee determined that the project was successful although some of the findings indicated no significant improvement. Given this success, the project expanded from the TB use case to address additional high-priority infectious diseases. Subsequently, the rapid deployment of CDS was used with success for creating the measles and yellow fever modules at the request of the infectious disease governance committee. The simplicity of copying the base module and adding requested criteria assisted with supporting the rapid deployment model and indicates the success of the project with the overall intent to develop a modular approach for rapid deployment of CDS for infectious diseases.
Limitations
The results of this study describe decreasing time for patients who self-identify previous exposure to TB by alerting clinicians to possible exposure. While the project examined a small sample size (n = 16) with limited cases in a single test facility, the results can likely inform improvement strategies beyond this institution. Despite the small sample size and challenges of implementation, findings indicated that clinicians were satisfied with the new processes. Perhaps more important, this approach can be used to create a modular approach to fast-track infectious disease digital guidelines into practice smoothly and efficiently. Critical to success is end-user SMEs fully engaged in the process.
CONCLUSION
This project deployed HIT-enabled QI using strategies to build CDS within rapidly modifiable modules for infectious diseases and is an essential contribution to bioinformatics. The CDC has a project titled "Adapting Clinical Guidelines for the Digital Age," which is focused on designing national strategies to automate how rapidly CDC clinical guidelines can be deployed in the field, from conception to implementation. Their goal is to trim timelines considerably and to address rapidly evolving infectious disease crises such as the Ebola incident. This project will help to inform those national strategies and will test methods to evaluate CDS builds for usability and end-user satisfaction. This project also has the potential to guide efforts for building the Sustainable Medical Apps, Reusable Technology (SMART) on Fast Health Interoperability Resources (FHIR), currently under development as part of the Adapting Clinical Guidelines for the Digital Age project; SMART on FHIR will have artifacts that will contain CDS criteria maintained at the national level.
Recent real-world scenarios include the emergence of the COVID-19 virus. Utilizing the rapid deployment model described in this QI study, the base module was modified and tested in 2 hours with emergency communication submitted to the infectious disease governance committee. Full approval from the governance committee was received 12 hours later, and the newly developed module was fully implemented. The untimely disease process provided for full complete evaluation of the rapid deployment model was discussed.
Finally, the authors recommend several lessons learned from this process for organizations considering a modular approach to integrate rapid deployment of CDS for infectious diseases into the clinical workflow: (1) establish a governance structure including infectious disease specialists, clinical informaticists, IT professionals, and end users; (2) select a use case such as TB to inform the build strategy; (3) map workflows with current state and future state with usability as the goal; (4) solicit end users and SMEs throughout the process; and (5) consider use of the Health QI Toolkit6 and the TURF model5 to inform evaluation of the build.
Acknowledgment
The authors would like to thank Dr Stephanie Hoelscher for her countless hours of insight and advice; Dr Lisa Campbell for guidance navigating the grant application and public health aspect; and Dr Jacob Nichols, Dr Tammy Williams, and Renae Yates for support and expert guidance from respective institutions.
References