Natural language processing (NLP) programs use machine learning to convert text or transcribe conversations into interpretable usable data to answer a variety of research questions.1 Natural language processing has been primarily used in relation to the electronic health record (EHR) and other healthcare databases to convert unstructured data to structured data2 Additionally, NLP programs have been used to extract patient information from nursing narrative notes.3 Natural language processing has been used to identify vulnerable at-risk adults through the use of algorithms to scan the EHR for key words and sentences in clinical notes.4 An additional application of NLP includes predicting the risk of hospital-associated infections through analysis of clinical notes.5 Natural language processing has rarely been used in healthcare to analyze how healthcare providers are communicating essential patient information from one provider to another or how the language that is used influences patient outcomes. The analysis of healthcare provider communication, both written and verbal, has the potential to decrease errors and improve patient outcomes.
An innovative within-methods triangulation data analysis was first used to analyze nurses and physicians communicating patient information.6 This method of analysis uses traditional content analysis and an NLP program to provide a deeper contextual comprehension and understanding of the same transcribed text data.6
OBJECTIVE
Building on the Renz et al6 within-methods triangulation data analysis method, the focus of this feasibility study is to explore the use of an innovative analysis method to uncover the contextual and linguistic meaning of the nurse-to-nurse change-of-shift hand-off communication. Previous qualitative studies examining hand-off communication have traditionally analyzed the data using content and thematic analysis.7 These two analysis methods are effective in identifying key themes, codes, and categories in communication text data but provide limited insight into how the message is communicated. The within-methods triangulation data analysis of using an NLP program and content analysis has the capability to provide new perspective and knowledge about how the content of the nurse-to-nurse hand-off is communicated. The purpose of this article is to present the outcomes from a feasibility study that sought to test within-methods triangulation data analysis on nurse-to-nurse hand-off text data.
BACKGROUND AND SIGNIFICANCE
Nearly 100 000 patient deaths occur each year in the United States due to healthcare provider miscommunication.8 Communication failures in healthcare place the patient at high risk for disruption in the continuity of care, resulting in direct and indirect harm.9,10 Miscommunication between healthcare providers during the transfer of care accounts for 80% of all sentinel events in the healthcare setting.10,11 Errors in communication were identified as the root cause of 86% of delays in treatment and 75% of hospital-associated infections and complications.12
One known source of miscommunication is the transfer of patient care from one provider to another, referred to in several ways as "hand-off," "handover," "nurse-to-nurse hand-off communication," and "change of shift report." Nurse-to-nurse hand-off communication is the process of transferring patient care and responsibility from one nurse to another.13-15 The threat to effective communication during hand-off is a consequence of the nature of the hand-off as a cognitively intense process occurring in a sociotechnical system requiring the registered nurse (RN) to simultaneously analyze and synthesize a vast amount of data, information, and knowledge.14,16 Effective nurse-to-nurse hand-off communication is essential in the prevention of adverse patient outcomes. The majority of the studies focused on the structure, form, and consistency of the hand-off communication to the exclusion of its content that addresses the "what" and "how" of the communication.7 The methods we are currently using to research hand-off communication are not effective in initiating the change needed to impact patient outcomes. What is needed is to approach the same problem from a different perspective using innovative data collection and data analysis methods.
Due to the nature of hand-off communication process and the known risks of miscommunication to patient safety, we sought to test a novel method for analyzing hand-off data. Within-methods data analysis triangulation was used to determine the effectiveness in analyzing hand-off text data. Here, we used Linguistic Inquiry Word Count (LIWC, Austin, TX), an NLP program and content analysis, to learn the "how" and "what" of the nurse-to-nurse hand-off communication for patients who have experienced a clinical event. A clinical event (CE) is an unexpected change in the patient condition in following areas: fever, pain, bleeding, changes in output, changes in respiratory status, and changes in level of consciousness.17,18
WITHIN-METHODS DATA ANALYSIS TRIANGULATION
Linguistic Inquiry Word Count
Nursing has not applied NLP programs as a data analysis tool in research as frequently compared to other healthcare disciplines.6 The study conducted by Renz et al6 was the first to utilize both NLP and content analysis as a within-methods triangulation data analysis method in a qualitative nursing research study. Renz et al6 used LIWC, an NLP program, to nurse interview text data. The LIWC program was developed to analyze text data from multiple sources such as emails, tweets, natural speech, novels, newsprint, and blogs.19 This program provides insight into how individuals are communicating. The language used in text and verbal conversation reflects what the individual is thinking about, what is important, his or her feelings, and how he or she is synthesizing and analyzing the events occurring in the environment.20
Text data are analyzed by LIWC using predefined word dictionaries to code the text data into the following categories: analytical thinking, clout, authenticity, emotional tone, psychological processes (ie, affective, cognitive, biological, social, and drives), time orientated, personal concern, and informal language such as fillers, netspeak, and assents.19 The LIWC Operator's Manual21 and the Development Manual19 provide guidelines on how to interpret the raw LIWC scores. When using NLP to analyze the data, the researcher must be cognizant that the computer is unable to determine subtleties, context, or the meaning of a term or phrase.22 The onus of interpreting and applying the LIWC results to the study's research questions, aims, and theoretical framework remains with the researcher.
Content Analysis
Content analysis involves systematic coding and categorizing of text data using both inductive and deductive techniques to identify trends and patterns.22-25 In addition to identifying trends and patterns in the data, content analysis provides basic quantitative results in the form of frequencies or number of occurrences of the codes and categories occurring in the text.22-25 Content analysis provides the researcher with the ability to identify and describe the content characteristics of the text.22-25 Advantages of using content analysis are that it enables the researcher to become immersed in the data to gain a deeper understanding and that it is unobtrusive to the participants. The potential limitations of content analysis are that it is a time-intensive process and that researcher bias may exist.22,24,25
There are several different coding strategies and schemes associated with content analysis in qualitative research.25,26 The specific strategy or scheme used depends on the type of data and the research question or questions.25 The content analysis strategy used for this feasibility study was hypothesis coding. Hypothesis coding is the application of a researcher's a priori generated list of codes to text data.23 The codes are developed based on a prediction of what will be discovered in the data prior to the data being collected or analyzed.23 This type of coding is appropriate for content analysis when exploring the data for explanations.23
Triangulation
The triangulation technique in research is the use of multiple strategies for data collection, data analysis, or both in a single study.23,27 The goal is to select strategies that will provide complementary insight into the phenomenon being studied. There are four types of triangulation: (1) data source (time, place, space); (2) method (within methods or between methods); (3) researcher (ie, additional investigators); and (4) theory (eg, the use of two or more theories to examine a hypothesis).23 Miles et al23 added a fifth type of triangulation termed data type using a combination of both qualitative and quantitative data (ie, mixed methods). The use of triangulation in analysis has the potential to provide the researcher with a deeper and broader understanding of the findings.
METHODS: APPLICATION OF USING NATURAL LANGUAGE PROCESSING AND WITHIN-METHODS DATA ANALYSIS
Research Questions
This feasibility study examined the practicality of the data collection and the within-methods triangulation data analysis to explore the content of the nurse-to-nurse hand-off communication. The research questions of this feasibility study were as follows:
1. Are the data collection methods of digital audio recording and observing the hand-off suitable to generate the data required to explore the nurse-to-nurse hand-off communication?
2. Is the within-methods data triangulation, using content analysis hypothesis coding and LIWC, a suitable method for analyzing the data generated from the nurse-to-nurse change-of-shift hand-off communication?
Methods
Institutional Review Board
Approval to conduct the feasibility study was obtained through the University of Arizona's institutional review board and through the healthcare organization's (data collection site) institutional review board committee. Additionally, permission to conduct the study on the medical-surgical unit was obtained from the healthcare organization's chief nursing officer, institutional research department, and unit nursing manager.
Setting and Participants
The setting of the feasibility study was the Mountain West region. A convenience sample of RNs was obtained from a medical-surgical unit. The RNs in the feasibility study met the following inclusion criteria for the study: (1) working full-time in a medical-surgical unit; (2) have at least 3 months of experience in the medical-surgical unit and are not currently on orientation; (3) have provided care or continued care for a patient who has experienced a CE; and (4) are able to speak and read in English.
Data Collection
The primary investigator (PI) arrived at the selected unit 45 minutes before the end of the day shift. The PI talked with the charge RN to identify patients who had experienced or were experiencing a CE and the RNs providing care to those patients. The PI approached the RN providing care, explained the study, and obtained informed consent from the RN in a private location. CE is defined as an unexpected and sudden change in the patient's condition in the following areas: bleeding, pain, fever, change in output, change in respiratory status, and change in level of consciousness. CEs are precursors to sentinel events.17 The same procedure was repeated for the incoming RN assigned to provide care to the same patient. When the RNs were prepared to begin the hand-off, the PI started the digital audio recorder and stepped away. When the hand-off communication was complete, the RNs signaled to the PI, and the PI stopped the recorder. Additional data on hand-off communication were collected through observation. The observation data were collected concurrently with the digital audio recording. Two nurse-to-nurse hand-offs were observed for the feasibility study.
Data Analysis
The digitally audio-recorded hand-off communications were transcribed verbatim and verified for accuracy by the PI. Once the accuracy of the transcriptions was verified, the digital recordings were deleted from the digital audio recorder and the PI's computer. During the transcription of the hand-off communications, the confidentiality and anonymity of the nurses and any patient's protected health information were ensured by removing such information through deidentification strategies. Deidentification strategies were completed as follows: the patient's name and room number were transcribed as "patient," nurse's name as "nurse," provider's name as "provider," and so forth.
Content Analysis
Content analysis using hypothesis coding was used in this study to analyze "what," or the content of the hand-off communication in terms of data, information, and knowledge. The PI used the LIWC to analyze "how" the nurse-to-nurse hand-off was communicated. Thematic analysis was used to analyze the observation notes to identify emerging themes related to environmental factors and nonverbal communication potentially affecting the nurse-to-nurse hand-off communication.
The content analysis strategy used was hypothesis coding, which is an exploratory method. Hypothesis coding uses predefined codes, developed theoretically, about the content of the text data.23 The PI theorized data, information, and knowledge were being communicated in the hand-off communication. The PI conducted a literature review to determine the most relevant and current definitions of the concepts of data, information, and knowledge. The definitions for data, information, and knowledge were developed prior to data collection.
Data points are raw facts, which are unconnected and mean nothing individually.28-30 Information is at least two data elements connected together, creating meaning and a relationship that organizes the data into information.28,29 Knowledge is the cognitive processing and connecting of meaningful data/information relationships to other meaningful data/information relationships, thus creating information that has context and conceptual understanding, knowledge.28,29Table 1 provides the definitions and exemplars of data, information, and knowledge.
Linguistic Inquiry Word Count Analysis
An NLP program, LIWC provides insight into the psychosocial processes, cognitive processes, and the emotional context of the message being communicated.20 The transcribed nursing hand-offs were prepared for analysis according to the LIWC program recommendations. The preparation included saving each text data file as a Word document using a systematic naming procedure.21 The transcribed text files were "cleaned" prior to data analysis. Cleaning of the data files included reviewing for misspellings, abbreviations, and jargon to ensure that those words or phrases would be included in the analysis. In addition to checking for misspellings, abbreviations, and jargon, filler words, for example, "like" were changed to "rrlike" for the LIWC program to correctly analyze the word in the correct context.
Within-Methods Data Triangulation
Within-methods data triangulation was utilized to analyze the change-of-shift hand-off communication data. The within-methods triangulation integrated the content analysis results with the LIWC to provide a deeper contextual analysis of the same transcribed text data.6 The content analysis hypothesis coding provided the "what" of the hand-off content in terms of data, information, and knowledge. The LIWC analysis provided insight into how the content (words and language used) of the nurse-to-nurse hand-off message was communicated.6,20 The results of the LIWC and content analysis were combined and provided holistic three-dimensional analysis and representation of the participants' words and meaning. The within-methods triangulation data analysis process is an iterative process of continually analyzing and comparing the results from the content analysis to the LIWC analysis and synthesizing the results.
Results
Content Analysis Hypothesis Coding
The content analysis of the hand-off communication provided results related to the "what" of the hand-off content in terms of the data information and knowledge. The results of the content analysis hypothesis coding of the hand-offs provided the frequency and percentage of data, information, and knowledge in the nurse-to-nurse change-of-shift hand-off communication. The hand-off was comprised of 54% information, 35.5% data, and 10.5% knowledge. Table 2 contains the mean frequencies for the hypothesis content analysis. Examples of data elements in the hand-off included the following: a blood pressure value, individual diagnostic test results, a pain score, and a discontinued medication. Exemplars of information in the hand-off included the following: "Had an additional diagnostic test due to abnormal lab values"; "Specialist consult ordered due to symptoms"; and "Vital signs are normal." The communication of knowledge was present in the hand-off in the following exemplars: "On potassium protocol, but the potassium level was within normal limits, so I held it" and "Had repeated scans and compared it to back to previous scans, and said it was evolving."
Linguistic Inquiry Word Count
The transcribed nurse-to-nurse change-of-shift hand-off communication was analyzed using the LIWC. The LIWC analysis produced a significant amount of data on how the content (data, information, and knowledge) of the nurse-to-nurse hand-off message was communicated in terms of the meaning of the language used to communicate the message. The feasibility study focused on the ability of the LIWC to provide insight into the following data analysis variables: analytical thinking, clout, authenticity, emotional tone, time orientation, and drives. Table 3 presents a comparison of the identified LIWC variables' mean scores of hand-off communication to the LIWC program mean scores. The analytical score for the nursing hand-off was 21.49, indicating that the nurses were using a more personal and informal narrative storytelling thinking process to communicate the message.21
The nursing hand-off had a clout score of 64.13, indicating that the nurses were speaking as experts but were also anxious in the communication.19 The analysis of the hand-off for authenticity score was 14.54, indicating that the nurses were communicating patient information and not personal information. The LIWC variable of emotional tone for the hand-off scored at 23.57, indicating that the nurses were expressing more negative emotions (anxiousness, sadness, or fear) in the communication.21 The LIWC analyzes the communication for past, present, and future tense verbs. The nurses were proficient in using language to communicate past and present assessments, interventions, and evaluations but used minimal language to communicate future or anticipated patient events (Table 3). The next LIWC variable analyzed for the hand-off communication was the drive or motivation to communicate the message or information. The LIWC analyzed three areas of drive power, reward, and risk. The drive of risk was the focus because in each hand-off, the patient had experienced a CE. The drive of risk for the nursing hand-off communication score was 0.15, indicating minimal drive to communicate based on risk, danger, or fear.
Within-Methods Triangulation Data Analysis
The results of the content analysis hypothesis coding and the LIWC results were compared, contrasted, and then combined to provide a description of the nurse-to-nurse hand-off communication. The description of the hand-off message included what was being communicated in terms of data, information, and knowledge and how the hand-off was communicated, that is, the meaning of the words and the language used by the participants. The description was based solely on the words and language used by the nurses.
The hand-off consisted of primarily data and information with minimal knowledge being communicated. The hand-off message was communicated using an informal and narrative storytelling style of what the patient had experienced throughout the shift. The nurses communicated the hand-off with confidence and as experts. The hand-off was focused on communicating patient information rather than the nurses discussing personal information. The overall emotional tone of the hand-off message was more negative and anxious. The hand-off message was focused on communicating past and present event but not future or anticipated patient events. The drive to communicate the hand-off was minimally based on a risk, danger, fear, or concern despite the patients having experienced a CE.
Discussion
This feasibility study sought to address two aims: (1) examine the suitability of the data collection methods of digital audio recording and observing the hand-off to generate the data required to explore the nurse-to-nurse hand-off communication; and (2) explore the usefulness of within-methods data triangulation using hypothesis content analysis and the LIWC method for analyzing the data generated from the nurse-to-nurse hand-off communication. The data collection methods of observation and digital audio recording of the change-of-shift hand-off communications generated ample suitable data for the analysis.
Within-methods triangulation data analysis is an iterative process of comparing and contrasting the content analysis results with the LIWC results and referring back to the participants' words. Within-methods data triangulation provided a three-dimensional perspective of how the nurses communicated essential patient information such as clinical events. The content analysis hypothesis coding enabled the PI to discern what was being communicated in terms of data, information, and knowledge in the hand-off. The LIWC provided the PI with insight into the cognitive processes, psychosocial processes, and emotional processes the nurses used to communicate the hand-off. This method of analysis provided insight into not only the content communicated during the hand-off (data, information, and knowledge) but also the intentions, motivations, prioritization, emotional context, and thought processes of the nurses.
The secondary analysis of nurse-to-nurse communication in the acute care setting conducted by Renz et al,6 using the same within-methods data triangulation analysis, produced similar results. Renz et al6 were able to achieve a clear comprehension of the nurse's voice and perspective through what the nurse revealed and how the nurse said it. Additionally, Renz31 used the same analysis methods to assess nurse-to-physician communication in the nursing home setting. The results from that study provided an in-depth understanding of what was communicated and how the information was communicated between nurses and physicians in the nursing home setting.31
Limitations and Strengths
A potential limitation to analyzing the data is interpreting the raw output from the LIWC program. The first time reading the LIWC output is challenging due to how the raw data are visually presented. The two LIWC program manuals provided clear and concise examples and instructions on how to interpret the LIWC output.
A strength of this method was that no additional training was required to use the LIWC NLP or the within-methods data analysis. Other strengths of this feasibility study were the methods used to collect the hand-off communication data and the analysis of that data. An additional strength of the within-method triangulation data analysis was that this method requires the nurse researcher to fully immerse himself or herself into the data. The feasibility study provided the required evidence to proceed with future studies.
Recommendations
The data collection methods provided a wealth of quality data for analysis. The within-methods data triangulation analysis using an LIWC NLP and content analysis provided a three-dimensional depiction of the hand-off communication. We recommend for future studies to use the within-methods data triangulation and LIWC program to research the following types of healthcare communication: nurse to another healthcare provider (nurse practitioner, physician assistant, and physician), interfacility hand-offs, intradepartmental hand-offs, and physician-to-physician hand-off communication. Additionally, we recommend using the LIWC program to analyze nursing narrative notes to provide insight into the cognitive processes and the drive to communicate patient information.
CONCLUSION
The use of an NLP program in healthcare research is not limited to data mining the EHR or converting unstructured data to structured data. An NLP program such as LIWC is a viable and valuable research tool to analyze how nurses are communicating essential patient information from one nurse to another. The within-methods data triangulation analysis of using content analysis and the NLP advances the researcher's capability to gain greater breadth and depth of the context and meaning in text data to address the research questions.
The data analysis triangulation method has now been tested in two separate nursing focused studies: a secondary analysis of transcribed nurse interviews6 and transcribed nursing hand-off communications. This triangulation method demonstrates the ability to analyze communication from a different but complementary perspective compared to the traditional forms of content analysis. The within-methods data triangulation can be implemented in various nursing research studies utilizing diverse forms of text data. The information obtained from this study has informed the research protocol and methods for a larger study examining nurse-to-nurse change-of-shift hand-off communication.
References