Assessment and management of pain, agitation, or some combination of both is an essential part of neonatal nursing care. Pain is subjective, and unfortunately, a vulnerable population such as medically fragile and chronically ill infants cannot self-report pain.1 The challenge of assessing and managing pain in those unable to self-report has led to the development of at least 65 observational pain scales, including several scales for use with critically ill infants.2-4 The selection of appropriate infant pain scales requires a consideration of the specific characteristics of the neonatal intensive care unit (NICU), including types of patients and the preferences of NICU care providers.5-10
The Neonatal Pain, Agitation, and Sedation Scale (N-PASS) was developed to systematically assess pain, agitation, and sedation levels in critically ill infants with acute and/or ongoing pain.5 This pain scale was originally validated in a small population of ventilated and/or postoperative neonates.5,6 N-PASS construct validity has since been established for infants in cardiac intensive care units and NICUs.11-14 Variable validity has been reported for the pain portion of the N-PASS in specific subpopulations, including mechanically ventilated infants,4 ventilated preterm infants,15 chronic pain in mechanically ventilated infants,16 long-term pain and sedation in nonventilated patients, and acute postoperative patients.17
In our institution, the valid and reliable Neonatal Infant Pain Scale (NIPS) was used to assess pain but required a separate tool to assess the need for sedation. Additionally at the time of study, our institution did not have policies in place to order sedation goals based on an infant's clinical situation. The sedation portion of the N-PASS has been criticized for a lack of evidence to support its validity, particularly with infants who are only mildly sedated.9 Although a number of studies have examined the use of the N-PASS among ventilated infants,6,10,18-21 none have examined its use in medically fragile or medically complex and chronically mechanically ventilated infants. Infants with medical complexity have been described as "children with multiple significant chronic health problems involving multiple organ systems, which result in functional limitations, high health care needs or utilization, and often requires need for, or use of, medical technology."22,23 For this study, the terms "medically fragile and chronically mechanically ventilated" were used, and were defined as infants with at least 3 severe chronic conditions or diagnoses, and/or a requirement of over 30 days' ventilation support.
In addition to being valid and reliable, infant pain or sedation scales should be evaluated as useful by the nurses who use them. Therefore, we examined the clinical validity, feasibility, and utility of the N-PASS compared to the NIPS when used by bedside nurses to evaluate medically fragile and chronically mechanically ventilated infants in an urban level IV NICU. The study aims were to:
1. Examine associations between patient characteristics and N-PASS scores (N-PASS pain and N-PASS sedation scores).
2. Evaluate agreement of bedside nurses' perceived assessment of pain/agitation and sedation with N-PASS scores.
3. Evaluate associations of bedside nurses' N-PASS and NIPS scores.
4. Calculate interrater reliability between bedside nurse volunteers' and bedside nurse investigators' NIPS, N-PASS pain, and N-PASS sedation scores.
These study aims were aligned with recommendations from the American Society for Pain Management in Nursing to use a hierarchy of pain assessment techniques by determining agreement of behavioral pain scores (N-PASS and NIPS) with proxy reports of pain (bedside nurses' perceptions).24 The nursing professional practice model, Relationship-Based Care,25 provided the guiding framework for this study to generate meaningful data that will assist in decision-making for infant pain and sedation care management.
What This Study Adds
* The N-PASS provided an objective measure of sedation and pain that reflected nurses' assessments.
* Our study provides more evidence to support the validity and clinical utility of the N-PASS to chronically mechanically ventilated and medically fragile infants with complex medical histories in a level IV NICU.
* N-PASS pain and NIPS scores were strongly associated and nursing surveys indicated their clinical utility for guiding pain management.
* Training to achieve good to excellent interrater reliability for N-PASS pain and sedation scoring took approximately 10 minutes.
METHODS
In this prospective observational clinical study, infants in a 64-bed level IV NICU were assessed by 9 bedside registered nurse investigators and 15 bedside registered nurse volunteers for pain, agitation, and sedation using the N-PASS and the NIPS. This study was developed by the NICU Nursing Research and Evidence-Based Practice committee composed of the 9 nurse investigators who had a mean of 5.6 years of NICU nursing experience. After receiving Nursing Research Council and Institutional Review Board (IRB 2018-2061) approval, bedside registered nurses with at least 3 months of NICU experience were recruited to participate in this study. Written consent was obtained from bedside registered nurse volunteers to participate in this study who were colleagues of the bedside registered nurse investigators. These 15 bedside registered nurse volunteers were required to complete the N-PASS training and meet interrater reliability criteria to participate in the study. The study was conducted from November 2018 to August 2019.
Participants and Setting
Fifteen bedside registered nurse volunteers representing all nursing shifts, with a mean of 3.8 years' NICU nursing experience (range = 6 months to 15 years) who met interrater reliability standards, assessed infants' pain/agitation and sedation in a 64-bed level IV NICU between November 2018 and August 2019. The NICU is part of a 364-bed, free-standing, university-affiliated, not-for-profit urban children's hospital in Illinois that cares for neonates with complex medical needs. Assessments were obtained for all infants except those younger than 24 hours, as newborn transitioning can last for several hours.26,27 Infants more than 12 months' postnatal age were also excluded because that is the usual age of transition from this NICU. In addition, infants with suspected neurological injury were not included until after the 72-hour period of induced hypothermia, as this treatment may mask physiologic responses to pain and/or agitation.
Sample Size
Power analysis was performed using [kappa] agreement as the primary analysis. Based upon an assumption of conservative estimates of Cohen's [kappa] (0.5-0.6) and proportion of agreement (0.7), a sample size of 200 observations was needed.
Instruments
Neonatal Infant Pain Scale
The NIPS is an interval scale that measures pain by quantifying 6 behaviors, from 0 (no pain behaviors) to 7 (most possible pain behaviors), originally validated in preterm (gestational age <37 weeks) and full-term newborns (gestational age >37 weeks to 6 weeks after delivery) with acute postoperative pain and pain in response to medical procedures,28 but has been used in NICUs up to 1 year of age.15,28,29 Five categories-behavior, facial expression, breathing patterns, motor activity (arms and legs), and state of arousal-are each scored a 0 or 1. The sixth category, cry, is scored 0, 1, or 2, for a maximum NIPS score of 7. Content, construct, and concurrent validity (with a visual analog scale) have been well-established; the reported internal consistency (Cronbach's [alpha] 0.87-0.95) and interrater reliability of the NIPS are excellent (Pearson's correlations 0.92-0.97).15,29,30 However, the NIPS has not been validated to assess chronic neonatal pain.15,30
Neonatal Pain, Agitation, and Sedation Scale
The N-PASS is an interval scale with 2 subscales, pain/agitation and sedation.6 Five behavior and physiologic categories-crying/irritability, behavior/state, facial expression, extremities/tone, and vital signs (heart rate, respiratory rate, oxygen saturation [SpO2], and/or blood pressure)-are each scored a 0, 1, or 2 for a maximum N-PASS pain score of 10 for infants whose gestational age is more than 30 weeks. Like the Premature Infant Pain Profile (PIPP),31,32 N-PASS pain scores incorporate a scoring adjustment for preterm infants less than 30 weeks' gestational age, allowing for a maximum score of 11 to correct for preterm infants' limited ability to maintain robust behaviors. Construct and concurrent validity (with the PIPP) have been established, test-retest reliability is demonstrative (Spearman's [rho] correlation 0.874), internal consistency is very good (Cronbach's [alpha] 0.84-0.89), and interrater reliability of the N-PASS is excellent (Pearson's correlations 0.95-0.97).6,7
The N-PASS sedation score is derived from the same 5 behavior and physiologic categories as the pain score. Each category is scored a 0, -1, or -2 for a maximum score of -10 and a score of 0 indicating no sedation. Construct and criterion validity (with the University of Michigan Sedation Scale) have been established, internal consistency is high (Cronbach's [alpha] 0.85-0.94), and interrater reliability of the N-PASS is excellent (Pearson's correlations 0.95-0.97).6,7 Factor analysis indicated that 2 factors, pain/agitation and sedation, accounted for 80% to 88% of the variance, confirming the 2 subscales of the N-PASS.33
Bedside Nurse Volunteers' Assessment of Pain/Agitation and Sedation
A 7-question survey was developed for this study by the bedside nurse investigators to record bedside nurse volunteers' assessments of pain/agitation and sedation. The survey includes 2 multiple-choice questions asking whether changes should be made to the infant's pain or sedation medications, and a third multiple-choice question asking about nonpharmacologic methods used to console/soothe the infants. In addition, the survey includes 4 Likert scale questions to assess the clinical utility of the NIPS and the N-PASS, meaning to what extent does each instrument capture an infant's level of pain/agitation and sedation (1 = not at all, 2 = minimally, 3 = somewhat well, and 4 = very well) (see Supplemental Appendix 1, available at: http://links.lww.com/ANC/A137).
Bedside Nurse Volunteers' Evaluation of the N-PASS Versus the NIPS Post-Study Data Collection
A 9-question survey was developed for this study by the bedside nurse investigators to record bedside nurse volunteers' assessment of the clinical utility of the N-PASS and the NIPS after all data were collected. The survey includes questions about amount or time taken to use the scale, ease of use, and training. Each question is rated with a Likert scale (from 1 = easiest to use/shortest time to complete to 5 = hardest to use/longest time to complete). The survey also includes 2-multiple choice questions: "What could have made N-PASS training easier to learn and/or understand?" (Select all that apply-better technology, videos that better represent our population, more one-on-one practice using the N-PASS on real patients, or other); and "Would implementation of the N-PASS improve patient care?" (Yes, no, or maybe.) Both questions are followed up with an open-ended question to solicit a free text response.
N-PASS Training
NICU bedside nurse volunteers were trained to use the N-PASS by first completing a 5-minute online training module that was created by the bedside nurse investigators. Bedside nurse volunteers' then assigned N-PASS scores to 3 videos: 1 term infant not in pain and not sedated, 1 preterm infant mildly sedated and in pain, and 1 term infant heavily sedated. Bedside nurse volunteers were considered adequately trained if N-PASS pain and sedation scores for all 3 videos were identical to predetermined values or differed by only 1 point. Training was reinforced between bedside nurse volunteers and bedside nurse investigators by discussing the rationale for N-PASS video scores.
NIPS Training
NICU bedside nurses were trained to use the NIPS during NICU orientation by both an educator in a class format and their preceptor during unit orientation. Prior to this study, the NIPS was the pain scale routinely used to assess pain on all patients until they transitioned out of the NICU, so the bedside nurse volunteers were very familiar with NIPS use and documentation. Nurses do not receive additional training on the NIPS after their NICU orientation, thus no additional NIPS training was provided to the bedside nurse volunteers prior to the start of the study.
Study Procedures
While providing NICU patient care and immediately after routine infant assessments, a bedside nurse volunteer and a bedside nurse investigator simultaneously and independently assigned N-PASS and NIPS scores. Nurses independently recorded scores, but scores were not totaled or compared between the bedside nurse volunteer and bedside nurse investigator until study completion.
At the time of assessment, bedside nurse volunteers completed the 7-question Bedside Nurse Volunteers' Assessment of Pain/Agitation and Sedation. At the conclusion of the study, after all data were collected, bedside nurse volunteers who participated in this study were sent an electronic survey to obtain additional feedback about the clinical utility of the N-PASS compared with the NIPS.
Data Analysis
Data were reported as frequencies and percentages for categorical variables and as means and standard deviations for continuous variables. For comparison of outcomes, each assessment was considered. Agreement was dichotomized for N-PASS scores and nurses' recommendations for pain/agitation and/or sedation medications.
N-PASS scores and nurses' recommendations for pain/agitation treatment were coded as:
* agreement if the N-PASS score was 0, 1, or 2 and the nurse recommended a decrease/no change in treatment;
* agreement if the score was 3 or greater and the nurse recommended an increase in treatment; and
* disagreement for all other combinations.
N-PASS sedation scores and the nurses' recommendations for sedation treatment were coded as:
* agreement if the sedation score was 0 and the nurse recommended a decrease/no change;
* agreement if the score was between -1 and -5 (mild to moderate sedation) and the nurse recommended no change or to increase or initiate medication;
* agreement if the score was greater than -5 and the nurse recommended a decrease or no change in sedation medication; and
* disagreement for all other combinations.
Repeated assessments on some infants occurred on different days. To calculate a confidence interval (CI) between 2.5 and 97.5, a bootstrap method was used,34 where a random sample of 88 patients with replacement from the original data set was obtained, and all assessments from randomly selected patients were used. The [kappa] statistic for the random sample was calculated, and the process was repeated 500 times.
Study data were collected and managed using the Research Electronic Data Capture (REDCap) tool hosted at Northwestern University.35,36 REDCap is a secure, web-based software platform designed to support data capture for research studies, providing (1) an intuitive interface for validated data capture; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for data integration and interoperability with external sources. Data analyses were conducted with SAS version 9.4 (SAS Institute Inc, Cary, North Carolina).
RESULTS
Bedside registered nurses performed 202 assessments of 88 infants in their care during their shifts. All nursing shifts were represented. Infants' median corrected gestational age at the time of observation was 42.42 weeks (Table 1), with 81.6% of the observations completed on infants with a history of prematurity. Of the diagnoses highlighted in the patient characteristics table, infants had an average of 3.3 diagnoses (Table 2).37 Most assessments (90.8%) were completed for infants who required respiratory support; 69.8% required invasive ventilation, including conventional ventilation, high-frequency jet ventilation, or high-frequency oscillatory ventilation (Figure 1). Of those ventilated, 21% of assessments were performed for infants who were tracheostomy- and ventilator-dependent. At the time of assessment, infants required invasive ventilation support for a mean of 68 days (SD 67.89) after removing an outlier of 321 days. Additionally, infants had a mean postoperative status of 21 days (SD 20.46) after removing an outlier of 125 days. There were no significant associations between N-PASS pain or N-PASS sedation scores and infant characteristics, such as prematurity, presence of bronchopulmonary dysplasia, or number of days ventilated.
There was a strong association between N-PASS pain and NIPS scores among nurses (intraclass correlation coefficient [ICC] = 0.9439, P < .001). There was also good interrater reliability between bedside nurse volunteers' and bedside nurse investigators' N-PASS pain scores (ICC = 0.83, 95% CI = 0.78-1.13), and NIPS scores (ICC = 0.85, 95% CI = 0.80-0.88). There was also moderate agreement between N-PASS pain scores and bedside nurse volunteers' recommendations to initiate, increase, decrease, or make no change in current pain medication therapy ([kappa]= 0.52, 95% CI = 0.41-0.63). During 96% of assessments, bedside nurse volunteers reported N-PASS scores captured pain "well" or "very well." In 92.6% of assessments, nonpharmacologic interventions were provided to comfort infants (see Supplemental Appendix 1, available at: http://links.lww.com/ANC/A137).
For N-PASS sedation scores, there was excellent interrater reliability between bedside nurse volunteers and bedside nurse investigators' scores (ICC = 0.94, 95% CI = 0.92-1.25). There was also strong agreement between N-PASS sedation scores and bedside nurse volunteers' recommendations to initiate, increase, decrease, or recommend no change in infants' sedation treatment ([kappa]= 0.988, 95% CI = 0.95-1). During 93% of assessments, bedside nurse volunteers reported that N-PASS scores captured sedation well or very well.
Nine of the bedside registered nurse volunteers completed the poststudy survey (67% response rate). Two nurses left the institution before the completion of the study and therefore were unable to complete the survey. Overall, the bedside nurse volunteers reported that it was possible to be trained to use the N-PASS tool during routine shift hours. Further, the bedside nurse volunteers reported that the N-PASS demonstrated better clinical utility than the NIPS, particularly for use with infants who are chronically intubated and have complex medical histories (Table 3).
DISCUSSION
Consistent with previous studies, we found the N-PASS to be valid for use in infants experiencing acute, postoperative, and prolonged pain.6,7,19,38-40 Our study provides more evidence to support the validity and clinical utility of the N-PASS to chronically mechanically ventilated and medically fragile infants with complex medical histories in a level IV NICU. In addition, we provided evidence of the concurrent validity of the N-PASS pain subscale with the NIPS,15 our NICU standard for pain assessment at the time of this study. Similar to a recently published quality improvement project, we found a strong association between the N-PASS and the NIPS, which indicated that the N-PASS is at least as sensitive as the NIPS to evaluate pain.15
The goal of observational assessment tools is to standardize the approach to pain and sedation assessment among bedside nurses and across shifts to improve infants' pain and sedation management. Thus, we also surveyed bedside nurse volunteers' expert opinions about the extent to which the N-PASS and the NIPS captured infants' pain and sedation. Bedside nurse volunteers reported that the N-PASS captured infant pain/agitation and sedation well or very well. Surprisingly, there were no associations between N-PASS pain and sedation with infant characteristics.
Pain tools that are reliable, valid, and feasible are necessary.1 Although there are many infant pain scales that have been validated for different types of pain,1,4,16 it is not feasible for nurses to use multiple pain scales for different clinical situations. Training to achieve good to excellent interrater reliability for N-PASS pain and sedation took approximately 10 minutes, which demonstrated the feasibility of introducing N-PASS pain and sedation subscales in a level IV NICU with medically fragile and chronically ventilated infants. Further, bedside nurse volunteers indicated through surveys that the clinical utility of the N-PASS, which measures both pain/agitation and sedation, exceeds that of the NIPS, which does not account for agitation or measure levels of sedation.
In addition to validity and reliability, infant pain scales must also demonstrate good clinical utility,1 and also be preferred by clinicians who use them in neonatal pain assessment.19 N-PASS pain and sedation scores were in moderate to strong agreement with nurses' pain and sedation treatment recommendations. Thus, this study suggests that the clinical utility of the N-PASS as a measure of both pain and sedation exceeds that of the NIPS, which only measures pain.
N-PASS Pain
A recent systematic review examined the validity, reliability, and clinical utility of different pain scales for ventilated infants.16 The review concluded that the N-PASS pain score is effective for pain assessment in mechanically ventilated infants, but other scales may be more useful for assessing acute and postoperative pain.16 In contrast, we found that bedside nurses were able to use the N-PASS without difficulty for infants who were both postoperative and/or chronically ventilated (mean ventilation 67.23 days). Another systematic review reported good to excellent reliability of the N-PASS for assessing acute pain in preterm to term infants, but none of the studies in the review examined infants beyond 47 weeks' postconceptual age.17,40 Our study included infants with a corrected gestational age of 24.9 to 75.9 weeks.
N-PASS Sedation
The N-PASS is unique compared with other objective pain scales because of its sedation subscale. Contrary to previous criticism that the sedation portion of the N-PASS lacks evidence to support its validity for use in infants who are mildly sedated9 or adequately and undersedated,41 we found excellent agreement between sedation scores and bedside nurse volunteers' recommendations in infants who did not need sedation or were adequately sedated. Adequate levels of sedation and recommendations to initiate or increase use of sedatives are variable and depend upon an infant's clinical status.4,40,41 At the time of the study, it was not our practice to have a standardized process for desired level of sedation. The nurses' recommendations were considered goals for desired levels of sedation. We found that the agreement between N-PASS sedation scores and nurse recommendations demonstrated that the N-PASS provides a systematic and consistent method to guide sedation management for critically ill infants. Further, the NICU bedside nurse volunteers' recommendations to alter sedatives depended upon an infant's clinical condition. For example, in our study, an infant who had been on continuous pain and sedation infusions since birth, and was restless despite nonpharmacologic comfort interventions, had an increased heart rate and desaturation levels to the 70s. The bedside nurse volunteer assigned a sedation score of 0 and recommended a demand dose of sedative. It was the nurse's expert assessment that an extra dose of sedative was necessary to reestablish an effective level of sedation and maintain optimal levels of ventilation and oxygenation. This case suggests the need for a standardized sedation scale in NICUs to address case-based goals of sedation (ie, need for mild, moderate, or deep levels of sedation).
Nonpharmacologic Treatments
We also assessed the use of pharmacologic and nonpharmacologic pain and sedation treatments in ventilated and nonventilated NICU patients. The efficacy of nonpharmacologic pain and comfort strategies, such as nonnutritive sucking, facilitated tucking, skin-to-skin care, and human milk, is well established.42-44 Nonpharmacologic interventions were provided to comfort infants during 92.6% of study assessments. Nonpharmacologic interventions used were nonnutritive sucking, swaddling, repositioning, patting, holding, soothing sounds, and oral sucrose. These interventions may have helped to alleviate pain or agitation while avoiding the sedating effects of pharmacologic interventions.43
The prevalent use of nonpharmacologic comfort strategies emphasizes the importance of using a multimodal approach to pain/agitation treatment. Historically, nurses have been noted to be less consistent about charting nonpharmacologic interventions.45,46 As a result of our study, the electronic health record was updated to facilitate documentation of nonpharmacologic interventions more readily for pain/agitation management.
Study Limitations
This study has several limitations. First, a convenience sample in a single setting was used; therefore, the results are not generalizable for N-PASS use among infants with medical conditions not described in this study. Second, repeated assessments on some infants occurred, but the N-PASS was used at different time points throughout an infant's NICU admission, and bootstrap methods were used to control for multiple assessments. Third, agreement rules for sedation assessment were set, but at the time of the study, patient-specific sedation goals were not routinely used to guide sedation management in our NICU. Fourth, our NICU uses a primary care team model. Therefore, nurses' familiarity with infants may have influenced their responses to patients' behaviors and their survey responses. Nevertheless, such familiarity is important for guiding therapeutic care that includes a well-informed balance of nonpharmacologic and pharmacologic interventions. Finally, of the almost 200 bedside nurses in the NICU at our institution, only 15 volunteered to participate in the study and only 9 completed the poststudy survey, which was an anonymous evaluation of the N-PASS and its clinical utility. Participating bedside nurse volunteers in the study represented nurses with 6 months to 15 years of NICU nursing experience, who worked both day and night shifts. However, these nurse volunteers were a self-selected population, which reduces the generalizability of our findings to all NICU nurses and nurses in other units who routinely assess infant pain.
Impact on Patient Care
Based on the study results, pain and sedation policies at the study site have been changed. The NIPS has been replaced with the N-PASS as the NICU's standardized pain and sedation assessment tool. In addition, case-based goals for sedation are now part of an order set for infants receiving sedatives. Moreover, a more comprehensive list of nonpharmacologic interventions has been added to our electronic health records.
CONCLUSION AND NEXT STEPS
Our findings suggest that the N-PASS provides a valid, reliable, and clinically useful objective measure of pain and sedation that reflects nurses' expert assessments. In addition, ours is the first study to suggest validity of the N-PASS for use with medically fragile and chronically ill infants hospitalized in a level IV NICU. Replication of our study to confirm validation in this vulnerable population is recommended. Our findings also demonstrated concurrent validity of N-PASS pain with the NIPS; however, bedside nurse volunteers reported that the N-PASS demonstrated better clinical utility in the NICU. Further evaluation of the N-PASS and its influence on pain and sedation management for infants, preterm infants, and infants requiring chronic ventilatory support is needed.
References