Introduction

Mechanical ventilation (MV) is used in the intensive care unit (ICU) to maintain life by assisting or replacing spontaneous breathing. However, prolonged MV increases the risk of complications such as ventilator-associated pneumonia and ventilator-induced lung injury (Ghauri et al., 2019; Trivedi et al., 2022; Worraphan et al., 2020). Furthermore, early MV withdrawal may cause breathing problems that lead to further distress and injury and extubation failure. Reintubation also exposes patients to life-threating complications (Sanfilippo et al., 2021; Torrini et al., 2021). Therefore, it is important to time MV withdrawal to coincide with the point at which patients are capable of self-breathing without undue effort (Sato et al., 2021). A readiness-to-wean assessment should include different factors that evaluate the respiratory, cardiovascular, and neurological statuses of the patient (Deschamps et al., 2019; Fathy et al., 2020). Thus, complex clinical decision making based on accurate assessment is required (Starnes et al., 2019).

In Japan, several issues affect the practice of MV weaning. Although a standard MV weaning protocol was established in 2015 (Japan Society of Intensive Care, Japan Society of Respiratory Care Medicine, & Japan Critical Care Nursing Association, 2015), our experience indicates that this protocol is insufficient to enable adequate patient assessment by nurses. Moreover, no assessment tools are currently available in Japan for predicting MV weaning success. Therefore, a valid and reliable assessment tool that can be used by Japanese nurses is urgently needed. The results from such an assessment tool may be shared among multidisciplinary ICU teams to improve outcomes in terms of decreasing MV duration and improving extubation success rates.

The Burns Wean Assessment Program (BWAP) is a valid and reliable assessment tool that has shown robust results in predicting the success of MV weaning (Burns et al., 2010). The BWAP (also known as Burns' weaning; Burns et al., 2000) was developed by researchers at the University of Virginia in the United States and is a comprehensive clinical checklist and scoring tool for MV weaning. This assessment program aims to support the multidisciplinary team in implementing safe MV weanings (Burns et al., 2010). The BWAP includes 26 factors (Table 1) known to interfere with safe ventilator weaning (Burns et al., 2010). A 5-year study of the BWAP was previously conducted across five adult critical care units. Advanced practice nurses who conducted the BWAP assessments received extensive training and reported a high level of interrater reliability (Burns et al., 2010). The results showed that 88% of 1,889 adult patients who had been on MV for >= 3 days and who had a BWAP score of >= 14 points achieved successful MV weaning (Burns et al., 2010). In addition, Yazdannik et al. (2012) established the reliability of the BWAP with a Cronbach's alpha of .85. Furthermore, the BWAP has been used successfully in multiple countries (Al-Faouri et al., 2014; Jeong & Lee, 2018; Jiang et al., 2014).

Table 1 Burns Wean Assessment Program (BWAP) Factors and Modified Points for Japanese BWAP

The BWAP involves conducting a comprehensive nursing assessment of the 26 factors based on patient status (Table 1). For example, the factor of hemodynamic status stability (pulse rate, cardiac output) is judged using the following criteria: stable heart rate and rhythm and blood pressure without the use of vasoactive agents or any oral agents administered on an as-needed or stat basis to control the heart rate and/or blood pressure. In Japan, specific criteria are followed for using lower vasoactive agent dosages. In addition, parameters specific to the healthcare environment in Japan may be added to the BWAP to make assessment easier. This justifies the development of a Japanese version of the BWAP (J-BWAP). In addition, it may be possible to use e-learning to teach nurses from various hospitals regarding proper J-BWAP implementation. However, although e-learning has been shown to be effective in improving knowledge of amplitude-integrated electroencephalography (aEEG) in a neonatal ICU (Poon et al., 2015), it remains unclear whether (a) it is an appropriate teaching modality for Japanese nurses to learn and competently apply the J-BWAP and (b) it is a feasible tool in terms of acceptability, demand, implementation, and adaptation (Bowen et al., 2009).

Therefore, this study was designed to develop J-BWAP e-learning materials and to assess its usage by ICU nurses involved in clinical MV weaning. Furthermore, in this study, we examined the effect of J-BWAP e-learning materials on the MV weaning assessment competence of ICU nurses.

Two hypotheses were posited in this study: (a) The intervention groups will achieve a higher learning effect (i.e., a higher total J-BWAP score) than the control group (Hypothesis 1), and (b) Intervention Group 2 will achieve a higher learning effect than Intervention Group 1 (Hypothesis 2).

Methods

Study Design and Sample

This study was a pretest-posttest, nonrandomized control group feasibility study. The primary outcome measure was the difference in online pretest and posttest total scores between the intervention groups and the control group. A purposive sample of six hospitals located in suburban areas was selected. All had over 300 beds. Hospitals A and B were university hospitals, and Hospitals C, D, and F were general hospitals in areas contiguous to Tokyo. Hospital E was a university hospital located in Tokyo. The participants were all nurses who were purposively sampled from each hospital. The inclusion criteria were as follows: working in the ICU for more than 1 year and having a Clinical Ladder II-III rating (clinical experience of approximately 3-9 years).

Measures

Stage 1: develop the Japanese version of the Burns Wean Assessment Program

Stage 1 was conducted from March to June 2016. Permission was first obtained from the original developers of the BWAP (S. M. Burns, personal communication, March 19, 2016) to create and use the J-BWAP. The original BWAP was translated by the researchers and then adapted for Japanese patients under the supervision of one anesthesiologist, two critical care certified nurse specialists (CCNSs with master's degrees), and a critical care nursing graduate student. Consent was obtained to make minor changes in several item criteria and definitions from the original version for the Japanese version (S. M. Burns, personal communication, March 19, 2016) to make it easier for nurses to understand the program and to better reflect Japanese clinical practice standards.

Stage 2: develop the e-learning teaching materials

Stage 2 was conducted from July to December 2016. The J-BWAP e-learning materials were developed by the lead researcher (R. K.) in collaboration with experts in MV weaning, including one anesthesiologist and two nurses (one CCNS with a master's degree and one critical care nursing graduate student). An e-learning site with web content was developed based on the created teaching materials. A web design expert provided advice regarding the development and operability of the web content. The educational content addressed the following three topics for each of the 26 J-BWAP factors: (a) assessment indicators, (b) the relevance of the factor to ventilator weaning, and (c) the knowledge necessary to assess the factor. The course may be completed in approximately 1 hour, and the e-learning materials are accessed by pointing and clicking the cursor. The content includes illustrations and video commentary on respiratory assessment (e.g., a video that teaches the proper assessment of accessory muscles). To establish content validity, six critical care nursing graduate students with sufficient knowledge and experience in MV weaning evaluated the content of the e-learning materials using either a protocol established by the Japan Society of Intensive Care, Japan Society of Respiratory Care Medicine, and the Japan Critical Care Nursing Association (2015) or the J-PAD (Japanese guidelines for the Management of Pain, Agitation, and Delirium; Committee for the Development of Japanese Guidelines for the Management of Pain, Agitation, and Delirium in Intensive Care Unit, Japanese Society of Intensive Care Medicine, 2014). The final version was created after the contents were found acceptable for clinical use in Japan.

Stage 3: pretest and posttest

Stage 3 was conducted from January to April 2017. The pretest and posttest were developed to measure the educational effects of the e-learning materials. The test assessed the accuracy of participant scores on the 26 items of the J-BWAP based on e-learning information on simulated patients. For each item, a correct answer was scored as 1, whereas an incorrect answer (including "cannot assess") was scored as 0, for a maximum possible total score of 26. The participants assessed and entered their answers on the web. Six nurses (critical care nursing graduate students and CCNSs with a master's degree) with sufficient knowledge and experience in MV weaning evaluated and established the content validity of the pretest and posttest results.

Feasibility Evaluation

To determine the feasibility of the developed J-BWAP and e-learning materials, the participants were asked to respond to five open-ended questions based on the feasibility criteria of Bowen et al. (2009). The questions addressed the following four parameters: (a) acceptability, "How do respondents react to the educational intervention through the J-BWAP e-learning?"; (b) demand, "Would the respondents use the J-BWAP in practice?"; (c) implementability, "Is the J-BWAP feasible for implementation in clinical situations?"; and (4) adaptability, "Are modifications of the e-learning materials adaptable?"

Data Collection

Data were collected from August to December 2017. To minimize intergroup contamination, the participants were assigned to intervention and control groups based on hospital affiliation. The intervention groups were from University Hospitals A and B and General Hospitals C and D, whereas the control group was from University Hospital E and General Hospital F. All of the hospitals were located in suburban areas of Tokyo except for Hospital E, which was located in Tokyo proper. Participants in the intervention group were assigned to either (a) Intervention Group 1 (one intervention per week) or (b) Intervention Group 2 (three interventions per 2 weeks). All of the participants received questionnaires and pretest access by email. Only the intervention groups were provided access to the online e-learning materials website. On the basis of group assignment, the intervention groups were instructed to use the e-learning materials either once or 3 times during the 1-week period. After 1 week, the participants in the intervention and control groups received an email regarding access instructions for taking the posttest.

Sample Size

To the knowledge of the authors, no previous studies have measured the learning effectiveness of e-learning in the context of content similar to this study. Thus, we referred to a previous study by Kato et al. (2015), which reported a 17% increase in knowledge after a 23-item knowledge test for continuous learning was conducted in an e-learning development study for midwives. Therefore, the sample size in this study was calculated using G*Power (University of Dusseldorf, Dusseldorf, Germany). The difference in pretest and posttest total scores between the intervention groups and the control group was estimated as 3 points, with a standard deviation of 3. Using a significance level of alpha = .05 and a detection power of 0.80, the required sample size was thus calculated as 14. On the basis of an estimated dropout rate of 10%, the sample size of each group was set at 16 participants. The sample was divided into three groups, namely, the control group, Intervention Group 1 (one intervention per week), and Intervention Group 2 (three interventions per week), giving a total required sample size of 48.

Statistical Analysis

Quantitative data were analyzed using the Mann-Whitney U test, Kruskal-Wallis test, covariance analysis, Steel-Dwass test, and Fisher's exact test using the statistical package EZR (Kanda, 2013). The level of significance was set at p < .05. If there were dropouts between the pretest and posttest, the analysis was performed as initially assigned, and a statistical intention-to-treat analysis was used to reduce any resulting bias. Qualitative data analysis was performed based on the feasibility criteria of Bowen et al. (2009).

Ethical Considerations

This study was approved by the Ethics Committee of St. Luke's International University (No. 17-A033) and was conducted in accordance with the Declaration of Helsinki. The participants were fully informed regarding the objectives of the study and rules of participation. Voluntary participation, right to withdrawal, and confidentiality were all assured.

Results

Participant Characteristics

Forty-eight participants met the inclusion criteria and completed the pretest. Thirty-five participants (dropout rate: 27.1%) completed both the questionnaires and pretest, and 32 (dropout rate: 33.3%) completed the posttest. The responses of the 32 participants were examined, and an intention-to-treat analysis was conducted. Of the 32 participants, 28 completed the posttest questionnaires (Figure 1). The demographic characteristics of the participants in the intervention and control groups are presented in Table 2. The only significant differences between the intervention and control groups were that the control group had a relatively higher educational background and relatively more additional participation in outside-hospital educational programs.

Figure 1. Flowchart of Participants

Table 2 Demographic Data of Intervention and Control Groups

Changes in Total Scores Between Pretest and Posttest

Between-group differences in total pretest score

The average pretest scores were 19.7 (SD = 1.8) in the intervention groups and 20.4 (SD = 2.0) in the control group. The Mann-Whitney U test revealed no significant difference in pretest scores between the intervention groups (median interquartile range [IQR]: 20 [18-21]) and the control group (median [IQR]: 21 [19-22]).

Between-group differences in total pretest and posttest scores

The average posttest scores were 21.5 (SD = 3.0) in the intervention groups and 19.0 (SD = 2.6) in the control group. The Mann-Whitney U test showed the improvement in test score from pretest to posttest to be significantly higher in the intervention groups than the control group (median [IQR]: 2 [0-3] vs. -1 [-3-0], p = .019). Covariance analysis using the pretest total score as the covariate yielded no interaction (p = .17).

Comparison of Total Pretest Score Among the Three Groups

The average pretest scores were 18.0 (SD = 3.1) for Intervention Group 1, 19.7 (SD = 1.8) for Intervention Group 2, and 20.2 (SD = 2.1) for the control group. The Kruskal-Wallis test found no significant difference among the scores (p = .256), with the median of the Kruskal-Wallis test showing no significant differences among Intervention Group 1, Intervention Group 2, or the control group (median [IQR]: 1.0 [0-2.75] vs. 2.0 [0-3.0] vs. -0.5 [-3.25 to -0.75], respectively). Therefore, two sets of groups were formed to compare and contrast the score differences using the Steel-Dwass test (Figure 2), with the results showing a significant difference between Intervention Group 2 and the control group (p = .049).

Figure 2. Comparison of Pretest and Posttest Total Score Differences Among the Three Groups Using the Steel-Dwass Test

Feasibility Parameters of the Japanese version of the Burns Wean Assessment Program

Acceptability

To investigate the acceptability of the J-BWAP, the participants were asked about its clinical application and contents using the following question: "Would you like to use the J-BWAP clinically?" (Figure 3). A Fisher's exact test between the two groups showed that the number of affirmative answers was significantly larger in the intervention groups (p = .049).

Figure 3. Comparisons of Intervention Group and Control Group Responses to "Would You Like to Use a Japanese Version BWAP Clinically?"

Demand

Demand was assessed with respect to how or why nurses would use the J-BWAP. The participants (n = 15) were asked "Why is a J-BWAP necessary?" Eight participants selected "J-BWAP provides weaning criteria," two participants selected "Provides a common language among medical professionals," three participants selected "Useful as an educational tool," and two participants selected "Helps to understand the risks." Two of the participants deemed the J-BWAP unnecessary under the following conditions: when there is a need for urgent extubation (n = 1) and when the patient's overall condition is too critical (n = 1).

Implementation

The feasibility of implementing the J-BWAP was assessed using the question "What should be improved when making a J-BWAP available in the future" (multiple answers were acceptable). Five participants selected "Create a workplace environment that includes new information," three participants selected "Include knowledge" (i.e., information concerning unfamiliar items such as assessing inspiratory and expiratory pressure), five participants selected "Improvement of J-BWAP" by (for example) making it a more suitable and convenient tool in Japanese clinical practice by reducing the number of items, and one participant selected "Training to become familiar with the program." With respect to the question "Is education necessary to use the J-BWAP?" most of the participants in the control group (n = 9) responded that education was necessary to use the J-BWAP, whereas the remaining (n = 2) responded "undecided." No participant answered "not necessary."

Adaptation

Adaptation of the program was assessed based on changing program contents to make them appropriate to new situations (Bowen et al., 2009). A 5-point Likert scale for the question "Was the teaching material difficult?" yielded "neutral" or "slightly difficult" responses. The open-ended question on material difficulty yielded the responses "[I need] more knowledge about anatomy and physiology," "there were many words I did not know," and "many factors are related to the respiratory system."

Discussion

The results of this study showed that, despite the significantly higher levels of education and continuing education in the control group, a positive educational effect was observed in the intervention groups. The changes in the J-BWAP total score in the pretest and posttest showed a significant difference between the control and intervention groups, which supports Hypothesis 1. However, when the intervention groups were divided into Intervention Group 1 (one intervention per week) and Intervention Group 2 (three interventions per week), a significant difference was observed only in the scores between Intervention Group 2 and the control group. Therefore, a learning effect may not be achieved after only one intervention, and thus, repeated interventions are necessary.

These results are similar to those of Poon et al. (2015), who provided e-learning education on aEEG to neonatal ICU nurses and physicians, with the comparison group taught using traditional teaching methods. The pretest and posttest for aEEG through e-learning indicated that both subjective and objective knowledge of aEEG were significantly higher in the intervention groups than the comparison group. The results of this study also found a significant difference between the control group and Intervention Group 2, and a more-positive educational effect was achieved with three e-learning sessions than with one e-learning session. Thus, Hypothesis 2 was also supported. Most of the participants were motivated to learn proper MV weaning techniques and had received in-house MV weaning education. Moreover, approximately 40% of the participants had participated in related external education (e.g., seminars) and independent learning. As learning about MV weaning requires on-the-job training, in-house learning and independent learning using e-learning materials about MV weaning represents a feasible new option. Although the number of participants who wanted to use the J-BWAP was significantly higher in the intervention groups than the control group, the participants noted that the tool should be more convenient and include fewer evaluation items.

Previous studies have verified that the rate of extubation success is significantly higher when at least 14 of the 26 items are assessed (Burns et al., 2010). Given that BWAP scores may influence extubation decisions, accurate scoring is essential. This study was designed to assess the feasibility and usability of the J-BWAP tool by general ICU nurses, whereas previous studies assessed the scoring of patients by experienced/well-trained physicians and nurse practitioners (Burns et al., 2010; Jeong & Lee, 2018; Jiang et al., 2014). Therefore, the contents of the e-learning materials should be revised and refined to meet the learning needs of ICU nurses. Additional training for nurses may also help improve their competence in using the J-BWAP. Furthermore, the J-BWAP items require revision. Summarizing the main results of the feasibility parameters of the J-BWP reveals the acceptability, demand, and adaptation of this tool to be generally favorable. For the key points of future scale revisions, implementation of the J-BWP should be modified to be more clinically relevant, and the method of J-BWP implementation and support should be carefully specified.

This study revealed a need to use the J-BWAP with patients with weaning difficulties and to provide a common tool in the multidisciplinary ICU team. The J-BWAP is useful in evaluating patients in instances where it is difficult to determine the appropriateness of MV weaning. The optimal timing for MV weaning is currently not indicated in the Japan Society of Intensive Care, Japan Society of Respiratory Care Medicine, and Japan Critical Care Nursing Association (2015) guidelines. Thus, we initially evaluated the feasibility of developing J-BWAP. The next step will be to verify the reliability and validity of the J-BWAP, and its real-world applicability should be evaluated using a randomized controlled trial.

The strengths of this study include the development and testing of the J-BWAP and the use of e-learning as a feasible teaching modality. The successful use of e-learning in other countries provides a strong indicator that this approach may also be implemented in Japan. However, as the authors could not confirm whether the e-learning was performed 3 times, the effective number of repetitions is unknown. Nevertheless, the results suggest that greater engagement with the e-learning materials should improve outcomes.

This study is affected by several limitations. First, the dropout rate was high in Intervention Group 1. However, there were no intergroup differences in characteristics because of the random assignment method used. The cause for the discrepancy in dropout rates is unclear because we did not follow up with those who dropped out. However, it is possible that only one intervention had an impact on motivation, although we had explained to both intervention groups that the intervention might be implemented 3 times. In future studies, it should be explained to the participants that all groups will receive the same level of education after the study is completed. Second, the sample size in this study was small because of the high number of dropouts. The dropout buffer of 10% was insufficient to compensate for the actual dropout rate of approximately 30%. The preliminary data obtained from this study may be referenced when calculating the sample size for subsequent studies. Finally, the participants were assigned to the intervention and control groups within each hospital rather than cluster assignment by hospital to eliminate the influence of hospital characteristics. Therefore, the possibility of bias cannot be ruled out.

Conclusions

This study validated the feasibility of developing a J-BWAP that teaches MV weaning techniques using e-learning to ICU nurses. The increased need to use J-BWAP weaning criteria has increased the need to develop a common language and educational approach. Importantly, the content of the J-BWAP requires further revision to make it more applicable to clinical practice in Japan. Clinical application methods suitable as educational tools in conjunction with facilities' protocol should also be further examined.

Acknowledgments

We thank Dr. S. M. Burns for her permission to create the Japanese version of the Burns Wean Assessment Program and Dr. Miyasaka and the critical certified nurse specialists for their cooperation in developing the Japanese version of the Burns Wean Assessment Program. We also thank the six hospitals that participated in the research. Dr. Sarah E. Porter and Dr. Edward Barroga provided native English language editorial assistance.

Author Contributions

Study conception and design: All authors

Data collection: RK

Data analysis and interpretation: RK, NH

Drafting of the article: RK

Critical revision of the article: NH

References