Nurses from a variety of educational backgrounds are expected to engage in evidence-based practice (EBP). To prepare nurses for EBP, nurse faculty often use professional guidelines such as the American Association of Colleges of Nursing's (AACN, 2008) Essentials of Baccalaureate Education for Professional Nursing Practice and Quality and Safety Education for Nurses (QSEN) Competencies (Cronenwett et al., 2007; QSEN, 2012) to guide curriculum development and inform instructional strategies. However, the extent to which prelicensure nursing students and licensed nurses enrolled in higher degree programs possess the knowledge required to enact EBP has received little systematic investigation.

To address this gap in the evidence, the purpose of this study was to evaluate what prelicensure nursing students and licensed nurses enrolled in baccalaureate nursing (BSN) through doctor of nursing practice (DNP) programs know about EBP. This study is necessary to provide baseline evidence on nurses' EBP knowledge levels to help inform curriculum and instructional planning across the spectrum of nursing education programs.

BACKGROUND

The baccalaureate degree, focused on preparing bedside nurses with basic leadership knowledge, is often the baseline educational requirement for managers, supervisors, or directors. Leadership roles require nurses to expand the use of EBP from the patient to systems levels by leading and mentoring evidence-based change, content that is most often taught at the graduate level (e.g., master's [MSN/MS], DNP). For that reason, academic leaders must look more broadly at EBP curricula, considering not only within particular programs (e.g., BSN) but also across levels of education (BSN, MSN, DNP) to prepare nurses for expanded roles in the provision of evidence-based care.

Although EBP is often taught in undergraduate and graduate nursing programs, this approach to care may not be fully integrated across courses, applied in clinical, and mentored into use. Furthermore, studies show that nursing students and nurses often report insufficient knowledge as a barrier to evidence-based care (Connor, Dwyer, & Oliveira, 2016; Saunders & Vehvilainen-Julkunen, 2016; Williamson, Almaskari, Lester, & Maguire, 2015). To evaluate the effectiveness of curricula and teaching, programs most often utilize existing EBP-related measures that rely on self-rated EBP knowledge, skills, attitudes, beliefs, implementation, and/or use (Melnyk, Fineout-Overholt, & Mays, 2008; Upton & Upton, 2006). However, a growing body of evidence from medicine (Davis et al., 2006; Lai & Teng, 2011) and nursing (Wonder et al., 2017) illustrates a lack of correspondence between self-reported and objectively measured EBP knowledge measures, and between self-assessments and more objective assessments of ability more generally (Zell & Krizan, 2014). To date, EBP knowledge has not been objectively evaluated among students enrolled across a broad range of nursing education programs, creating a significant gap in the nursing education evidence base.

Finally, if the nursing profession is to address the call from the Institute of Medicine (2001) to improve patient care through EBP, methods to evaluate the effectiveness of embedded and specific EBP education through the study of individual EBP knowledge and the development of EBP knowledge postlicensure are essential. The aims of this national, multisite study were to: a) describe the levels of EBP knowledge present among a sample of nursing students enrolled in BSN through DNP programs; b) examine the relationships between objective and subjective EBP knowledge measures; and c) describe the correlations between educational and demographic factors, respectively, and EBP knowledge. This study also sought to further evaluate evidence of validity and reliability for the Evidence-Based Practice Knowledge Assessment in Nursing (EKAN) among students enrolled in BSN through DNP-level programs.

METHOD

This multisite, cross-sectional, correlational study employed a large-scale survey approach to recruit eligible students enrolled in accredited BSN (including BSN completion programs for licensed nurses), MSN/MS, post-master's certificate, and DNP programs from five different universities across the United States. Participating universities were located in the Midwest, Northern Midwest, and South Central regions of the United States. Eligible students were required to be at least 18 years of age, actively enrolled in an eligible program at a participating study site during the data collection period (January to March 2017), and able to use a computer to complete the study questionnaires. Institutional review board or equivalent approvals were obtained from the investigators' home institutions and from host sites, when necessary.

The research team emailed an invitation to each eligible student's institutional email address. Each email contained information about the study and a link to additional information and the instruments. Because no personal identifiers were collected, completing the study instruments constituted consent to participate in the study. All study instruments were housed in Qualtrics(R), a secure, online data-capture platform. The first 700 respondents who completed all study instruments were offered a $10 incentive gift code to a popular online retailer. Responses to the study questionnaires were anonymized by the survey platform; incentive gift codes were delivered automatically via email to each respondent after all questionnaires were completed.

Measures

Respondents completed three questionnaires in the following order: 1) a demographic questionnaire; 2) the Evidence-Based Practice Questionnaire (EBPQ; Upton & Upton, 2006); and 3) the EKAN (Spurlock & Wonder, 2015). The demographic questionnaire was used to describe the characteristics of the sample (e.g., age, gender, primary language, educational background, confidence with EBP) and to assist in psychometric analyses.

EVIDENCE-BASED PRACTICE QUESTIONNAIRE

The EBPQ was used to measure respondents' self-reported EBP knowledge/skills, EBP attitudes, and EBP practice/use on a 7-point response scale, with higher scores indicating more positive self-assessments of EBP capabilities (Upton & Upton, 2006). The EBPQ was initially developed and tested among nurses, producing an estimate of internal consistency reliability for the entire questionnaire (Cronbach's [alpha] = .87). Cronbach's [alpha]s for subscales were as follows: EBP Knowledge/Skills, [alpha] = .91; EBP Attitude, [alpha] = .79; and EBP Practice/Use, [alpha] = .85 (Upton & Upton, 2006). These internal consistency reliability estimates are consistent with findings from the current study: entire questionnaire, [alpha] = .94; EBP Knowledge/Skills subscale, [alpha] = .94; EBP Attitudes subscale, [alpha] = .76; and EBP Practice/Use subscale, [alpha] = .95.

EVIDENCE-BASED PRACTICE KNOWLEDGE ASSESSMENT IN NURSING

The EKAN (Spurlock & Wonder, 2015) is a 20-item multiple-choice test developed to measure nursing students' and nurses' knowledge of EBP concepts derived from the AACN's (2008) Essentials and the QSEN Competencies (Cronenwett et al., 2007; QSEN, 2012) to promote continuity with the structures and guidelines commonly utilized by nursing education programs across the United States. Just as classroom tests are scored, the EKAN is scored as the total number correct (20 points maximum). The psychometric properties of the EKAN were established using the single-parameter item response theory Rasch model to produce estimates of how well each test item performs relative to an examinee's ability (knowledge) level (Meyer, 2014; a more extensive discussion of this modern psychometric approach is available in Spurlock & Wonder, 2015).

The EKAN was initially tested among 200 students enrolled in one of two traditional BSN programs in two Midwestern states. The initial psychometric properties were reported by Spurlock and Wonder (2015): Difficulty was M = 0.19 (range -2.0 to 2.8); weighted mean square infit was M = 1.01 (range 0.95 to 1.06); standardized weighted mean square infit was M = 0.33 (range -0.7 to 1.6); unweighted mean squares outfit was M = 1.02 (range 0.93 to 1.14); and standardized unweighted mean squares outfit was M = 0.34 (range -1.08 to 2.00). The item separation index was 7.05; the person separation index was 1.66 (Spurlock & Wonder, 2015). Item reliability was .98, and person reliability was .66 (Spurlock & Wonder, 2015). These values showed strong item performance but indicated restriction in trait range, which was likely due to homogeneity of the subject pool (Linacre, 2012; Spurlock & Wonder, 2015).

Rasch psychometric parameters from the current administration of the EKAN were as follows: mean item difficulty, M = 0.0 (range -1.98 to 2.0); weighted mean square infit, M = 0.98 (range 0.88 to 1.16); and unweighted mean square outfit, M = 1.00 (range 0.83 to 1.37). The item separation index was 11.16; the person separation index was 0.99. Item reliability was .99; person reliability was .50. Linacre (2012) noted that, although person reliability in the Rasch model is analogous to scale reliability under classical test theory (most commonly evaluated using Cronbach's [alpha]), Rasch person reliability estimates tend to underestimate reliability while Cronbach's [alpha] tends to overestimate it.

Though the EKAN was developed under Rasch measurement theory, several measures of classical test theory scale reliability were calculated to further examine reliability. Overall EKAN reliability estimates were KR-20 = .745, split-half forms r = .605, Spearman-Brown coefficient = .754, and Guttman split-half coefficient = .750. Similar psychometric results for EKAN have been documented in recent studies of practicing nurses (Wonder et al., 2017) and BSN students (Cosme, Milner, & Wonder, 2018).

Data Analysis

All continuous data were examined and screened for outliers and normality using visual and analytic methods prior to the planned statistical procedures. Descriptive statistics were used to summarize sample characteristics. Overall EKAN sum scores and EBPQ mean scores were calculated to support logical subgroup comparisons of scores by type of educational program in which the respondent was enrolled, level of program completion, and highest earned nursing degree. Relationships between mean EKAN sum scores, EBPQ scores, and demographic characteristics were examined using correlation procedures appropriate to the level of measurement of the variables being compared. Hierarchical linear regression models were used to examine the extent to which educational and demographic factors and EBPQ subscale scores are associated with EKAN total scores. All null hypothesis statistical tests were evaluated using a Bonferroni correction for familywise error rates, holding [alpha] = .05 across all tests. Apart from the Rasch analysis of the EKAN, all analyses were conducted using SPSS v. 24.

RESULTS

Though responses were received from 701 eligible students, 27 were excluded from planned analyses due to high risk for response bias based on the speed with which these respondents completed the study questionnaires. Thus, the final sample size included in the analysis was N = 674. Because of missing or incomplete data for some of the educational and demographic variables, all analyses involving those variables were conducted on an available case basis. The sample was predominantly female (n = 604, 89.6 percent), Caucasian/white (n = 574, 85.2 percent), and spoke English as their primary language (n = 660, 97.9 percent). The mean age was 28.9 years (SD = 10.12, range 18 to 62 years). Most respondents (n = 393, 58.3 percent) were enrolled in traditional prelicensure BSN or RN-BSN programs; 21.7 percent (n = 146) were enrolled in traditional BSN to MSN/MS programs, with the remainder of respondents enrolled in accelerated BSN or MSN entry programs (10.2 percent) and postmaster's graduate certificate and DNP programs (9.8 percent).

Because some program types were represented by a small number of respondents (e.g., n = 9 in RN to MSN/MS programs), nursing education program types were collapsed into four primary categories for the purposes of further analysis: 1) BSN (traditional prelicensure and RN-BSN), 2) accelerated (accelerated BSN and RN to MSN/MS), 3) MSN/MS (with prior completion of BSN), and 4) post-master's/DNP (graduate certificate and DNP) programs. Table 1 provides details on the educational and demographic characteristics of study respondents.

Table 1 Description of Study Sample (

For the entire sample, the mean EKAN score was 10.45 (SD = 2.70, range 3 to 18). Planned subgroup comparisons revealed numerous differences in EKAN scores across groups. First, a small (<1 point) difference in mean EKAN scores was found among the five study sites (F = 5.96 [4, 696], p < .001, [eta]² = .03); mean scores from each site ranged from M = 9.97 (SD = 2.71) to M = 10.87 (SD = 2.73). Differences in mean EKAN scores were also found across the types of programs in which respondents were enrolled, such as with students enrolled in BSN (M = 9.49, SD = 2.44) versus post-master's certificate/DNP (M = 12.19, SD = 2.73; F = 42.79 [3, 697], p < .001, [eta]² = .156) programs. Differences in EKAN scores were also found when examining the recentness of undergraduate research (F = 34.01 [4, 639], p <. 001, [eta]² = .175) and statistics (F = 29.81 [4, 665], p < .001, [eta]² = .152) courses. There were no statistically significant differences in EKAN scores for respondents who reported having completed up to three or more days of specialized EBP training over the past year when compared to others who had not completed special EBP education during that time (F = 0.21 [3, 697], p = .893, [eta]² = .001). Table 2 contains more detailed information about differences in mean EKAN scores across subgroups.

Table 2 Subgroup Comparisons of Evidence-Based Practice Knowledge Assessment in Nursing Scores (

Respondents provided uniformly positive self-ratings on each of the 7-point EBPQ subscales (Knowledge/Skills, Attitudes, and Practice/Use) and overall (M = 4.92, SD = .99). There were no statistically significant differences in EBPQ scores by study site. EBPQ attitude scores were highest (M = 5.46, SD = 1.12) followed by EBP Knowledge/Skills (M = 5.03, SD = 0.95) and Practice/Use (M = 4.27, SD = 1.74). EBPQ scores were positively associated with the highest earned nursing degree, the level of program in which respondents were enrolled, and the recentness of undergraduate research and statistics courses, but effect sizes were small ([eta]² range = .001 to .049).

Bivariate correlation analysis revealed that years of experience as an RN was positively associated with scores from all three EBPQ subscales (r = .12 to .22, p < .01) and with EKAN sum scores (r = .30, p < .001). Scores from each of the EBPQ subscales were positively and moderately associated with scores from the other EBPQ subscales (r = .32 to .53, p < .001). Although EBPQ Practice/Use and Attitudes were not associated with scores on the EKAN (r = .06 to .09, p = ns), EBPQ Knowledge/Skill subscale scores were positively though weakly associated with EKAN scores (r = .13, p = .001).

Hierarchical linear multiple regression was used to further estimate the extent to which EBPQ subscale scores predict EKAN scores after important covariates such as study site, highest earned nursing degree, and years of experience as an RN were accounted for in the regression models. Study site alone accounted for just under 4 percent of the variance in EKAN scores, R² = .036, F(4, 666) = 6.19, p < .001. In Step 2 of the regression, the highest earned nursing degree was added, increasing R² to .138, F[DELTA](1, 665) = 78.56, p < .001. In Step 3, years of RN experience was added, improving R² to .143, F[DELTA](1, 664) = 3.88, p = .049. In the final step, EBPQ subscale scores were entered into the regression, improving R² to .150, F[DELTA](3, 661) = 2.016, p = .110. Thus, the most parsimonious model for predicting EKAN scores would include only study site (to account for site-based differences in EKAN scores) and respondents' level of nursing education degree preparation.

DISCUSSION

The results highlight several important points for EBP and nursing education. First, higher levels of nursing education attainment were associated with higher scores on the EKAN. This finding suggests that the knowledge provided on enacting EBP at the patient and systems levels increases as nurses move from one level of nursing education to the next. Nurse educators are well positioned to assist in full implementation of evidence-based care in practice settings by understanding that EBP knowledge is developed over time and through formal study in graduate-level nursing education programs. Teaching EBP in this manner will require faculty, especially faculty teaching in programs without an associated graduate-level program, to work collaboratively to progressively build students' knowledge within and across programs.

The weak correlations between EKAN and EBPQ knowledge scores were consistent with prior studies that also found low, trivial correlations between objective and subjective knowledge measures (Wonder et al., 2017; Zell & Krizan, 2014). To prepare nurses capable of enacting EBP at the patient and systems levels and to achieve the longstanding goals established by the Institute of Medicine (2001), nurse faculty should use measures with strong validity and reliability evidence to calibrate EBP curricular and instructional methods across all types of programs. The availability of an objective measure with documented psychometric performance data across levels of nursing education enables faculty to gauge the effects of students' exposure to curriculum and teaching innovations. Furthermore, more objective measures of EBP knowledge enable faculty to evaluate how exposure effects (e.g., dose, frequency) vary among student populations (e.g., BSN, MS/MSN, DNP) and geographic areas (e.g., Midwest, West Coast).

Over time, educators can evaluate the effectiveness of EBP content, teaching strategies, and frequency of educational offerings to sustain and advance the EBP knowledge of students and nurses from a wide range of nursing education programs. Future research should include a focus on the relationship between objective measures of EBP knowledge and the self-report measures now commonly used, and how the incongruity between these measures impacts the interpretation of research and evaluation findings. This study provides support for organizational structures that can help improve EBP at the patient and systems levels (e.g., greater access to MSN/MS-prepared nurses to mentor the use of EBP, financial assistance for nurses seeking MSN/MS or DNP). Finally, as EBP content is more fully integrated across all levels of nursing education programs, the need for faculty development is clear. Furthermore, evaluation of the EBP knowledge levels of nursing faculty may be considered as academic leaders strive to maximize development efforts and student outcomes.

CONCLUSION

This study found that objectively measured EBP knowledge level increased positively as respondents achieved higher levels of nursing education. The additional validity evidence for the EKAN instrument derived from this study enables faculty to develop and test teaching innovations and to collaborate in evaluating the effects of these interventions on different populations of students and practicing nurses. To bolster the use of EBP in nursing practice, ongoing, rigorous evaluation is needed to calibrate curricula and teaching across all levels of nursing education.

REFERENCES