Should results of quality improvement (QI) projects be considered evidence and be used to build the knowledge foundation for evidence-based practice projects?
Healthcare institutions are responding to the crisis in quality and safety by simultaneously implementing multiple projects at great cost to improve quality. Clear evidence that would assist them in choosing the activities most likely to work is badly needed. QI recommendations claim some evidence base and wave the banner of evidence-based practice. In truth, however, only a small portion of this evidence meets standards of validity: internal validity-Is there actually a relationship between the presumed cause and the observed effect and external validity? Can we infer that this relationship is present in other settings and with other presentations of cause and measurements of the effects (Cook & Campbell, 1979)? Responding to this uncertainty, projects attempt to improve the quality and safety of healthcare and to evaluate the impact of producing the badly needed evidence.
By their nature, QI efforts are organization specific, attempt to minimize disruption, and try to keep costs of implementation constrained. To justify the organization's investment in the project, there is a desire to show that the project had the intended effect. Furthermore, the directors of the project often want to capitalize on the work by publicly reporting the results. Because of these multiple goals, the projects often have only low-cost superficial evaluation efforts that are then reported as evidence with an emphasis on outcomes supporting the intervention and omission of those that did not. Those QI studies may have significant bias and can cause harm by disseminating results that lead healthcare institutions to invest in activities that may not improve quality while ignoring others that could (Pronovost & Wachter, 2006).
As QI activities cannot be tested with rigorous and controlled research, poorly conducted studies of efforts to improve quality and safety cannot be accepted because valid knowledge is crucial. We need to develop a QI science that will enhance the internal and external validity of the results by substituting complex analytic methods for control over confounding factors (Berwick, 2005; Lamb, 2007; Mosser & Kane, 2007). Practitioners' distrust of research and its accompanying statistics and the disdain of researchers for the messiness of QI research must be tempered with better understanding. Collaboration between directors of QI and systems researchers would maximize the potential for producing evidence from these field studies.
Candid reporting of QI results is crucial to developing valid knowledge. To maximize learning, these reports must include both the intended and unintended outcomes, the robustness of the outcome measures, and thorough descriptions of the intervention, its implementation, and the organizational context. Guidelines for the publication of QI projects may assist in achieving this thoroughness and transparency (Davidoff & Batalden, 2005). Knowledge from QI projects is not only possible, but it is also necessary for future evidenced-based practice.
Mary A. Blegen, RN, PhD, FAAN
Professor and Director of the Center for Patient Safety
School of Nursing, University of California San Francisco
[email protected]
References