There's little denying the utility and value of patient-reported outcomes (PROs) when it comes to cancer research and care, according to Andrew Vickers, PhD, Attending Research Methodologist at Memorial Sloan Kettering Cancer Center (MSKCC) and Co-Director of the Patient-Reported Outcomes, Community-Engagement and Language Core at the facility (which has the mission of supporting patient-reported outcomes efforts across MSKCC).
But there are sometimes problems with the tools used to collect PRO data, he noted-in the way that they're validated in clinical trials and in the way they're used in clinical practice. That's why Vickers recently penned an editorial in the Journal of Clinical Oncology titled "Validation of Patient-Reported Outcomes: A Low Bar" (2019;37(23):1990-1992).
Vickers believes the problem is that tools are validated in studies that evaluate whether a particular tool (such as a questionnaire) collects information about a particular set of patients. But those studies don't necessarily investigate if such tools are the best tools for measuring those outcomes, nor do they always collect information from a large and diverse sample. And even if they do include a large, diverse sample, sometimes shortcomings of the tool are not accounted for.
If, for example, a validation study investigates whether patients understand the questions in a questionnaire intended to collect information about patients' quality of life following breast cancer treatment, the sample patient population should be a diverse one. It's inadequate to test the understandability of those questions in a group of patients who all have a college education, he explained. "I would like to see that questionnaire given out to people whose English was a second language or who had a high school or pre-high school education."
And when those tools get used in the clinic, there needs to be room to tailor them to be appropriate for the patients they're being used for. "Questionnaires designed for research studies are often inappropriate for use in the clinic," Vickers told Oncology Times. "I want people to think about these validation studies with a little more common sense in the way they develop and use them." Here's what else Vickers said about the editorial and his views on PROs and PRO tools.
1 Why did you decide to write this editorial now?
"This editorial is about a specific paper [about the validity of a PRO tool for measuring sexual interest and satisfaction in men following radical prostatectomy] (J Clin Oncol 2019;37(23):2017-2027). I can't really criticize the authors because they did what everybody else does. They took a new measure and asked [patients] to complete the questionnaire.
"They found that the questionnaire correlated to a bunch of things they expected it to do; and they found the questions didn't correlate so well with a bunch of things they didn't expect it to correlate with. Therefore, they found it's valid and we should use it.
"I'm trying to say that's not enough. That's why I say in the editorial it's a 'very low bar' for validating PRO tools.
"A correlation measures the average for a whole bunch of people. But you have to go a little deeper than that and look at the specific effects on individuals. And then there's this problem of how do people use the results of this methodology [of PRO tools]. For example, if you validate a tool and you can't make any modifications to what's been tried [and tested in the validation study], that's not good enough. We need to move away from the mentality that the only thing we need to know is that a tool has been validated. We have to make it work for the individual patients."
2 What would you like to see happen to better validate PRO tools and better use them in clinical practice?
"I think when we give patients a PRO tool to use [for the purpose of validating its effectiveness in research], we need to do better debriefing than is currently done. After they have filled out a questionnaire, we need to ask: What do you understand about that question? What about specific words? Why did you answer the question that way? It's called debriefing-I think it needs to be done.
"We have to have more diversity [in the populations we test tools on], especially if you're talking about issues of personal issues or intimate issues, such as sex. For example, there are 8 million people in New York City and they all use the same word for knee or elbow, but they use many different words for 'penis' or 'vagina' or 'sex' or 'erection.'
"We need to use a little bit more common sense [when developing PRO tools]."
3 What's the takeaway message about validating PRO tools and using them in smarter ways?
"The big change that we're seeing is that PROs used to be something that you did in research, but were difficult in practice. But they're being used much more commonly in clinical practice as well as in research. But I would say that the questionnaires we use in research have to be adapted for use in the clinic. And we have to break out of this very rigid way of thinking about validating tools.
"We might need to take out a whole section [of a questionnaire when it gets used in clinical practice]. It may have been valid, but let's use our proper judgment, common sense, getting more data if we need it, and think about how we should use these-rather than taking a research tool and shove it in front of the patient."