1 on 1 interview: Bringing Clinical Trial Rigor to Cancer Treatment Data Collected in the Real World
FOCUS: Tell us about the study you were conducting and about the data inconsistencies you identified.
Andrew Klink: In 2016, after recently completing an RWE study to determine patient response to cancer therapy, our team was surprised by the findings. When we analyzed the physician inputs in aggregate, across a cohort of patients, we found that the proportion of patients with complete or partial response to therapy was much higher than the responses reported in the drugs’ randomized clinical trials.
We wanted to understand the reasons behind this variance, so we sought out qualitative feedback from the participating physicians, to learn what measures or tools they were using to evaluate patient response. We found that most participating physicians weren’t using standardized criteria at all; they were primarily using their own clinical judgment to evaluate patient response. For example, perceived clinical benefit demonstrated by improved lab values or less symptom burden might have contributed to a more subjective interpretation of response.
While the treatment of most forms of cancer can definitely be both an art and a science, it was inarguable that these methods for measuring patient response were far more subjective, and less consistent, than the gold standard, RECIST. The RECIST classification compares changes in a patient’s lesions and tumor sizes over time, and is the methodology typically used to measure solid tumor response to therapy in clinical trials. We hypothesized that the more objective RECIST approach would be likely to deliver more accurate patient response results than those formulated based on the individual clinical judgment of each participating physician.
FOCUS: What steps did you take to test that hypothesis?

AK: The data we were seeking wasn’t available in an off-the-shelf EMR [electronic medical record] data set, couldn’t be automated, and needed to specifically be provided by a treating physician. So, we went back to the physicians who participated in the study and asked them to provide data from each patient’s tumor imaging results. That information isn’t always in a patient’s record, so in some cases the treating physician needed to send a request for the imaging results in order to complete the assessment. That’s an important nuance, because a research nurse or chart abstractor wouldn’t typically have access to this information. We then asked the treating physicians to specifically enter data including actual tumor/lesion measurements taken from patient scans, into a HIPAA-compliant online data capture form.
Our team then analyzed these new physician-reported data and used it to measure patient response to therapy using a process analogous to RECIST, the gold standard classification. We found that physicians’ more subjective methods for measuring patient response mirrored those found using the RECIST methodology, only 43% of the time. Overall, the trend was that physicians overestimated the positivity of the patient response.
FOCUS: Is this methodology identical to the RECIST process used in clinical trial settings?
FOCUS: How has this finding shaped how you collect RWD?
AK: Our goal is consistent with the FDA’s; we always aim to obtain the most accurate and reliable data possible. So, this research served as the impetus for us to now use the RECIST classification for evaluating RWE that measures patient response to therapies that treat solid tumors.
We’ve also tested a similar approach to generating RWE in the hematologic malignancy setting, using the Lugano classification (Cheson) for patient response to therapy. We found a similar trend, regarding the overestimation of positive patient response by physicians, and we’re currently working on publishing the results of that research, too.
FOCUS: The new data collection approach you’ve developed will likely take more time and investment to facilitate. Does the improved accuracy of the data justify those time and cost investments?
AK: The ultimate end goal of generating RWE is to ensure it’s actually useful to the FDA and other healthcare decision makers for labeling and other decisions. That means accuracy is key. Manufacturers are well served to report RWE in a way that’s as close as possible to the gold standard FDA reviewers are accustomed to dealing with in clinical trial settings.
In December 2018, the FDA released a framework for using RWD for regulatory decisions. In general, the framework shares criteria for improving the accuracy, reliability, recency and speed of data collected in real-world settings. The approach we’ve developed models the RECIST approach, but within real-world parameters, and allows for all the FDA’s recently released criteria to be met.
FOCUS: Can you provide some specific examples of how this type of RWE would be useful to manufacturers?
AK: Sure. Let’s start with its usefulness in premarket studies, when manufacturers are seeking to understand the current competitive landscape. This research indicates that more subjective physician-reported measures tend to overestimate patients’ positive responses to therapies currently on the market. Relying on those subjective measures alone can potentially make it more difficult to prove that new therapies can address unmet needs. Using this novel, more rigorous means of patient response evaluation not only delivers more accurate results, but also is likely to prove far more useful to manufacturers in search of data that can help them demonstrate that their therapies are addressing unmet needs.
Post-approval, we believe that manufacturers will be able to use this new method to demonstrate how their therapies are being used in the real world, in a manner that’s consistent with new FDA guidance, to ensure accurate and reliable data but at a fraction of the cost to conduct similar research in a clinical trial setting.