This weeks Paper of the Week is brought to you by: Professor Sir Muir Gray
Author’s Conclusion
“In July 2018, the Journal published the results of TAILORx (Trial Assigning Individualized Option for Treatment).1 This randomized trial conducted by Sparano et al. showed the noninferiority of endocrine therapy to chemoendocrine therapy with respect to invasive disease–free survival among women with hormone-receptor–positive, human epidermal growth factor receptor 2 (HER2)–negative, axillary node–negative breast cancer who had a recurrence score (based on a 21-gene assay) of 11 to 25 (scores range from 0 to 100, with higher scores predicting a higher risk of distant recurrence)…
Less than a year later, we are publishing additional results from the same data set showing that adding “clinical risk” (i.e. tumor size and histologic grade) provides additional prognostic information…
Readers may be surprised to learn that the classic prognostic markers were not directly analyzed in the 2018 article, unsurprised to learn that they contribute prognostic information, and curious as to why it took two articles in the Journal, published almost a year apart, to impart this information. The answers lie in the system for conducting and reporting on clinical trials, and they have some implications for the incorporation of new research findings into clinical practice….
Thus, practice-changing suggestions are being made on the basis of a subgroup analysis of a secondary objective in the trial, published as a follow-up to the report of the primary objective. The promise of “precision” medicine has collided with the rather messier world of using all available evidence to try and make educated guesses to improve patient outcomes…
Not only is the term “precision medicine” a misnomer when applied to common complex diseases (since we need accurate predictions of response to therapies, having “precise” predictions that are inaccurate may be misleading), but the vision of “N-of-1” prediction can be obtained only with large statistical uncertainty, and to reduce statistical uncertainty, we need to classify patients into larger groups within which there will be substantial heterogeneity…
The presentation of additional analyses from important data sets will remain a fundamental component of evidence synthesis. Distinguishing between results that warrant a change in practice and those that do not will not be a “precise” process. Medicine in the molecular era will be no more “precise” than in prior eras — evidence synthesis, clinical judgment, and patient preferences all factor in. Although regulatory bodies such as the Food and Drug Administration may continue to insist on statistically significant results from randomized trials in licensing decisions, communicating information from trials beyond prespecified and primary objectives remains important, and judicious use of these data should continue to inform clinical practice.”
3VH commentary on the Implications for value improvement
HEADLINE – Evidence based caution about polygenic scores
By happy coincidence the week after our podcast on the need for caution when presented with enthusiastic claims about the benefits of polygenic scores a blockbuster three-page editorial is published in the New England Journal of Medicine on evidence to support this caution. Interestingly, the evidence comes from two papers in the NEJM by the same authors. It is to the great credit of the NEJM they published the second paper and then drew attention to the fact that the first paper was not as significant as a paper in the NEJM would be assumed to be and publishing this unusually long editorial.
The editorial is highly instructive. It points out the need to combine all the data that are available and not to jump to conclusions too quickly. It reinforces the need for hubris to be replaced by realism and the need for developing even further the critical appraisal skills of every decision maker. Not only does the statistical method need to be appraised but also the completeness of the reporting, particularly if polygenic analysis is part of the intervention.
This is challenging for the pregenomic generation to which most decision makers belong that there is an urgent need for educating people who pay for or manage healthcare in the critical appraisal of papers about the value of these developments. However, the field is moving so fast that the developers, clinicians and geneticists, need to be included in the networks taking collective responsibility for all the resources used for, in this example, women with breast cancer. The value of an innovation needs to be assessed with respect to all the other interventions currently being offered by the system of care, from screening through to long term care, certainly in countries committed to universal health coverage. So, both the cost-effectiveness and the opportunity cost effectiveness need to be considered, and of course before that the effectiveness needs to be established and this is complex, as this saga shows us.