Patients Rising NOW offers the following comments on ICER’s May 22nd draft report, “Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy.” Our comments about the draft report are organized below into sections concerning: Patient and Family Perspectives and Issues; Assumptions of Future Effectiveness; ICER’s Framework for Ultra-Rare Diseases; and Additional Points.
Patient and Family Perspectives and Issues
Duchenne Muscular Dystrophy (DMD) is clearly a devastating progressive disease as described in the draft report. And families and patients need new treatments since “Untreated children with DMD usually progress to a loss of ambulation by age 10-12 years…. Children may also have developmental delay and behavioral issues, as well as impaired growth, delayed puberty, and gastrointestinal complications (e.g., dysphagia and gastroparesis) from the loss of muscle contraction. Orthopedic complications such as contractures and scoliosis occur in most patients, typically after loss of ambulation. Bone health is a major long-term problem, as osteoporosis frequently occurs in later-stages of the disease, with a high risk of fractures. Fatal respiratory or cardiac complications commonly develop in the second or third decade of life, with many deaths occurring in the setting of an acute infection such as pneumonia or after surgery. However, treatment with corticosteroids and advanced supportive care such as assisted ventilation (non-invasive and invasive), spinal surgery, and prevention and management of cardiomyopathy-related heart failure have led to delays in disease progression and improved survival of patients with DMD such that some patients are now surviving into their 30s or 40s.”[1]
Because DMD affects children into young adulthood, it also has major implications for their parents and family members, so it is troubling that there are not good quality of life measures for caregiver impacts, i.e., “there is not currently a standard instrument that is used across studies and that DMD patients and their caregivers have a complex quality of life profile that may not be fully captured by current standard tools.”[2]
The draft report notes those and other challenges in monitoring disease progression and evaluating the effectiveness of treatments because of the variability of the clinical parameters used to determine disease progression. For example, ambulation is effort-based and scoring can be subjective, and “Caregivers and patient groups expressed that, in terms of impact on quality of life and caregiving, the time a patient requires to complete the 6MWT may not be as important as whether the patient can complete the test at all.”[3] We also appreciate the draft report’s discussion about the potential for using videos for monitoring ambulation, which should provide greater real-world assessments of function and improve the current analytical fuzziness, e.g., “…there appears to be a gap between currently reported trial outcomes and video evidence shared by patients, particularly for eteplirsen. Videos of patients on eteplirsen appear to show clinical improvement in function that was not reflected in the clinical trial outcomes. The reasons for this discrepancy are unclear, but possibilities include flaws in study design and execution, flaws in data collection during the trial, non-systematic collection, and scoring of video data, the fact that a subset of patients may benefit substantially from the drug while others do not benefit at all, or choice of clinical outcomes during trials that are not sensitive enough to detect subtle changes in clinical status.”[4] We are hopeful that such tests will become better standardized and validated for DMD (and potentially other neuromuscular conditions), and could be an opportunity for the use of artificial intelligence for analyzing such videos to quantify various aspects of ambulation, activities of daily living, and quality of life – particularly in real-world situations and through longer-term observations.
We are also struck by how the challenges in determining disease progression (and treatment effectiveness) contrast so markedly with the scientific underpinnings of the exon-skipping treatments. Not only are exon-skipping treatments a very cutting edge and fascinating scientific intervention – modifying RNA’s translation into an amino acid sequence that is different than what the underlying DNA codes for[5] – but it also produces a very specific and measurable biologic response, i.e., a change in the presence of the dystrophin protein. We realize that eteplirsen currently only produces very low levels of dystrophin,[6] and that long-term clinical improvements have not yet been definitively demonstrated from an increased presence of this protein, but our point is that there is a dramatic difference in the specificity and accuracy of those two measurements: ambulation (and other measures of disease progression) and levels of dystrophin.
We raise that comparison because it illustrates a serious and fundamental flaw in ICER’s draft report arising from the very limited and imprecise measurements and data of disease progression and treatment effectiveness for DMD. That is, because of the rarity of DMD and the imprecision of the measures of clinical effectiveness of existing treatments, ICER’s analysis in this draft report is by necessity overwhelmed by uncertainty and assumptions. Basically, plugging numbers based upon very little data and extensive assumptions into a model means that the results are only as valid as the uncertainty of the underlying data, which in this case are very, very uncertain. And the inconsistency between the lack of robustness of the available data and ICER’s assurance of its own conclusions are exemplified by two statements from the draft report:
- “…there is a lack of long-term data for exon-skipping therapies and thus the potential long-term benefits and harms of these drugs is unknown…”[7]
- “However, as demonstrated by our extensive scenario/sensitivity analyses, our conclusions were extremely robust at the current treatment prices.”[8]
The challenge for ICER – and others looking to evaluate disease progression and treatment effectiveness of DMD at the population level (as opposed to for individual patients and their families engaged in shared decision making with their care team) – is to synthesize the available data within the context of its limitations. The draft report distinctly oversteps that threshold. As James Collins, the author of “Good to Great” said, “The signature of mediocrity is chronic inconsistency.” This is true for ICER’s dogmatic framework process overall, and how in the draft report ICER attempts to quantify results based on models and algorithms despite the immense imprecision of the underlying data.
Assumptions of Future Effectiveness:
We appreciate that the limited expression of the dystrophin protein from eteplirsen represents a quandary for analysis since it is a below normal level of a not fully functional protein. And, in the same vein we appreciate the “Future Therapies” section of the draft report providing a brief overview of potential treatment approaches being investigated. That brief discussion is very important in the context of this draft report since it illuminates the challenges of modeling into the distant future, (i.e., beyond 5 years), since patients, their families and clinicians are all hoping that better treatments will be coming out of that research pipeline. We believe this is a very important contextual consideration since the clinical situation with DMD is similar to the early days for developing new and better treatments for other complex diseases where initial improvements were small, but they provide insights that are then built upon – sometimes with combination treatments acting in unison or synergistically. That has been the historical nature of finding better treatments for common and rare conditions. In addition to the interventions described in the Future Therapies section of the draft report, we could speculate that in the future, the addition of an adjuvant agent – perhaps an already approved small molecule drug – could increase the amount and effectiveness of the dystrophin protein and thus dramatically increase the clinical effectiveness of the exon-skipping treatments without significantly increasing costs. We don’t know if that will happen, and neither does ICER – and that’s the point.
Declaring the initial efforts as low value is like concluding that the Wright brother’s first airplane was not worthy of pursuit because it would only fly hundreds of feet – and was not capable of supersonic speed. Failure to recognize the step-wise progress of science and technology will result in stagnation and lack of improvement for clinical care, which would be detrimental to patients and society.
ICER’s Framework for Ultra-Rare Diseases:
Concerning ICER continuing to ask companies about input development costs for specific treatments (which is part of its dogmatic ultra-rare framework modification), we would appreciate ICER responding to these questions:
- How do development costs affect the clinical value of a treatment to a patient, payer, or society?
- How do development costs affect the economic value of a treatment to a patient, payer, or society?
- Where in ICER’s economic modeling would input/development costs be inserted as part of the calculation of the economic value of “long-term value for money”?
- How would ICER propose that any research and development costs for a second generation treatment – such as a second exon-skipping treatment – be allocated if such an analysis would be conducted? That is, should the knowledge – and the costs of acquiring that knowledge through research and analysis – be attributed to the first such treatment, or should it be divided between the first and any subsequent treatments that were based upon that knowledge base? And if the costs should be divided, in what ratio, and what would be the process for retrospectively adjusting the modeling “input costs” for the first treatment after a second treatment is created using some of the same basic knowledge?
Since we continue to be baffled by why ICER thinks that R&D or input costs are a valid facet for exploration in determining economic value – or acceptable price – of treatments, we look forward to ICER’s responses to those questions.
Additional Points:
- The Draft Report provides a link to the list of stakeholders from whom ICER requested input,[9] but not those from whom it actually received input. That list should be provided.
- The draft report indicates ICER requested information about “potential cost-saving measures” that could “create headroom in health care budgets.”[10] However, we challenge ICER’s use of the term “headroom” since it implies a fixed ceiling of health care spending. We recognize that other countries have more pre-determined national budgets for government run health care programs, but the only examples of that in the United States are in the Departments of Veterans Affairs and Defense, which comprise less than 5% of U.S. health care spending.[11] Since words do have meaning, and the imagery of the word “headroom” in association with “health care budgets” is misleading, we urge ICER to recognize this reality and change its rhetoric accordingly.
- In previous reports ICER has stated it could not find coverage policies for experimental agents not yet approved by the FDA. We are glad that ICER has finally recognized the inherent futility of such research and has stated that “At the time this report was published, the FDA had yet to issue a decision on golodirsen, precluding a survey of its coverage policies.”[12] Similarly, the description of the coverage policies in the draft report (pages 15-17) illustrates the complex multi-payer reality of the U.S. health care financing and reimbursement system. We would hope that ICER will expand its inclusion of this reality into its overall work by not implying that there are national payment policies or an overall fixed budget for all health care services or for any subtype of treatment options such as FDA approved biopharmaceuticals.
Conclusions & Recommendations
Patients Rising Now believes that ICER’s Draft Report on DMD attempts to reflect patients’ and caregivers’ perspectives, but because of the lack of available data and insights about those perspectives, the draft report’s analysis and conclusions are flawed because it attempts to assert precision via modeling when the underlying data is very imprecise. This inconsistency makes the draft report’s conclusions misleading and inaccurate. Thus, we are forced to conclude that ICER’s draft report on DMD ventures into the realm of fiction when its “analysis” is essentially a series of “What If” propositions, stuffed with assumptions about possible long-term clinical outcomes, and its process is gilded by an economically-driven aspirational world view.
We hope ICER recognizes the fundamental inconsistencies in the draft report’s analyses and conclusions during its public discussion, explores it more fully and forthrightly in its final report, and somehow can bring itself to incorporate the realities of data uncertainty – and the pluralistic nature of the U.S. health care financing system – into any future updates to its framework process.
[1] “Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 1.
[2] “Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 2.
[3] Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 38.
[4] Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 38.
[5] Lim et al., “Eteplirsen in the treatment of Duchenne muscular dystrophy,” Drug Design, Development and Therapy, February 28, 2017, pp. 533–45.
[6] “Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 34.
[7] Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 62.
[8] Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 62.
[9] “Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. iv.
[10] “Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 14.
[11] CMS, 2017 National Health Expenditures.
[12] “Deflazacort, Eteplirsen, and Golodirsen for Duchenne Muscular Dystrophy” Draft Report, May 22, 2019, p. 15.