Biopharma's dirty secret revealed: science is fragile; forecasting is unreliable. Now deal with it.

Article Highlights

  • The continued reliance on forecasting and prediction in drug development is disappointing

    Tweet This

  • Drug development requires more sophisticated, integrated, holistic view of patients and diseases

    Tweet This

  • Core of medicine and medical research is and must be the patient

    Tweet This

In 2008, Nassim Taleb and I wrote a commentary for the Financial Times arguing that the fundamental reason new drug development hadn’t lived up to the expectations of the genomics revolution was “the mismeasure of uncertainty, as academic researchers underestimated the fragility of scientific knowledge, and pharmaceutical executives overestimated their ability to domesticate scientific research,” and in particular placed excessive confidence in their ability to forecast sales.  “Spreadsheets are easy, “ we wrote, “science is hard.”

"The core of medicine, and medical research, is and must be the patient, and the success of future drug development will depend upon our ability to keep patients in the front of our minds and at the center of our efforts."--David Shaywitz, M.D.

Two recent articles emphasize the continued relevance of these concerns – an outstanding piece in today’s WSJ discusses the fragility of science (a topic I’ve discussed here), and a recent Kahneman piece in the NYT (which I’ve reflected upon here) highlights our tendency to make overconfident predictions.

Our Financial Times piece also discussed potential strategies that could be employed, such as an approach focused on a greater number of smaller but higher-probability shots (i.e. rare disease that are well-defined) with the idea that cost of developing such products is relatively constrained, yet the upside could be significantly higher than anticipated.  We also highlighted the importance of enabling bottom-up innovation, and pursuing the most promising ideas, rather than a top-down view of the “right” TAs and disease states.

Over the last three years, the industry seems to have adopted aspects of these strategies – through not always exactly as envisioned.

Certainly, a number of companies are currently pursuing the rare disease space (e.g. Ultragenyx seeking 10 treatments for rare diseases in 10 years), as well as Novartis’s pathway-based approach, seeking initial approval for a rare, highly-specific indication to demonstrate proof-of-mechanism (e.g. Ilaris, essentially an IL-1β sponge, for Muckle-Wells, a rare hereditary disease caused by excess IL-1β), then hoping for subsequent approvals for more common indications such as gout (which may be driven in part by IL-1β).

The assumption that rare-disease approaches are relatively cost-constrained seems to have held up, as rare diseases represent in essence the most clear and extreme expression of personalized medicine.  However, drugs for these indications have generally not found an unexpectedly large market (and Ilaris was rejected by the FDA for the treatment of gout), nor is it clear that this can be done without significant additional expenditure.  Moreover, the market for a number of these rare diseases is actually getting quite competitive – although a vanishingly small number of orphan diseases actually have available therapeutics, several of these already have multiple treatments; this is wonderful news for patients (especially given their dependence upon these products), and an important reminder for companies.

The most common way big pharmas seems to have expanded their R&D approach is by significantly cutting back on internal research, relying increasingly on the external world to provide the products that they will pull through development and commercialize.  This pretty clearly reflects a lack of internal confidence in home-grown research; evidently, large companies increasingly feel there’s little reason to believe what their own scientists develop is likely to be particularly better than what’s available on the open market.  Bill Joy may have famously said, “no matter who you are, most of the smartest people work for someone else,” but in the case of bench scientists, big pharma seems determined to prove this true.

You could argue this trend reflects not just an appreciation by industry for the complexity of science (something most biotech investors recognized a long time ago), but also an acknowledgement of the difficulty of prediction: by in-licensing relatively late-stage products, executives don’t need to look quite as far into their crystal balls, and have a better sense of how potential products might fit into the market.

On a more operational level, pharma companies (and even VCs, as Bruce Booth writes here) maintain their healthy skepticism towards fresh scientific data, and generally require rigorous internal validation and confirmation before accepting a published basic science result as true – especially since so many publications seem to defy replication (see this paper from Bayer scientists).  At the same time, big pharma is embracing academic collaborations like never before, viewing university researchers as unparalleled sources of innovation.  Maybe big pharma views university researchers like bad-boy boyfriends or bad-girl girlfriends –alluring and captivating yet not entirely trustworthy, and requiring serious diligence before being introduced to the parents.

I confess that I’m disappointed, though not surprised (see here) by the continued reliance on forecasting and prediction that remains a central part of drug development, both in terms of project selection and prioritizing – including deciding which projects to kill for “strategic” reasons, an increasingly common occurrence in big pharma.  Most management consultants seem to feel that big pharma needs to prioritize even more fiercely, to select therapeutic areas even more critically, to avoiding spending effort and treasure on areas unlikely to be commercially successful.

I’ve never agreed with this, and I worry the greatest disservice consultants have done to the industry is preaching this top-down mindset (probably pretty smart considering who pays the bills, however), rather than helping to build a far-more nimble structure capable of opportunistically advancing projects in a far great variety of therapeutic areas and indications.  The result is that largely as a consequence of their reliance on the same small group of consultants, most big pharmas are clustered in the same several TAs, and engage in a remarkably similar groupthink (i.e. viewing payors as the most important customer, or as a monolithic customer – see this smart recent piece by Roger Longman in his new and immediately essential “Value and Innovation” blog), while wide swaths of unmet medical need remain, and a wide variety of potential alternative approaches and scenarios remain unexplored.

I continue to believe that what drug development continues to require, above all else, is a much more sophisticated, integrated, and holistic view of patients and disease, as they exist in the real world (see here and here, for example).  On the basic side, there’s an urgent need for the study of human physiology, and to understand how disease exists, even at the molecular level, in the context of a whole person.

On the clinical side, as I’ve been arguing, it’s critical to develop improved ways of measuring and monitoring real-world health (i.e. here).  I worry that without a mechanism of capturing how patients are really doing, without a way of meaningfully measuring this, it’s difficult for doctors to track this, and for companies to develop products that can improve these parameters.  Two therapeutic approaches that look similar in traditional, highly-regimented clinical studies may offer profoundly different benefits in real-world situations, and it’s important to have a way of recognizing and capturing this.  It’s heartening to see that the FDA has prioritized “assessment science,” as I’ve discussed previously.

The core of medicine, and medical research, is and must be the patient, and the success of future drug development will depend upon our ability to keep patients in the front of our minds and at the center of our efforts.

David Shaywitz, M.D., is an adjunct scholar at AEI

Also Visit
AEIdeas Blog The American Magazine
About the Author

 

David
Shaywitz

What's new on AEI

Making Ryan's tax plan smarter
image The teacher evaluation confronts the future
image How to reform the US immigration system
image Inversion hysteria
AEI on Facebook
Events Calendar
  • 01
    MON
  • 02
    TUE
  • 03
    WED
  • 04
    THU
  • 05
    FRI
Wednesday, September 03, 2014 | 9:00 a.m. – 10:30 a.m.
From anarchy to Augustus: Lessons on dealing with disorder, from Rome’s first emperor

We invite you to join us for two panel discussions on how Augustus created order from chaos 2,000 years ago, and what makes for durable domestic and international political systems in the 21st century.

Event Registration is Closed
Wednesday, September 03, 2014 | 12:00 p.m. – 1:30 p.m.
Multiple choice: Expanding opportunity through innovation in K–12 education

Please join us for a book launch event and panel discussion about how a marketplace of education options can help today's students succeed in tomorrow's economy. Attendees will receive a complimentary copy of the featured book.

Thursday, September 04, 2014 | 12:00 p.m. – 1:30 p.m.
How conservatives can save the safety net

Please join us for a luncheon event in which our panel will discuss what conservatives can learn from how liberals talk and think about the safety net and where free-market economics, federalism, and social responsibility intersect to lift people out of poverty.

No events scheduled this day.
No events scheduled this day.
No events scheduled today.
No events scheduled this day.
No events scheduled this day.
No events scheduled this day.