Computer program aims to rank vaccine development decisions

Abstract

Aligning the priorities of all stakeholders involved in vaccine development can be a convoluted and thorny process. An international health organization might emphasize a candidate vaccines expected health benefits for disadvantaged populations, a government agency might be more focused on its own backyard, and a drug company could be driven by its monetary bottom line.

N E W S

Computer program aims to rank vaccine development decisions

npg 2012 Nature America, Inc. All rights reserved.

WASHINGTON, DC Aligning the priorities of all stakeholders involved in vaccine development can be a convoluted and thorny process. An international health organization might emphasize a candidate vaccines expected health benefits for disadvantaged populations, a government agency might be more focused on its own backyard, and a drug company could be driven by its monetary bottom line. With so many competing interests, what experimental product does it make the most sense for these partners to pursue?

Soon, a mathematical model thats particularly good at weighing complex alternatives may be able to help. Its at the heart of a new computer program, called the Strategic Multi-Attribute Ranking Tool (SMART) for Vaccines, that scores potential avenues for vaccine research and development according to the priorities fed into its algorithm. Members of the US Institute of Medicine (IOM) panel behind the new tool, who discussed the algorithms prototype at a meeting here on 2 November, hope it will establish a shared vocabulary that will allow everyone working on preventative vaccines for infectious agents to better understand and share their own perspective. Were creating a common language for people to talk with, instead of everyone having their own language, says IOM committee member Charles Phelps, a health economist at the University of Rochester in New York.

In the past, the IOM simply released reports that encouraged vaccine developers to prioritize tackling certain diseases on the basis of the balance of expected health benefits, the costs of developing and administering the vaccine and the projected savings from the preventative medicine. For instance, in the most recent report, published in 2000, the IOM strongly favored targeting influenza, a virus that kills up to 49,000 people each year in the US at a cost of tens of billions of dollars annually to the countrys economy. In contrast, the bacteria responsible for Lyme disease, a far less prevalent pathogen with a smaller economic burden, fell much lower on the priority list.

The IOM had intended for vaccine developers to take its rankings into account when making decisions. However, according to Paul Offit, chief of infectious diseases at the Childrens Hospital of Philadelphia and a co-inventor of the rotavirus vaccine,

such lists tended to justify choices that had already been made. When the IOM puts a list out, he says, [vaccine manufacturers] feel that validates what theyve done.

With the SMART tool, any organization can generate its own priority rankings, custom-tailored from a list of 29 different vaccine attributes, including the number of premature deaths expected to be prevented from immunization, the availability of other medical interventions and whether the targeted disease has been stigmatized. A vaccine maker could give more weight to economic considerations such as the costs of clinical trials and licenses, say, whereas a defense-related agency could flag diseases that tend to afflict military personnel serving abroad. Out pops a numerical score for each candidate under consideration, thanks to a computational method also used to weigh complicated options for expanding Mexico Citys airport decades ago. Each score is broken down to reveal how much the chosen priorities contributed to the final number.

The SMARTest algorithm in the room? Advocates for this approach hope that the tools flexibility and transparency will render it more likely to be used when different decision makers sit down to talk. However, it has received mixed reviews.

Some attendees of last months meeting questioned the usefulness of boiling down the complexity of a hypothetical vaccine to a single number, or they worried that a high score could be mistakenly used as proof positive that a particular vaccine candidate is, objectively, the best option. You can tweak the system in a way that gets the outcome you want, cautioned Joachim Hombach, acting head of the World Health Organizations Initiative for Vaccine Research in Geneva, at the meeting. As such, the score will probably not be useful.

Although no one outside of the IOM panel has yet gotten a chance to test drive the SMART Vaccines software, critics doubt its ability to compete with other specialized tools already in use. The disease model with which we evaluate different TB [tuberculosis vaccine] candidates is quite a bit more sophisticated and detailed, Ann Ginsberg, head of scientific affairs at the Maryland-based, tuberculosis-oriented nonprofit Aeras, told meeting attendees. According to Ginsberg, Aerass propriety tools can check for interactions between diseases, as well as simulate how

a tuberculosis epidemic might play out over different time scales stretching out to centuries.

GlaxoSmithKline, the London-based manufacturer of vaccines targeted against influenza, diphtheria, rotavirus and hepatitis, similarly examines diseases on a finer scale than the SMART tool can provide, says Genevieve Meier, a health economist at the drugmakers North America Vaccines Division near Philadelphia. The companys models take into account differences between urban and rural populations, for example, and factor in how a disease works on a broader range of scales than does the IOMs modelfrom how a virus interacts with a cell to how a pandemic spreads across the globe.

Still, Meier supports the development of the SMART tool for others who lack the resources to develop an in-house alternative. This could be a big step forward for a lot of countries that are trying to make decisions with a limited understanding of a disease in their own population, she told Nature Medicine. However, with limited understanding often come limited data, especially in the developing world. The challenge is to populate this tool with credible numbers and assumptions, Alex Palacios, of the Geneva-based GAVI Alliance, said at the meeting.

Thats why Carole Heilman, director of the Division of Microbiology and Infectious Diseases at the US National Institute of Allergy and Infectious Diseases in Bethesda, Maryland, encouraged the IOM committee to focus its efforts now on creating an open, centralized data library, rather than further tweaking the software. Vetted information stored in that library could feed into any tool used by any vaccine developer, she said. To me, this is the most valuable aspect of any future direction that the IOM panel should consider, she advised the committee.

With this feedback in hand, the IOM panel will modify its prototype. The final version, due out late next year, is planned to be an open-source project that can continue to be modified. According to Carter Diggs, senior technical adviser to the Malaria Vaccine Development Program at the US Agency for International Development in Washington, DC, What were seeing now is the first infant steps of what will happen over the coming decades.

Devin Powell

1720 VOLUME 18 | NUMBER 12 | DECEMBER 2012 NATURE MEDICINE