Faulty assumptions lead to faulty conclusions.
Jeremy D. Bailoo, Sangy Panicker & Allyson J. Bennett
January 12th 2021
A new entry in the seemingly eternal quest for universal adoption of European approaches to the regulation of research animals appeared today in the journal Scientific Reports. The author, US veterinarian Larry Carbone, estimates from a sample of 16 facilities (less than 2% of the US total) that 99.3% of the vertebrate animals in research and testing in the US are mice and rats, with 111.5M used in US research in 2017-18. Carbone uses the paper to argue that the US should prioritize a public accounting of these animals through policy change.
With its singular focus on counting, however, the paper manages to steer clear of balancing fundamental ethical considerations that shape much decision-making about animal research. It also fails to provide supportive empirical evidence for some claims about how a different counting system would improve animal welfare. And last but not least, its methodology raises questions that threaten the validity of the results.
Image credit: Jackson Labs
Speaking of Research (SR) has a long history of posting international reports on the number of animals in research as well as providing the critical context needed for their interpretation. At the same time, SR has written extensively about the use of counts and numbers as an inadequate proxy for animal welfare and, further, as a problematic decontextualized surrogate for consideration of global changes in the use and need for animals in research and testing (e.g., here, here). In the case of the US, we have written about the ways in which rats and mice, along with other animals, are covered by federal regulation, external oversight, and mechanisms for public transparency (for example, here). As an organization concerned with global science, animal welfare, medical, and scientific advances, SR strives to consider not only research in single countries or world regions, but also the intersections between them. Thus, we read the paper with interest and provide a synopsis, as well as a detailed analysis below.
Global science, global context
Source: Public sector biomedical R&D expenditure in 2012.
The paper adopts a European-biased lens, failing to provide serious and full consideration of whether (and why) the US—a global leader in public-sector expenditures in biomedical R&D—should model countries with similar, increasing, or decreasing investment in research and development. For example, an international comparison showed that the US accounts for 44% of global biomedical R&D, with Europe at 30%, and Asia-Oceania at 23% (Chakma et al., 2014). Further, R&D investment trends show acceleration in some Asian countries, like China, and deceleration, or slow growth, in some European countries like Germany and France (see graph below). These trends matter if we consider the future rather than the past. While overall biomedical R&D investment data are not specific to animal research, they do highlight critical context for thinking about the impact, feasibility, and longer-term consequences of practices and policies that affect research. Such data may also raise questions about what—if any—effect policies have on the sustainability and growth of research. At minimum, in comparisons of animal numbers across countries, the size and scope of research provides the balancing context in which those numbers should be viewed.
Finally, a more inclusive consideration is requisite to avoiding the international bias that has no place in science. While Carbone begins the paper with “Alone among Western nations, the United States” he never returns to address the question of whether and why the policies of Western nations should be those that best serve global public health and animal welfare. The assumption is one that deserves thoughtful, serious, critical evaluation for contemporary science policy that can serve global public health now and in the future.
Redrawn from Source: Original caption reads “Data are for the top eight R&D-performing countries and the EU. Data are not available for all countries for all years. Data for the United States in this figure reflect international standards for calculating gross expenditures on R&D, which vary slightly from the National Science Foundation’s protocol for tallying U.S. total R&D. Data for Japan for 1996 onward may not be consistent with earlier data because of changes in methodology. Data for Germany for 1981–90 are for West Germany.”
**EU = European Union; PPP = purchasing power parity.
Problematic assumptions and methodological issues undermine usefulness
A central aim of Carbone’s paper is to produce an estimate of how many mice and rats are in research and testing in the US—with little consideration given to how, why, or when those animals are in research or testing. The article’s overall premise is that counting the number of rats and mice in research will lead to improved animal welfare. Unfortunately, the premise isn’t fully evaluated and the analysis is inexplicably very limited, with appearance of flawed assumptions and bias. In short, it is not clear that the findings are valid or generalizable, which is unfortunate given that science and animal welfare deserve clear thinking and strong evidence for decision-making. The number of animals in research and testing could increase or decrease for any number of reasons—emergence of new threats to global health, emergence of new technology, country-level declines in research investment, and so on. Narrow, decontextualized focus on counting animals does not provide a sound basis for public policy and is a disservice to thoughtful consideration of animal welfare, science, and public health. As it stands, the paper reads a bit as a political statement posing as an empirical paper.
Precise but not accurate data lead to flawed conclusions
The paper’s goal is presented as developing a basis for estimation of the number of mice and rats in research in the US. The author argues that knowing the number will inform analysis of the resources needed if the Animal Welfare Act (AWA) were expanded to include rats and mice.
The author analyzed 16 institutions—less than 1.7% of 966 non-federal research institutions registered with the USDA. The author’s analysis accounts for only roughly 5% of AWA-covered animals in research in the US in 2018 (the year assessed). Based on the numbers of rats and mice (RM) that were privately reported from each institution he concludes that 99.3% of the animals used in these institutions are RM.
The small sample analysis predictably leads to wildly variable estimates with no attempt or evidence of generalizability to the remaining 98% of US research facilities. If the paper’s findings were true, a handful of private companies that account for tens of thousands of publicly-reported AWA animals would have 1.4M rats and mice per every 10K AWA animals.
The data used in the analysis were available primarily via open records through state laws or the federal freedom of information act and were composed of a highly skewed sample of institutions with some of those highest in NIH funding. It does not include non-academic private companies and research facilities. Seemingly, if the goal were to estimate the number of RM used for research, an effort to contextualize the limited 2% sample with at least the proportion of public vs private institutions should have been made. The identity of both private and public institutions, as well as their annual AWA animal number reports, are easily accessible via the USDA website. No FOIA request is needed.
By our analysis, seven private facilities account for just over one-quarter of the US total AWA covered animals used for research in 2018—the top 50 extramural NIH-funded facilities account for just over 10% of the US total AWA-covered animals in 2018 (N=83,416).
In other words, analysis of only a single private company would cover more than twice the percentage of US animals as the current analysis of 16 facilities. Another way to say that is—a single private company covers more animals alone than 20 of the top NIH-funded institutions and two private companies have more AWA animals than the combined total of all 50 of the top NIH-funded universities.
If Carbone’s assumption is that requiring counts will lead to reliable analysis of changes in numbers over time, or to better animal welfare, he should be clear about the consequences of very narrowly limiting consideration to a very small proportion of those animals.
Instead, the author’s rationale for analyzing 16 large academic institutions is a combination of convenience sampling and assumptions about generalizability that may, or may not, be supportable. By the principle of parsimony, similarity in RM ratios for those 16 universities might simply be explained by the likelihood that these 16 universities may have more in common with each other in terms of funding and other resources than other academic institutions that were not analyzed.
In other words, the data may be highly precise and specific to the 16 institutions analyzed—but it is unclear that they are accurate.
The author does not appear to have even attempted to test these critical assumptions, despite the fact that data from public institutions across the spectrum of NIH funding could have been requested and included.
Visualization of the purported ratio of rats and mice in relation to 10,000 AWA covered species.
From his analysis Carbone extrapolates that RM account for between 97.3-99.9% of the total number of vertebrate animals in research. What that would mean is that for every 10,000 AWA-covered animals, a research facility would have 1.42 million rats and mice. In reality, however, there is no evidence that the estimate is generalizable and the assumption of similarity not only between academic institutions, but more broadly, between private companies and academic institutions is not tested. Given that the mission of private companies can be very different from academic institutions, it is entirely possible that the proportion of different types of animals may vary widely between them. For instance, a veterinary medicine company that develops and tests products may have many guinea pigs in research or testing, but proportionally fewer mice than a university with a large basic, or fundamental, research mission.
Misrepresenting the 3Rs? Does counting = better animal welfare?
As we’ve written about a great many times, the welfare of mice and rats used in public research institutions are guaranteed by regulations other than the AWA. If mice and rats were to be included in the AWA, critical consideration of the numerous costs in relation to the purported associated benefits requires thorough evaluation.
It is not universally accepted that mice and rats need to be included in the AWA nor that their inclusion provides substantive benefit to the animals.
In addition, the author misrepresents the 3Rs, emphasizing reduction—by itself—as the goal that indicates ethical goodness—i.e., less animals equals “better”, or “more” ethical. In fact, the core ethical consideration should be the balancing of animal welfare with respect to the scientific objectives to serve public health. Specifically, the number of animals used for research can increase or decrease for any number of reasons—emergence of new threats to global health (e.g., COVID-19), emergence of new technology (e.g., CRISPR), country-level declines in research investment, and so on. Narrow, decontextualized focus on counting animals does not provide a sound basis for public policy and is a disservice to thoughtful consideration of animal welfare, science, and public health.
Speciesism—what is the public view?
Overall, from the perspective of serious consideration and transparent public discussion, the paper misses the point that informs broad public policy. The core ethical consideration is not the number, but rather, whether humans should use other animals, for what purpose, under what conditions, and for what benefit.
The US, like a great many other countries, engages in animal research and testing in order to advance scientific knowledge and medical discoveries that benefit global health—for humans and other animals. When animals are required for research, their welfare is balanced with scientific objectives. When alternatives to animal use are available they are used, but ultimately, humans are prioritized over nonhuman animals in accord with many long-standing international ethics codes that include the Nuremberg Code, Declaration of Helsinki, and Belmont Report.
In other words, the relevant ethical consideration is not how many mice are in research in the US, but whether the nation has scientific capacity to advance knowledge about disease, to develop effective and safe prevention or treatment, and, when that depends on animal research or testing, how it is balanced with consideration of animal welfare. There are, of course, absolutists who believe other animals, including mice, should never be used by humans, regardless of the consequences of that decision (some examples here). For these individuals, the only answer to the numbers question is zero—no mice, no rats, no animals in research even if that means medical progress is limited or humans are used to test the safety and efficacy of new treatments, vaccines, and medical devices.
The question for global society is which alternative they would choose.
Like this:
Loading…
