Last week, while working remotely, I saw a story pop up on the home page of Google.com. It was an article on Sciencex.com, a site where researchers can write popular media articles about their own research. The title of the article was very provocative: “No home where the buffalo roam? Birds decline after bison return to conservation grasslands.”
The article, which you can read here, was written by two of the authors of a peer-reviewed journal publication on the same research project. Because I was short on time, I skimmed through the article and was immediately bothered by the way the authors seemed to be interpreting and extrapolating their results and made a note to read the full journal article later. When I had the time to focus, I re-read the article on ScienceX, as well as the original journal article, which you can find here.
Because the authors were writing about topics I’m well-informed on (grassland birds, bison, grazing, and prairies), I had a number of concerns about how they portrayed their results. To keep this relatively short, I’m only going to present a few of those concerns here:
- The basic finding of their research was that high intensity (as judged by the researchers) year-round grazing by bison was correlated with reduced numbers of both adult and juvenile bobolinks within that management unit. Those bird numbers came from years of sampling bird populations (before and after bison were introduced to the site) by using mist nets to capture/release them.
- While their research focused on the response of a single bird species (bobolink) to bison grazing at a single site, the headline of the article in ScienceX is written such that a casual viewer would understandably assume the research documented declines of multiple grassland bird species across a range of bison grazed grasslands. To be fair to the authors, headlines are often written by the publication, not the authors, but it’s a bad start. Many people don’t read beyond the headline of articles like this, so headlines matter.
- In the ‘Conclusion and Recommendations’ section of their journal article, the authors make some pretty big extrapolations about their results. Those include this statement: “grasslands with high bison densities are not compatible with the objective of managing grassland for bird species of conservation concern such as Bobolinks.” In the ScienceX article, the authors consistently framed the description of their research in ways that suggested implications far beyond the very limited scope of their results.
- In both the scientific and popular articles, the authors cited two other studies that purportedly identified declines in grassland bird species following bison reintroductions. I’m familiar with both of those studies. One showed mixed results over a four year period and concluded “bison did not negatively impact nest success of grassland nesting birds immediately following their reintroduction (<5 y)” at a site in Illinois. The other showed bison grazing in a Kansas prairie decreased numbers of Henslow’s sparrows and dickcissels but increased numbers of upland sandpipers and grasshopper sparrows (all four species are of conservation concern).
- At the end of the ScienceX article, the authors did grant that bobolinks can do well in rotationally-grazed cattle pastures and that if the bison grazing at this site was managed to provide periodic rest periods from grazing, bobolinks would likely respond more positively. They then strongly implied that bison management of that type ‘may be beyond the capacities’ of organizations/sites outside of places like Yellowstone National Park.
So. Bobolinks, which prefer habitat with a relatively high amount of vegetation cover, declined in number within a grazing system that provided less cover than they prefer. That is not a shocking result. We don’t know how other grassland bird species did within this particular study area because the research either didn’t measure or report that. There is a tremendous amount of other research, though, that links bird species presence and abundance to habitat structure. Some birds like their habitat short/sparse, some like it tall/dense, and others like something in the middle. Anyone who has a reasonable amount of experience with grassland birds can walk into a prairie and tell you what bird species are likely to be present, just based on the height and density of the vegetation.
However, these scientists, while being careful to make accurate statements about their own research, consistently slanted their overall discussion of their study to imply that grassland bird conservation is not compatible with bison grazing – outside of huge landscapes such as Yellowstone National Park. They implied this based on their study of a single bird species in a single situation at a single site. They also presented incomplete, if not biased, portrayals of other research to back themselves up.
Bison grazing, like any grazing, is something that can be managed in many different ways, including at scales much smaller than the Greater Yellowstone Ecosystem. It’s silly to say it’s not possible to manage the timing or density of bison in smaller locations. That’s done all the time, including at sites I’m very familiar with.
On the other hand, it’s completely fair to look critically at how a particular site is being managed. We should always be evaluating the way species and communities are responding to prairie management treatments and adapt as we learn. However, that evaluation has to be conducted and applied sensibly. it wouldn’t make sense to broadly advocate against prescribed fire just because it is being used in a way that negatively impacts one bird (or butterfly, or whatever) species in one location. The same should apply to bison grazing or any other management strategy. The effectiveness of management strategies should be evaluated across a wide range of sites and assess a broad spectrum of implementation approaches.
People often get frustrated with scientists for being ‘wishy-washy’ in the way they talk about their results and how they can be applied, but there’s an important reason for that. Science is an iterative process. One study doesn’t tell us much – it’s the aggregation of evidence across many studies that starts to show us what’s really happening. Rarely does a single research project provide enough evidence to justify making strong statements about its implications, and when it happens, it’s usually an expansive study that examines a topic both broadly and deeply. As a result, scientists are wary of overstating the implications of a research project because they know it’s just one piece of a very complicated puzzle.
There is incentive, however, for scientists to make bold or controversial statements about their research – it’s a great way to get attention. That is displayed in this case, in which a very limited study of a single bird species at a single site has generated a lot of interest. I’ve seen the ScienceX article pop up several times in various feeds and conversations, which is why I decided I needed to write about it.
Most scientists resist the urge to strongly promote the implications of any single study because they recognize the danger in applying too much significance to or inappropriately extrapolating the results of any one project. They also work hard to collect and present their data with as little bias as possible and let their results be interpreted in the context of other similar work. In this case, I think the authors of the study and the publishers of both the journal article and the popular article gave in to temptation in a way that isn’t helpful – either to conservation or to the overall and growing distrust of science and scientists these days.
The Prairie Ecologist 
Great work, Chris. I’ve shared with the NGP team. Hope you’re doing well.
You’re breathing down my neck on Instagram. Gotta post more or you’ll catch me, haha.
Clay
[1461786929562_bethevoice_emailsig.jpg] Clay Bolt | Senior Program Officer, Communications Lead, WWF Northern Great Plains Program 13 South Willson Ave, Suite 1 | Bozeman, Montana 59715 clay.bolt@wwfus.org | 406.582.0236 ext. 114
Fellow, Linnean Society of London Senior Fellow, International League of Conservation Photographers ________________________________
Oh, I’m coming for you, pal. Watch your back… : ) Thanks, and I hope you’re well too. Keep up the great work!
Thank you for unpacking that study. It is always disturbing when scientists are not rigorously honest in their interpretation of data. High standards are good standards.
Chris, I wonder if you’ll have the occasion to interact with these researchers (or were previously familiar with the long-term study) because the research site is in the Platte River valley. From my perspective, you raise some excellent points about the generalizations and false correlations in the articles. I’m surprised that peer-reviewers did not call to question some of the conclusions or implications the research might have on management.
From Laura Domyancich-Lee: “I’m surprised that peer-reviewers did not call to question some of the conclusions or implications the research might have on management.”
A possible explanation for why peer reviewers (and editors) didn’t question some of the things presented in the paper is that the journal (https://www.mdpi.com/journal/animals) may not have a pool of editors and reviewers who were sufficiently familiar with many of the specifics of the subject matter.
The journal appears to be relatively “young”, publishing its first articles in 2011 (https://www.mdpi.com/journal/animals/history). It also appears to use a pay-to-play business model (https://www.mdpi.com/journal/animals/apc), which concerns some/many people because the pay-to-play business model can incentivize a journal to accept manuscripts, regardless of quality, because acceptance of a manuscript is acceptance of payment. Rejection of a manuscript is rejection of income. This contrasts with a business model in which consumers of the published product (journal subscribers, libraries, academic and research institutions, etc.) pay for the right to consume it. That business model presumably puts a higher premium on quality because consumers would presumably not be willing to pay for a low-quality product.
Finally, the journal seems to have a core audience that doesn’t consist of wildlife ecologists, but instead might be comprised of people in the veterinary sciences. The journal’s home page declares they are a JCR Q1 journal (ranked in the top 25% of journals) for veterinary science journals and a CiteScore Q2 journal (ranked in the top 50% of journals) for general veterinary journals. Those are reasonably good journal ranks and points towards the journal being reasonably high quality – from a veterinary science standpoint. But if their editors and reviewers are primarily well-versed in the veterinary sciences, it doesn’t speak well to the ability of their editors and reviewers to know much about the interaction of grazing and grassland birds in the Great Plains. And a blatantly cynical take-away of that possibility is that perhaps they don’t need to be well-versed in those topics, because perhaps they’ll accept payment to publish a manuscript regardless of whether the editors and reviewers can do a sufficient job.
Finally, I’ll offer some additional critical speculation, but I’m doing this from the standpoint of someone who has published in some very low-tier (low-ranked) and obviously non-specialized journals. I’ve done that simply because some of the research I conducted generated some results that just weren’t exceptional or even noteworthy, had substantial limitations, and was rejected by one or more higher-ranked journals. I ended up publishing them in lower-ranked journals because I wanted to get the information out there one way or another for others to see and consider. And this leads to a question that can be asked about the article we’re discussing – why wasn’t it published in the Journal of Wildlife Management, the Journal of Animal Ecology, Applied Ecology, the Journal of Applied Ecology, Rangeland Ecology and Management, Conservation Biology, The Auk, Condor, or the Journal of Field Ornithology, all journals where this type of research is regularly published? For that matter, why wasn’t it published in Great Plains Research (https://www.unl.edu/plains/publications/GPR/gpr.shtml), a journal published by the institution one of the authors is affiliated with (https://www.unl.edu/plains/)? Maybe the manuscript was submitted to these or other journals and rejected. Again, that’s speculation on my part but these are all the type of questions we should always be ask as we assess how much we should hang our hats on anything that is published.
In the authors’ defense, perhaps they wanted to publish in that journal, which appears to be based I Sweden (https://www.mdpi.com/journal/animals/editorial_office), because the author who has an affiliation with an academic institution in Sweden was familiar with that journal, had a high regard for it, and convinced the other authors it would be a good journal to publish in.
But all-in-all, I too am concerned with the headline that popped up in my various newsfeeds and the subsequent popular-audience piece I read about the research. I haven’t finished reading the original journal article, but so far I’m scratching my head about so much that wasn’t addressed and how the results aren’t sufficiently framed in the correct context, a context that is decidedly broader and richer. As was pointed out by Chris in his original post, grassland birds respond to vegetation structure, and there are birds that respond positively or negatively to any of the various levels of structure. You could “overgraze” a pasture with season-long grazing by bison or cattle, or any other large herbivores, and upland sandpipers would love it. But Henslow’s sparrows would hate it. You could remove all grazing by animals, very rarely burn, and let the accumulated vegetation biomass build up year after year and Henslow’s sparrows would love that while upland sandpipers wouldn’t think too much of it. Somewhere in between those two extremes you might find meadowlarks (perhaps edging towards the half of the gradient with higher amounts of vegetation biomass) and grasshopper sparrows (perhaps edging towards the half of the gradient with lower amounts of vegetation biomass). Here’s research that demonstrates this in a cattle-grazing system with breeding grassland birds (https://doi.org/10.1890/1051-0761(2006)016%5B1706:SHBTBF%5D2.0.CO;2) and non-breeding grassland birds (i.e., wintering; https://esajournals.onlinelibrary.wiley.com/doi/full/10.1890/ES14-00062.1). Figure 5 in that second one (non-breeding birds) is a nice graphical depiction of the concept. Figure 1 in this article is another nice graphical depiction (https://bioone.org/journals/rangeland-ecology-and-management/volume-62/issue-2/08-008.1/Livestock-as-Ecosystem-Engineers-for-Grassland-Bird-Habitat-in-the/10.2111/08-008.1.full).
Bringing back bison means coyote #s go down, furbearers like raccoon go up then everyone says “What happened to the Savanna sp and UPSANP?” By my observations, manmade rules involving hunting restrictions has made this problem what it is.
It seems like maybe this study is getting at the problems with reintroducing one species/part of an ecosystem while ignoring others.
The study itself even pointed out that practicing high-intensity, short-duration rotational grazing management would probably mitigate some of the negative impacts on the bobolinks, and a lot of other research has suggested that in a pre-settlement Great Plains ecosystem, bison would be constantly moving in right herds in order to escape predation by wolves.
I wonder if reintroducing bison also “requires” reintroducing wolves in order to recreate traditional bison grazing patterns?
Thanks for sharing your analysis with us, Chris. I’m glad you were already familiar with the other two studies cited.
Well written, Chris. And what you are speaking specifically about applies to many other spheres of life as well. The universe is a messy, complex place. Which is what makes it so fascinating and wonderful!
I hope you were able to comment on their work where they could read it! Thank you for being diligent and pointing these discrepancies out for us.
I appreciate your analysis. You’ve brought our attention before to the relationship between type of habitat and the kind of species that inhabit it. Thanks for providing an example of how to use our education and experience when perusing scientific (or any other) articles.
This also caught my attention after it was published in UNL’s pocket science series. https://news.unl.edu/newsrooms/today/article/losing-ground-dense-bison-herds-may-threaten-nesting-bird-species/
I believe part of the problem is how the study was promoted in the first place; the editor’s need for a dynamite headline; and the pressure researchers are under to get recognition and hence, grant money to continue their work. None of these factors contribute to academic integrity.
What the ….?! Thank you for calling out this shoddy science. I’m angry and sad that stuff like this can get through peer review, and then hit popular media with even more misleading summaries. I will try and do my part to also help staunch this misinformation as well.
PLEASE send this as a letter to the editors of that publication. Hope they print your points!
An excellent summary of this conundrum and of the scientific process. Thank you for your attention and reasoning.
Thank you for this article and the explanation. One term in your article taught me something I did not know.
The iterative process is the practice of building, refining, and improving a project, product, or initiative. … You can think of an iterative process as a trial-and-error methodology that brings your project closer to its end goal.
When I was teaching at a university I tried to get students to think critically when reading or listening to “research” and “facts.” I know that i got through to some of them, but most were as unaffected as water down a duck’s back. Sadly, people who try to be raised on professional news are constantly bombarded by commentators who do not understand math, much less analysis. They read cue cards prepared by people who also do not have the skills required.
Your point about promoting themselves and their work is very accurate. I would go further by adding to that by saying that many “researchers” are not really that competent (in a good research sense). They are unfortunately rewarded by publicity not detailed professionalism and precism.
Two key phrases “intensive grazing”
and “many ways to manage grazing”
I am NOT a prairie expert, nor do I have more than a nodding acquaintance with either bison or bobolinks. But I am an amateur observer of ecological systems, and in general I think a lot about systems, having worked on computer networks for a lot of time.
My understanding of bison behaviour is that they wandered in large herds, sometimes horizon spanning in scope. My guess is that you would have very unhappy bobolinks in their wake.
Grazing confined to a small preserve is not going to have the same impact as the occasional visit of a wandering bison herd jugernaut.
Cows when confined to pasture quickly make trails — patterns of movement between areas to graze and source of water. Do bison do this when they are confined? This is going to create edges and edges are important.
In moderately steep terrain, both cattle and sheep will walk the contour. The covers hills with ‘steps’ or ‘terraces’ This will have a significant effect on how water runs off the land.
I’d like to see some comparison studies:
* Vary the stocking density. At one end have stocking done at levels comparible to pasture raised cattle in terms of tonnes of cow/acre. At the other end stocking comparable to the prairie in the 1800’s
* Vary the recovery time. Lot of mention in regenerative agriculture about grazing intensely for short periods, then giving that land a rest. This is more comparable to the roaming herds.
From your description, these guys have put one datapoint down. More work needs to be done.
I agree with your suggestions for comparison studies, they would be important in deriving useful implications for management. This study seems to only scratch the surface on the issue of declining grassland bird populations when most research on this large-scale topic can only really be considered as a piece of the puzzle. As you said, this seems to be one data point where more work needs to be done.
How’d this get past peer review?
Thank you!
I’d love to read about prairie bird evolution and bison in North America. Maybe there’s a book you know of, or a bibliography? There are birds that need dense grass cover and others that abhor it (we find them using turf farms), and others that need both or one or the other at different times of the year.
Some good info here: https://www.nrcs.usda.gov/Internet/FSE_DOCUMENTS/nrcs144p2_054067.pdf
Geez, I agree. Why don’t you run for President!!
Pingback: Noisy Boys of the Prairie | The Prairie Ecologist