The result held. The data grain changed what we could test.

A headline worth testing

In 2023, Nature published a study with a plain title: the recovery of European freshwater biodiversity has come to a halt. It was led by Peter Haase at the Senckenberg Research Institute in Germany, with a large pan-European team that included the Laboratory for Freshwater Ecology and Inland Fisheries at NORCE Norwegian Research Centre in Bergen. The team assembled 1,816 long-term records of freshwater invertebrate communities across 22 countries, some running from 1968 to 2020. Their analysis found that the recovery seen during the 1990s and 2000s had slowed substantially around 2010.

That kind of result is worth testing outside the original analysis environment. We loaded one of the public files into AlkhemYst and built a reproduction pipeline to see what this specific input could support.

What we checked

Our input was the site-level summary file: 1,816 rows, one per monitoring site. Each row contains long-term trend estimates for the biodiversity measures and environmental information around the site. The canvas connects that input to an audit, two trend checks, a driver sensitivity analysis and a machine-readable reproduction boundary.

Every independently calculated result and every figure below comes from the platform's deployed run. The published estimates shown in the comparison are reference values taken from the original study.

The long-term direction agrees with the paper. Taxon richness is rising at 70 percent of sites and falling at 30 percent. Abundance is falling at 39 percent of sites. The last two values match the proportions reported in the paper.

We then ran a separate study-balanced sensitivity check. It calculated the mean trend within each of the 47 source studies and resampled those study means 5,000 times. The four published estimates were compatible with the resulting intervals.

This is supporting evidence, not an exact reproduction of the authors' hierarchical Bayesian model. The two approaches use different estimators and should not be treated as interchangeable.

What this input could not test

The paper's central result about the slowdown around 2010 comes from moving a ten-year window across site-by-year observations. Our first input contained one long-term trend per site, not the yearly observations behind that trend. It could therefore check the overall direction, but it could not calculate the moving-window result.

That is a limitation of the file selected for this pipeline, not a failure by the authors to publish their materials. The wider public record includes a site-by-year file, moving-window outputs, driver outputs, raw composition data and the annotated R code. The public trend output also includes the uncertainty associated with each site estimate. Reproducing the full Bayesian and moving-window analyses would require bringing those additional inputs and the original model specification into a second pipeline.

The same distinction applies to our driver analysis. The selected input did not contain the uncertainty values required for the paper's weighted Bayesian model. We therefore ran an unweighted clustered regression as a sensitivity check and labelled it as such. It is useful for comparison, but it is not the published driver model.

The analysis unit is part of the method

The important lesson was not that the data were missing. It was that two public files from the same study answer different questions.

A row representing one site cannot be treated as if it represented one site in one year. In particular, the column recording when a series started is not an observation year. Feeding it into a moving-window calculation would produce an answer, but the answer would have no scientific meaning.

The corrected pipeline records that boundary explicitly. Its reproduction record states what the selected file supports, what requires another input and which substitution must be avoided. Anyone opening the run later can see the decision beside the results instead of reconstructing it from memory.

This is where reproducibility becomes an infrastructure problem. Keeping files open is essential, but openness alone does not explain which file belongs to which analysis. The question, the unit of analysis, the transformation, the model settings and the resulting figure need to remain connected.

For this study, the platform preserved that connection. It reproduced the checks supported by the site-level summary and made the next reproduction stage clear without overstating what had been completed.

Why the trail matters

A 2015 study estimated that more than half of US preclinical biomedical research was irreproducible, representing about 28 billion US dollars in annual spending. That estimate belongs to preclinical biomedicine and should not be transferred directly to freshwater ecology. It does, however, put a scale on the practical cost of losing the path between data, decisions and results.

How many hours did your team lose last year rebuilding a dataset or an analysis that someone had already completed because the trail behind it had gone cold? Put a number in the comments. The honest total is often higher than expected.

---

Study: Haase et al. 2023, The recovery of European freshwater biodiversity has come to a halt, Nature, doi 10.1038/s41586-023-06400-1.

Data: figshare 10.6084/m9.figshare.22227841, CC BY 4.0, and the authors' public repository at https://github.com/Ewelti/EuroAquaticMacroInverts.

Figures and independently calculated results: deployed AlkhemYst run run_MJObLC01xJje.

Cost estimate: Freedman, Cockburn and Simcoe 2015, The Economics of Reproducibility in Preclinical Research, doi 10.1371/journal.pbio.1002165.