Overly benevolent reviewers: what AI is already doing to science

According to Science, the number of scientific papers doubled between 2020 and 2025, which conceals an even larger number of submissions^[3]. On the funding side, I had watched from the front row in 2025 as the number of applications to Horizon Europe schemes rose massively^[4].

Funders are beginning to take measures that are explicitly motivated by generative AI. In July 2025, the American NIH, the world's largest public funder of biomedical research, capped applications at six per researcher per year, citing the influx of AI-generated proposals^[5].

And what is arriving upstream of the system suggests that this is only the beginning of a much deeper transformation. Matthew Schwartz, a physicist at Harvard, recently described an experiment conducted while in residence at Anthropic: in three months, with nineteen co-authors and Claude's intensive help, 36 manuscripts produced across 18 different disciplines^[6]. The signal is clear: even leaving AI slop aside, the capacity to produce papers and proposals has changed scale.

Evaluation, which still rests on the unpaid time of reviewers, was in truth already at breaking point before AI arrived. On the funding side, the same exhaustion can be read from the other side of the counter: spending dozens of hours on an application whose success rate is measured in single percentage points, with writing codes and expectations that are sometimes opaque to researchers, inevitably pushes people to save time with the tools that allow it. So AI is not bursting into a healthy system: it is rushing into a system at breaking point. That is what explains the speed of its adoption: for its 2026 edition, AAAI already ran an AI review pilot, and nearly 23,000 papers received, alongside at least two human reviews, a review generated by a model from the GPT-5 family^[1].

That is also what makes the question of tool choice, including through the constraints set by institutions and oversight bodies, so difficult to raise: an exhausted system that adopts the tool that relieves it is in no position to examine the tool. Usage is developing much faster than decisions about frameworks. According to a survey of 1,600 researchers reported by Nature, more than half already use AI tools in their reviewing work, most often against journal guidance^[7]. Most journals still bar their reviewers from using AI while they work out a framework; in the meantime, as one researcher interviewed by Science puts it, "people are using these tools in a clandestine, unregulated way. It's kind of the Wild West out there now"^[3]. I see the same thing in grant writing: American frontier models are used across French and European institutes, by researchers and support staff alike, including where official guidance proclaims sovereignty. And what circulates through these uses is far from trivial: unpublished manuscripts, confidential reviews, projects that have not yet found their funding. In other words, it is tomorrow's European research that is being handed over to the infrastructures of a few private American actors. Institutional constraints, today, do not decide which tools enter the system: they are chasing the usage.

Artificial intelligence is already transforming the entire system of scientific research: the writing and evaluation of papers, the drafting and assessment of funding applications, and even the conduct of research itself.

The values hidden in the tools

This is where the article by Nitin Verma, an information science researcher, delivers the result that should give us pause^[1]. He tested six OpenAI models on 137 manuscripts submitted to the ACL 2017 conference, asking them to evaluate the papers. Four of the six models accepted nearly all of them, and the two most recent about two thirds, where the conference's actual acceptance rate was below 25%.

This behaviour is a direct product of how LLMs are designed: they are meant to be agreeable, helpful assistants. Verma notes that the models tested strongly express values such as benevolence, care and tolerance. These are excellent values for a conversational assistant. They are almost exactly the inverse of the values on which peer review rests, as theorised by the sociologist Robert Merton: organised scepticism, disinterested critique^[8].

The same diagnosis appears in AI-assisted science. Schwartz describes his teammate in these terms: "Claude loves to declare victory", "Claude loves to please". This eagerness to please often yields results that are technically correct but scientifically unremarkable, until an expert in the field steps in to reset the direction^[6].

Two independent observations, one from the evaluation side, one from the production side, of the same phenomenon: a model's values shape its results.

This debate about values reminded me of the training sessions I used to run for innovation and R&D teams in large industrial groups, at a time when their profession was going through another profound change: integrating environmental issues into their technical development processes, and therefore learning to see the values at work in their technical choices.

I used a two-stage foresight exercise. First, the teams imagined the typical day of a persona 10, 20 or even 30 years ahead, built around a product innovation they anticipated in their sector; the technological proposition was told not through its technical specs but as an anecdote of daily life. The second stage was more uncomfortable: I imposed a set of values on the teams and asked them to rewrite exactly the same story. How would the same scientific knowledge, the same technical capabilities, be transformed in the service of other values? In every session, the same thing happened: the future changed completely, and sometimes the technical object itself changed in nature.

In the case study from the perfume and flavour industry, the participants had imagined Samantha, twenty-eight, living in a self-sufficient tower in Paris, and her everyday technology: her intelligent personal assistant picks up on her irritation and decides to fill the kitchen with a soothing scent, via a capsule slotted into something resembling an espresso machine. Then the imposed values arrived: "nature first" and autonomy. The teams rewrote the scene. The personal assistant disappeared. Samantha, still irritated, starts up a terrarium herself, and the scent now comes from the odour-producing bacteria living inside it. The R&D teams, and their fine-grained knowledge of scents and their effect on emotional states, are the same. But the values did not only change the use, they changed the object: a capsule machine driven by an assistant in one case, a biotechnology you start up yourself in the other.

This exercise was particularly powerful for participants from R&D and research departments: it is not enough to think through the technical developments, you also have to choose the values they rest on.

Without taking into account the values of LLMs, optimised for an entirely different use, scientific evaluation becomes complacent and stops doing the job we expect of it.

Taking back control: building tools that answer to our values

In a previous essay^[9], I described two design choices that structured an AI tool I worked on for three years: a world logic, that is, an explicit description of the objects that exist in a domain and of their relations; and information gates, that is, the list of the pieces of information that must be gathered before the system acts (how many objects are known, how they relate to one another), with a refusal to move forward while they are missing.

Review-assistance tools already exist and are improving fast; the Science article surveys several of them, and I do not claim there is only one way to build them well^[3]. But one possible approach, to avoid the pitfalls already documented (notably complacency and the recurring difficulty of judging novelty), is to formalise the world logic of this profession and to define its information gates: the list of the information that needs to be in hand before a solid evaluation can be produced. A study published in PNAS, for instance, which examined "autonomous AI scientists", identifies four methodological flaws that are nearly invisible in the final papers produced, and recommends requiring the system's execution logs and code to be submitted along with the paper^[10]. As new ways of producing knowledge develop, the definition of the information needed for a sound evaluation will change profoundly.

Producing this world logic is deep work that requires the practitioners of the profession. It is by choosing the gates that you choose the values that will sit at the heart of how the tool works, which brings us back to Verma's finding.

This work has, in fact, been done by Schwartz, even if he does not name it that way. Faced with his teammate's flaws, Schwartz did not trust the model: he built BootLoops, an open source framework in which a computational result only stands once it has passed explicit acceptance gates: thirty digits of numerical agreement between independent methods, adversarial review between agents, certified arithmetic designed so that computations "cannot be silently wrong"^[11]. It is a written world logic, with its information gates, dictated by the values of scientific computation: verifiability, exactness, scepticism by default.

If this work has been done nowhere for peer review, it is not only a question of means. The scientific world does not like talking about its values: it built itself on an ideal of neutrality that relegates that vocabulary to politics or morality, and the questioning has always been unwelcome.

AI no longer allows it that comfort. The values are already in the tools, with measurable consequences.

Peer review as we practise it is a historical artefact: it was formalised in the mid-twentieth century for a world where manuscripts were scarce and expert attention relatively available. AI inverts that ratio, and asking models to imitate yesterday's reviewer amounts to automating a system designed for a world that no longer exists. Schwartz says as much at the end of his experiment: he no longer knows how to train his doctoral students, and he asks why anyone would apply for three years of funding for a problem a model might solve overnight^[6]. These are not questions of tooling. They are questions about the organisation of the scientific system, and they are now on the table, whether we answer them or not.

The time has come to ask the redesign question for real: not adding AI to yesterday's system, but rethinking evaluation, funding, training and infrastructures for the world in which these tools exist. That is the question now facing our scientific institutions, and it goes beyond laws and regulations: it is a question of norms and values.