arXiv draws a line on AI slop

In other news, arXiv’s CS section chair Thomas Dietterich announced that authors who submit papers with “incontrovertible evidence” of unchecked AI content (e.g. hallucinated references or leftover chat content like “here is that manuscript you requested”) will be banned for a year. After that, future submissions will be required to clear peer review before being posted on arXiv. This caused a bit of a stir online, with some researchers supportive and many strongly opposed. One opposition camp was frustrated that this move shifts arXiv even further away from the openness and lack of gatekeeping that made it valuable in the first place and is needed going forward. I’m sympathetic to that concern, especially with regard to the post-ban requirement that submissions pass peer review. But another camp seems to find it unreasonable to expect authors to verify their claims and citations, which I am less sympathetic to. It is certainly true that references have included mistakes or misattributions since long before AI, and these are usually relatively harmless (though they don’t suggest great citation practices on the part of authors). But it’s never been good when the knowledge ecosystem is flooded with unsourced claims, and AI allows that to happen at speed and scale previously unthinkable; if researchers can work with AI to write a paper that is accurate and rigorous, fantastic! But having AI write an inaccurate paper that is never carefully reviewed is not fantastic. Holding authors accountable for the accuracy of the work they’ve put their name on seems worthwhile.