When NeurIPS Flags Your Paper as AI‑Generated: How Trustworthy Is AI‑Assisted Review?

NeurIPS 2026 has begun using an AI detector, Pangram, to decide whether a submission violates its AI‑use policy, and the detector’s output can lead to desk rejections. A Reddit user complained that their paper was rejected by the Position Paper Track on the basis of a high Pangram score, prompting a broader discussion.

The track uses Pangram, a closed‑source AI text detector, and considers both the detector’s score and the author’s AI‑use declaration when making rejection decisions. This creates a potential circular argument: a high detector score is used to deem the author’s declaration inconsistent, and that inconsistency is then used to justify the rejection, making the detector more than an auxiliary tool.

NeurIPS’s own blog post (June 2) describes the tests they performed on Pangram, including audits on earlier ACM FAccT papers, synthetic AI‑generated position papers, and manually edited samples. However, the true target distribution—actual submissions to the 2026 Position Paper Track—lacks a known ground truth, making it unclear how the detector performs on real submissions.

The key question is the false‑positive rate on the real‑world distribution. A false‑positive rate measured on one dataset does not automatically transfer to another. If the submission pool shows an “abnormally high” proportion of flagged papers, it could indicate distribution shift or calibration problems with the detector.

To illustrate the detector’s behavior, the author ran Pangram on several recent papers authored by track chairs, obtaining scores of 69% AI, 45% AI, 36% AI, and 24% AI. The author stresses that these scores alone do not prove the papers were AI‑generated, highlighting the core issue.

NeurIPS policy states that papers must be primarily written by humans, with AI allowed only for polishing or peripheral assistance. The Position Paper Track chair adopted a conservative stance, arguing that excessive AI writing offers limited benefit to the research community and can obscure the authors’ intended meaning, shifting verification costs to reviewers and raising questions about contribution attribution.

To enforce the policy, NeurIPS partnered with Pangram under an enterprise‑level data agreement that guarantees no data retention during model use, and conducted multiple independent analyses to validate the model’s accuracy and minimize large‑scale misclassifications.

The final statistics reported were:

178 submissions (18.4% of total) were directly rejected.

123 submissions (12.7% of total) were asked to provide evidence of sufficient human involvement, otherwise facing possible rejection.

One of the Reddit commenters was among those directly rejected. The discussion also raised fairness concerns, with some participants labeling the detector as a “dead‑weight” tool, while others pointed out shortcomings in Pangram’s ability to detect AI usage.

Overall, the controversy reveals not only the risk of mis‑rejecting authors but also a deeper issue: as AI becomes embedded in scholarly writing, the academic community must define the line between reasonable assistance and excessive ghostwriting, and consider whether delegating this judgment to a black‑box detector merely postpones the fairness debate.