The AI Trust Gap in Mathematics

Mathematician Andreas Thom has raised serious concerns about OpenAI's transparency after the company provided conflicting accounts of how it uses unpublished research data. While OpenAI initially denied using specific user conversations for training, later admissions suggest that de-identified data is still utilized to improve their models. This has led to a broader community debate about the risks of using AI tools for sensitive, pre-publication mathematical research.

Key Points

• Andreas Thom argues that OpenAI lacks transparency regarding whether unpublished mathematical research shared with ChatGPT is used for model training or reasoning.
• OpenAI's conflicting statements—denying direct access to user data while admitting de-identified data may improve models—have created a trust deficit among researchers.
• Community members point out that 'opting out' of data training may be insufficient, as data can still be harvested through feedback loops, safety triggers, or older conversation threads.
• The controversy suggests that the communal process of mathematics could be damaged if researchers cannot trust AI tools with sensitive, unpublished work.
• The discussion was prompted by the release of major results concerning finite-time blowup for incompressible porous media, Boussinesq, and 3d incompressible Euler equations.

Sentiment

The overall sentiment is skeptical and critical of OpenAI, with many commenters agreeing that the incident damages trust in AI tools for confidential mathematical research. The thread is divided on whether the facts establish plagiarism or misconduct, but there is broad concern that OpenAI's assurances are hard to verify and that existing privacy and attribution norms are inadequate for AI-mediated research.

In Agreement

• OpenAI functions like an untrusted collaborator if researchers share unpublished ideas with its products and cannot verify how those interactions influence later model behavior or research outputs.
• Even without direct prompt access, de-identified logs, feedback channels, safety systems, evaluation data, or training pipelines could contaminate models in ways outsiders cannot audit.
• Hearing that mathematicians were close to a major result and then using superior compute and internal model access to pursue the same target is viewed as front-running and ethically corrosive.
• Academic mathematics depends on priority and credit, so ambiguous attribution harms careers and research incentives more severely than it might in routine software work.
• The episode fits a broader pattern where AI companies extract value from human knowledge and user interaction while returning little control, compensation, or verifiable accountability to contributors.

Opposed

• OpenAI's stated position distinguishes direct use of private prompts from the unlikely possibility of indirect de-identified product data, and some commenters see that as a meaningful ethical difference.
• If OpenAI believed the problem had already been solved and tried to coordinate credit rather than preempt publication, its actions may be defensible or at least not clearly unethical.
• Claims of plagiarism require stronger evidence, such as a prior proof or substantial overlap with the OpenAI result, rather than suspicion based on timing and distrust.
• Some commenters interpret the backlash as part of a broader discomfort with AI systems becoming capable in domains where experts previously expected human exclusivity.
• A few argue that learning from interactions is not categorically different from human learning, though others strongly dispute that analogy because corporate ownership and scale change the incentives.