The blog posts on this website are few and far between these days, as over the years I have created different sections of this website to share various goings-on in my life, both mathematical and non-mathematical. The content of today's very long contribution, however, most surely belongs in a blog post, as it is largely dedicated to expressing my opinions on a mathematical "current event", so to speak.
My recent papers
Since my last blog post, which was almost a year ago, I've posted two papers to the arXiv, both of which were collaborative. The first was "Entropy lower bounds and sum-product phenomena", which was written in collaboration with Lampros Gavalakis and Ioannis Kontoyiannis at the University of Cambridge, and the second was "Communication complexity of point-line incidences over the reals", written with my PhD supervisor Hamed Hatami. This second paper draws upon the construction used in the recent disproof of the sum-product conjecture over the reals, the broad idea behind which can directly be traced to OpenAI's disproof of the unit distance problem. Naturally, working on it has got me thinking a lot about how (un)comfortable I am with AI playing a role in the mathematics I produce.
My stance towards AI
If you know me personally, you'll already know that I try to interact with AI as little as I can. I won't delve too deeply into why; the title of this post is not "Why should someone become an AI vegetarian?" My reasons are not very creative, and others have argued for similar opinions better than I can. But using AI doesn't seem fun or enjoyable to me, and I find the practices of AI companies abhorrent enough that I don't want to have anything to do with them, even if I can eke some personal gain out of using their tools. Many forms of AI are hard to get away from without quitting the internet entirely, but not using generative AI is as simple as not typing in any prompts. So my stance on this form of AI has always been clear: I just won't use it. I have never entered anything into a large language model, and I hope that I will never be forced or pressured into doing so. On this topic of forcing the use of AI, I'll share an anecdote.
Back in 2023 before I started my PhD, I worked as a software developer intern in a small company. At that time, none of my colleagues were regularly using LLMs to perform coding tasks, and the team had a policy against the use of such tools, mostly due to security concerns. There was a single exception: on a certain week, the higher-ups asked everyone to use AI as much as possible (without pasting sensitive code directly into the prompts, of course). At the end of the week, everyone gave their reports as to what AI could do and what it couldn't. I chose not to participate in the exercise, which was something I felt like I could do, as a temporary intern with a PhD position lined up. If I had instead been a permanent employmee, I don't think I'd have been as confident declining a company-wide order to use a certain tool that way. This exercise certainly made me feel like I was making the right choice in returning to academia, where (I hoped) an emphasis on slow and deliberate understanding would take precedence over frantic hysterical productivity. (As a side note, the employee reports following this AI Week painted a fairly dismal picture of the coding capabilities of AI. I think in early 2023 that was perhaps accurate, but the tone of some reports made me suspect some respondents were deliberately downplaying the LLMs' strengths. In any case, judging by the current state of LLM coding tools I would be surprised if anyone at that company wasn't using AI today.)
The first two years of my PhD went by without AI significantly impacting me in any way mathematically. Sure, now and then someone would show me an AI-generated proof, but these "proofs" were mostly laughable. I actually encountered AI most during teaching, when a student would come into office hours demonstrating large gaps in their understanding, and would admit to having "studied" by asking AI conceptual questions about the material. (What they didn't admit to was using AI to directly solve homework problems, but I suspect many students in my classes did that as well.) It was less than a year ago when I first heard about LLMs actually helping my colleagues develop key steps in their proofs. And since the new year, these reports have come with a recommendation: "You should really start using AI too."
I did not invent the term "AI vegetarian" that appears in the title. I first saw it in this blog post, which traces the origins of the word. The opinions expressed in this linked post are close to my own, so I won't repeat things that are better explained there. I'm not going to try and convince you to be like me, or convince the world at large to stop the spread of AI into every domain of life. This post will mostly be confined to discussing practical aspects of adhering to this AI-free lifestyle while also trying to make a career out of mathematical research.
AI and math careers
It is the "making a career" part that is concerning to me; the old-fashioned way I do the actual research hasn't magically become easier or harder now that other people are using AI. But recently there has been evidence to suggest that most mathematicians using AI to some extent see some measurable increase in productivity, and it is unclear how one should treat those who forgo it entirely. There is certainly some coherent logic to support the view that people who abstain from using AI choose to do so at their own risk—when comparing two candidates, the fact that one uses AI and one doesn't shouldn't favour the committee any which way, because in theory both candidates have the same access to the AI tools. One of them is simply not making use of a tool that the other uses. (It is also not at all difficult to imagine a world in which the mathematical candidate who does use AI is favoured over the other, just as a mathematician being adept at, say, computer programming is generally seen as a desirable trait.)
On the other hand, one has to admit that asking an LLM to do math seems to require a very different skill set than actually doing math. Because I haven't actually prompted an LLM before, I'll have to rely on the experiences of others to demonstrate this. I've picked one that came across my path, but it is not my intention to pick on this paper specifically; this choice is meant to represent a new but rapidly growing class of heavily AI-assisted mathematical papers. The paper in question claims that the only mathematically meaningful input from the authors is the prompt: "Try to use the proof idea [link to OpenAI's unit-distance PDF] to improve the bound in [link to another PDF]" (see the end of Section 1 in the paper for the precise prompt, and a description of the follow-up prompts). The workflow of simply asking a question, obtaining something somewhat nonsensical from a non-human entity, and then asking it to improve its answer until I get what I desire is an extremely different from the traditional method of doing math that I practise. It is very rare for me to turn to an non-human entity and ask it questions, but so far these entities (the ceiling, the sky, etc.) have not done me the favour of replying. And even then I'm not usually so bold as to ask full question outright; I usually break it down into smaller questions.
(Another tangent: I have long been a user of the website MathOverflow, where I post questions related to my research when I get really stuck. This has something in common with prompting an LLM, in that one can "prompt" an outside entity into giving insight on a problem. The main difference is that on MathOverflow, one is asking people to help them, and this interaction is public, so the exchange of ideas is traceable (hence one is expected to acknowledge helpful interactions on MathOverflow if they lead to something that ends up in a published paper). The fact that questions and their askers are public also prevents anyone with a sense of shame from overusing this question-answer system for their own gain.)
My doom-and-gloom outlook towards the prospect of early-career mathematicians who do not use AI is based on a certain view of the economy of mathematical academia that some might call overly cynical or pessimistic. Many mathematicians are funded—in whole or in part—by public funding agencies. The actual value that society at large gets by funding mathematicians is at best hard to describe, and at worst negligible. The closest encounter that average middle-class taxpayers get with a math researcher is when their kids go to university and get served up canned mathematics by their professors. Of course, mathematically trained people do all sorts of useful things in the world, so a mathematician who teaches a 300-person class how to do calculus has some distributed indirect impact. The amount of value a mathematician produces in this way is hard to measure, so agencies usually judge a performance by the number of research publications one has written. Mathematicians like doing research anyway, so they do it, and the ones who produce lots of good results get rewarded with more funding in the future. But these research papers themselves have an extremely nebulous impact on the ones who are paying the mathematicians. The average math paper is read by only a small handful of people, most of whom are just other researchers playing the same game.
If AI-generated mathematics continues to improve at its current rate, it would undermine this economic structure by allowing some people (the ones with access to strong language models) to produce research publications faster than before. This in some sense exploits the reward system of academia without actually producing more value for the "consumer". In light of this, it is clear the economic system is unstable, but where it will find its new equilibrium is unclear. A best-case scenario might be that we emphasise research output less when determining a mathematician's worth—maybe points on a CV that were previously more important for teaching positions, such as outreach and mentorship roles, will become more important factors even when hiring for research faculty positions. But I'm not sure how this can be carried out at scale without much more time and effort on the part of hiring committees.
The worst-case scenario seems to be if the committees take an extremely laissez-faire attitude towards this new increase in research "supply". Assuming roughly the same amount of research still gets published but the average number of papers per person increases by some constant factor, the number of people who make it to any given milestone in academia decreases, which is actually a good thing for the publish-or-perish economy, since universities can produce the same number of papers while paying salary to fewer faculty members. This means fewer jobs for someone like me who will be applying to faculty positions within the next five or so years. Once again, this is speculative in the worst-case direction, but doing a PhD, or applying for a postdoc for that matter, involves a great deal of sacrifice, time- and money-wise. Most people are only comfortable with this opportunity cost if there is some decent promise of subsequent renumeration, and in the wake of recent advances in AI-generated math, speculations about the state of the field in the short-term future differ widely.
The ethics of abstaining
Consider the following scenarios:
- You are a doctor that many patients on a daily basis, and there is an AI tool that helps you diagnose twice as many patients in a day, and the diagnoses are accurate 50% more often.
- You are a low-level employee at an office job, paid for eight hours of work a day, five days a week. Your work is not a matter of life and death, but produces value to some interested parties. Implicit in this hourly wage structure is that you're expected to do as much work as you reasonably can every day. You have access to an AI tool which allows you to produce twice as much value at the same effort cost.
- You are a digital artist commissioned to preparing some visual art that will go on some organisation's website. You have the opportunity to use generative AI tools to produce in a day what would have taken a week in the past.
My judgement is that in Scenario 1, it's unethical not to use AI. In Scenarios 2 and 3 there is not so clear an ethical judgement to be made, but in Scenario 2, it appears to me that there is an implicit expectation that you should be producing as much value as you can with your time on the clock, so if I were the CEO of the company in Scenario 2, I might be annoyed if my employee didn't use AI. In Scenario 3, it feels as if there is an implicit expectation that how the art is made matters as much as what is actually produced, so if I were the client, I'd be annoyed if the artist did use AI. This is all admittedly very fuzzy justification, but this is my broad feeling.
Math lands somewhere on this spectrum. Others might disagree, but I would say it's somewhere between Scenarios 2 and 3. As discussed above, while mathematicians are judged by their research output (and this is the thing AI can help best with at the moment), I don't think this fully encompasses why people pay them their salaries. Hence I don't think it's unethical for a mathematician to abstain from using AI, since the mathematician is not directly paid to produce as many papers as they can in any given time period.
My boundaries
Here is my attempt to draw up a set of boundaries that I will adhere to for the forseeable future. I had hesitated to write this publicly because AI is so polarising that reading this pseudo-manifesto is likely to discourage some from wanting to hire me or collaborate with me. (My only consolation is that I'd likely be unhappy working with anyone who is willing to discount me on the basis of these AI boundaries anyway.)
- I will not use generative AI in any way in my research, i.e., I will never ask something like ChatGPT or Claude any questions. I do use search engines (with the generative feature disabled), which puts me into incidental contact with AI-generated content from time to time, but even then I will use alternative human-generated sources of information whenever possible.
- I will collaborate with users of generative AI in so far as they respect my continued commitment to abstaining from AI tools.
- AI-generated math papers that are human-verified I will treat I would any other math papers. I will examine their content closely, draw inspiration from them sometimes, and cite any influence they have on my own human-generated work.
- I will not put my name on a paper that presents any AI-generated mathematics.
The fourth item is certainly the most extreme. It subsumes the first item but I thought all the points were worth stating separately; its relation to the second item may be somewhat confusing, so I will try to clarify my stance by means of the following two scenarios.
- I work with someone who uses AI now and then. Between our meetings, my collaborator asks questions to ChatGPT to point them towards related papers and to deepen their understanding of the topic. Maybe they even fall under the temptation to ask the agent to prove a lemma outright, but the agent returns nonsense, so none of this AI-generated output makes it into the final paper. After the paper is written by the two of us, my collaborator feeds the output into ChatGPT to see if it can spot any errors. It cannot, and we publish the paper.
- Same scenario as above, except that when the collaborator asks ChatGPT to prove the lemma, the machine produces a short proof of the lemma that we cannot trace to any other papers. In this case I would not put my name on the paper if the lemma appears in it. Maybe the result can be expanded to something that my collaborator can post separately and that we can cite in our present work. If no such compromise can be found, then I would simply remove my name from the paper.
The first scenario represents the maximum contribution that I am comfortable with AI having in a paper of mine. The second scenario differs from the first only by the circumstance of the agent finding the proof of a lemma, so I hope to be able to communicate with any of my current and future collaborators that I'm not keen on them using AI in any way that could possibly lead to the second scenario. I'm even a bit uneasy with them asking for AI feedback on a paper, but I understand that's so common these days that it's not something I can really take a stand against. (In any case, this act does not seem sufficiently "generative" for me to feel too strongly against it.)
The big question
This leads me to the question in the title of this blog post. Can someone as junior as me in the math world operate under such restrictions? How should the community treat such people? At this stage in my career, all but two of my research papers have been collaborative, and since I am a PhD student, most of these collaborations have been with researchers more senior than I am. It can be argued that I stand to benefit much more from any potential joint work than my collaborators in these cases, so it seems somewhat petulant and even ungrateful to impose rules such as these upon the collaboration—it's not my place. But I also cannot imagine loosening my stance on this. I would sooner stop doing mathematics.
I will leave you with a bedtime story that heavily dramatises how I feel about this situation.
Once upon a time there was a boy. This boy lived in a kingdom where some people rode bicycles and some did not. Whether could ride a bicycle well was not a significant factor in determining one's worth in society, but most people learned the basics of cycling at a young age. The boy was no different in this regard, but towards the end of the boy's schooling, some of his teachers noticed he was fairly athletic, with good leg muscles, so they encouraged him to cycle on the school team. He improved quickly, made the duchy's cycling team, and soon started to care a lot about the world of cycling. He read autobiographies of famous cyclists of the past. His team's coach sent him to training camps around the world, where he met and befriended some of the best cyclists in the world, including some that he idolised.
At some point, the boy needed money for his training and sought work at a cycle rickshaw company. He took a break from training and spent several months ferrying people around the city. Having been a competitive cyclist, he was good at driving a cycle rickshaw. But after a while he found that being a cycle-rickshaw puller was not to his taste, and rejoined the world of competitive cycling. It was worth giving up the salary to do something he had come to love.
The boy, now a young man, regularly goes to competitions. Based on his performance, it seems unlikely he will ever break any records or become famous on the world stage, but he is certainly good enough to justify his place on local teams, and all evidence points to him being a good cycling coach in a few years, to continue to work on his own skills while mentoring others in his duchy.
This is what our young protagonist is doing when all of a sudden, a pharmaceutical company introduces a new type of pill that—among other things—drastically improves many users' cycling performance. The drug is widely available, but teams with better sponsorships can afford better versions for their athletes. At competitions now, it is unclear who is using the drug and who isn't. There is talk of banning the drug in the sport, but it would be hard to enforce. Some prominent cyclists are even of the opinion that the drug is good for cycling—ordinary people can now easily go on bike rides exceeding 100 miles, and are falling in love with the activity as a result. Cycle-rickshaw customers have come to expect the new speeds that drug-fortified pullers provide.
Many pro-drug cyclists advocate further development of the drug out of sheer fascination and curiosity. How the drug operates is largely a mystery, and they hope that understand its biological mechanisms will help push the human-bicycle interface to its absolute limit. Some well-known cyclists are receiving money from the pharmaceutical company. Some of the famous cyclists that our young man idolised in his developmental stage are leading the pro-drug charge.
The young man does not like to use the drug. Some of his friends use it but he does not like the effect it has on them. His coaches generally do not use the drug themselves; it has been a long time since their performance in races has really mattered. But they have tried it enough to know that it helps, and strongly recommend the young man start regularly incorporating the drug into his life. Some have even offered to help pay for his prescription. The man has declined, but worries he will not place well in upcoming competitions. The future of the sport is unclear.