Is Math a Canary in the AI Coal Mine? (with Bryna Kra)

View Full Episode Transcript

What happens when AI can produce a mathematical proof before any human understands it? And what does that mean for the rest of us?

Northwestern mathematician Bryna Kra, a former president of the American Mathematical Society, joins host Jess Love to explore how AI is changing mathematical discovery. Generative AI has already brought about extraordinary breakthroughs, but its use is also unsettling the culture of collaboration, upending how young mathematicians learn, and raising a deceptively difficult question: is solving a problem the same as understanding it?

Kra explains why some mathematicians are becoming wary of sharing unfinished ideas and how she cautiously makes space for AI in her own research. Despite the disruption she still enthusiastically encourages students to enter the field.

Life, Automated is a production of the Ryan Institute on Complexity, at Northwestern’s Kellogg School of Management. The host is Jess Love, senior director. Distributed by KQED. 

Deep theorems were scarce and difficult and so became an effective mechanism to identify deep thought. AI has broken this system. Bryna Kra, TerryTao.wordpress.com

How an AI math breakthrough ignited a controversy. Celina Zhao and Adrian Cho, Science.

This is a computer-generated transcript. While our team has reviewed it, there may be errors.

Jess Love [Voiceover]: This is Life, Automated, the show where we explore what it means to live, work, and make decisions in a world increasingly shaped by machines. Bryna Kra is a mathematician here at Northwestern University, where I work. She’s the real deal. She’s been elected to the prestigious National Academy of Sciences and spent a few years as president of the American Mathematical Society.

And she works on the kinds of math problems that are nothing like what you saw in high school. They’re weird problems, heady problems, like Nivat’s conjecture, which basically says, bear with me here, that if you have an infinite 2D grid, like an infinitely large floor with square tiles, and no matter where you look on that floor, the local patterns have surprisingly little variety, then the entire floor must repeat itself, at least in one direction.

If you’re not totally following why, don’t worry. These kinds of problems can take decades to solve. Someone will put forth a conjecture, and then a bunch of mathematicians or teams of mathematicians will work together to solve bits and pieces of the problem. And as of a few months ago, Bryna was one of the mathematicians who’d done just this.

Along with a coauthor, she’d put forth one of the major partial results for Nivat’s conjecture. And then, this September, everything changed. 

Bryna Kra: So starting a few weeks ago when there were new models released by the AI companies, those of us who’d worked on it began to receive proofs. And the proofs came from people who none of us knew. That’s great. You know, mathematicians tend to welcome new people into the field. 

Jess Love [Voiceover]: But there was something unsettling about these proofs. 

Bryna Kra: Well, they were very bad at tracing where the results came from. You know, what they were using from previous results. They didn’t highlight what was novel, which when you come to a breakthrough, you really want to write in the introduction, “This is what’s new.”

Jess Love: Right. It was like missing the big story. 

Bryna Kra: Yeah. It’s sort of putting everything on equal footing. And they seem to be correct. The problem is solved, but without any understanding of what was going into it.

Jess Love [Voiceover]: Bryna wanted to talk to the mathematicians behind the proof.

Bryna Kra: I wrote back to these people, and I asked, you know, “Would you like to meet and explain the proof?” There was always a reason why not. 

Jess Love [Voiceover]: And Bryna came to suspect that the mathematicians couldn’t explain the proof because they didn’t really understand it because, well, it sure looked like they had gotten AI to just do it all for them.

Bryna Kra: One of them disclosed full AI use. That’s fine. You know, they disclosed what they did. They said they were undergraduates who didn’t really know much math. 

Jess Love [Voiceover]: Now, Bryna thinks there is some value in knowing that the conjecture can, in fact, be proven, some satisfaction even. But mostly she seems to think of it as a missed opportunity to actually understand something new, gain a tool, a new technique, a new way into the next problem.

Because for a long time, this is what solving a proof meant. It meant you now understood something you didn’t before. And the Watts conjecture was hardly the biggest mathematical story of September. OpenAI announced that its models had solved one of the field’s crown jewels, the Navier-Stokes problem. On today’s episode, I speak with Bryna Kra about how AI is rocking the math world.

And even if you don’t care about math, maybe you still have PTSD from Algebra 2, Bryna argues you should still care about what is happening here, because today AI can put forth solutions to problems that have long stumped the field’s best and brightest, all without a single human ever reading or understanding that solution, which means while AI might be solving a bunch of problems, we humans might not be really learning very much, all while destroying the ecosystem that has long led to new discoveries.

So what does this mean for the future of math, for the humans who’ve devoted their lives to doing it, for the next generation of humans who are still learning it? And how does that apply to other intellectual pursuits? Because what happens in math doesn’t stay in math.

This is Life, Automated, a project from Kellogg’s Ryan Institute on Complexity, distributed by KQED. I started by asking Bryna about why OpenAI’s solving of the Navier-Stokes problem is such a big deal. 

Bryna Kra: So this problem is one of the millennium problems, which was posed in 2000 with a million dollars prize for each of them. There are seven millennium problems, only up till now, only one had been solved. That was by a human. 

Jess Love: Thanks for clarifying. 

Bryna Kra: And some time ago. And the others were open, and they really guide the community in what directions we’re going to do research. So it’s about more than just solving the problems. It’s about understanding the methods, understanding stopping places along the way, partial results. It’s a much bigger picture than just solving this problem. What happened in early September was that they announced the solution, and it was immediately filled with controversy. There were some mathematicians who were working on it using AI, fully disclosed that they were using AI to work on it.

And they had made significant progress, and there were a lot of questions immediately as to how the results from OpenAI were released, what access they had to what the mathematicians had been doing, was it independent, and how this was a complete change in the norms of the field. Mathematics typically is slow. We check each other’s work. We share results even along the way, partial, and this was just a very different way of doing math. 

Jess Love [Voiceover]: Part of what makes it different is that apparently OpenAI rushed to solve the problem after learning that two mathematicians, including one associated with Anthropic, its big corporate rival, were close to a breakthrough. This is according to a published account by NYU mathematician Tristan Buckmaster, one of the collaborators. OpenAI apparently decided they wanted to beat the team to the punch and diverted massive computing resources, including thousands of AI agents, to solving the problem. 

Bryna Kra: This is turning math into a competition in a way that it never has. Our community, the math community, has for ages been a very collaborative place, a place where we share our results before they are published. We put them on the archive, which is an open registry for this, where people can upload the results even before they’ve been reviewed by journals for publication by referees. We have an extremely rigorous refereeing policy, and this was just a completely different way to do it. 

Jess Love: Yeah, so it’s almost destabilizing for several different reasons. There’s the sense that, “oh my gosh, it actually solved this problem.” Like, but then there’s also the sense that it’s being used potentially to impress investors, to get them to pour more money into some of these companies. And so you’re taking these problems, and instead of letting them be guideposts to where mathematicians can orient their own efforts, it’s taking that and using it for something, like hijacking it for a completely different purpose. 

Bryna Kra: Correct. There’s a complete misalignment in the interests of the AI companies and the interests of the mathematical community. For the AI companies, this is a way to benchmark, to show their investors that they have done something hard. Right. Most people agree math is hard. 

Jess Love: Yes. 

Bryna Kra: So this is one way that they can show that they’ve really achieved something that the humans did not do, that they’ve gone beyond what humans were capable of. For the math community, our incentives are quite different. Of course, we wanna solve problems, and I think most mathematicians would agree it’s amazing that these problems have been solved, and we’re all delighted to see that these problems are solved. But it goes well beyond just solving problems. We wanna understand what the solutions are and what they tell us and what they give us as guideposts for future research and future directions.

Jess Love [Voiceover]: I wanted to know how Bryna would compare these discoveries in math to those in, say, drug development, where the goal seems like it should be to discover as much as possible as quickly as possible. 

Bryna Kra: So we love to discover. Mathematicians love to uncover new, deep results. This is great. But there isn’t the same pressure to hurry the discoveries out. Let’s say the solutions in Navier-Stokes had come out a month later, giving the humans time to finish writing up their portions, the humans involved in the discovery that led to the huge breakthroughs by OpenAI, to write up their part and carefully explain it to the community and finish their work versus coming out exactly as it did under pressure in a tweet, in a press release. This… Math is not done by tweet or press release. 

On the other hand, had OpenAI come up with a cure for cancer, the rush to announce it, to share it with the world, I think is completely understandable. This is just a very different culture of how math is done and the immediate importance of the result versus something like a drug company.

Jess Love [Voiceover]: This point about the immediate importance of a mathematical result versus, say, a cure for cancer is really worth spending a bit of time with, because as abstract and frankly out there as some of these problems sound infinite grids, they really can lead to these enormous breakthroughs; but the breakthroughs don’t just follow from the solution. They come from understanding that occurs along the way. Solutions need to be absorbed, mulled, and picked apart, and mulled again, and the normal pace of math has always allowed for that because what’s the rush? The practical uses from these breakthroughs when they do occur could be decades away. Techniques developed by the mathematician Joseph Fourier two hundred years ago help to power medical imaging today. Or here’s another example.

Bryna Kra: Well, you probably use your credit card online, all the time.

Jess Love: I do, too much. 

Bryna Kra: Behind that is deep number theory that was developed, some of it 100 years ago, some of it 50 years ago, and some of it is still being developed. Number theory is about the positive integers and their properties, like the numbers one, two, three, four, five, and so on, and what kind of properties they have. But inside that study of number theory, there were these tools that were developed that allow the encryption that allows you to use your credit card securely and most of the time without getting the number stolen, right? So, so that is a tool that was developed for the sake of understanding- 

 Jess Love: How fascinating … 

Bryna Kra: …And yet it gets used every day all the time now. So we don’t know all of the applications that are coming out of the foundational research that is happening in mathematics, and we cannot predict that. 

Jess Love [Voiceover]: Bryna’s fear is that reams of AI-generated proofs hitting the field all at once will lead to little actual understanding, fewer new tools and insights, all while discouraging human mathematicians from continuing their own investigations into these conjectures.

Bryna Kra: What it did is close off this area. What’s the incentive now for somebody to go through and understand that? Unless it’s gonna lead somewhere new. 

Jess Love [Voiceover]: Bryna thinks that simply having AI systems that can solve, and maybe in their own way understand the math behind a proof, isn’t adequate. We need humans to understand the math in order to keep directing the field, keep inquiring. 

And she does see some ways of using AI that could bring that about. In these cases, the AI follows the human’s lead, helps them follow their own intuition and curiosity to discover new things, rather than AI just doing everything and leaving humans to deal with the aftermath. 

Bryna Kra:So, I have started using it to explore new directions. I use it if there’s something I’m reading and I don’t understand. It’s a fantastic way to ask it, like, “Explain how you get from this statement to that statement.” It’s really good at that. Or, “Explain where this comes from, this idea.” 

And if you ask it to go search for an idea, because I like to understand where the ideas come from, it’s actually very good. It says, “Well, there seems to be something similar in this paper from 10 years ago, and another one from 20 years ago,” that I would probably not have found. I’ve started using it to code up things. And it can write computer programs and execute computer programs for me, and test thousands of examples in a way that would take me months, if not years to do.

This is not just saying, “Solve this problem.” You direct it. You say, “Well, now explore that direction, because that’s interesting.” And that’s where the human understanding of what’s going on in the math is super important, because many of us know things that cannot work for some reason, or are not the right direction to search, and it might be an intuition or it might be an actual math fact.

But we can direct it to look in these areas because we have spent so much time thinking about these problems. And so we can really guide the AI to help us look, here’s where you might be thinking about a technique, a tool, an approach, or a counterexample. 

Jess Love: So even as there’s more human-machine collaboration, do you wonder what will happen to human-human collaboration? And I just kept thinking that the second you present one of these partial results where you’re kinda like, “Hey, we got to this new point in the road,” does it then just trigger a bunch of AI-enabled mathematicians? Or maybe forget the mathematician, just AI-enabled AI to go ahead and solve the rest. Does it make it less likely to want to discuss the things that you are learning along the way?

Bryna Kra: Yeah, so this is exactly one of the issues that the field is grappling with now, that it’s a complete change in our culture. A year ago when I gave a talk at a conference, I would always pepper it with open questions and other directions of research because I would be delighted if someone in the audience would go and work on those and come back to me and say, “Hey, you asked this question, and I came up with this answer. Can we discuss?” Or if they write it on their own, either of these things would be delicious and wonderful. 

Jess Love: Right. Right. 

Bryna Kra: This is now complete change. If you go and you pose a problem as part of a natural open, you know, next direction that you are thinking about, and you don’t know the audience, and there’s somebody there who wants to just take it and put it in AI, of course, they’re welcome to do that. But if it’s a problem, for example, that I want to work on or that I want one of my students to work on, this is now a very dangerous thing to do. 

Jess Love: Yeah. 

Bryna Kra: So it has destroyed the trust that the community has in other people because there are people there who are just wanting to take it and put all the problems into AI and keep moving forward.

That cultural norm of openness and sharing has really been destroyed in an instant, and I really hope we find a way to go back to that because it is one of the things that made mathematics such an amazing place to be. The community is a great bunch of people that shared and worked together in the collaborative environment, and if we destroy that, then I think we will really have destroyed the role of human mathematicians.

On the other hand, there are many of us fighting hard to keep that. This can be local. We have a local seminar, working group, which has about 10 to 15 people each week, and we have all agreed that nothing goes into AI… unless the speaker agrees. So nobody will be doing that in our local group, and we all trust each other. 

I think people have to come with those types of agreements, with their collaborators. There are different types of collaborations, and whether or not you include AI, put the problems in or not, that is something every collaborator needs to discuss.

Jess Love [Voiceover]: Different agreements, tortured discussions about when and how AI will be used and disclosed. People genuinely afraid to present their results at conferences because analyses that would have taken months can be done in hours, making the likelihood of getting scooped that much higher. This sounds very familiar because I’m seeing this play out in so many academic disciplines.

Working at a university, I have a front row seat to the ways different fields, different subfields, different individuals are approaching this moment. I walk through campus and can feel the undercurrent of shock and wariness and excitement and anxiety. I can hear an entire lecture hall hush as everyone reckons with the magnitude of the changes that are coming, changes that may already be here now that the work of knowledge production no longer happens at human scale.

And it’s not just academia. This is happening wherever ideas are generated and codified and shared with others. AI is coding up complex software and designing sophisticated marketing campaigns and writing prize-winning literature. It’s just that it’s all happening so clearly in mathematics with like actual proofs attached.

In a recent blog post, Bryna described her field as, “a canary in the coal mine.” We discuss after the break.

Jess Love: So one of the things that really caught my attention in your blog post about recent events is that you mentioned that AI’s abilities in mathematics are a bit of a “canary in the coal mine” for a lot of different disciplines. Can you say more about what you mean? 

Bryna Kra: Right. So the issues that mathematics is dealing with right now are not going to stay just a problem for mathematicians. It is already happening in theoretical computer science, which, you know, is part of math in some way, another close field. But I do not believe that it will stay there. I think it will come for all intellectual fields. It’s coming for law already. I think people understand how much the AI can actually absorb case law and produce what read and look like briefs. And so I see no reason why it won’t develop for other ones as it gets better. Math was first in some way because our training data was so excellent and because it was such a good proving ground for the companies to benchmark of what they’re doing.

But, I think it’s naive to think that it will stop with math and theoretical computer science and law. Fields will need to decide what their values are, which ones they want to keep with the changes, it can’t be all because they won’t all get kept, and then figure out the ways to implement keeping those.

So what steps does the field take to make sure that their values are still there at the end of this AI revolution? 

Jess Love: Right, which might involve really at the core making sure that the incentives of the broader field are in some way aligned with the incentives of individuals. 

Bryna Kra: Correct. Right now, we have a complete misalignment on many levels. We have a misalignment between what the AI companies want from mathematics and what mathematicians want. We have a misalignment between the ways we use mathematical results to judge people and what can be produced now, making it very hard to judge people. So the math community is grappling with these issues.

Right now, there’s recently a report that came out of a group, there was a conference at Harvard on the future of the PhD in mathematics. Mathematicians are deeply concerned about how we will train the next generation because we really do believe that this is a time, if anything, for more mathematical input and more mathematical insight, the human type, not just the AI type.

And if we have cut ourselves off at the knees and don’t have a way to train the next generation, how will we have that? So there’s great concern on this. 

Jess Love: Right, because you might be, instead of relying on a PhD student or two to assist you with your research, it can be very tempting to go to AI. 

Bryna Kra: Well, no, I think it’s a different issue.

Jess Love: Okay. 

Bryna Kra: In mathematics, it’s quite different than the lab sciences. PhD students typically don’t assist us with our research, but rather it’s the opposite. We kinda give them, at the beginning, things that we sort of know how to do, maybe not the details- certainly not all the details, but that we see the path forward.

There’s a lot of work to do, and they gotta go do the work themselves. But we kinda see that. That’s how we start training them. And then after they’ve worked on such a problem, we help them explore new directions so they find their own way- to find their own problem that they can do, and we still help them.

Jess Love: And so how does… Can you just flesh out how AI disrupts that? 

Bryna Kra: The AI disrupts this completely. 

Jess Love: Yeah, I’d love to hear how. 

Bryna Kra: Completely, because that first problem that we sort of see the path, and it’s missing some hard work. Every problem I can come up with like that, if I were to put it into the AI, it would just give me the solution.

There’s a big difference between coming up with it on your own, and therefore having to do the work, the pen and paper work, and actually reading somebody else’s. And we are trying to exactly train them to do that, the difference between reading and understanding something that you get from elsewhere, and being able to develop it yourself and writing it up.

Jess Love: I see. So it’s almost like the types of problems that a PhD student would even be able, like a new PhD student would be able to handle, are the exact types of problems that AI can one-shot. 

Bryna Kra: Correct. So I think we have to change our training programs. We have to train the students more to think about how do we go outside the box? What are the right questions to be asking about this, rather than just getting them from their advisor, “Here’s a good question to ask.” Because it’s, it- that’s a skill that we have not had to develop as early in a student’s training. 

Jess Love: Would you still advise a student who is very interested in mathematics to go into the field?

Bryna Kra: So to me, somebody who’s interested in mathematics, the answer is yes. Absolutely, resoundingly yes. I spoke to some students and I said to them, “You know, I’d like to think about it this way. The first, few lines of, ‘A Tale of Two Cities’ – 

Jess Love: “It was the best of times, it was the worst of times.” 

Bryna Kra: Right. And so in some sense, in mathematics, it was, this is the best of times. We’re going to see huge conjectures fall. We’re going to see new understanding of areas that we couldn’t have possibly hoped to understand 10 years ago. This is amazing, and for a student to be at the forefront of that and really in that, great. The part that is the worst of times is that we need to completely adapt and do it really quickly of how we’re doing this.

So it’s gonna be a turbulent time, so that’s gonna be complicated. But I think the community is working hard to do that, to ensure that there are jobs for the next generation, to ensure that we have a way to train the next generation, and to meet the need because I think we need more mathematicians right now.

We don’t want AI systems that we can’t understand at all on any level. The math that developed these systems now needs to be turned towards developing safety for these systems. There are new uses of math that are coming, and so yeah, in some sense it is the best of times for mathematics.

Jess Love [Voiceover]: It was the best of times, it was the worst of times. It was the age of falling conjectures. It was the age of new conjectures rising. It was the epoch of dauntingly capable AI systems transforming math. It was the epoch of new kinds of math desperately needed to make these systems safe. It was the season of human irrelevance.

It was the season of humans reasserting themselves, reminding everyone who this is all supposed to be for. I feel this all so much. Maybe you do too, because right now it seems true for a lot of fields. Take my own world of science communication. I’ve spent 15 years trying to become excellent at my work, making complex ideas accessible and helping others do the same, and the truth is, AI can do this really well now at superhuman scale, at least the technical parts of it.

I got into this work because I felt it was important, so on some level, an on-demand complexity demystifying machine that everyone can use is amazing. It is the best of times. But on another level, it feels terrible. Will another generation of science writers even bother learning this craft? Will they be able to connect human to human to the people they’re doing this for? What is the role for humans in these kinds of intellectual pursuits? 

Jess Love: So we talked about the difference earlier between discoveries in math versus, say, cancer research, drugs development. I wanna bring a different discipline into the mix, art. With art, the process is the point. So fully AI-generated art is certainly already having commercial uses, but it’s hard to imagine it ever fully replacing human-generated art because how art is done, by whom, when, why, all of these things, like, that’s the ballgame with art.

That’s why we still love oil paintings even though we have cameras. That’s why my eight-year-old will draw a squiggle and I will love it, but it will have zero market value, and then Picasso will draw a squiggle and it has… You know, I might also love it, but it will also have market value. So there seems to be an interesting continuum, a different one, between disciplines where the outcome is really what matters, and disciplines where the process is what matters.

Bryna Kra: What do you think of this way of thinking about it, and how would math fit into that? So I’m an optimist about the future of mathematics. I believe that the human role will continue, and it will remain key. And, one can do exactly like AI-generated art. There’s a lot of AI-generated math out there. And it’s overwhelming the community because who’s gonna read it all? 

Jess Love: Yeah. 

Bryna Kra: And who’s gonna check it all, and who’s gonna use it all? But it still seems to me that the role for human understanding and human use of this mathematics is extremely important for many of the same reasons.

See, we get to decide what are the interesting directions for research. We get to decide what are the interesting questions that we want answered. I don’t think anyone knows now if the AI companies will reach superhuman intelligence in mathematics. Maybe they will, but I find it hard to believe that even with superhuman intelligence of problem-solving, that the AIs will develop the same interest of what we find interesting, and what are the right directions to search. 

In this sense, it’s very different than, something like the game of ‘Go’, which an AI is just simply better at- and developed new – 

Jess Love: Chess.

Bryna Kra:  ‘Go’, ‘Chess’. Coming up with the next direction, what’s the next interesting thing, to me seems unbounded in a way that is quite different from those games. And so for this reason, I really believe that there will be a human role in the future. 

Questions, asking the right questions, and going in the right directions may become a new proxy for the field for judging things. I don’t know. But it’s certainly much more important than it has been in the past.

There were always the people who were great at posing questions. Erdős is a famous mathematician who had thousands of publications and thousands of co-authors because he was so good at posing questions all over the place, and his influence is seen in so many parts of mathematics now.

And so many applications outside of math. You know, questions he posed relate to sphere packing, and sphere packing relates to how your cell phone works in a crowded room, for example. And his ability to pose questions was not the same as his ability maybe to build theories and prove things.

So I think this role for humans continues as to posing questions and building theories and finding what we think is interesting. To me, that doesn’t change with AI. 

Jess Love: Well, thank you so much for chatting with me today. 

Bryna Kra: Thank you. It was really a delight to talk with you.

Jess Love [Voiceover]: For fun, here’s our credits in the form of a mathematical proof. Let Jess Love be your host. Given Life, Automated is a project of the Ryan Institute on Complexity at the Kellogg School of Management at Northwestern University. Assume distribution by KQED. This assumption is justified by the fact that we are distributed by KQED.

By the combined efforts of Jesse Dukes and Tom Dixon, the show was produced. Add music by Steven Jackson and recording assistance from Will Feeney and George Christensen. To extend our results to a wider audience, apply the marketing efforts of Ananya Mallapragada and Lauren Ko. We acknowledge Laura Pavin and Rob Mitchum, whose contributions merit special thanks, though the full demonstration is omitted here for reasons of space.

Therefore, this episode exists. Whether you understood it is left as an exercise for the listener.