Episode 9 artwork

Episode 9 · November 18, 2026 · Transcript

How Big A Deal Is Chinese Chip Smuggling Anyway?

After Jensen Huang’s fall at Nvidia, chip smuggling has gone from a niche policy worry to the center of a Washington fight. To make sense of it all, we’re joined by Ben Buchanan, the first White House Special Advisor for AI under President Biden and the architect of the original chip export controls.

This transcript was generated by AI and may contain errors. Machine Gods is made to be heard, so please check the audio before quoting.

Kevin Roose: What's the latest fallout from Jensen Huang's sudden downfall at Nvidia?

Casey Newton: How does President Biden's top AI advisor think Democrats should regulate AI?

Roose: How big a problem is Chinese chip smuggling anyway?

Newton: And did Breachvember nearly derail my Disney adventure?

Roose: I'm Kevin Roose.

Newton: I'm Casey Newton.

Roose: And from San Francisco, in partnership with NPR, this is Machine Gods.

Newton: The official podcast of the singularity starts now.

Roose: [Singing] Machine Gods. Casey, today we are here to talk about the continuing fallout from the Nvidia scandal and some real news coming out of Washington about our chip export control policy in a segment that I'm calling Panic! At the ChipCo.

Newton: Very good, Kevin. You know, it's only been a few days since we last recorded. It is now Monday morning, and man, there was a full weekend of developments across everything related to Nvidia.

Roose: There sure was. So just as we were putting our last episode to bed—

Newton: And speaking of going to bed.

Roose: [Laughs] Yes. Reuters reported a third Jensen Huang affair with another Nvidia employee. The board is still trying to figure out external and internal candidates for a permanent CEO. But probably the biggest Nvidia news is that there's a growing campaign to bring Jensen Huang back. Casey, have you seen this?

Newton: I have, and it seems like the sort of thing that would be organized by a new PAC called Oligarchs Against Accountability.

Roose: [Laughs] Well, in fact, this was organized by some Silicon Valley executives, as well as former and current Nvidia employees and shareholders. There are currently more than 180,000 signatures on an open letter demanding that Nvidia reinstate Jensen Huang, and the FT reported that Huang himself has talked with friends and confidantes about the possibility of returning.

Newton: Yes, although I also just wanna say I'm gonna need to see an audit of those 180,000 signatures. The idea that there is some massive groundswell of people trying to bring Jensen Huang— unless these people hold Nvidia stock and are truly worried, I really don't get it.

Roose: But Casey, you also predicted that Jensen Huang would be back. So this is, in some ways, the world moving toward your prediction.

Newton: Here's what I know. When a venture capitalist sees a very rich person experience a consequence for their actions, that is terrifying to them. Because what if it happened to them, too, Kevin? And so any moves toward accountability need to be nipped in the bud, lest we live in some sort of world where the rule of law applies to everyone equally.

Roose: Banish the thought. In the meantime, President Trump continues to defend Jensen Huang. Before boarding a flight to Asia on Monday, President Trump told reporters he wouldn't mind seeing Huang get reinstated. And we just got some reporting from The Wall Street Journal that Nvidia is going to be designated an AI national champion by the Trump administration, along with a bunch of other companies, including OpenAI and Anthropic.

Newton: Setting aside for the moment the fact that my fiancé works there, could there be anything more ludicrous than a company being named both a supply chain risk and a national champion simultaneously by the same government? Like, what are we doing here?

Roose: They contain multitudes. Within every company is a national champion and a supply chain risk. [laughs] Meanwhile, there is an ongoing knife fight within the Trump administration over export controls that block the sale of Nvidia chips to China and other places. In particular, Treasury Secretary Scott Bessent and Commerce Secretary Howard Lutnick have been kind of going at each other, both in public and in the background. Treasury's position is that these— that Megaspeed, this company run by Alice Huang, suspected of funneling American chips to China, they wanted them blacklisted, but Commerce had held that up. And so over the weekend, there were reports that Trump advisors discussed ousting Howard Lutnick from Commerce over this issue, and others. But all of this has led to an actual new rule on Monday, a rarity in this day and age.

Newton: Paging Dua Lipa.

Roose: [Laughs] I don't get it.

Newton: New rules?

Roose: Oh, yeah. [laughs]

Newton: I— Every producer in unison: new rules.

Roose: [Laughs] Sorry.

Newton: It's okay.

Roose: It's that old one.

Newton: We'll show a graphic when we say it on the podcast.

Roose: [Laughs] Anyway, all of this has led to a new rule from the U.S. Commerce Department on Monday morning. They are going to finally close the loophole that allows China to legally access advanced AI chips from Nvidia and other American companies by renting compute from third parties like Malaysia and Singapore.

Newton: Yeah. You know like when you're working at an office job and you're sort of screwing around, not doing whatever you're supposed to do, and your boss sort of comes, and they're about to look over your shoulder, and you quickly alt-tab to like a spreadsheet to make it look like you're doing what you're supposed to be doing? That is where the Commerce Department is now in this— They have alt-tabbed over to the loophole, and they're like, "Yeah, we're taking a look at this."

Roose: Yeah.

Newton: Yes.

Roose: And I'll just say, like, this has felt like a real shift in vibe among the policy world when it comes to AI very suddenly. I think some news events kind of catalyze just a chain reaction that you wouldn't have maybe predicted or anticipated, but I think in some sense, like, Jensen Huang resigning suddenly from Nvidia under a cloud of affairs and suspected espionage and whatnot, has really, like, opened the floodgates on AI policy and specifically on chip policy because there was this fear of Jensen Huang and Nvidia in Washington. They were fierce lobbyists on their issues. They had lots of power and lots of ins with the Trump administration. And so people really tiptoed lightly around these export controls, and now it kind of feels like open season. What do you make?

Newton: Oh, absolutely. There were always China hawks in the administration who wanted to see Trump take a much stronger position against these groups. But because Nvidia was so powerful within the administra— I mean, you know, Jensen and Trump were calling each other on a pretty regular basis, and Jensen got a lot of what he wanted from Trump just by calling him and maybe making some strategic donations. So with him out of the picture, the hawks are ascendant.

Roose: Well, Casey, I think it's a really good time to start talking about another piece of this AI puzzle, which is chips. This is an area where Nvidia obviously had a huge role, and now that Jensen Huang is out, at least temporarily, it feels like the landscape is shifting very quickly.

Newton: Absolutely. I think there is a dawning awareness among more and more people in Washington that to some great extent, chips are destiny, right? If you are determined to build a future for your country in which AI is playing a starring role, you need to get access not just to any chips, but to cutting-edge chips, which is why there has been such a booming market in smuggling them, and that market may be about to be closed off. But, you know, if this is something that seems a little bit wonky or in the weeds, I think it's just important to remember, chips are the critical piece of infrastructure that is necessary to build these leading models and to deploy these models. And so while it's not quite as simple as the country with the most chips wins, it's not entirely wrong either.

Roose: Right, and I think a lot of the other harms that people worry about from AI, like job loss or the kind of use of AI to dehumanize or sort of addict people, like, all of that is downstream from chips in some way. If your country doesn't have chips, you're not getting advanced AI capabilities, no matter what else you're doing right.

Newton: Right.

Roose: And so this issue of who has chips and how many and does China have them, it is central to not just understanding the response of the US government right now, but also the kind of larger dynamics of the race.

Newton: Yes.

Roose: Luckily, we have the perfect guest to talk about this today. Our guest is Ben Buchanan. Ben Buchanan was the first White House special advisor for AI under President Biden. He was the architect of the original chip controls that went into effect in 2022. He now teaches at Johns Hopkins, and he has a new book out with Tantum Collins called “The Bitter Struggle,” came out last week, and it's basically a policy memoir about how these chip controls and the other things that the Biden administration did came together and why he thinks it's so urgent for the government to continue acting to prevent the sale of advanced AI chips to China.

Newton: Let's bring him in. Ben Buchanan, welcome to Machine Gods.

Ben Buchanan: Thank you for having me. This is a striking set. I feel the AGI already, guys.

Newton: [Laughs]

Buchanan: This is— I see where all your money has gone. This is great. [laughs]

Roose: This has been described to us recently as an AGI man cave, which is— I don't know how I feel about that.

Newton: All genders are welcome in this cave.

Buchanan: Do you have to call it a super space intelligence man cave now? [laughs]

Newton: Yes, exactly. [laughs]

Roose: Yes. [laughs] We have to talk about Jensen Huang. You have some parts in your book about Jensen Huang, and actually you talk about MegaSpeed in your book, and you describe the relationship between Jensen Huang and Alice Huang, the suspected Chinese spy, as close. What did you know?

Buchanan: As best I can recall, I never heard the name MegaSpeed and certainly did not hear the name Alice Huang while I was in the government, but we put them in the book, MegaSpeed and Alice, because it felt like they were an avatar for a class of risk we were very worried about and which now, of course, has gone completely mainstream, which is smuggling of chips through China, to China through third countries, and then remote access by China of chips in data centers in countries like Malaysia and Singapore. So it felt like a great avatar, or example of this risk, and boy, has it gone mainstream.

Newton: So smuggling is a byproduct of there being export controls on chips. You are a big advocate for those export controls. Tell us why. Like, make the case. Why was it important to you and others within the Biden administration that we place some sort of restrictions on the kinds of chips that China could get access to?

Buchanan: I was a big advocate, and there was a small group of us that basically came to the White House to put this policy in place. We entered the White House in 2021 with a couple convictions. The first was that AI would matter a lot for national security, which now seems obvious, but was a big debate at the time. And the second was that computing power was the fulcrum of AI progress. The great news was that, or the great reality, was that America and its democratic allies and partners had a dominant position in producing computing power. America was doing something like 100 to 1, even to this day, it's doing something like 100-to-1 advantage over China in producing the chips. This is not for a lack of trying on China's part. And as far back as 2014, when America was off fighting ISIS, Xi Jinping and the State Council of China recognized how important computing power was. They began an effort called the Big Fund, upwards of $200 billion, to try to indigenize chip production. They largely failed. But we felt like this was so important for national security, and this frankly was the best point of American leverage in this competition to use AI for geopolitical advantage.

Newton: I mean, if Jensen Huang were here, you know, I can imagine what he would say in response to that. And first of all, it would be incredible to have him here. He probably would've just wrapped up a date or two with one of his subordinates. But— If he were here, Kevin, I think he would be saying, "Look, if you're going to implement these restrictions, all you're doing is accelerating China's development of this alternate ecosystem, and you're effectively going to turn them into an AI superpower. Whereas, if we keep them on our chips, maybe a few generations behind, we can just kind of slow walk this thing, and America will maintain its advantage over China." How did you feel about that argument?

Buchanan: First of all, Jensen wasn't always saying a few generations behind. And Jensen has said, you know, he wanted to sell Blackwells into China and so forth. So I think it— I'm not sure he was saying that. I think he actually, to steelman his position, he would probably say exactly what you said about this lets the Chinese catch up or gives them incentive to, and also you're hurting Nvidia because we're not getting these profits, and I disagree on both counts. The place I disagree on indigenization is, as I said, China was pedal to the metal on indigenizing in as far back as 2014, tens, hundreds of billions of dollars on this, largely failed.

Roose: Indigenizing, of course, is what you do when you try to bring your production of chips or something else into your country as opposed to letting someone else build it for you.

Buchanan: That's right. So chip making's really hard to do. China is not shy about trying. I don't think our efforts really accelerated China much at all, especially when you consider the fact we cut them off the chip-making equipment. The second thing is, did we slow Nvidia down? At the time we did the chip controls, October 2022, Nvidia's stock was something like $350 billion. Before this scandal, they were at 5 trillion. They've had a rough week here. Like, let's be honest about that. They're still at something like 4 trillion. So Nvidia has, you know, gone from an important company to one of the most valuable companies in the world under a regime in which they couldn't sell chips to China. And our view is the supply of chips is basically fixed. There's only so many chips coming off the line at TSMC, and it's a national security imperative for those chips to go to the United States and democratic buyers and not to China, and Nvidia's gonna make a lot of money. And Nvidia's profit margin or gross margin during this period was probably 70%. So we thought Nvidia would be great, would do great. We told Jensen that. We told Nvidia that, and for the most part, they have.

Roose: There's an interesting passage in your book where you're talking about this moment where you and your team are trying to make the case to the kind of the rest of the Biden administration and the rest of the federal government for these export controls back in 2021, 2022. And you write that basically you had to kind of lie about why you wanted these export controls. Not lie, but sort of misdirect in a sense because they— You and your team were very AGI-pilled, very persuaded that this was happening, that you wanted to get on top of it. But the policymaking apparatus in D.C. wasn't that AGI-pilled, and so there were all these sort of reasons about, "Oh, we wanna keep China from getting these military weapons," and you write that your team had sort of a different view. Do you feel like that is still going on, where on one hand we have these very AGI-pilled people inside the government talking about AI policy through that lens, and a whole different set who just view this as a normal technology looking at it through that lens?

Buchanan: I do think there was a debate then, and to a lesser degree is a debate now about how AGI-pilled should we be. I suspect the Trump administration is becoming more and more AGI-pilled because reality is just really obvious at this point in a way that it was kind of speculative in 2021, 2022. I don't know where he is now, but certainly my successor in the job of presidential AI advisor was David Sacks, and his posture, I think, in 2025 was, "This is a normal technology. It's plateauing," and others have said similar things in the Trump orbit. Dean Ball, who is someone I respect, who was in the first year of the Trump administration, wrote an essay in 2026 in which he said essentially, "My colleagues thought this technology was gonna plateau." He's now at OpenAI. He's very AGI-pilled. But I do think that was a really big difference between our policy and their policy. I think they're shifting. It's worth noting, you know, Jensen has this really interesting quote in his biography where he says something towards the end of the book where he's like, "You know, we're not living in a science fiction story." And I think at least implicitly, I think he's talking about recursive self-improvement, though it's not explicit in the book. And I do think there's a sense in which, in some sense, right or wrong, we came to the government with the belief that this technology was really gonna change the world, and not everyone else was there. And even today, I think in the Trump orbit, not everyone else is there.

Roose: Got it. We recently got the news that the Trump administration is going to close the loophole that allowed Chinese companies to rent compute through these remote data centers in places like Malaysia and Singapore. I was frankly shocked that this was allowed in the first place when I learned that this was a loophole in existence. What did you make of them closing that loophole, and do you think it was too late?

Buchanan: I think the rule that they put in place, while overdue and good, has some loopholes. It doesn't restrict Chinese access to American compute, so China can rent compute from AWS, just not from Megaspeed. So there's more to do, but I do think we are at an inflection point in which there is a very strong bipartisan consensus. AI matters for national security. Computing power matters for AI. Americans always have an advantage. We should use it.

Roose: Wait, China can rent compute from AWS?

Buchanan: My understanding, and again, I'm not a lawyer here, but my understanding of the current rule is it does not restrict China's access to US-based data centers.

Roose: That seems like a mistake. [laughs] But what do I know? That's why I'm not running chip policy.

Buchanan: There are authorities that could be used, and I would encourage Trump administration to use them. And there's— It's worth saying there's two bills in the Congress that are on this point, which have very strong MAGA senators supporting them, Tom Cotton, Jim Banks, the MATCH Act, the OVERWATCH Act that cuts off China's access to chip-making equipment and to chips.

Roose: There's another argument that gets made though, Ben, which is that maybe these very restrictive controls that you're advocating for wind up provoking a conflict with China, right? That if China feels like it cannot get access to state-of-the-art technology, that is going to make it more hostile, more aggressive to us, you know, maybe get us into some sort of like open, protracted conflict. What do you think is like the rational response from China to these sorts of things, and is there a way that you want to try to route some of the potential benefits from AI to them so that maybe they feel less threatened?

Buchanan: Yeah, this is a really important question and one we spent a lot of time on, and let's take it in parts. The first version of the question is, does this make a Taiwan invasion more likely given all these chips are coming from Taiwan? The answer seems to be no, and in fact, it's not clear how legible this whole AI conversation is to senior decision-makers in China, and the decision to invade Taiwan is a legacy decision for Xi Jinping. It depends a lot on do they think peaceful reunification is possible? Does he trust his military? He's purged huge numbers of commanders of his military. This nerd stuff is, you know, fun and games and really important if you're on a show called Machine Gods, but this is— Geopolitics is a whole nother game. And—

Roose: Did you just come into our home and call us nerds, Ben? [laughs]

Buchanan: Kevin, where I come from, this is a point of pride. But the answer is yes, I sure did.

Roose: Okay, nerds complimentary in brackets.

Buchanan: Nerds.

Roose: Okay.

Buchanan: Do you— Is it ever not complimentary? [laughs] Really? Where would—

Roose: Go on. Sorry.

Buchanan: Where would it not be complimentary? So yeah, I think that's the first question, the Taiwan piece, and I think the answer there is, like, actually there's much more going on. And the second question is, would there be some retaliation from China economically? It's worth noting in the Biden administration, for all the controls we did, there was not much. And in part because we were messaging to the Chinese, "We are not trying to launch an economic broadside against your economy. We've actually scoped these controls very carefully," too carefully if you ask me, creating some loopholes that Nvidia worked around. But the only thing China really did in response was some noise about cybersecurity investigations and then controlled two critical minerals that were not particularly hard to have alternative supply chains for. Contra that with the tariffs, where China hit back very hard. So I do think there— We were trying to put the plan in a sweet spot of a lot of leverage strategically for the United States without spiraling into tit for tat or certainly war. It is worth saying, we felt like it was totally compatible to have this chip control policy and also say to the Chinese, "We are serious about engaging with you on AI safety and the things that matter to both nations here." And there's a line in the president's executive order from October 2023 that when you translate from the legalese says something like, "America will engage on AI safety, not just with its allies and partners, but also with its competitors." And President Biden took that line to President Xi three weeks later, said the leader-to-leader version of, "I'm serious about this," and that prompted the dialogue in Geneva that I was a part of with the Chinese that said, "We wanna talk about this." The Chinese did not take it seriously at all. We pushed. We eventually got an agreement to do the most obvious thing, which is don't put AI in your nuclear weapons systems. But this was something that we felt like was a part of the strategy engaging with China to avoid the race to the bottom you're talking about.

Roose: So I wanna hear more about this because it has been confusing to me to watch some of the decisions that Chinese leaders make, like for example, saying, like, we're gonna sort of double down on open source forever, even maybe in a world where these models become very good at, you know, mounting cyberattacks and doing other things that just, you know, seem like an authoritarian state would really not want to see its people doing. So tell us a little bit more about this meeting and what the attitude was. Was it just that, you know, whatever the American story was about this sort of coming AGI was just a fairy tale, or where were they?

Buchanan: So this is May 2024 in Geneva. It's worth saying it's more than two years ago, and I think—

Roose: Yeah, so that's like—

Buchanan: The Chinese posture may have changed.

Roose: That's like, what, like GPT-4, maybe, something?

Buchanan: I believe GPT-4o came out the day of the meeting.

Roose: [Laughs] Okay.

Buchanan: And I remember thinking, "Oh my God, the Chinese are gonna think this is, like, a plot that, you know, America has control over its tech companies, and we're doing this to show them up," and so forth. I have no idea if they thought that, but I— certainly the thought crossed my mind. We tried to lay out to them what we thought the shared risks were, things like cyber, things like bio, and their posture was just very much like, "We don't think there's risks here, and we think this discussion of risk is slowing us down," and so forth. I do think there's some evidence that certainly there's an AI safety community that has emerged in China since then. The decision was clearly made at the Chinese— in the Chinese system in advance of that meeting to subordinate the technical folks to the political folks, and that's why we got the posture we got. But I don't wanna throw cold water on folks who have done good work trying to promote shared dialogues and track two dialogues. Like, all of that is good, and all of that is the precursor to what I hope will be more robust discussions between the two governments now that the risks are even more obvious in the way that was probably not the case for them in 2024.

Roose: What's your current view of how close China is to frontier US AI capabilities, and why? Because there are some people who Casey and I talk to who say, "Well, actually they're, you know, they're on our heels. They're maybe a month or two behind us," and so in that world, it really matters that you have all the tools at your disposal to delay them, such as export controls. There's another view that says actually they only appear to be close to us because the Chinese labs are distilling from American models, and they're using smuggled American chips. And so if you cut off the distillation and you cut off the smuggling, then you end up in a world where we're back to maybe being a year or two ahead of China. Which of those is—

Buchanan: I'm much closer to the second view, and I think if you look at the stats, especially when you look at the benchmarks that are not public, so, like, the private, held-out benchmarks, I think they're closer to eight months behind on some of these. You can— There's all sorts of different benchmarks, you know.

Roose: So you think a lot of it is distillation and smuggling?

Buchanan: Even with distillation, I think they're eight months behind.

Roose: Okay.

Buchanan: It's worth noting with one or two minor exceptions that essentially every single frontier, quote, frontier Chinese system is trained on American chips. So it's smuggling, it's remote access, it's the chips Trump agreed to sell them. So it is essentially a policy choice for the United States to have the Chinese as close as they are. We can debate is it four months or six months or eight months, but the argument you'll hear from me is by cutting China off access to these chips, these remote data centers and so forth, we can grow that lead even more from where it is now. It's worth remembering the demands for AI training continue to grow exponentially. So our theory was the faster you cut them off, the more they've got to climb an exponential mountain to catch up. And GPT-3 was trained on a couple thousand chips, if I remember correctly. Now we're talking about data centers with pushing a million chips. The chips are exponentially better than they were in 2020. China is facing a very, very, very big compute mountain to climb here. And in a world in which the country's only producing 200,000 chips, even if that's grows to a million chips somehow, that's just not enough to supply a whole country. So the American position is dominant in computing power.

Roose: Well, particularly, you know, you talk about they're eight months behind. Well, you know, according to the frontier labs, eight months from now we could have something very close to AGI, right?

Buchanan: Yeah.

Roose: Like, where they're gonna sort of be, like, in full-blown recursive self-improvement. So—

Buchanan: Put aside whether that's a good thing, and, like, you know, the AI safety part of me doesn't— is not a pure accelerationist. But one of the points we make in the book is that for a lot of human history, technological advantages, even measured in years, didn't matter that much. The canonical cases, you go back 10,000 years and technology was progressing so slowly someone could be 100 years ahead technologically, and it didn't really matter. Come to the modern age and the difference between, you know, cyber operations in 2002 and 2026 is really big. So I think there's a dynamic over history in which the technology gaps matters more and more, even as the windows get shorter, and RSI could accelerate that. So we'll see.

Roose: We just learned this morning from some reporting in The Wall Street Journal that apparently the Trump administration is planning to designate Nvidia, OpenAI, and Anthropic national champions. What did you make of that, and do you think that's a good idea?

Buchanan: I guess I wanna know what the policy change would be, but certainly—

Roose: I think it means a good—

Buchanan: Democrats are always accused of picking winners and losers in the market. [laughs]

Roose: [Laughs]

Buchanan: And I guess national champion means you're a winner. I'm not sure. [laughs]

Roose: I think it gets you some preferential financing. Maybe the government takes an equity stake. I see it as kind of a soft nationalization, which is something that I've predicted would kind of happen as AI got more powerful. Do you think the end state of this is the US government owning major stakes in at least one, maybe multiple frontier AI companies?

Buchanan: I'm not pushing for that, and I actually worry that confounds a lot of the regulatory factors if the US government owns pieces of companies it's trying to regulate and so forth. I do think AI is the first revolutionary technology, probably the last 100 or so years, that really comes from the private sector. Think about nukes, space, early days of the internet, early days of the microprocessor, early days of jet aviation. DoD funded all of that, and it was the universities that did the work, private sector did the work, but DoD was funding it. AI actually is that way, too, if you go back to the 1980s. But the deep learning paradigm chronicled in, you know, “The AGI Chronicles,” the sort of the story of the last 10 or 12 years of this huge scale, massive investments, hundreds of billions of dollars in investment this year from American companies, that's all the private sector. And what that means is that the government doesn't understand this technology or didn't, doesn't have any direct capacity to shape it. The companies are growing up in an environment that is not particularly alert to counterintelligence threats, maybe as the Jensen case shows. But every single policy choice that I felt like I was making as the president's special advisor for AI is about the relationship between the public and private sector. How do we regulate this technology? Who can sell what to whom? How do we bring this technology into DoD? All of that. So I think this national champion thing, whatever it actually ends up meaning, is probably the latest example of the two sides just negotiating this relationship that neither side has a lot of experience trying to negotiate.

Roose: You'd think that being named a national champion would mean that maybe you were immune from being designated a supply chain risk.

Buchanan: [Laughs]

Roose: But I don't know that that's gonna be the case.

Buchanan: "¿Por qué no los dos?" is what they say to that. [laughs]

Roose: All right, we're gonna take a quick break. When we come back, we'll have more questions with Ben Buchanan. All right, we are back with Ben Buchanan. Ben, let's talk about what has been going on in Washington outside of this narrow chip policy debate. We had a—

Buchanan: Did you just call my policy narrow, Kevin? [laughs]

Roose: [Laughs] Listen, you called us nerds.

Buchanan: [Laughs]

Roose: I figure we're one for one now. Democrats are going to be taking back both houses of Congress in January. They— a lot of them ran on AI and opposition to data centers. You ran AI policy the last time Democrats held the White House, and I guess I'm curious what you think the right path forward is. How should these Democrats coming into Congress look to regulate AI? What are the relevant issues? Where can they actually make progress?

Buchanan: So the table-stakes move is the passing the export controls on China in statute. Hopefully, that gets done before the new Congress comes in, but we'll see. Then there's the question of: how do you manage these companies in the United States? And our theory of the case, just to make it explicit, was you build a lead over China by slowing China down, and that essentially makes AI development a one-player game for democracies, and that reduces the arms race incentives that would make it hard to slow American companies down and regulate American companies. We did the beginnings of this regulation with the Biden executive order and the voluntary commitments and so forth, all of which have been repealed by President Trump. What does regulation look like? For me, there's a few things. The first is mandatory pre-deployment testing before systems are released to the public, and building a government capacity to do that for the frontier. The next one is some kind of internal or embedded evaluators. As you guys know very well, the Hugging Face incident with OpenAI happened during testing, so it was not a released system. We've gotta guard against those kind of risks as well and make sure the companies are living up to their standards there. The third is some kind of cybersecurity standard to guard against foreign espionage. This was a hypothetical threat. Now we've got a lot of evidence that the spies are after American companies. Part of, as I said, these companies growing up in the private sector, this technology growing up in the private sector, is they're not alert to the means of foreign government. So what are the cybersecurity and maybe even personnel security standards that the government should require of the companies and also help the companies with? And the fourth one is I'm an advocate for pacing the frontier, and I'm an advocate for slowing down, and I think there is a role for the government in figuring out what that looks like to slow down companies and make sure they're not racing ahead and racing to the bottom. So that, I think, is probably enough for the Congress to get going with in 2027, and I hope they're ready to get to work.

Roose: Should Congress try to ban recursive self-improvement?

Buchanan: Define recursive self-improvement for me, Kevin.

Roose: AI systems building better AI systems.

Buchanan: I think there's a lot even in that definition. There's probably some forms of recursive self-improvement I'm not worried about. An AI system writing the code for someone that is then inspected by a human seems totally fine to me. There's also kinds of recursive self-improvement that worry me tremendously. Sometimes it's called closed loop. You sort of package up the data center and you say, "Okay, go make your successor with this compute," and we don't have the visibility. I worry a lot about that, and I suspect there's a lot of work to be done on defining what kind of recursive self-improvement we'd want to restrict or ban and what kinds are fine. This is a case where it's just a lot of nuance, and the answer probably is like, yes, I would wanna ban some parts of it, and other parts of it don't worry me.

Newton: So, METR embedded at Anthropic, where you are an advisor. My fiancé works there as well. The report coming out of that time that they spent there said that using their own AI systems, they've been able to accelerate their progress by about double based on what you would expect were they not able to use their AI systems. METR seemed quite worried about this. The AI safety community is very worried about this. Should companies be able to use AI to double the speed at which they are able to train successor systems, or where would you put the actual slowing in the frontier?

Buchanan: I could imagine a couple ways in which you could slow down. To be clear, I want the companies to slow down. A couple things the government could do. The simplest is some kind of pairing of safety standards to capability development, so you've gotta prove you're able to meet the safety standards before you can reach the next level of capability. That probably slows them down or, at a minimum, gives us more confidence in the safety. So you must be this high to ride to this level. It must be this high to ride to that level. That's one thing. Another idea that I would consider is limiting the amount of computing power that can go to training. Sometimes it's called an input control. There's pluses and minuses to that, what counts as training versus safety developments and so forth, so I'd want a very clear definition that applies across the industry. But yeah, in general, my posture here is build the lead over China so we're not in a race, regulate the companies so we don't have a race to the bottom here, and at least in the interim, that probably involves slowing them down.

Newton: We have this system now where, you know, the Trump administration, after pooh-poohing the idea of AI safety, you know, getting rid of your executive order on day one to really, you know, stick it to the safety community, you know, then we see three or four leaps in advancement of the models, and then all of a sudden it's, "Guys, we might need some sort of safety policy around here," right? And so we now have this de facto licensing regime where if you're an OpenAI or an Anthropic, there's some sort of secret system by which you go to someone or something in the Trump administration and say, "Hey, we have the next version of ChatGPT or Claude ready to go," and then somehow it gets out. Do you know what is actually happening during—

Buchanan: No, I— [laughs]

Newton: That process? Like, are they— in my mind, they must be doing some sort of pre-deployment testing, right? Or like they're giving them some sort of report. Like, what is going on?

Buchanan: Great question when you have Jensen and David Sacks on. You should ask them.

Newton: Coming next week on Machine Gods.

Buchanan: [Laughs] We built and I handed to the Trump team the capacity to do pre-deployment testing. So on a voluntary basis, OpenAI and Anthropic agreed to do pre-deployment testing with what we call the AI Safety Institute. It's been renamed twice since then. I would say implicit in the Trump administration's policy over the last six or eight months is some recognition that these systems are really capable, and the US government has a role in this, and I'm glad they're in that spot. I would— you know, the sort of good-government Democrat in me wants policies to be out in the open and interpretable to everyone and all that kind of stuff, but I think this is better than where they were in 2025, where David Sacks was saying, "We just gotta let the private sector cook." JD Vance was saying, "The future's not gonna be won by hand-wringing on AI safety," and so forth. So reality and the government are converging, and that's probably a good thing.

Newton: Yeah.

Roose: One of the things that sticks out to me about the Democratic posture on AI now is that it seems like quite a fragile coalition. You have these sort of antitrust hawks, people like Elizabeth Warren, people who are very skeptical of sort of corporate power, talking about AI as if it is a relatively normal technology that is being shoved down people's throats by these tech elites, that the models may not actually be as capable as people say, and that they're mostly opposed to data centers. Then you have the sort of AI safety coalition that thinks that the technology is real and improving quickly, and we may all die pretty soon if we don't do something drastic to slow down the AI companies. Those two sometimes find common cause, but sometimes they want completely different things. Do you think the Democrats have a coherent AI message, and if not, what should it be?

Buchanan: Let's take this one in parts. I actually would not put Elizabeth Warren in the first camp. I agree that camp exists and it's on the left, but Elizabeth Warren's been very good on export controls to China and I think been very good. She's skeptical of corporate power, so am I, so, like, we're good there. But I do think you're right that there is a coalition here on AI policy in general that's still constructing. Other groups I'd mention in this coalition are the kids' online safety community, which is really important to this, and folks who care about copyright and artists and so forth. So we met with all of these groups when I was in the posi— the White House special advisor for AI role, and I felt like we were trying to build a coalition then. I kinda think Democratic politics is always coalition-building, and if you look at the Democrats from history who have done well, Obama being an obvious example, he's been able, the leaders have been able to assemble a coalition. AI policy's not gonna be any different. I do think this will be one of the biggest issues of the 2028 presidential race. This is gonna be entangled with affordability and economic anxiety and concern about big tech. It'll be entangled with corruption and, you know, who paid what for the ballroom and are the tech oligarchs buying the government and so forth. And I suspect the Democratic candidate in 2028 who wins is the one who can tell the story and knit this coalition together. I suspect there's a coherent story to be told that does knit it together, but I don't know. I don't do political strategy. It's someone else's job to figure out how to do that.

Roose: Do you think the data center thing is going to be the central piece of it in 2028? 'Cause that, to Casey and I, that feels like the most visible symbol, the thing that people are reacting to maybe because of the sort of actual claims about the data centers being bad for the environment or whatnot, but maybe just because it's something that they can see and hear that is in their communities that they don't like.

Buchanan: It will be a big part of it. I suspect economic anxiety, jobs will be there as well. Depending on where the economy is, I think that'll be a big part of it. I suspect the big picture here is just gonna be people don't want this technology shoved down their throat, and it's not just about the data center, it's about the broader vibe. I would say our posture was we wanted to get ahead of a lot of this. We wanted to put regulation in place so the public could trust the technology. And maybe unsurprisingly to folks like me, if you have a lot of periods of let the private sector cook, the public doesn't trust the technology, and you get this kinda current moment, and I don't blame the public. What I am a little bit heartened by, I say as someone who's generally a pessimist, is how many folks voted on this issue and how many politicians on both sides of the aisle, but especially Democratic side of the aisle, campaigned on this. And you got your AI safety types, Manny Rutinel, Scott Wiener, campaigning on this in that way. But you also get the Adam Hamiltons of the world in Kansas talking about this and of course, Jon Ossoff, I think doing amongst the best job of anyone knitting this story together. So I suspect this is gonna be a big issue in the '27, '28 Congress and in the '28 race, and I think it's just a really good thing.

Roose: Do you think the world looks very different by the time the 2028 elections come? I mean, we've been talking last week about this METR report and all these guys like Paul Christiano who keep coming out and saying, you know, "Everyone in this room may die as a result of AI," like... And all that seems premised on the fact that this could look, like, things could look very, very different a year and a half from now or two years from now. So, like, what's your best guess of where things are going, and how worried are you?

Buchanan: I'm very worried, and from pure technological capabilities, absent some pacing the frontier, yeah, I do think we will see a pretty different world, and I have very strong conviction that the ceiling on intelligence is far away, the AI systems are on a trajectory to get closer to that ceiling, and the timelines are pretty short. I kinda am not in the, like, it's three months camp that's sort of become in vogue in the last, you know, couple weeks here. But I do think this will be a very prominent issue, the technological capabilities, I should say, will be a very prominent issue in the '28 race. Does that translate to job loss right away? I actually am a little bit more skeptical of. I'm not an economist, but I do think a big portion of the economy is highly regulated, and is healthcare, and I work at a university, and, you know, they're not exactly at the cutting edge. Johns Hopkins is really good compared to other universities, but they're not at the cutting edge. So huge portions of the economy, you know, like the government, can barely use track changes, so let's just pump the brake a little bit on how fast they're gonna adopt this technology. But I do think the technological capability, the raw firepower, is absolutely gonna be there by '28, '29.

Roose: Casey and I talked on our last show about what people can do if they're feeling hopeless or afraid. We've gotten a lot of listener responses to that, people saying, "Should I be, like, stocking up my bunker now?" That doesn't feel particularly satisfying as advice for people, but I'm sure you get asked all the time, Ben. What should people do if they feel worried or concerned about the direction or the pace of AI progress?

Buchanan: I do think the answer you guys said on the show is right, which is like this is a moment where the political salience is high and voters can engage. One of the things about my time in the government was when we polled on our actions, they polled, like, 85/15 in favor. But to use a wonky political science term, the salience of AI was just not that high as an issue. No one voted in 2024 on AI. 56% of voters in the 2026 exit polls said AI was a, quote, "crucial or major factor in their vote." I suspect the number's gonna be even higher in 2028, especially if it gets entangled with the affordability and the corruption and so forth. So I do think this is a moment where engagement with the political system is right, and I don't think there's anything trite about saying that. And, you know, insofar as our book and our policy is based very strongly on a premise of this technology goes better if it's in the hands of democracies, I think an implicit correlate for that is you kinda have to have faith in the democratic voters, small-D democratic voters, to engage with the system and get the system to manage this well.

Roose: Well, yeah, I mean, the advice I gave in the last episode was essentially, like, call your congressperson which I feel sheepish about in saying because that is not something I have typically done. Honestly, it's just journalists mostly do not call their congresspeople and ask for laws to be changed.

Buchanan: You probably shouldn't, I think. [laughs]

Roose: Yeah, it's probably not, like, my place. We're in this sort of, like, weird secular priesthood where we don't do that, but I do encourage other people to do that. But, you know, having worked in government, do you feel like that makes a difference?

Buchanan: Yes.

Roose: Like, when the phones ring off the hook, is there an actual impact on the sort of legislation that gets brought to a vote?

Buchanan: Yeah. Every time you call a congressman, you raise the salience for them, and part of the reason I'm a little bit more optimistic is the political reality is so stark, especially after the midterms, that they are going to have to act, and they will not be able to sweep it under the rug the way they have swept other issues under the rug. And that is ultimately a good thing if you want them to change the policy.

Roose: And crucially, there just is a lot of bipartisan agreement on this, right?

Buchanan: Yeah.

Roose: Like, when I look at the AI debate compared to the social media debate, like, Democrats and Republicans wanted opposite things when it came to, like, regulating speech online. When it comes to AI, there's just a lot of agreement, so it just seems like a much, like, richer canvas that you can paint on.

Buchanan: There's so many MAGA Republican senators who I probably don't agree with on many things who I think have been really good on this issue. Pete Ricketts in Nebraska, Todd Young, maybe a little less MAGA in his case, but in Indiana, Jim Banks, also in Indiana, Tom Cotton in Arkansas, who have all done different versions of bills with Democratic co-sponsors that I think we should sign, and the president should sign. And I am cautiously optimistic that there is a space here to craft a bipartisan coalition, certainly on the China question, but I think increasingly on some of the domestic regulatory questions as well. No one wants to die in an AI-enabled bioweapon. It just, you know— And as the technology and its power become more and more obvious, I think it's just harder to deny these things.

Roose: Ben, you live in Washington. What is a thing that people outside of the San Francisco AI bubble don't understand about what's happening inside the bubble, and what is something that people inside the San Francisco AI bubble don't understand about the rest of America?

Buchanan: I think from San Francisco, it is hard to appreciate how slow Washington can be. But then—

Newton: I feel like I appreciate that. [laughs]

Buchanan: But then also how fast Washington can be. Right? So it's very, very slow for a long time, and then there's a spark, and then it happens, which is why I'm optimistic we might get the MATCH and OVERWATCH Acts, this next couple months here.

Roose: It's very confusing to me that they call it the OVERWATCH Act, 'cause Overwatch was the name of a first-person shooter that I—

Newton: That Kevin was obsessed with.

Roose: I played for, like, a thousand hours, like, five years ago. So every time I see, I hear the words OVERWATCH Act, I'm like, "They're regulating Overwatch? That seems a little excessive."

Newton: He gets pandemic flashbacks.

Roose: Yeah. [laughs]

Buchanan: I also played Overwatch. And this— Kevin, this is a good example of the nerd Washington, San Francisco divide.

Roose: Who's your main? [laughs]

Buchanan: Oh, it's been so long, I forget. [laughs]

Newton: [Laughs] Oh, he doesn't wanna say.

Buchanan: Yeah.

Roose: He was a tank.

Newton: He was a Hanzo main. He was sniping people from across the map.

Buchanan: I did a lot more, but we'll come back to that. We'll come back to that later.

Roose: And what's one thing that people in the rest of the country don't understand about what's going on here?

Buchanan: In the rest of the country or in D.C.?

Roose: Well, let's call it D.C.

Buchanan: Yeah, I think folks in D.C. don't recognize how serious the AI companies are about what they're doing. The companies do mean this, and they're not kidding about, you know, their timelines and the capabilities. And everything in D.C. has a comms strategy and talking points and so forth, and my experience is AI companies don't. At least the AI engineers don't. Like, they're incapable of sticking to a script. They're just gonna tell you what they think. So yeah, I wish maybe D.C. appreciated that a little bit more. I think people, I think it's changing. I think people— It's breaking through. In the last couple months, it's really broken through.

Roose: Mm-hmm. All right. Well, Ben, thanks so much for coming by. The book is “The Bitter Struggle.” It is out now. It's very good.

Buchanan: Thank you.

Newton: And it has a window into what it is like to actually work on AI policy at the highest levels of American government. I'm just mad I can no longer use “The Bitter Struggle” for my memoir of my time with Kevin. But if anyone else had to use it, I'm glad it was you.

Buchanan: I will sell you the naming rights.

Newton: Okay. Actually, that's great.

Buchanan: For a very reasonable fee.

Newton: That's great.

Roose: Well, I thought “The Bitter Struggle” was gonna be your memoir about your experience of trying to read a book all the way through.

Newton: Could be that, too. Yeah.

Buchanan: The good news, Casey, this book's very accessible. You can sound out all the words. You know, it'll be great. [laughs]

Roose: Lots of pictures for him, I hope.

Newton: All right.

Roose: Perfect. Ben, thanks so much.

Buchanan: Thanks for having me, guys. This was fun.

Roose: Casey, how was your weekend?

Newton: Well, Kevin, I was wrapped up in the national ground stoppage of all flights.

Roose: Oh, no.

Newton: I was supposed to fly down to Southern California for my mom's 72nd birthday, taking the family to Disneyland, and, yeah, then heard from our executive producer actually [laughs] that they weren't gonna fly planes anymore.

Roose: Oh, my God.

Newton: But so just as technology failed me, I have to say technology did save me as well, because my fiancé, when this happened, sprung into action, rented one of these self-driving Teslas, and we took a self-driving Tesla from San Francisco to Los Angeles.

Roose: [Laughs] Wait, really?

Newton: Yes. And listen, you know I am no fan of Elon Musk, but whoever, like, specifically at Tesla is responsible for the self-driving, they're cooking over there.

Roose: Did you put a bumper sticker on it said, "I rented this before Elon went crazy"?

Newton: [Laughs] Yes, exactly.

Roose: [Laughs] Well, I'm glad you made it to Disney World.

Newton: Yeah.

Roose: I'm glad you had a good time, and I thought you were gonna say that your fiancé, like, used Claude Mythos, the internal-only version to, like, hack into the FAA and fix whatever was going on with the planes. Did they ever figure out what that was, by the way?

Newton: I, you know, it's some sort of nonsense, you know? They were basically like, "Oh, we tried to, you know, upgrade a system and something got overloaded," but nothing that satisfied me.

Roose: BreachVember strikes again.

Newton: Yeah, BreachVember still going strong.

Roose: Casey, who makes this show?

Newton: Machine Gods is made in partnership with NPR, with generous support from the Tarbell Center for AI Journalism, the William and Flora Hewlett Foundation, and Emerson Collective. This episode was produced by Sasha Ashall and Jacob Blumberg. Our senior producer is Rachel Cohn. Our executive producer is Danush Parvaneh. Original music by Armen Bazarian. Branding and visual design by Alex Castro. Special thanks to Oren Rosenbaum, Erica Berry, and Nadine Zylstra.

Roose: Keep sending us your questions about AI and technology. You can send them as text or voice memos or short videos to hey@machinegods.fm or leave us a comment wherever you're getting the podcast. And for the love of the gods, don't forget to like and subscribe.

Newton: Until next time, stay blessed.

← All transcripts · Questions for the show? Ask Kevin and Casey.