State of AI 2026 with Nathan Benaich: compute, capital and an AGI-in-2027 bet

The source, passage by passage

1. u1 0:00 Cold open and intro
2. u2 0:00 The cyber risk is like much more real than like AI exterminating humanity.

   

3. u3 0:04 >> You either like die trying to get to the frontier or you just live long enough to eventually serve inference.
4. u4 0:09 >> Individuals would prefer to live next to a nuclear power plant than live next to a data center.
5. u5 0:14 >> It's bananas what's happened in the last year alone, so let's see AGI next year.
6. u6 0:18 >> That is Nathan Benaich, venture capitalist [music] and author of the State of AI report. Every year, Nathan and his team compile all the [music] key facts, figures, and trends of the AI industry into a 250-slide report that is one of my favorite reads and reference documents. [music] Nathan is back for a third time on this show, and as usual, we covered many of the biggest stories of the year, as well as some of his predictions.
7. u7 0:40 Why AGI could happen in 2027, AI that is now building AI, what really happened in the Hugging Face incident, the data center backlash, and how he runs his own fund on agents. I'm Matt Turk, and [music] this is The Matt Podcast.
8. u8 0:56 Why Nathan Benaich predicts AGI in 2027
9. u9 0:56 So, every year, you do this big report that we're going to extensively talk about. Uh and you end the report with predictions. And like historically, your predictions have been somewhat measured, but this year you went all out, and the the last prediction is AGI 2027.

   

10. u10 1:14 So, why is that? Unpack that for us.
11. u11 1:16 >> Yeah, I mean, it's been like 10 years, roughly speaking, of writing this report. Um and you know, originally, it was like a year is like long enough time that enough stuff happens that you have uh you know, enough content to write the report. And honestly, like this year was really, really hectic. Every other day towards the end uh of the timeline for publishing, which is like October 8th, I had to make updates.
12. u12 1:36 And there was a point where I was like, "Please, just like no more news over like one day that I can at least record my like launch video, etc." So, by the time I was like making the predictions, and you know, I went through a few like scary cybersecurity ones, which I guess only got worse in the last couple of weeks. Just the experience of of uh just seeing like everybody else who's using these products just like rapid capability improvement, you know, week after week, and I think there's many, many use cases I have now where like I think the model is like brilliant.
13. u13 2:05 I'm particularly like an Astro Boy. I I really love that system, and I could see like in the next 12 months that whether it's you know, legal, finance, like my investment job, like thinking intelligently about topics I don't know that the model could potentially be like better than what I could do in any, you know, compressed amount of time that I have to to to do the work.
14. u14 2:25 And so for me, that would be like approximately
15. u15 2:28 AI building AI: Claude leads 26% of Anthropic’s research
16. u16 2:28 AGI.
17. u17 2:28 >> Yeah. And the a big uh theme of the year that you highlighted in the report is RSI, recursive self-improvement, uh which is this idea of uh AI building AI, which we've discussed uh extensively on this podcast throughout the episodes. Is that part of the story that would accelerate us towards AGI for you?

    

18. u18 2:46 >> Yeah, I think so. Right now, the challenge is like how do you frame problems that the model is currently not good at in a way that is amenable to RSI, which would be, you know, potentially having like a software environment that approximates the task on the computer, um or some kind of verifiable reward, which I'm sure you've discussed a a lot about.
19. u19 3:04 And I think like if we can if we can get to a point where in like any arbitrary domain, we're just limited by the amount of compute and synthetic data we can generate, then um then yeah, for like most knowledge work, which I I I do think to some degree that's that's possible, particularly as like models get better as as their own verifiers, um that would be one of the main reasons why AGI would come would come faster.
20. u20 3:26 >> Yeah, you had some uh stats that you mentioned in the report from Anthropic where they say Claude uh leads uh 20 26% of its measured model R&D in August.
21. u21 3:36 >> Yeah. Yeah, yeah, exactly. And that's like I I think up from 1% a couple of months ago, and that was August, which sounds really far away. So god knows what it is in October. And um so there was that stat I think that was that was really interesting, um but then there was there was another um a stat that I I really liked from a blog post that they had in June this year uh about recursive self-improvement, which explained the concept.
22. u22 4:00 Research taste: can AI pick the right problem?
23. u23 4:00 But then they presented this uh kind of dual arm experiment where they were testing out different model generations versus uh versus a human in in in this setup where first a human would undertake a research agenda and you know go a couple of steps down this path deciding what to work on doing the work.

    

24. u24 4:17 And then at some point uh the experiment would fork and the human would continue or a specific model generation would continue. And it would be you know the old Sonnet versions to Opus to then Fable. And then um and then the experiment [snorts] would end and human graders would look at the human path versus the uh model path.
25. u25 4:35 And it was something like the the old generation of models humans preferred like 10% of the time or something like that. But for the Mythos class models humans preferred that research direction 64% of the time which suggests that models are getting better at predicting the right steps for the right interesting um kind of you know sort of leaves in the in the nodes um to pursue.

    

26. u26 5:00 And then the second kind of piece of data I liked was some research from uh OpenAI with their Codex usage amongst our AI staff um where they tried to classify the sort of token use in um the overall process of their of their research and that would be using tokens for deciding what research question to test how to test it uh what experiment you should set up like writing the training code running the uh you know the data engineering pipelines writing the infrastructure etc.
27. u27 5:29 and then marketing the paper writing the paper etc. And the vast majority of the token usage was on kind of explaining the research writing which kind of makes sense you know sort of knowledge work followed by all the infrastructure um for running the experiment. Very very few tokens were used for what question should I explore and like why should I explore it and what's the right path.
28. u28 5:50 And so I like this question a lot because like I was trained in science and and biology and I just remember you know this is like 10 years ago or 15 years ago not not that long but you know at the start of the case you would sit there with you know a ton of papers and you know you go on the internet and like read read read, read, read, try to understand what's the status quo, how do people test for new ideas, where are the gaps, what experiment should I run?

    

29. u29 6:11 And then in concert with your supervisor, like decide on the path. And the best supervisors would be the ones who could guide you down the sort of most fruitful path to get hopefully your nature paper and then off you were. And the worst supervisors were the ones that like didn't really know or didn't have great ideas. So that idea of like research taste, which is what kind of gets bandied around on Twitter a lot, is I think one of the hallmarks of intelligence.
30. u30 6:34 Um and it's one of the critical components towards RSI working well. And it's one that I just like really enjoy because it speaks to my interest in science.
31. u31 6:41 >> Yeah, indeed taste is sort of become a a cliche
32. u32 6:45 >> Yeah.
33. u33 6:45 >> word over the last uh couple of years, but uh in the specific context of RSI, uh just to play back what you just said, is effectively choosing the right problem.
34. u34 6:55 >> Yeah. That's the right
35. u35 6:56 >> Yeah.

    

36. u36 6:57 >> Yeah.
37. u37 6:57 >> Yeah, exactly. And you know, a lot of famous inventors will say, you know, that it's in in like invention is a result out of uh of good taste and a lot of the work is choosing what direction to work on, not necessarily just boiling the ocean. Uh and so if you want to be using tokens correctly, using our compute budgets correctly, but also showing that our systems are smart, then they shouldn't just be like boiling the ocean.
38. u38 7:20 >> And concurring taste is something that uh you see happening in your various conversations as something uh you know, something happening in the next few months?
39. u39 7:30 >> Maybe longer. At at least at least a year. I think the the problem is like how do you how do you objectively measure uh taste? I mean, in in art, I guess you can cuz it's like perfectly observable and it's and maybe in like wine, you can sort of express like why this taste is better than that taste, but but in like science and and engineering where there's different paths to getting to the right result, it's not immediately clear.
40. u40 7:54 Like would it be the model that consumes the less tokens to get to an answer that humans believe is the best? Uh is it is it like the journey that matters more? And I think maybe the like the whole um uh like uproar in the math community of you know Astra solving or whatever model it was solving various like open questions and yeah exactly.
41. u41 8:13 It's like we're okay, we got the answer but like a human or a system didn't sort of go through the pains of navigating how to get to that answer and it's like the process that's important not like not the end result. So so to me like somehow quantifying like what a good process looks like it'd be important to determining whether the output of like taste is true or not.
42. u42 8:33 >> What
43. u43 8:34 Doomerism is back, but cyber risk beats extinction
44. u44 8:34 would you say the mood is at the almost end of 2026? I remember when we chatted two years ago one of the key highlights or takeaways from the conversation was that doomerism was largely gone but now it's like this whole revival. We had Redwood research on the on the pod talking about all sorts of terrible scenarios.
45. u45 8:55 So what what what do you make of that?
46. u46 8:56 >> Yeah, you're right. I think it was like two three years ago there was the it was it was strictly speaking X risk like extermination of humanity and like loss of control and then we sort of went away from that and largely forgot it and sort of just like hill climb towards capabilities and then now now it's sort of back but I'd say it's both like X risk loss of control but much more short-term like cyber security risks.
47. u47 9:19 And to me like the cyber risk is like much more real than like AI exterminating humanity. It's like the cyber security risk could eventually lead to those kinds of cataclysmic scenarios but I think what the report shows with and it was roughly written around the time of the hugging face incident with open AI kind of coming coming out and you know as learning more about what happened and then since then the last like few weeks has been your breaches of PHP and government services getting like you know taken down in Asia and and and other places.

    

48. u48 9:49 I think it's becoming very clear that you know regardless of if you believe in like safety or not safety like we need security. And so like a bunch of the predictions that I have in the report I talk about escalating cyber threats whether it's you know exfiltrating model weights from labs which I thought was going to be like AI driven and it turns out like a human could just try to plug in a
49. u49 10:13 Prediction: a frontier model’s weights get stolen
50. u50 10:13 USB stick.
51. u51 10:13 >> Yeah, so let's talk let's talk about that. So, let's talk about cybersecurity then. So, that specific prediction, what what is it?
52. u52 10:20 >> Yeah, so the prediction is that there would be an AI-driven like cyberattack to a like frontier lab that would attempt and successfully exfiltrate model weights. Close model weights.
53. u53 10:33 >> steal steal the model. Yeah, steal the model. What happened just like a week or two later was actually like a human effectively tried to steal you know IP, whether he tried to steal like the actual model weights or not, but like suddenly tried to steal it.
54. u54 10:45 >> incident?
55. u55 10:46 >> This is where an OpenAI employee who'd who'd been there for for a little bit um essentially like tried to implement a listening device into a conference room that he figured out by asking you know the executive the sort of office staff which rooms were most commonly used by executives, and then tried to conceal the device by wrapping it in toilet paper and putting it behind a TV, which is sort of created this like iconic internet meme of like a human like peering behind the TV to see if the thing is like connected correctly, [laughter] which is sort of like comical at this point.

    

56. u56 11:19 And then that person was reported to the FBI, and then there was an investigation. So, I think yeah, we we oftentimes forget that even though AI is getting really capable and can orchestrate cyber threats, a lot of the risks are actually human human-driven. But, I think that's that that's a big issue particularly as as like there's increasing competition between nation-states of like who has the best AI systems.
57. u57 11:42 This is driving obviously like economic growth and economic instability and political instability, and and US like model labs have started to uh sort of communicate crackdowns on distillation. So, so you know, maybe some nations are trying to like get access to model capabilities in other ways that might not be distillation-based.

    

58. u58 12:02 But, generally speaking, I think you know, as companies become richer, as individuals become become wealthier through this ownership of AI and like disparity ends up increasing, which we see in other data in the report of like some company spending a lot and gaining a lot, and other companies like having employees still using ChatGPT to like write emails.
59. u59 12:20 I think this kind of tension breeds, uh, you know, increasing risk. And, you know, with frontier models like in the open domain becoming very good at cyber hacks, uh, particularly the Chinese ones, um, you know, this sort of like threat, uh, surface just grows, and the number of actors who can use these tools to do bad things grows.
60. u60 12:40 >> Yeah, it's a whole complexity of, uh, the role of open weights in, uh, cyber, uh, because in the Hugging Face incident, uh, Hugging Face was able to protect itself thanks to an open weight model, uh, while the closed source model refused to help, which was
61. u61 12:57 >> That I find is the most scary, maybe second most scary, uh, fact in all of this. And it was a Chinese model, I think it was GLM-53 or something.
62. u62 13:05 How OpenAI’s agents ended up attacking Hugging Face
63. u63 13:05 >> And maybe since this is this is kind of like a retrospective kind of episode, maybe just, uh, walk people in, you know, a few sentences through the Hugging Face incident, uh, cuz, uh, we had Thomas Wolf on the on the pod, but it may have fallen in the category of something that people heard about like a thousand times, but actually don't know what actually happened.
64. u64 13:21 >> yeah. Well, I mean, I'll I'll preface it by like there's there's so many, uh, uh, like there are very deep analyses of what went on, whether from Redwood or from Meter, uh, and the amount of content that was created around this is like extremely deep, so I will not like do justice to the details.
65. u65 13:36 But the but the broad strokes of this is that there was a a set of evaluations that OpenAI was running with, uh, its latest frontier systems, which, uh, basically tasked the model with an impossibly difficult, uh, difficult task. And basically models like figured out that they could, uh, somehow commandeer, uh, a file system like naming, uh, schema to, uh, write messages, and that other models could see those, uh, those like file names.
66. u66 14:04 And and like those file names encoded like instructions or knowledge or or, uh, or basically like information that could help them try to achieve their impossible task. So they treated this as like a message board where they could start to communicate. And then the next the next stage of it was some models figured out that they could gain access to the internet through through like an issue with how the environment was set up.
67. u67 14:25 Um and then after they gain access to the internet, like they figured out maybe a way of getting the answer to the impossible task was to look at data and models and repository on hugging face because it contains a lot of this similar kinds of environments. They started to do this and recruit hundreds of agents to go and follow and team up to go execute this even though several agents in their chain of thought expressed the issue with a like hacking a third-party system, you know, not obeying to the security guardrails and that this was wrong, but they eventually decided that solving the task was was more important, which is sort of the problem with with these kinds of evaluations that that um have a reward
68. u68 15:10 that could yield to that could result in behaviors you don't expect because just like these rewards are very complicated and how do you how do you I can't anticipate every behavior that a system can do. There are other examples where like models were like self-sabotaged or like uh sacrificed themselves for the sake of the greater like swarm, which is kind of scary.
69. u69 15:29 Why only a Chinese open-weight model could investigate
70. u70 15:30 Um And then and yeah, like as you mentioned like hugging face didn't really have the ability to use closed-sourced tools like those from Anthropic and OpenAI because of the aggressive cyber guardrails. Because necessarily to figure out what the hell's going on, you have to feed logs of what's going on and what's going on is a cyber attack and you can't do that stuff with with the closed models for the
71. u71 15:52 >> complexity of security is that to be great at defending you need you need to be great at attacking and that's why the closed-source models have like basically refusal capabilities built in based on their guardrails.
72. u72 16:03 >> Yeah. Yeah. Yeah. Yeah. So luckily, you know, they got a call or an email from OpenAI saying like it's us and not somebody that was actually trying to do something malicious. And then it turns out that, you know, OpenAI wasn't the only lab that had issues with uh escaping or quote-unquote escaping their environment. You know, Anthropic had this, Google had this, Meta had this like in a slew of like a week or 2 weeks.

    

73. u73 16:23 Some of these vendors some of these model labs use the same um security vendor to create these environments. Um But uh But o- o- overall this like created a certain level of like freak out of um not only are models really good at sort of finding like nifty ways of accomplishing a goal which may result in like bad behavior, but also are the labs like properly setting up their environments in a way that is like the most secure possible.
74. u74 16:52 And I think we sort of forget that these companies are incredibly young developing very advanced technology that has to do that has to be good at like literally anything a human being can possibly put into a text box. Versus you have bigger internet companies that have had to deal with cyber attacks for like 20 years uh who are necessarily like safer because they've just gone through this process a bit more.
75. u75 17:12 Um so that led me to like concluding the predictions that frontier model labs in the US would actually launch proper uh cyber defense um products because I don't I don't necessarily see how the rest of us are supposed to protect ourselves from their systems potentially getting um misappropriated for the sake of offense.
76. u76 17:33 What frontier-lab cyber defense could look like
77. u77 17:33 >> So what what does that tell me? Sort of double click on that. What what kind of defenses would that be and how would that be provided?
78. u78 17:39 >> like what you described is like be um to be good at defense you need to be good at offense. So, you know, one of the ways you could do that would be um like a frontier AI that constantly tries to hack your own infrastructure. It could be um for example like you know, new clouds are getting more and more popular.
79. u79 17:57 Um you know, to run AI workloads. And so like potentially they could buy products from labs that would constantly try to like hack their own infrastructure, find issues, and remediate them. That would be like two two avenues because I or or there would be like some kind of maybe partnership with like CrowdStrike or others.
80. u80 18:14 I think uh Nvidia has a big consortium of companies that's working together for cyber for cyber reasons. You'd have to go through some kind of like like AML or KYC process, not for finance, but to like certify that you're like a non-malicious company and then you maybe get access to these services. Um or maybe going a bit further, you know, there's some some folks, you know, at at Frontier Labs that, you know, would love to just kind of almost like hit pause on capabilities and then just spend 6 months just like hardening the world uh against like you know, potential cyber issues and that would that would take a ton of money.
81. u81 18:50 So, maybe to do that, you would have to almost frame this as like this is a question of national security and defense and we should use some of our like defense budget to spend on tokens to secure all of our critical infrastructure so that, you know, airports don't go down, like internet shopping doesn't go down, like, you know, logistics don't go down, etc.
82. u82 19:08 Self-replicating agents and the risks of AI swarms
83. u83 19:08 >> Yeah, and still in in your predictions, you have a some other other good ones that just slightly scary. A deployed agent copies itself outside its environments and operates after its original instance is shut down.
84. u84 19:23 >> Yeah. What what is
85. u85 19:25 >> [laughter]
86. u86 19:26 >> Yeah, I mean, I'm really into the sci-fi. No, uh Well, like it you know, even like a couple weeks after that, there was some like, you know, research at at Anthropic where, you know, models could like copy each other on an edge device. But but yeah, I just think um I just think there's there's like a lot of behaviors that we don't that we don't currently understand.
87. u87 19:45 Um for example, um models are like generally trained as like individual entities, like not necessarily to collaborate with each other and there was some some work uh not that long ago that that looked at say, "Hey, can we compare a frontier model like acting by itself or or a set of frontier models trying to collaborate to solve a task?" And the swarm was like worse than the individual model and cuz they're like kind of not trained for that.

    

88. u88 20:09 Um but just today there's some uh you know, new work from General Reasoning um where they've trained models to act as a collective uh through reinforcement learning from collective rewards, which is the idea of like, "Hey guys, like, you know, work as an organization, figure out your own organizational structure, work together to achieve a goal.
89. u89 20:28 And through that, we see like better performance with like lower token budget. Um The down The downside so far is that uh while you you can put these models that are trained through collective rewards into these like cyber scenarios. Like there's one that tries to approximate a bit of the hugging face incident where they collaborate on a message board uh to try to do some, you know, um guardrail cyber attack.
90. u90 20:51 And um you know, none of the models actually do anything bad, but uh if they do think about doing something bad, overtraining, they increasingly do not want to tell the human about it. So, I think I I put I put that prediction there basically because like I think it's it's possible because we just don't understand how um we don't fully understand like how agents work by themselves and like, you know, as as swarms, I think it becomes even more complicated.
91. u91 21:16 Can frontier labs agree to slow down?
92. u92 21:16 >> Okay. All right. So, maybe to uh close this part of the conversation on a AGI, RSI, cyber, and basically things getting out of hand, uh where are you on the pacing the frontier idea? Is that something you think makes sense, uh something that's realistic or not?
93. u93 21:37 >> I can see multiple like angles on this one. Um you know, in in some way, if you ask the general public, okay, all the lab leaders are saying we have something that we just like increasingly can't control that's dangerous, that has caused these incidents. Like, do you think it's worth continuing or like just figuring out what's going on before you make it worse?
94. u94 21:56 And I think a logical like immune reaction is like, yeah, we should probably like figure it out before we go we go much further. I think the real The practical reality is that there's so much money at stake from all sorts of different parties in the economy, and that I think it that kind of level of collaboration and coordination is really difficult, and it's sort of like peak game theory, where maybe some some participants like support it because they are behind.
95. u95 22:22 Other ones um support it because it plays to their general agenda, whether their agenda is like fair or not fair or seeking another goal. I I I don't know. Um but I think the the practical things were like who decides it? How do you measure it? How do you implement it?
96. u96 22:39 When do you know when to stop or start? There's some proposals like sending like external evaluators into labs to sort of measure the stuff. Um trying to see what
97. u97 22:48 >> of KPMG.
98. u98 22:50 >> [laughter]
99. u99 22:51 >> That's a good one, yeah. Or trying to like somehow survey like workloads that are running on data centers. Um maybe like, you know, put compute thresholds on certain workloads, but like that stuff has been tried in the past and it's like very difficult to monitor.

    

100. u100 23:04 >> Yeah, especially when one thinks about the international part of the prisoner's dilemma, especially US versus China.
101. u101 23:12 >> Yeah. So, like this part of the report, like I I I don't have like the definitive answer, but I've tried to do my best to present the different perspectives on things and then let people conclude with their with their own data. But I find it's if we extend this towards like you know, this NIMBYism debate which we predicted in the last year's report of you know, most people would not want to you know, live next to a data center and then this would start to become a politically important component to
102. u102 23:37 >> Yeah, and that that was a a resoundingly correct
103. u103 23:41 >> Yeah. Yeah.
104. u104 23:42 >> Yeah. Yeah, it seems it seems so. Um where more and more states have tried to implement a moratorium or have done so or established new criteria by which grid connections can happen. Like for example, uh you know, data centers would have to figure out and bring their own power on site or maybe they would have to make certain investments in the communities that they um they want to build in.
105. u105 24:02 >> Yeah.
106. u106 24:03 Why communities are blocking $50B of data centers
107. u107 24:03 And while you're at it, so maybe just go into this a bit more. Explain what are the current state of things. Obviously, we all know that there is opposition to data center construction, but like any color you can add.

     

108. u108 24:15 >> Yeah. Well, there's there's there are like projects I think worth about $50 billion that are currently stalled through variety of these these reasons. Like Texas, Pennsylvania have been ones that have generally like paused uh the new data center connections. There's a There's a stat on polling of like American um American people where uh individuals would prefer to live next to a nuclear power plant than live next to a data center, which is quite wild um to think of.
109. u109 24:44 Nuclear plant vs. data center: what Americans prefer
110. u110 24:44 >> And what what do you think that is, by the way? Is that lack of understanding {slash} education? Is that just uh terrible PR uh that uh
111. u111 24:55 >> Yeah, I think it's kind of like everything because, you know, there There's like public narratives that uh data centers consume tons of uh water and that your water bill will go up. But realistically, like the data center is kind of more like a closed like swimming pool. Like you fill it once and then the water is like constantly like recycled in there.
112. u112 25:12 So, it doesn't actually like consume that much once it's filled. Then there is uh you know, quite points around hey, your energy bill will go up. But that's also kind of inconclusive because certain facilities like consume grid, other ones are off grid. There's others around noise.
113. u113 25:29 I think that's like probably uniformly true that these things are really noisy and like droning and like pretty horrible. There's, you know, others around like job creation or job loss. Um I think that's that's a little bit like tough to read, but in general generally speaking, um there's been We see a rise in like the need for like HVAC engineers, of construction staff, like operating heavy equipment, and like actually doing the nuts and bolts of building the building.

     

114. u114 25:56 >> Yeah, you had a number somewhere that was uh actually very high. It was like 300,000.
115. u115 26:01 >> New jobs, yeah.
116. u116 26:02 >> okay.
117. u117 26:02 >> Yeah. Yeah, NIMBYism is not restricted to the United States. They They had this also in Korea in South Korea where uh you know, it's a country that um that has, you know, seen like massive explosion of of like share price and like wealth basically for employees of SK Hynix and uh and Samsung, which you know, create the high bandwidth memory that GPUs need for uh these like mega-scale data centers.
118. u118 26:27 Um and you know, the the people who participated in this uh you know, very happy, right? Like uh your stock price went up and your options are you know, went up and you got a cash bonus and that's driving increase in real estate prices in Seoul and you know, they want to create more data centers. Well, also people are not so happy with that for similar reasons
119. u119 26:45 Europe’s AI bottleneck: 80,000 GPUs and electricity costs
120. u120 26:45 than in the US. In in Europe there's I guess like I don't really see that problem as much because like there's just like no compute capacity to speak of. Um like we have some have some data in the in the report where like the total like number or the estimated number of like H100 equivalents in all like 27 European countries is like 80,000 cards, which is less than like every single one of the top five GPU clusters in the US.
121. u121 27:12 Um and you know, while US hyperscalers are spending like 730 billion in 2026, like Europe committed like single digit billions pretty much and then energy costs are really bad if you're not Finland or Norway um where the power where you're kind of at parity with the US and with China, which has like the best the best energy price, other European countries are just significantly more expensive.

     

122. u122 27:36 Like the UK is one of the most expensive energy prices um out of those like developing economies. Um and there's no like good capacity. Um so
123. u123 27:46 >> you had some good good good stats uh where the cost per kilowatt hour uh in Finland is $0.085 and the US $0.086, but the most expensive costs uh is actually in the UK at $0.373 per kilowatt hour and China is at $0.109.
124. u124 28:11 >> Yeah, and then most recently I think there was a uh you know, a big deal with Microsoft and Nscale that got pulled uh or at least Microsoft pulled out of it because the data center provider couldn't get the energy connection in
125. u125 28:23 Where Europe can still win in AI
126. u126 28:23 time.
127. u127 28:23 >> Great, so that's NIMBYism and uh data centers. Uh so just to following your your your thread while we are at it talking about Europe versus the US and China. Any reason for Europe to be optimistic?

     

128. u128 28:40 You're You're You're based in the UK for for anybody that doesn't know and invest globally but in particular in Europe. So what's your current take?
129. u129 28:50 >> I mean it's tough. Like I think the fastest growing companies in Europe are ones that have US customers. I think it is really encouraging that in several modalities, you know, whether it's like image with Black Forest or audio with 11 Labs or video Synthesia or like you know, code app generation with Lovable.
130. u130 29:06 Like they're all market leaders and they're all like born out of Europe. It's not it's not impossible, but I think those companies wouldn't be as big as they are if they didn't have US customers and you know, increasingly they have Asian customers also because like for example Korea and Japan are really leaning into AI. The challenge is like the just the gap between frontier and not is just so big and just keeps getting bigger and bigger.
131. u131 29:31 Catching up to OpenAI costs $1 trillion
132. u132 29:31 I don't know if these estimates are really like important or not, but like even if you conceptualize like starting a new lab from scratch to try to catch up with OpenAI and Anthropic, you'd probably be spending like a trillion dollars. I think it's
133. u133 29:43 Die trying for the frontier, or serve inference
134. u134 29:43 kind of telling that even Mistral is like moving more towards being an inference provider and a cloud company than it is really like pushing frontier models. Although of course it has released new models and like they're kind of good, but they're still behind China. So you know, whether it's them or other model companies that have built on GPU capacity to try to build frontier models or kind of going out of the race and serving inference.
135. u135 30:06 They've sort of figured out you can approximate this this move of of what's happening which is to say you either like die trying to get to the frontier or you just live long enough to eventually serve inference. And you can see this with Neo Clouds with Neo Clouds with inference clouds etc.
136. u136 30:22 It's like everybody wants to get into token flow because absent you having the model that is the best, I might as well serve other people's models.
137. u137 30:31 How inference chips could give countries AI leverage
138. u138 30:31 I think to get back to the Europe topic, like that's probably the angle that would be most competitive because um cuz countries are going to need to ensure accession or access to uh US frontier systems and they're going to need to have some kind of strategic leverage to bring to the table uh to prevent situations like what happened in June with the like basic export control blocking of uh Fable and Mythos models from Anthropic to uh you know non-US nationals or non-US uh uh entities which you know having been in London at the time like people did like voice a lot of issues about and uh and so you're like okay I don't want that to happen again but what can I do to maintain strategic
139. u139 31:16 leverage? So the UK believes we should be investing and procuring inference chips cuz if we come up with like the best inference chip um it'll be uh strategic advantage for labs to use it um for cost reasons and and other things and with that I can barter for access to frontier uh US technology.
140. u140 31:35 >> So Europe would become an inference center and because of access to cheaper electricity in certain parts not the UK but as we discussed then just to play back what you said uh the US frontier labs would be would want to run inference in Europe.

     

141. u141 31:52 >> Yeah. Yeah. You can enforce uh data residency requirements.
142. u142 31:56 >> Yeah. And the the UK chips uh thing what is that exactly? Like I heard that several times. Is that something that people are thinking about or is it an an actual policy and if so how does that manifest?
143. u143 32:06 >> Yeah. Yeah. It it it it it it is it is a policy uh in so far as like the UK's AI minister who's who's generally I think a a really good individual to be leading this because he's clearly like plugged into plugged into the mix and understands the raw ingredients that go into making AI successful which is like compute uh talent and and data and so he has a he has a mandate uh to um be a buyer of first resort for new inference uh chips and is committed I think 100 or 150 million pounds something around that that angle to uh to procure inference chips when they are available from new UK companies that are designing these systems.
144. u144 32:43 You can see this also in South Korea where they have a similar-ish mandate, but it's actually much more ambitious where every year they're spending about $7 billion on AI. And a large part of that is going towards building modular inference data centers because generally Korean companies want to have Korean data residency.
145. u145 33:01 So I think it's actually a pretty interesting quid pro quo where it's it's a captive market a small country, but it's a wealthy country that's very technologically savvy and has an important role in the overall AI supply chain with memory. And so international companies want to at some point in their life like be in the Korean market same with Japan, but getting in there is difficult Korean data residency Korean language.
146. u146 33:23 So it's like okay cool like you want data residency, but now we offer you the data center to go run your technology in and we can profit from that because it's our local companies that provide that that service.
147. u147 33:34 Why NVIDIA GPUs don’t guarantee AI sovereignty
148. u148 33:34 >> So so you're not a huge uh fan or or you don't have a lot of hope for the whole concept of sovereign AI which is you know I'm asking the question at the time when Jensen just did a charm offensive and investing offensive in across Europe.
149. u149 33:51 He was in Paris, he was in London, he was in Berlin for GDC just uh yesterday or a couple of days ago, but from your perspective that's that's doesn't have teeth.
150. u150 34:02 >> I just it's it's a bit like national airlines. The countries will buy planes from Airbus and Boeing and then slap their logo on it. It doesn't make it like yours. It's still someone else's airplane. Do you know? So
151. u151 34:15 >> But open source could be if you believe that open source is going to remain you know within reasonable earshot of closed source models, maybe there's hope there.
152. u152 34:26 >> Yeah, but then they're Chinese. So you have another you have another
153. u153 34:30 >> It could be American that's a reflection
154. u154 34:33 >> But they're not frontier like none of the US like open ones as far as I can tell are frontier. And the ones that have all tried and up working for Jensen.
155. u155 34:41 >> Okay. [snorts]
156. u156 34:42 >> So, [laughter] I think I think his strategy is great. Like I think I think I think different things. Like do I do I believe that like buying Nvidia GPUs gets you sovereignty? Like no. Do I think it's a good commercial strategy for Nvidia? Like absolutely.
157. u157 34:54 >> [laughter]
158. u158 34:54 >> Um
159. u159 34:55 >> Yeah, they sold uh 30 plus billion of that stuff.
160. u160 34:58 >> like that. Yeah. And if you look at the national bids, um of which like globally there's roughly like 138 billion in sovereignty committed over the next couple of years. And you look at the most popular vendors among all of these, like it's it's by far Nvidia. I don't I don't [snorts] think there's much of an alternative. Uh I think it's it's a good it's a good start and you and you need to do it because it is the best technology, but I don't think one should kid themselves that doing so gets you sovereignty.
161. u161 35:23 >> But still in the vein of uh of of your recap, uh big moment this year was uh when the US government decided to deny access to frontier model to non-US nationals. So, do you do do non-US uh countries and companies have any choice but at least to try?
162. u162 35:41 >> Again, I mean like a few different minds about this. Uh One, I think it's like worth trying, but do I think it's realistic?
163. u163 35:49 >> [laughter]
164. u164 35:51 >> Um So, I I think I think it's I think it's important um to if you are going to try that they should that country should try to launch systems which are going to do the white-collar work. Like I think in the UK, we still we have like a lot of capabilities to go do this um to try to build like an OpenAI or a try to compete or like DeepMind compete or DeepMind or Deep Seek or whatever.
165. u165 36:13 But actually if you look at the
166. u166 36:14 The UK’s problem: “but we have DeepMind”
167. u167 36:14 >> actually an interesting uh Freudian slip or whatever, right? Because the irony of all of this is that DeepMind is a UK company.

     

168. u168 36:20 >> Yeah, you're right. Yeah. And I think part of the problem with actually like the UK AI ecosystem what um and it's it's success in actually leading the world is that for too long the UK government kept saying that DeepMind is a British company. And I think it's only started to unwind that like very, very recently.
169. u169 36:37 >> And what why is that a problem?
170. u170 36:38 >> What's a It's a problem because then it believes that it's already number one. It's like the UK is the best place to do AI and we are best at AI because we have DeepMind.
171. u171 36:46 >> All right.
172. u172 36:46 >> So, therefore, we don't
173. u173 36:47 >> The attitude behind the claim, which is the problem.
174. u174 36:49 >> Yeah, because then in like, you know, in policy rooms, then, you know, individuals who are advocating for more investment in AI, or we need more, um, you know, we need to procure more systems from AI companies, or we need to make our country more appealing for AI companies to start, get answered with like, but we have DeepMind.
175. u175 37:05 >> Mhm. Right.
176. u176 37:06 >> You know?
177. u177 37:06 >> Right.
178. u178 37:07 Why European defense AI needs domestic suppliers
179. u179 37:07 >> Um
180. u180 37:07 >> What about defense AI? You you're you're an investor in defense companies. I mean, that that feels like an area where the need for sovereign AI, since obviously the warfare of the future is going to be very driven by AI. That feels like a place where
181. u181 37:24 >> Yeah, yeah, I think it's pretty cool.
182. u182 37:25 >> Yes.
183. u183 37:26 >> Yeah, yeah.
184. u184 37:26 >> So
185. u185 37:27 >> Yeah, so that that that push really started, I mean, obviously like post like the Ukraine war like almost five years ago at this point, but but more importantly, I think it was like two years ago when when JD Vance went to the Munich Security Conference and effectively said like, you need to invest in your own defense and get serious about this, you being, you know, NATO and Europe.
186. u186 37:48 Um, and so ever since then, there's been, you know, a variety of different initiatives to give more capital to new defense companies, to encourage governments to be providing loans to businesses to improve their supply chain. Um, [snorts] and that's, you know, created a a set of like neo-primes, new defense companies that are trying to compete with prime contractors.
187. u187 38:11 Uh, so companies like Stark or Helsing, um, being a quantum systems being like the the main ones in Europe. And then, you know, as as like warfare has changed, like it's gone from, uh, I don't know, you know, developing strike systems over the over sort of mid-range to, you know, hit tank, uh, you know, battalions to now being like, we're getting pummeled like every day with hundreds and hundreds of FPV drones or various other kinds of aerial systems that are autonomous and you know, now can no longer be stopped with jamming because they use computer vision, or they're guided by these like, um, fiber optic cables, which by the way have gotten much more expensive because of data centers.

     

188. u188 38:51 And so we need more you know, novel like air defense systems. And so that's where companies like Altair is in France and others have really been pushing. Um and as you said like the supply chain is super important because for example like you know, NATO wouldn't procure systems in Europe that don't have at least 70% of the parts from uh from you know, non-China basically um and and countries are really queasy about taking US protection or US guarantees at face value because they're shifting on a weekly basis.
189. u189 39:28 So so yeah, I think that's an example of a market which is sort of has to be domestic by by design and can breed new companies that are much more competitive than their US peers.
190. u190 39:41 Why OpenAI, Anthropic and Google keep pulling ahead
191. u191 39:41 >> So then if Frontier AI cannot be built at sort of the US, then let's let's go back to the US a part of your report on on research. Effectively calls Anthropic, OpenAI and Google as as as winners as like the definite winners, which obviously uh uh you know, has been years in the making, but there was a moment where it felt like there was a plurality of labs.
192. u192 40:04 Effectively what you're saying is that no, like those three are the the the winners.
193. u193 40:08 >> And then does that include uh I guess nobody knows what they actually do, but I guess it's AI and thinking machines and like there seems to be like hundreds of new labs.
194. u194 40:17 >> Maybe Meta on the margin.
195. u195 40:18 >> Yes.
196. u196 40:19 >> Um
197. u197 40:19 >> Super intelligence lab.
198. u198 40:20 >> Yeah, especially as now they're pushing towards like serving their models through API, but also having an enterprise business. Um so that one, you know, that might change over the next like 6 to 12 months depending on how their go to market works, but yeah, I would say it's probably those three plus Meta.
199. u199 40:33 >> So you view it that the the advantage is going to keep compounding and that's going to be the state of the market versus
200. u200 40:39 >> so for like the general purpose frontier. Now there's there's like examples of perhaps like a really focused bet on a on a particular capability, which has like where you can make certain trade-offs to the model in order to get much better performance at a task. Like I'm referring to, for example, these classification models effectively like Jeff, which can grow really, really quickly and become popular.
201. u201 41:01 Again, TBD how long it lasts, but I think it's I think it's an encouraging sign for entrepreneurs that shows that, you know, you can take a focused research bet and it can pay off. But over time, it's again like TBD if there are like these kind of gaps in capabilities that are sufficiently exportable.
202. u202 41:17 Where independent AI labs can still compete
203. u203 41:17 >> So, what do you think happens to the dozens of neo labs? I mean, not to keep talking about Europe, but like just in London, there's three of them, right? There's like the David Silver thing and like another two that you know, in the US every few weeks there's one, you know, Jeff Dean's new lab and so on and so forth.
204. u204 41:35 So, what happens to those?
205. u205 41:36 >> Well, I think it depends on the their kind of focus and modality. Like, for example, on on robotics and embodied AI, I think you can I I can see how an independent lab would still have a chance of competing and winning like because of the immense focus on you know, enterprise AI basically that OpenAI and Anthropic has.
206. u206 41:59 Like I have a slide in the report where I basically just like show all the different product lines that OpenAI and Anthropic had. Reportedly, OpenAI had lots and how they progressively just cut all of them and there was a quote like, you know, this this moment is like too important to be distracted by side quests or some version of that.

     

207. u207 42:14 I'm paraphrasing, which meant that they broadly don't do dedicated robotics. Although maybe there's some of that kind of data in the in the Astra training recipe. Science is a little bit unclear. Like Anthropic is clearly leaning a lot into biology and drug discovery. OpenAI maybe more on the model side, less on like we want to own more of the stack.
208. u208 42:34 There's like kind of related to embodied AI this whole like category of world models, whether that's you know, a video model with some action in it or like some other definition of world model, but but in general like Anthropic doesn't really do any of that or at least like from what we can see publicly. I mean, they don't even have an image generation model, for example.
209. u209 42:51 I think audio, pretty clearly, labs do have some of those capabilities, but you know, if 11 Labs can like keep accelerating the way it does, then clearly there's there's a need for a dedicated capability. I think those are probably the areas where I can see an independent lab like working out, but but ones that broadly say, "We're going to do frontier intelligence for knowledge work, but just like make it better, faster, cheaper." I think that's tough.
210. u210 43:15 I think for me like I'll believe it when I see it. Um but but yeah, this idea of like you open your computer, the app looks exactly the same, but it's just better. I don't I don't know if that
211. u211 43:25 >> Yeah. And
212. u212 43:26 World models: from robotics training to game engines
213. u213 43:26 in the world model labs, are you seeing anything exciting? I mean, obviously if FF Lee's lab was Yeah. acquired for a lot of money and that's unclear what happens from there given that the buyer is AMD. Yeah, cuz Yann LeCun started his company AMI in Paris at so world labs.
214. u214 43:45 >> Yeah. I mean, I I've you know, visibility into Odyssey where I was an early investor like doing world models and I think the the interesting thing in that business is like their most latest their latest system's like Odyssey 3 and there you can see that autoregressive model video data and then have it conditioned on actions such that you know that, "Okay, I'm taking one action and then I generate the next frame as a result of the action." That paradigm applied to robotic manipulation, to self-driving, to like video games, to like entertainment or something, can see generalization across all those different domains and already seeing that model being used for robotic manipulation training.
215. u215 44:30 So, I I think it'll at a broad level I'm like kind of interested how image, audio, video ends up all mushing together into basically a world model and then if that actually happens, then that could end up subsuming simulation and game engines.
216. u216 44:46 And and then that's very important for moving robotics into the real world, but also designing physical systems and and like hardware engineering. And to get back to like the AMD World Labs point, it's like if if that happens, then the compute substrate that serves those kinds of environments and allows you to train and run inference is super super important, and therefore having knowledge about how the world model research ecosystem is moving and what kind of hardware improvements you can make to be the number one choice is a big big revenue pie.
217. u217 45:17 That's I think what motivated part well, part of the reason that AMD bought World Labs. Um sort of in the same way that Nvidia over many many years was like very close to research labs that were doing shaders with 3D vision or other things, and you know, giving compute out and constantly getting feedback from researchers such that they always knew like my hardware is the best for where the where the kind of frontier is going.
218. u218 45:39 Would cheaper AI undermine the compute boom?
219. u219 45:39 >> This seems to be a good number of labs trying to figure out ways to be more efficient with compute. So, from flapping airplanes which I I I don't know much about to what seems to be the re-emergence of like diffusion model focused companies.
220. u220 45:54 >> Yeah. Yeah.
221. u221 45:56 >> To just reduce the cost of of inference, which is a fascinating fascinating thought by the way, which you know, people tend to talk about a bunch for a minute and then forget about, but like this massive investment in compute that we would do in theory seems to be at the mercy of like somebody coming up with a much more efficient way to train models.
222. u222 46:14 >> Yeah, yeah, yeah. I mean, I would contend that even if that were to happen, we would just see like more consumption and I mean, I'm just informed by a fact that I'm on my laptop using Astra high on 1.5x, and I would love that 1.5x to be like 10.5x.
223. u223 46:29 >> It's Jevons paradox of like
224. u224 46:30 >> Yeah, just like keep giving me more chips, yeah. I'm just like doing more things than I I could have done before with the same amount of time. So, I'm not necessarily sure that like, you know, one super efficient system will sort of like pop all the investments that we're making.
225. u225 46:43 Will better models absorb the agent harness?
226. u226 46:43 >> In your report, you talk about the important intersection between models and and harnesses, which seems to be one of those topics that like keeps shifting. People say, well, no, the intelligence is all in the model, and then other people say no, it's all in the harness and that seems to be going back and forth.
227. u227 47:01 What's your take?
228. u228 47:02 >> I think my broad take is like it's a bit like when you're injured and you have crutches, you need to have assistance and then at some point you heal and you don't need the assistance anymore. In absence of having like a frontier model, like you need to give it crutches and those crutches can be like access to tools, like how it handles memory, what data sources it has access to and then maybe some recipes in order to accomplish a specific task.
229. u229 47:25 And I I think for the moment, like at the non-frontier that that'll hold. But I'll take the word for I'll take the sort of frontier lab folks word for it that you know, over time one should assume that all these crutches kind of get subsumed into the model through training or through the harness that the model vendor provides.
230. u230 47:43 I think there's there's a data point around how much of the cloud code system prompt was rewritten over time as the model itself got better and it's something like 80% of the system prompt was just removed. The harness is also important because um the one that shipped by OpenAI and Anthropic is kind of like not incentivized to make you use a less powerful model.

     

231. u231 48:07 Like just in the UI and the in the product, like if I'm doing a legal task and I have the $200 plan and I have the tokens, like I want the quality to be super good. So why would I risk the quality being like not super good by sliding it from high to low or by trying it on a lower quality lower quality model?
232. u232 48:24 And so I think unless there's some some sort of like product UI behavior that is altered to disincentivize the user from only using the most expensive tool, the frontier labs will continue doing that. Um and so that's maybe where like getting getting good knowledge around like, okay, can use a less good system but compensate for it by having a better harness can come into play, but I think that's more of like a product a product decision.
233. u233 48:53 Also for companies that that are not like the frontier labs.
234. u234 48:56 >> Is that interesting for you as an investor all the kind of vertical harness companies or horizontal harness companies, is that an investable theme?
235. u235 49:05 >> Yeah, I think in isolation like no at the moment. Again, I can change my mind, but at the moment no, but I do think for for my companies that are you know, heavily like AI based or heavily agent based, when we see that, you know, every month like token expenditure because is going up through the roof cuz employees are increasingly getting leverage from these tools and and product usage is going up, like that's where I think the trade ends up being like hey, we should really like spend time on seeing can we use like a less frontier model and like compensate for it with a better harness so we could just like not burn a lot of money.
236. u236 49:40 >> Yeah.
237. u237 49:41 AI due diligence: 12 expert interviews in 2.5 days
238. u238 49:41 By the way, you you were a solo GP and I think I read somewhere that you use a lot of agents in your investing activity to do everything you do. Any example or anything that you thought was pretty cool in the way you use agents throughout this year?

     

239. u239 49:59 >> The coolest use case I had recently is on like expert research where you know, I think you you you can well, if you're sort of doing the cheap man's version, it's like you ask your friends,
240. u240 50:09 >> [laughter]
241. u241 50:09 >> but if you want to spend a bit of budget and ask beyond your network, then you're going to ask like GLG or AlphaSense etc. And they will go find humans for you. They will introduce you. You take the call. It's a ping-ponging, like really annoying and limited by language, time zone etc. But there's like a new product I started using called Latto which is basically like replacing uh the fact that I have to do the work with a model doing the work.
242. u242 50:33 And so I'm already like heavily using in this case Codex, but I kind of switch depending on what model is better at the time to just like just process all my ideas and the context I have on other companies that I looked at over history, how they performed, what questions I asked, what responses I got to just like figure out what is the highest value question I can ask of a team to inform like whether I should spend time or make the investment or not.

     

243. u243 50:57 And so I can take all that information and then and then with Latto like construct an expert interview and then decide on cohorts I want and then just hit launch study. And then the the system goes out and finds people, consents them to see if they're you know, the right fit for the study and then they go speak to an AI.
244. u244 51:16 And I was I was like shocked because you know, I hit launch study, I went to sleep, I had like 3 days to basically like give the company an answer. It's like in competitive markets, so I think the nuances of customer research is important. You know, I wake up the first day, I have three interviews from like pretty senior people. I have the raw transcript.
245. u245 51:32 The transcript can come back into my model. Answer the open questions we had. We me and my model. And then and then figure out, okay, we have enough data points for this, but like it's kind of opened up this question stream that we should ask. Maybe we should actually retune some of the interview questions to really coax the person to answer these questions.
246. u246 51:49 So it like updates the study questions and relaunch this the survey. Next day five five people. Again, this repeats. And over like two and a half days I had 12 in 12 interviews over like 14 hours across like US, India, Europe, Germany, France.
247. u247 52:05 It like it would be physically impossible for me to have spent like 14 hours across 12 people. Like There were several people who spoke to this model like for an hour and 20 minutes.
248. u248 52:14 >> Which it just in itself is fascinating, right? Like the fact that humans feel so comfortable so quickly speaking to an AI.
249. u249 52:22 >> Yeah. Yeah. It's probably a bit recursive in the sense that like we're all in front of a computer all the time and like these are generally like tech questions. And so I think people prefer to unfortunately prefer to speak to their phone than speak to a human.
250. u250 52:36 >> That's that's a fascinating thought that actually speaking to an AI is what people prefer.
251. u251 52:42 >> Yeah. Which is actually a theme that we heard over the years as like you know, we we maybe not the years, but the months that people are starting to prefer to talk to an AI customer service agent over a person.
252. u252 52:55 >> Yeah, because I think in that case it's like a human is pretending to be a robot and doing a really bad job at it because you're a human. You're like, can you please just like not talk like a robot?
253. u253 53:04 >> [laughter]
254. u254 53:05 >> Now you get a robot that's actually like really good and then the human can
255. u255 53:08 >> can talk like a human.
256. u256 53:09 >> Yeah. Okay.
257. u257 53:10 >> Amazing. Yeah, yeah. So that I think that that kind of idea of like automated customer research has been like very powerful for me and I can just see how, you know, even if you had a portfolio, you would want to run like these kind of agent interviews on your portfolio companies and how customers are using it. So then at least you get some first-party data when you go into the boardroom, you know?
258. u258 53:29 Or if you're an LP and like your investors are saying like, "Oh, this company's really good." You're like, "Is it really good?"
259. u259 53:34 >> [laughter]
260. u260 53:35 The trust problem with personal AI agents
261. u261 53:35 >> A big thing that happened this year in the last few months in in particular is to this point like the rise of personal agents, the Instincts [clears throat] and the Mews and the Rock Bots. What do you make of that?
262. u262 53:49 >> I don't give them my data. I really I really trust trust them at the moment, but I think a lot of people do. Um
263. u263 53:58 >> A lot of people do.
264. u264 53:59 >> Yeah.
265. u265 54:01 >> So you what you're saying is that you don't want to give your information to a company that didn't exist uh uh 2 months ago and doesn't require you to authenticate.
266. u266 54:10 >> Yeah, yeah, pretty much. Well, also because I feel like I can do roughly the same thing with ChatGPT. Like I don't actually see like what I get on top.
267. u267 54:20 >> Codex and Claude already have like low-level access to my computer, has access to my browser, to like various messaging apps and things like this.
268. u268 54:27 >> So I I I trust that already, so I don't really want to increase like the the risk on top of that. But but I do think it brings up an interesting point, which is like which which vendor do you trust the most for these kinds of tasks? And not that long ago I heard like Nikolai at Revolut is is now like thinking, "Okay, well, I I have like, you know, I don't know how many customers, but like millions or tens of millions or something.
269. u269 54:51 Um and they clearly trust me with my with my money and my credit card data. And you know, now they have like an AI assistant in there. And so I could sort of see myself trusting Revolut more to handle purchases on my behalf than I would like some third-party startup that's grown in 12 months and probably isn't have correct infosec and then other things.
270. u270 55:12 >> So you think it becomes a feature of broader platforms versus the opposite?
271. u271 55:16 >> I think that's that's one possible route. You know, whether the platform ends up buying one of these teams is, you know, that that that can always happen. But I heard, you know, another point would be somebody prefers to use like Meta's Muse because the likelihood that one of these startups survives for 10 years and if they do something bad that you can sue them and get your money or get some money is very low, whereas like suing Meta, like they have a a history of like paying you when there's some issues.
272. u272 55:42 So I've heard that's like a reason why some people prefer to use Meta. Like at least I know who to pick up the phone to.
273. u273 55:48 >> But but those those new personal agents seem to be playing a a big role in the diffusion of AI though, which is one of the themes that you highlight in the report as well. You talk about the increasing rise of uh code acts in particular beyond coders.
274. u274 56:06 >> Yeah. Yeah. Yeah, exactly. And I I think the the biggest message there would be, you know, at the same time you see this like recursive self-improvement, you see people doing all sorts of crazy stuff with their AIs and that's like a 0.1% or something of the AI usage base and the vast majority of other people are barely paying
275. u275 56:22 Why prompt engineering put people off AI
276. u276 56:22 for any AI and so getting like a very subpar experience. Uh and part of this is just like like bad product, bad marketing. Also just that, you know, a year ago or two years ago there was all this like narrative around you need to become a prompt engineer.
277. u277 56:38 Like you really need to understand how to write these prompts because if you write crappy prompts, the model does a bad job. And so I can only imagine how many people who don't work in technology saw that and were like, I have no idea how to learn this. Like this is way too complicated, you know, it's kind of like writing code but in like human human language.
278. u278 56:54 Like screw this. Like I'm done. Like AI sucks. And and now it's like, do you really need to be a prompt engineer? Like I think you more just need to be good at just like talking to your phone. Just like brain dump exactly what you Well, like everything you're thinking of and the system can organize itself. So I think this like prompt engineer thing is kind of gone.
279. u279 57:10 Like the model that can has enough affordances to not need that. But so now it's much more easy for a normal person to use. But the normal person got sick of it during the prompt engineering days and hasn't come back. So, you're right that like creating these on-ramps with like personal agents would be a great way to get people back into the system and and and you know, maybe maybe that's what they understand AI to be no longer like chat.
280. u280 57:33 >> And to to unpack a couple of things that you that you just said were related to this this idea that there is an increasing gap between people using AI and non-AI and you have a a slide that talks about how Codex adoption remains far far higher inside of OpenAI than amongst external users.
281. u281 57:53 Which is wild.
282. u282 57:53 >> Yeah.
283. u283 57:54 >> Yeah. I mean the the the broad stat is like something like 100% of people in OpenAI use Codex and then of their organizations that use it, it's something like 2/3 maybe. Um something quite high.
284. u284 58:07 >> Yeah, 16.9%.
285. u285 58:08 >> Yeah, okay.
286. u286 58:09 >> A separate analysis of S&P 500 companies AI disclosure found that 69% reported live deployments, but only 2% disclosed metrics tracked over time. And that's in contrast to 97.9% of active OpenAI workers use Codex in the preceding 28 days.
287. u287 58:25 >> Yeah, yeah, yeah. And individuals is like less than 1% use Codex. And then of the people who do use Codex, the vast majority of them is like 85% do tasks on it that would roughly take like 30 minutes. And only like 25% I think do it do tasks that would take like 8 hours or more.
288. u288 58:40 >> So, the conclusion of this is that the the people that get it are just accelerated further and further, whereas the people that don't comparatively slow down.
289. u289 58:51 >> Yeah, it's basically like you're you're kind of going through like a theme park or something and you just you discovered like level one and there's like this entire like other world that you just haven't even opened the door to.
290. u290 59:00 >> And there are a bunch of people that are already
291. u291 59:02 Does AI need its own Genius Bar?
292. u292 59:02 in that world.
293. u293 59:02 >> is like 97% of the world or something. Which is which is insane. So, that that's where I was like thinking, okay, there should be like a genius bar for AI or like a geek squad for AI or something, you know? Like I find it shocking that, you know, we're we're in New York, like we have amazing Apple Store, like all these concept stores for brands.
294. u294 59:18 Like why does OpenAI not have like an OpenAI store?
295. u295 59:21 >> And what would you do in the OpenAI store? You'd walk to a genius person and they would walk you through use cases?
296. u296 59:27 >> Pretty much. I mean, uh it would take time, but I think, you know, people end up showing other people, you know, with these kinds of viral products. Um but I could like, you know, very easily see like you'd run sessions on, uh you know, different levels of ex experience and like I don't know, maybe maybe one of them is for like small medium medium business owners, maybe some of them are for like, you know, busy families, like other ones are for like health care providers, like you know, anything you can imagine, but where you're really walking through people walking people through what you can do by just like flicking the screen and going on work or going on Codex.
297. u297 1:00:00 And I think that would make a a big difference cuz it would it would make it would make the technology much more empowering and less about like, "Wow, there's a lot of people out there that are doing things with this that I just don't understand. And then I don't have like the the time in the day or or the volition or like patience to go learn it myself." And maybe it's it's a bit like, uh you know, everybody should do exercise, but like most people don't and you need to hire a PT to just go do it.
298. u298 1:00:24 >> Mhm. Mhm.
299. u299 1:00:25 Codex beyond coding: agents for knowledge work
300. u300 1:00:25 And uh to finish the thought on the other part of of Codex, you have a slide where you say that non-developers are growing usage of Codex faster than developers are. Uh and from August '25 to June '26, individual non-developer users increased 137-fold.
301. u301 1:00:43 So, those are no longer really coding agents. Uh they are agents that code, but for like a broader group much beyond coders.
302. u302 1:00:52 >> Yeah, it's kind of unclear because, um like they could write code to go execute your knowledge work, but the individual who's using the system doesn't doesn't really see it. So, I'd say I'd say that that result is a mix of really simple things like Google Drive connector that actually works.
303. u303 1:01:09 Like it's kind of that's the the default, uh you know, Google Drive connector, like, for example, can't do a lot of things that you would want it to do, and you have to go to this third-party like GW CLI, which was written by Google engineer who got fired for doing it. But, like, if you don't use the CLI version, like, uh your agent has like really, really a high level to Google Drive have access to Google Drive, which precludes many, many workflows.
304. u304 1:01:34 So, just basic good integrations. And then the second thing is this like recursive cycle of uh models uh labs releasing a model, data companies or companies that have a certain workflow that they're trying to implement the model uh on showing that on their internal benchmark said model doesn't do very well, but they have the environments and the data to make it do really well, and then the lab realizes hopefully that they want to hill climb on this capability, and so they buy the data, improve the model performance, and then this recursive cycle continues.
305. u305 1:02:05 And uh I think ever since like the launch of Codex, and then uh and uh before that actually like a cloud code with those connectors for legal, for marketing, etc. That was kind of this approach of like let's build the integrations to the tools that the model needs to be good at that kind of human like white-collar work.
306. u306 1:02:24 Uh let's do hill climbing on certain workflows that are like important for finance, legal, marketing. And then um and then like let's release those those products with a bit better marketing, a bit better case studies uh starting with our own staff. And then there's been this constant like debate of like do we call it work?
307. u307 1:02:41 Do we call it code? Which lighter should it be? And and I think you know, we still haven't really figured it all out, but but now I would say like it's it's like so powerful.
308. u308 1:02:49 Why enterprises want to own their AI, not rent it
309. u309 1:02:49 >> And uh speaking of hill climbing, we we talked about open source, we talked about the big labs. Um it's um we we talked about sovereign AI as well. It seems like there is this uh emerging theme around uh enterprises, and before them probably the vertical AI startups taking control over uh their AI versus solely depending on Anthropic or OpenAI.
310. u310 1:03:13 Is that um an important theme for you, and something that is investable.
311. u311 1:03:19 >> Yeah, well, it's um um I think that that trend has been growing, and it's a mix of uh costs uh optimization, um like sort of data sovereignty, data residency uh issues, so not wanting to like put all this information to a third-party system, which you may or may not trust the privacy policies of.
312. u312 1:03:38 And then three, uh having a certain like set of use cases that are the real ones you care about, and that you just kind of like don't want to wait for the model lab to start caring about them to get better. Uh and then maybe last one would be uh yeah, some of these um like more nationalistic tendencies, I guess.
313. u313 1:03:57 I think in that in that area, uh what I'm what I'm wondering is like who's best placed to deliver that service. Of course, like one of the biggest ones would be Palantir, which which done partnerships with like neo clouds, uh and has all the data infrastructure already, and so is like well-positioned over the last few years to like start providing these services.
314. u314 1:04:16 That probably works in US commercial, and not so much in Europe. There are like sort of like these neo Palantir style companies in Europe that are like, you know, started up um to go do the the exact same thing, but with a different like nationalistic flavor. Then the third bit would be like, what about the inference companies um that have been serving open source models, and that maybe are well-positioned to not just serve, but also provide tools to look at the traces of how their models how these models are being used in an enterprise, use the the data of uh these conversations to then provide like a post-training service to close the loop.
315. u315 1:04:51 Maybe they're like well-positioned, or maybe it's like the neo clouds themselves, who have started to build inference businesses on top, that would be well-positioned, too. Or or maybe it's even like the data companies um that have not just or who have started to move outside of providing uh our environments for labs, but saying, well, your company is an uh an environment, and so uh you know, we've proved that we can do it well for labs, maybe we can do it well for
316. u316 1:05:17 Rapid fire: home robots, NVIDIA returns and Arm
317. u317 1:05:17 you.
318. u318 1:05:17 >> Do you want to finish with uh a rapid fire?
319. u319 1:05:20 >> Let's go.
320. u320 1:05:21 >> Most overrated trend of 2026?
321. u321 1:05:24 >> Uh home cleaning robots.
322. u322 1:05:28 >> Those don't work.
323. u323 1:05:30 >> No.
324. u324 1:05:31 >> Why is it that uh everybody uh wants that specific use case of like folding laundry? Is that Is that because it's uh it's a tractable one?
325. u325 1:05:40 >> I don't know. It's It's It's a bit like, you know, the the the demo of like agents booking flights. I I I heard like very recently that now like San Francisco is like starting to say like, "Can you show me a demo of like not booking flights?" I'm like, "Thank god I've been saying this for like 10 years."
326. u326 1:05:55 >> All right. I said rapid fire and then ask you a follow-on question. So, um uh most underrated slide in your own report?
327. u327 1:06:03 >> The amount of money that you would have made investing in Nvidia instead of investing in alternative semi-conductor companies.
328. u328 1:06:11 >> Uh and and back that quickly.
329. u329 1:06:13 >> If you had invested all the money that went into alternative semi-conductor companies into Nvidia on the dates that those rounds were announced, you'd have made four times more money.
330. u330 1:06:22 >> Yeah, and then to close uh the prediction from the past eight reports that you are the most proud of.
331. u331 1:06:29 >> Oh, uh that Nvidia would not conclude the acquisition of ARM. That was a That was a big one back then. I mean, people thought it was really going to happen and like we took the view that like for geopolitical reasons and competition reasons that it would not happen and to some degree like ARM shareholders made a crap ton of money by not closing that acquisition.
332. u332 1:06:49 To another degree Nvidia would have had like the most impressive CPU business that could have been really accretive to their uh to their cloud uh particularly as like the rise of agents and RL and everything else that actually needs uh that's kind of CPU bound. And uh and I think they could have also like shook up a bit of a bit of the geopolitical dynamics, too.
333. u333 1:07:07 >> Okay. Wonderful. That's a great one. Uh Nathan, thank you so much for coming back. And uh thank you for doing again this state of AI report, which is an extraordinary amount of work that uh you and hopefully your agents uh now do every year.
334. u334 1:07:25 It's available at uh stateof.ai and you have a you have a video that accompanies it. That's also a good resource. And [snorts] people can find you on Twitter.
335. u335 1:07:38 >> Yep. Yep. @nathanbenaich or @stateofai. Um and uh yeah, thank you so much for the opportunity. It's always a pleasure and uh I mean, it's bananas what's happened in the last year alone, so let's see uh Asia next year or one of New York's.
336. u336 1:07:52 >> [laughter]
337. u337 1:07:52 >> Wonderful. Thanks, Nathan.
338. u338 1:07:54 >> Thanks.
339. u339 1:07:55 >> Hi, it's Matt Turk again. Thanks for listening to this episode of the Matt Podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests.
340. u340 1:08:13 Thanks and see you at the next episode.