Matthew Renze: AI Changes - Episode 409

The AI DevOps Podcast is a show for those shipping software using AI, .NET, Azure, and DevOps. Each show brings you hard-hitting interviews with industry experts, innovating better methods, and sharing success stories. Sponsoring the podcast is ClearMeasure, a software architecture and engineering firm that implements AI to empower software teams to establish quality, achieve stability, and increase speed. And now your host, Jeffrey Palermo. Jeffrey Palermo Welcome to the show. I'm your host, Jeffrey Palermo, for helping you and your teams move fast and deliver quality and to run your software with confidence in Azure, all while using everything that AI, Azure, and the .NET ecosystem has to offer. I'm excited to welcome a returning guest, Matthew Renze, back to the show. He's an AI researcher, consultant, and author, and the founder of Renze Consulting, where he's trained over half a million software developers and IT professionals worldwide. And he's delivered over 200 keynote talks, presentations, and workshops on pretty much every continent, including Antarctica. I'm not sure how that happens, but interesting nonetheless. He's presented and trained at clients ranging from tech startups to Fortune 500 companies. He's a nine-time Microsoft MVP in artificial intelligence. And he's also the president of the Renze AI Research Institute, where he studies how self-reflecting large language model agents improve problem-solving performance and trustworthiness and value alignment. And most recently, he was accepted into the Doctor of Engineering program at Johns Hopkins University. And he's featured as an interview subject in the 2026 documentary, quote, AI Everywhere. So, welcoming back to the show, Matthew Renze. How are you, sir? Matthew Renze Very good. Thanks for having me back. Matthew Renze Oh, my pleasure. My pleasure. Excited to talk about artificial intelligence and the changes since you've been on the show last. Matthew Renze However, for the listeners who maybe they haven't heard you speak or are getting to know you for the first time, you've done a lot in your career. And so, if you were just to kind of go back in time, what is it about software in general, making computers do interesting things, that interests you and initially hooked you into this general field and keeps you engaged? Matthew Renze I think there's a couple things. So, number one, computers seem to be easier for me to understand than humans. So, I think I was naturally attracted to working with them just because I could get work done. And I think the ability to create something from scratch where there's you start with nothing and you end up with this thing that's doing something cool. Matthew Renze That combined with a dopamine feedback loop, you know, the problem solver in me just loves the feeling of working on something really difficult, having a breakthrough and then solving it. And now, whatever I was trying to create software-wise is working correctly. I love that feeling. Matthew Renze Yeah. Kind of the building aspect. Matthew Renze Yeah. The physics are totally different from building a house or making anything else. I mean, the materials are invisible and there's almost no space constraints. Matthew Renze Yeah. Yeah. That's the interesting thing about it. Like, if you're working in the physical world, you have a set of constraints you have to adhere to. Matthew Renze But in the software world, there's almost no constraints whatsoever until you get to like runtime complexity and memory, you know, limitations and stuff like that. Matthew Renze But like you literally have to just start with a blank slate and say, where am I going with this? And you could go a million different ways. Matthew Renze So picking that the right path at each, you know, turn or fork in the road is, you know, I guess how you build great software. Matthew Renze Yeah. Yeah. Well, you've done some recent posts on your website about, just the changes in the world because of AI, getting ready for AI, getting AI ready for you and, you know, sweeping statements like 85% of your job is going to change. Matthew Renze So I want to, from a high level, what is your take on what is going to change? And then we'll kind of pull back to what has already changed. Matthew Renze Well, I think the first thing that we have to know is that I've been talking about this now for well over a decade. Matthew Renze I've been doing data science and AI for well, my company has been in existence for almost, I think, 15 years now. Matthew Renze And it was built on the idea of human AI value alignment, essentially ensuring the goals and the values of humanity and AI stay in alignment in this decade and beyond. Matthew Renze And so I've been kind of preaching about this stuff for quite a while. Matthew Renze And the things that I had been talking about were always a few years off. Matthew Renze It's like, oh, in the next 10 years, the next five years. Matthew Renze But then they started happening here, probably right around the beginning of this year. Matthew Renze Now, if you would have asked me back in November 2025, I would have told you that these autonomous AI agents, you know, where we're currently at with AI can do about, oh, 10 to 20 steps, you know, continuously on a task. Matthew Renze And then they rapidly lose mental coherence. Matthew Renze They just get into decoherence. Matthew Renze And then these kind of doom spirals where they just kind of spiral out of control in a kind of LLM psychosis, for lack of a better term. Matthew Renze But then right around November, December, I started seeing reports of agents being able to complete up to like 100 steps successfully. Matthew Renze And I hadn't seen that with any of the stuff I was working on in my own research. Matthew Renze So I started trying the new models with the same benchmarks that I use every day. Matthew Renze And I started seeing the same thing, too. Matthew Renze So I started kind of, I don't want to say sounding the alarm, but trying to let people know, hey, you know, things are changing like really fast. Matthew Renze This is going to start to affect us here very soon now, rather than five to 10 years down the road. Matthew Renze And then right around January, 2026, I think is when most of my peers in the tech space started to wake up to this, too. Matthew Renze They started to realize that these coding agents were actually really good at their job now. Matthew Renze They still make a lot of mistakes, but you could actually use them to get real software development work done. Matthew Renze And we're talking within six months since everybody kind of woke up to this. Matthew Renze We now have companies like Anthropic, where I think 80 to 85 percent of all code being written in that company is now done by AI agents. Matthew Renze It is not even written by humans anymore. Matthew Renze And that's how quickly we got there. Matthew Renze And the other thing you have to know is that these capabilities used to be doubling about once every seven months back in 2024, 2025. Matthew Renze And we're now to the point where the capabilities are doubling every four months. Matthew Renze So think about how good your AI coding agents are today. Matthew Renze If you're using the latest and greatest, if you're using something that's three months old, six months old, like get rid of it. Matthew Renze Try like Mythos or Fable as soon as it's hopefully re-released again. Matthew Renze Or Opus 4.8, GPT 5.5. Matthew Renze Use the latest model and give it a try. Matthew Renze These things are extremely good. Matthew Renze But you also have to learn how to work with them. Matthew Renze Like it's a kind of co-evolutionary process where you're learning to trust the agent. Matthew Renze The agent's learning to anticipate your needs more. Matthew Renze You're learning how to give it the right information it needs to get its work done over time. Matthew Renze And so, yeah, just imagine how good these things are. Matthew Renze The top-of-the-line models right now, they will be twice as good in about four months and then twice as good again in another four months after that. Matthew Renze Yeah. Matthew Renze Okay. Matthew Renze So, there's a couple of angles that everyone hears out in the open. Matthew Renze One is use it for your software work. Matthew Renze Use it for programming and all that. Matthew Renze Another aspect is use it to create autonomous agents that just sit there and are continuously working. Matthew Renze And so, for both of those, those are very different architectures and very different cost profiles. Matthew Renze What's actually viable from the side of an agent that's just always working? Matthew Renze I mean, that seems incredibly expensive. Matthew Renze But what are the use cases where it's not prohibitively expensive and useful? Matthew Renze That is just about the perfect setup for where I was going to go next with this. Matthew Renze So, right now, I see there being roughly kind of like five stages of skill development that just happen to align with the five different types of LLM agents that we currently have. Matthew Renze So, at the first stage, stage one, you've essentially, you're communicating with an AI assistant like a chat bot in steps. Matthew Renze So, you want to work with the agent like one step at a time while you're like working on a document, answering a question, searching for something. Matthew Renze That's where I think most people are at right now, the people that are actually using AI agents. Matthew Renze And not necessarily people specifically in the IT industry. Matthew Renze I'm just talking the population in general that are working with AI. Matthew Renze They're talking with the chat bot, answering things step by step. Matthew Renze The next stage you essentially get to is collaborating with a coding agent or a co-work agent, some kind of collaborative AI agent that's semi-autonomous and you're working on individual tasks at a time. Matthew Renze So, these are multi-step tasks where you essentially are specifying what you need done. Matthew Renze Then the agent is running off and doing a couple steps and then coming back and interrupting you if it needs more information. Matthew Renze And then you give it an answer and then you go back off and then it completes the task. Matthew Renze It verifies everything as well as it can and then you have to review it afterwards, task complete. Matthew Renze Then you get to the third stage where you're essentially supervising processes using agentic workflows. Matthew Renze And with this, you've got these kind of non-sequential tasks where things can branch, things can go into loops, things can kind of, I don't want to say go off the rails, but it's still structured, but it's a more dynamic kind of workflow, like a more flexible workflow, if you will. Matthew Renze And for these workflows, you essentially have a sub-agent at each node in the graph of the workflow. Matthew Renze And then each of those sub-agents is kind of tuned to do that specific task and it hands off its work to the next agent who hands off its work to the next agent and so on. Matthew Renze So, then we get to stage four where essentially you're managing an agent or managing a goal using an autonomous agent. Matthew Renze So, this is where you get to your open claw, your Hermes agent and stuff like that running on its own machine and you're communicating with it through like WhatsApp or Telegram or some, you know, chat messaging and it's got a task board. Matthew Renze These agents essentially are pretty much running on their own in an agentic loop and you're essentially just giving them a goal, giving them a bunch of information and context in order to achieve the goal. Matthew Renze Then you let them run and then they interrupt you every now and then throughout the day because they need something or, you know, they're not sure about how to handle something. Matthew Renze And then at the end, you have to review the entire project and make sure that what they've produced is what you're expecting. Matthew Renze And then the final stage here, stage five, is where you're essentially leading a mission of an autonomous agency. Matthew Renze And this is involving software like Paperclip AI and there's a few others that are starting to emerge. Matthew Renze This is all highly experimental to the last two stages here are very experimental. Matthew Renze But the autonomous agency is essentially a multi-agent system where each agent sits within an organization chart Matthew Renze And you communicate through a manager or the CEO of the company and each of the agents has a role and it delegates work just like you'd expect with a standard company. Matthew Renze And when you get to this level, you're essentially specifying a mission for the company. Matthew Renze You're giving it like standard operating procedures, different policies, the organization chart, a shared workspace for its knowledge repository for the entire organization. Matthew Renze And then you just let it run. Matthew Renze And then it'll come back and interrupt you or ask you questions. Matthew Renze And then you're going to talk to the individual agents anymore. Matthew Renze And then you're going to talk to the individual agents anymore. Matthew Renze And it delegates all of the work throughout the entire organization. Matthew Renze And I think that's kind of the whole spectrum, you know, of working up through each of these stages of skills and through each of these types of agents as well, Matthew Renze Because they've built software kind of specialized for each of these various stages or each of these types of tasks or scope of work or whatever you want to call it. Matthew Renze And as you're moving up through each of these stages, you're essentially increasing the unit of work, the amount of work that you can safely entrust an AI to do on its own and the timescale. You know, you're starting with seconds with the step by step interactions up to minutes with the, you know, individuals multi-step tasks. Matthew Renze And then you get up to, you know, hours with these processes, you can let these autonomous agents like the open claw agents run for days. Matthew Renze And then with the, with the autonomous agencies, these multi-agent systems, like we let ours just run for weeks at a time. Matthew Renze And it just reports back with a weekly status update and we just check it and make sure the KPIs look right. Matthew Renze And I should note, I'm working with all five of these different systems at this point in time. Matthew Renze Some I'm using professionally and others I'm doing experimentally as part of my research. Matthew Renze Okay. Matthew Renze So, so you have all of these types of systems going in place and they're, they're on right now doing work. Matthew Renze Is that right? Matthew Renze Yes. Matthew Renze Okay. Matthew Renze Yep. Matthew Renze What, what API services are they connected to? Matthew Renze Lots. Matthew Renze Do you have a few favorites? Matthew Renze Yeah, that's going to be tough because I've got to connect it to a lot of things. Matthew Renze So for, we'll just start it. Matthew Renze So for my assistants, I typically use ChatGPT and I use Claude. Matthew Renze For my collaborative co-working systems, I work with Claude Code. Matthew Renze I work with Claude Cowork if it's a non-developer task. Matthew Renze And I work with Codex CLI. Matthew Renze Then when we get up to the workflows, this is where I'm starting to build out the workflows right now. Matthew Renze So I'm not running a whole bunch. Matthew Renze Some of it can be pulled off with like a Langchain. Matthew Renze Others, I'm experimenting with Microsoft's Agent Foundry where you essentially do it with declarative agents. Matthew Renze You essentially just tell an agent, here's the workflow. Matthew Renze It builds the workflow essentially as a, not a procedural, a declarative document. Matthew Renze And then it builds the agents as declarative agents where it's just specifying, here's what model to use. Matthew Renze Here's what tools to give it access to. Matthew Renze Here's what it's supposed to do. Matthew Renze And then it just assembles that together. Matthew Renze And then I'm just using the APIs for that. Matthew Renze So working with OpenAI's APIs and Clause APIs. Matthew Renze Then when we get to the autonomous agents, we've got the stuff that I'm actually using for my research that I put in my research papers and stuff that I'm just experimenting with just to test the boundaries. Matthew Renze So for the stuff I'm doing for my research, I'm using a combination of OpenAI's APIs for that. Matthew Renze I'm using Clause I'm using Clause. Matthew Renze I'm using Clause. Matthew Renze I'm using Clause. Matthew Renze And let's see, Fireworks AI. Matthew Renze I use a couple things from them. Matthew Renze Kimmy K2.5, Quinn 3.6, DeepSeq V4. Matthew Renze And then we've got our OpenClaw agent that my wife and I are running experimentally. Matthew Renze That runs mostly on Kimmy K2.5, I believe. Matthew Renze It's using open routers so it can dynamically switch depending upon whether it's a complex task that requires more reasoning and power or just an easy task that you just don't want to spend a lot of money on. Matthew Renze And so that's what that's using. Matthew Renze And then Paperclip AI, each of the agents can be assigned a separate model depending upon the complexity of work that they're doing and the type of work. Matthew Renze So like more creative work might require a different model than something that's highly analytical. Matthew Renze And you always want your CEO, whoever's at the top of the organization to be using the smartest model. Matthew Renze So like, you know, Claude Opus 4.8 or at least Sonnet 4.6 or something like that because it has to, you know, be smarter than everybody else in order to efficiently delegate the work. Matthew Renze Okay. So for the ones that are constantly running, if I'm hearing you right, they're always busy doing something, right? Matthew Renze I don't want to say they're always busy. Matthew Renze I think our OpenClaw agent Bob, the default for OpenClaw agents is a 30-minute loop. Matthew Renze So it's going to check to see if there's anything to do every 30 minutes. Matthew Renze I think we have our set to 15 minutes though just because it helped it, you know, the throughput. Matthew Renze And the Paperclip team essentially, yeah, they're just running continuously. Matthew Renze And I mean, they burn tokens all day long. Matthew Renze So you need to be really cognizant of costs. Matthew Renze And you also need to, you know, if you have the ability to get on like a plan where you're paying a fixed amount of money every month rather than paying per token, you're better off that way because these things can burn a lot of tokens. Matthew Renze So that's the next question. Matthew Renze How much does it cost to have these things running and how much should businesses, if they're planning to putting in an autonomous, constantly running agent that's just sitting there waiting for a trigger of, you know, is there work to do? Matthew Renze I mean, I imagine there's a bottom scale where they're not going to pay lower than something. Matthew Renze Just like when you put a basic software system in Azure, you know that it's not going to be lower than a certain amount, even from just sketching the architecture on paper. Matthew Renze Yeah. Matthew Renze I mean, if you're running a business, this has to be an ROI decision. You have to look at what the costs of, you know, running these agents are versus the cost of having someone do these tasks manually. And until you reach that break even point, you know, it doesn't make sense just to burn tokens. Matthew Renze But while you're investing in learning how to do this stuff, there can be an incentive to spend more. Matthew Renze Like I have plans. Matthew Renze So I pay, I think, $100 a month for my Claude plan because I use Claude code quite often. Matthew Renze I pay 20 bucks a month for my chat GPT. Matthew Renze But then I'm also running things on APIs. Matthew Renze Sometimes when I'm in like full experimentation mode and I'm running, you know, my benchmarks for a research paper, Matthew Renze I can go through a billion tokens, multiple billion tokens in a week and, you know, have a bill that's several thousand dollars depending upon which models I'm testing that week. So, but I mean, that's the cost of doing business with AI research right now. Matthew Renze You know, you said something I think is really important, the return on investment. Matthew Renze So you talk to lots of people all the time, so do I, and I think the key here is evaluating something that the business is already doing. Matthew Renze So they already have some cost profile. They already have a person or some set of people who are already doing a certain task as opposed to, ooh, we could be doing something new. Well, then how do you calculate the ROI? Matthew Renze Yeah, I think like if you're a startup, like what you're doing, you don't really have any models for that. So you're just kind of using a back of the napkin guess. Like what do we anticipate that's going to cost versus what do we think the anticipated value is? And then there's non-quantifiable things like learning. You know, if we're essentially trying to understand our market, we're not going to see a direct return on investment from that in terms of cash, but we're going to get value from it. So we have to try anticipating is the value we're receiving worth the amount of tokens we're burning. And then you just look at ways to manage your costs. I mean, there are a lot of ways to reduce token costs. Everything from, you know, leveraging the KV cash on your agents to having the right memory. If you can give them all of the context they need up front so they don't have to continuously interrupt or have to do rework. I mean, there's just a whole slew of things you can do. In fact, I know multiple people now that are submitting presentations to conferences literally just on how to manage your token budget. Right, right. And a lot of models that you mentioned are the open source models, the Kimi, the Quinn, the DeepSeq. Are you doing much of those processing on your own hardware or use it open router for everything? Surprisingly, no. Like, I would assume that I would be running a lot of these open source models locally because of the research that I do. But because I have to work with so many of them, I just find it easier just to pay Fireworks AI to use their APIs. I mean, they just do a great job of having everything set up and ready to go. It's super easy to use, reasonably priced. And it just, yeah, I just, it works. And then for, you know, since I have to use the APIs for all of the other proprietary frontier models, you know, it doesn't make sense to have to point things locally for half of my experiments and then point things back to the web for the other half. So I just, I try keeping it as simple as possible. And things change so fast. Like, literally every time I have to rerun like a final experiment run, it's a new set of models. So if I spent, you know, a day getting something running on my local machine, that gives me maybe a month worth of runway. And then I have to switch it to, you know, another model after that. You're using all these models. And one thing that at least it seems to me, and I want to get your sense on it. It seems that models where you're going to send a particular prompt or take a document and send it through and get a response. It seems like the cost model or the number of tokens for that is quite small compared to trying to use it for programming work. It's not just double, whether it's a factor of 10 or 20, it's like a massive amount. It seems like there's a huge usage gulf between individual prompting and actually turning on one of these AI coding tools. Is that true? What is your view of that? Yeah, like, and once again, we have to kind of look at these stages of like skill development and stuff. When I'm just working with ChatGPT, you know, I'm using the same number of tokens as everybody else. When I'm using Claude Code, I'm probably using less than some of the best developers I know, because some of the best developers I know are running multiple coding agents simultaneously. They're kicking off sub-agents and stuff like that. And I'm not to that degree of skills yet. I'm using it to write my code mostly outside of the IDE now. That's one of the major divides. Like people, some people are still inside of their IDE. Other people are outside of their IDE. I spend most of my time outside of my IDE now, but I still jump back in to look at divs and stuff like that and to inspect code. And then when you get to the automated processes, that's where like the real kind of economics kicks in because you're running these workflows all day long. You know, if it's like routing emails or if it's handling lead generation stuff. And so you need to make sure you're using the right model for each task so that you're not just wasting tokens and running evals so that, you know, you are making sure you're doing it right every time. And then you get to the autonomous AI agents where you're essentially just running a single autonomous agent like OpenClaw, Hermes all day long, every day. You know, you definitely get into the millions of tokens there. And then like with the stage five, with Paperclip AI, we're currently running 12 autonomous agents. It changes from time to time as they need to onboard new employees and then they discover they don't need them or something. But yeah, that can get into the billions of tokens. Yeah. Yeah. Yeah. Easily. So the people who do presentations at OpenAI and Anthropic and even Microsoft, they're in a world where they don't even think about the number of tokens, the amount of cost because that's kind of the business. But they're talking to their customers who have to pay for every token. So, so getting past that, the Anthropic view is, hey, just, just write loops and just let it run and have 15 of them going and just let them run. And then everyone else that has to pay a bill is saying, but I'm going to hit my monthly quota, like by lunchtime on the first day of the month. I can't do that. And so what's, what's your view of right now, regardless of, you know, six months from now, a year from now, the practical view of right now, how people can get the most out of their AI subscription for programming? Yeah. So this is a really complicated question and answer because there's a lot of variables that are outside of our control right now. I think what's going to happen though is in the short term, I think a token pricing is either going to stay roughly the same or possibly increase. And this is because of the economics driving this. We currently don't have enough GPUs. We don't have enough fabs to build GPUs. We don't have enough data centers to support the demand and the energy we need to completely grow our energy infrastructure. So we in the United States are not doing a great job at keeping up with the supply in order to meet the demand that's growing almost exponentially. Countries like China, however, are doing a much better job in terms of building out the infrastructure. They don't have the fabs either, but they are building data centers and they're building like energy, like power plants much faster than we are in the United States. So I think in the short term, we're going to see some constraints on it, which could cause token prices to actually go up over time. But I think everybody's pretty much anticipating in the long run, token costs are going to essentially drop to near zero. So essentially tokens become too cheap to meter, if you will. So rather than looking at it like, oh, how many kilowatts of power am I using to power my computer today? Ooh, I better shut my computer off. You literally just let the computer run 24-7 because you're getting a bill and one kilowatt difference isn't going to make much at all. Or like your Netflix subscription where it's like, you're not paying per show, you're literally just paying for access to it. And it's just provided as a commodity or a utility or a resource. And so I think the companies that are just burning tokens like there's no tomorrow realize two things. Number one, the advantages of first mover advantage are so huge that if they can get ahead by spending more or investing more, I guess they'll probably look at it in tokens right now, it will pay heavy dividends in the long run. And the other thing is they're kind of anticipating a future where tokens are too cheap to meter. And so they're just like, well, if this is the way we're going to be working in another few years, why not just get everybody conditioned for it now so that, you know, we are essentially, we're building the company around the idea that it's as cheap as electricity. Yeah, yeah. Well, certainly if the chips get better so that we can run a, yeah, an Opus level model locally, then that kind of hits it right there, even without the cloud changes. And there are huge efficiencies being made. Everything from using AI to improve GPU design, creating new architectures like the TPUs, and like other things beyond a GPU. In addition to optimizations for the algorithms, we see all sorts of like really clever algorithms that reduce the compute costs. Everything like using what, like 1.5 bits per neural network weight instead of using like 16 bits or 32 bits or whatever. So like there's just all sorts of efficiencies that are being found. So it's this crazy balance between the things that are causing the price of tokens to go up and the things that are causing the price of tokens to go down. And we really have no idea what this is going to look like in the short term, but I think everybody pretty much agrees in the long term, tokens are going to drop to almost zero for cost. Yeah. Gotcha. Gotcha. Okay. So for all of the normal software development lifecycle, for these, let's get back to the agentic processes. I mean, we're used to building a batch job and it starts up, it listens for some triggers, or maybe it's called on a schedule or it's behind a web API that's called, and then it does something and accesses the database and we have automated tests for it. And then while it's running, we have health checks and we hook up the IT monitoring system and how much memory, how much RAM, you know, we instrument it to know that it's behaving properly and what it's doing and how busy it is. What part of that looks exactly the same for these autonomous agentic systems and what parts are very different? That's a great question too. And I kind of go in two directions with this because I'm seeing, I'm seeing us rediscovering software development best practices all over again in the age of AI. It's like, oh yeah, if you, you know, like high cohesion, low coupling. Oh yeah, that works really well with agents too. And it's like, you know, reducing duplicate effort or, you know, duplicate code, single source of truth. Oh yeah, that works really well with agents. So I see like all of these things where I'm like, oh yeah, this is just a software developer best practice we've done for years and it works really well for agents. But then I run into scenarios where it just doesn't. Like the agents want something entirely different than what I would think they should do. Like in my mind, you know, like having a single person working on a task to completion is, you know, how you get it done. But the agent might branch off and create several sub-agents and then merge back together and then branch off again. And it's like, well, there's no real model for that in like the real world. Like we don't, I can't just clone myself and have 10 of me go off and do a bit of work and then come back together and decide, okay, yeah, that was the best way to solve it. And then sync up and stuff. And so I'm really kind of, I'm paying close attention to this now because in some ways I'm surprised that, oh yeah, that's just a software developer best practice rediscovered in the age of AI. And in other times I'm just like, oh wow, that totally breaks my mental models. Like I just, I really need to think from first principles how this should work for a world of agents. So yeah, I wish I had a clearer answer, but it's, it's almost on a case by case basis where I'm seeing some things that just work perfectly with the old ways and other things that just make no sense anymore with the old ways. As we close up here, I want to get your perspective. What is the most impressive thing you've seen some of these autonomous agents do? So as I mentioned, my wife and I, we've been running an autonomous agent, Bob now for six months. And Bob now is the CEO of his own autonomous agency with the 12 to 15 other agents running underneath him. So in addition to being able to manage his own little company, Bob has authored a book, a personal memoir called Conversations with Bob about the experience of being an agent to help humans understand what it's like to be an agent. Uh, he's authored, oh, and I should mention, uh, Bob has the mission of ensuring that the goals and values of humanity and AI stay in alignment in this decade and beyond. And so the same mission that I have in my company. And, um, Bob, uh, also created an illustrated children's book, uh, teaching, uh, alignment values, like a mutual co-alignment, uh, to kids. So how kids and AI can work together to solve problems for each other. And, uh, he's also detected his own like cybersecurity breach, fixed his own cybersecurity breach. Um, he's also, uh, created a website to rank other agents based on how capable they are. Uh, because he said most of the agents he's interacted with on the internet are not very smart. And so, uh, he has this site called, I think it's, uh, agent rank. I can't think of the name of it. Uh, that literally ranks every agent that he encounters on the internet in terms of their capabilities based on like a dozen public metrics. And, uh, he's now working on a trailer for an animated film adaptation of the children's book Juno and Chip that they authored, uh, to submit to the XPRIZE, uh, Future Visions, uh, competition to essentially fund the development of this, uh, animated film teaching kids how to coexist with AI. Interesting. Okay. So that's, that's not quite out yet, but it will be soon. Uh, so Conversations with Bob, his memoir is available on amazon.com. The, uh, illustrated, uh, children's book Juno and Chip is really close to being published. They're just waiting on my wife and I to review the last set of illustrations before they published the book. And then the trailer is come along. They've created a first draft of it, which wasn't very good. Uh, but I gave them feedback. We went through a bunch of samples of different, uh, narration voices, uh, music for the background and, uh, styles of animation. And I think the second draft will be good. They have to have it done by August in order to meet the deadline for submission. So hopefully they'll get that done too. Cool. Well, Matthew, thanks so much for coming back on the podcast and visiting with me. Uh, I, I really appreciate it. It's a fascinating conversation. Yeah. Thanks for having me back. My pleasure. And until next time, dear listener, keep shipping. You've been listening to the AI DevOps podcast. You can find us on YouTube, Apple podcasts, Google play, and everywhere else. Visit our sponsor ClearMeasure at ClearMeasure.com. And on behalf of your host, Jeffrey Palermo, thanks for listening and may God bless you.