← Back to search
Okay, this unleashed my agent
AI Jason · 2026-10-29 · 17 min
Show full episode description
Learn more about your ad choices. Visit megaphone.fm/adchoices
✨ Episode Outline — click any point to jump to it in the episode
Problem solved
How to build a self-evolving agent that gets smarter with use, by comparing the memory and skill mechanisms in
Claude Code,
OpenClaw, and
Hermes Agent.
Benefits
- Separates harness-optimization loops (Auto Agent/Auto Research) from in-context self-learning
- Hot/warm memory split keeps system prompt lean while loading detail on demand
- Background memory consolidation (Auto Dream) prevents stale memory polluting context
- Searchable memory plus conversation history makes the agent remember across sessions
- Hermes autonomously creates skills from trial-and-error work without blocking the main agent
Use cases
- Auto Agent self-evolved harnesses that hit #1 on a spreadsheet bench and #1 on a terminal bench
- Claude Code auto memory saves user, project, feedback, and reference memories under .claude with a memory.md index
- Hermes spawns a background skill-reviewer subagent after 10+ steps without a skill being created
- OpenClaw's memory search tool spans memory files and raw conversation history
KPIs / results
- Auto Agent: #1 on spreadsheet bench and #1 on terminal bench
- Hermes skill reviewer triggers after more than 10 agent steps without a new skill
- Three memory pillars: memory files, skills, searchable history
Tools / build
- Hermes Agent skill manager + memory reviewer
- Claude Code auto memory and Auto Dream consolidation
- OpenClaw memory search tool and bootstrap.md
- Auto Agent / Auto Research self-improvement loop (program.md)
- HubSpot AI agent cheat sheet
Thanks HubSpot for sponsoring this video. For the past few weeks, a massive amount of progress has been made for making your agents self-evolving. From its Auto Agent Project, which is a concept evolved from Andrew Copsi's auto research that utilizing CloudCode codecs to self-evolve an agent harness for a specific set of tasks. And it achieved number one on the spreadsheet branch and number one on terminal branch. As well as from CloudCode's leaked source code where people found this hidden auto dream feature that is getting CloudCode to autonomously extract learnings and best practice. From the conversation, to super popular Hermes agent that almost took its growth away from OpenClaw because it's an agent that remembers what it learns and gets more capable over time. The question is, what is the actual mechanism behind all sorts of self-learning projects? And what is state-of-art implementation that you can take away for your own agent building? And this is what I want to take you through today. What's the state-of-art way of building a self-evolving agent that gets smarter the more you use it? So first of all, you should actually break down those different projects into two groups. Auto Agent and Auto Research is actually a very different creature compared with the rest of those self-evolving agent setup. Where Auto Agent or Auto Research is a mechanism to improve the agent harness or software itself. Which means the goal of Auto Agent is produce an agent harness that can complete a specific type of task better. While Hermes agent, Auto Dream, and many other self-learning skills are really focusing on the in-context learning or memory output. So they are serving very different purpose. Like with Auto Agent or Auto Research, fundamentally it's this for loop that is running. Where a user will define a vision or PRD in a program.md file that clearly explains what this agent or model should do. And you will get the latest agent harness like Cloud Code or Codex to read this program.md and make improvements to the system itself. Which can be the agent harness runtime itself or especially model and script. Then the agent will run the evaluation to compare the performance of this new version against the baseline. And decide whether it should keep or discard the improvements. And repeat this loop infinitely. And the system wants it's produced rather than a mechanism that makes your existing agent to continuously learning. And to run this loop, you will actually have to have this database of the task and programmatic way to evaluate and verify the performance. Which in many cases, you probably don't have that large database or deterministic way to verify the work. And in a not so appropriate comparison, this Auto Agent approach is almost like training or fine-tuning the model. Because output is model or agent harness itself. But once produced, this harness or model is kind of frozen. Versus what harmless agents or Auto Dream or other self-learning skills introduce. Is this mechanism for in-context learning. Memory mechanism. To making sure agent actually remember its action and feedback. So that it can make a better judgment call the next time. Which means you get this agent that grows smarter the longer you use it. And this second batch is the part that is much more practically useful today. So Cloud Code, Open Cloud, or Hermes Agent. They all have their own different setup for the self-evolving part. And we're going to take you through implementation for each one of them. So at the end of it, you have a good understanding about the difference between each implementation. And also form a good understanding of what a state-of-art implementation look like. Achieve this type of in-context self-learning mechanism. But before we dive into this. One thing I think a lot of people get wrong with agents right now. Is that they assume more agentic is always better. But in reality, there's a spectrum. That there are different ways you can deliver a large language model-based system. From just one single large language model call. To workflow-based system like chaining steps together. Just like how you do in Zapier and NAN. And on the other hand, you have fully agentic system. That can make decisions, generate skills, and evolve over time. And so FASQD builder don't always go for fully agentic system. Because it costs more token and can be slower. In fact, you choose the right architecture and setup based on the use case. Sometimes you want something deterministic and predictable. And other times you want something more flexible and adaptive. And that's why this AI agent cheat sheet from HubSpot is actually really useful. It covers the fundamental of different agentic systems. It breaks down and compares different production large language model systems. How they are architected. What they are good at. And what type of use case suits the best. So you can decide what kind of system actually fits your use case. So if you're building agents, it's a solid reference for you to learn and think through the architecturalizations. I put the link in the description below so you can download for free. And thanks again to HubSpot for sponsoring this video. Now let's get back to the right harness setup for in-contact self-learning. So at high level, to making sure agent actually continuously learn from its own action or feedback. There are three main pillars where empower that, which normally contain the important effects like user.md or cloud.md file. And normally there will be a separation between the hot memory, which is something that always loaded into the system pump of the agent, versus one memory, things that will be loaded on demand. And second one is skill. And skill quite often contains the domain knowledge for agent to execute a very specific type of task. And third is a history, which logs the raw conversation thread, so the agent can refer back. And each agent harness, like cloud code, open cloud, or Hermes agent, all touch different parts of those three pillars. And let's firstly take a look at a cloud code. How they implemented this three-layer memory system that many people didn't know about. So when cloud code was just introduced, initially we only have this cloud.md file. And whatever in this md file will be fed into agent system prompt. And this is where most people started to put a lot of preference, additional guardrail to adjust agent's behavior. But the problem is that then this file very quickly became too bloated and too large. Then a common practice is that people just put index or table content about also different other files into cloud.md. With a description to agent when to read and update which file. And from this, people already built this type of hot and warm memory setup, where hot memory is something that always part of system prompt. And warm memory is something that will be loaded on demand. And this setup is kind of like 99% of how people are using cloud code today. But many people didn't know, cloud code actually evolved a lot and has this three-layer memory system in place already. And there's one article from Arton where he gave a very detailed breakdown of how the memory system works, which is very useful. So I highly recommend it to go check out. But at high level, cloud code already introduced this auto memory feature that you can turn on. This auto memory feature is basically an instruction to the agent to ask to achieve something similar of what some of you are already doing. Once it turned on, it has this special prompt as part of cloud code system prompt in terms of when to save memory and what type of things should be considered as worth saving. And then agent will start saving those different memory files into the .cloud folder for each individual project. And it has very specific structure that has this memory.md file that's considered as an index or table of content of all the memory files. And cloud code has this organization convention for different type of memories. It could be something related to the user or it could be a piece of feedback they give or related to certain projects as well as reference doc. So if you open your .cloud code folder, projects, your specific project, you might see a memory folder that has this memory.md file that just logs the table of content. And that specific memory file contains the main details. So the process is basically you talk to the cloud code and because the special prompt it has, if cloud code notices there's something that's worth remembering about user, project, feedback, it will try to create a file and index in memory.md file. And this memory.md file will be automatically loaded as part of system prompt to the agent so that it will know what are all different memory exist and read those files on demand. And it kind of works in many situations. But the problem here is that this system is purely prompt based, which means to make it work, you have to make sure agent remember to create and update those memories, which we know large learning models can easily forget and skip some steps. And that causes a problem because that means those memories get outdated very fast. And this outdated information can actually pollute the context and impact the performance negatively. That's why they introduced this auto dream feature. And this auto dream feature is something exposed during the cloud code source code exposure. So people realize this hidden feature called auto dream, which is memory consolidation. It's basically a background process that will be triggered after a certain session finish. And we'll start this new cloud code session with this special prompt to ask cloud code. Firstly, look at what's already in the memory. Then check the conversation history to see if there's any memory that is outdated. Then consolidate all the different memory as well as update the index. And this process will be triggered while your cloud code is not running. To gather all the sessions, read the memory, store results, and consolidate. So this is three layer memory system cloud code has. It evolved from just a single cloud.md file to auto extract memory system. And now a background, a sync process that will autonomously keep the memory update. And even though it is actually pretty simple, but it's kind of represented state-of-art setup for the memory itself. Which means you should have hot and warm memory, where hot memory is always loaded into the system prop, which normally include index or table content of other warm memories that can be loaded on demand. And then you give agent instructions about when and where to write those memories, as well as some sync process to automatically update it. But the limitation with cloud code setup is also they mainly have mechanism to handle those kinds of fax memory. But there are also very important pieces need to be filled in, like the skill, which is domain knowledge, as well as an auditable history. And even though cloud code does have skill feature and also does have the raw conversation log, but the conversation log, for example, is not searchable. It is there, but it is not designed for agent to search across because it doesn't really make sense in the coding agent context. And skill, even though it's supported, but it is more or less relying on human to find some skill and equip cloud code with it. And it's those gaps that make people feel open cloud is so much farther than other agent when people first try it. Because they put those memory as a first class citizen. So they have a list of more defined memory files, each represent a different aspect. And they also have a bootstrap.md file, which will instruct agent to chat and collect those information proactively from user. And they also have the daily log, provide high-level snapshot of interactions between human and agent. And most importantly, they have this memory search tool out of the box. And this memory search tool will search across all those memory files, as well as a raw conversation history. And that's what made OpenClaw feel like it just remembers things across all different sessions. And also another aspect is skills. OpenClaw agent has very specific instruction to tell the agent that you can use CloudHub to search more relevant skills. And it can add and remove update skills on the go. And when you look at OpenClaw setup, it's actually a very simple still. But they just designed a whole system to make sure this type of self-improving is the core of their agent harness. However, it also still has problems. So when you use OpenClaw, you will notice all those memory creations, skill creation, and memory search still requires human to prompt it properly. And there's no async or proactive process that is automatically updating those memories. And this is a gap that Hermes agent comes in and tries to solve it. And they basically introduced two concepts that really made the agent feel much better. One is autonomous skill generation. Another is memory reviewer. And autonomous skill creation is a crux of the system. So Hermes agent has this mechanism that is counting the number of steps agent is doing. And every time when agent run more than 10 steps without creating any skills, it will spin up its new subagent that will not block the main agent process, but add background, but review what has been done and decide if there's any useful skills that can be created to make this complex process more stable. And the problem of a skill reviewer agent basically looks something like this. Review the conversation above and consider saving or updating a skill if appropriate. And focus on what is a non-trivial approach used to complete a task that requires trial and error or changing course due to experimental findings along the way. And from there, agent will create skill in format like this. So the agent is equipped with this skill manager tool that allows them to create a new skill, patch or add an existing skill, delete one, or write and remove files from a skill. And also add a proactive prompt in the main agent saying when using a skill and finding it outdated, incomplete, or wrong, patch it immediately. Don't wait to be asked. Skills that are maintained became liabilities. It is this flow and system that made Herms' agent just feel so much smarter in terms of extract its learnings and do it better next time. And because it is giving agents ability to create skills itself, they also add this concept of safety scan, which means when agent tries to create a new skill, it will go through the skill guard Python file, where they define a whole bunch of rejects pattern. And once those are detected, it will automatically fail and delete the skill and also send a message back to agents so that it can know how to adjust the skill. If it's all good, then it will be saved. So this is the first one of autonomous skill generation. It basically has this autonomous process to making sure domain or procedural knowledge is autonomous saved and maintained. And on the other side, they're also doing the same thing for the general memory effects. So Outbox Hermes agent has these four main tiers of different memories. They have user.md file, which mainly contains who user is, preference style, workflow habits, as well as memory.md file, which contain the environment effects about the project conventions, operation systems. And those two are part of the system problem every time. Then they use skill for the domain knowledge that will be loaded on demand, as well as role history. So every single conversation history will be saved to this local SQLite DB. They can be searched and retrieved using session search. And if you need, they also have a way for you to plug into a semantic memory layer like Meme Zero or Honcho. The main part agent is managing, apart from a skill and the role composition history, are just these two files of memory.md and user.md. And each one of them have very strict character caps that in total is less than 4,000 characters. So you can see that they really try to push agent to just use skill as a way to maintain most of task knowledge. And they have similar type of async background process to extract memory. It is counting the number of agent turns. And only up 10 turns, if there's no memory extraction happened before, it will respond to a new memory reviewer agent. With this special problem, has the user review things about themselves, their persona, desire, preference. Has the user express expectation about how you should behave? If so, save them to those two files. So let's see how Hermes agent works. You basically have the hot memory that is autonomously extracted every 10 turns, as well as warm memory of all sorts of different skills. That is again, atomously extracted every time when there's more than 10 steps. As well as large code memory for conversation history and semantic long-term DB the agent can search. And after going through this, you can basically map out how the different agents works and understand why Hermes agent feels just smarter. Because it has those async autonomous process across skill and memory creation updates. As well as a way for agent to search role conversation log. And this is kind of like the state-of-art implementation for you to build any kind of in-context self-learning aspect for your agent too. You basically use skill for capture domain knowledge, memory for facts, and its searchable and audible role history. And ideally, you have async process so we don't rely on agent or human to extract and maintain a snapshot of knowledge. And if you're already using OpenClaw, you actually don't have to change to Hermes agent to get this type of really good self-learning experience. There are different skills on the market that are already available for you to plug in and enhance your OpenClaw or CloudCode memory and self-learning setup. And here are three skills that I tested and found a pretty novel approach. I put the table here that can take a look in the detail. But their setup is very similar to what we just discussed before. Just implementation-wise, each one has its own points and points. And the most popular one is this self-improving agent skill. They introduced a simple memory structure. Apart from OpenClaw's own memory, they have this .learnings folder with learnings, arrows, and feature requests.dmd file. And they have pretty smart use of hooks to making sure this memory creation and updates are more following. For example, they use this user prompt submit hook. So every time after you send a message, they will capture that and feed a small piece of prompt to just make sure agents follow this memory generation pattern. Then they also have this post-to-use hook. After every bash command, they will check the result from bash command to see if they match with any arrow pattern. If it does introduce arrows, they will again append an arrow-detected reminder prompt as part of tool result. And for OpenClaw, when it is bootstrapped, they also have this self-improving reminder.md file that is injected as part of the system prompt. So if you already have an agent that you use for a while, you don't have to suddenly change your agent to another one. So the migration from OpenClaw to Herms agent is actually pretty simple. They have just one command to migrate everything over. So this basically is the art of how teams are achieving in-contact self-learning agent behavior. And as you can see, it's actually surprisingly simple. So if you're building your own agent harness, I hope this is useful. Meanwhile, if you want to learn more, I also have a more detailed breakdown of different agent memory and harness setup. We have a step-by-step module in AI Builder Club, where we have a group of top AI builders who are launching agent products. And we have a weekly workshop where myself or other industry experts will come and share the latest tips and practical learnings. And we recently launched this new platform called Coolit that is also a self-improving agent, but monitor all the critical data across all your business, prioritize growth actions autonomously, and every day and every week, review the results so you can drive the growth autonomously. It just gives you a company website, connect all your business data sources and integrations. They will analyze across different data sources and build the organization memory and start taking actions autonomously across content, leads, ads, or any other growth operations. And most importantly, it remembers all the actions that ever took, so you can review and improve them next time. We're opening early access to members in AI Builder Club. So if you're interested, I'll put the link of both AI Builder Club and Coolit in the description below, so you can check out. Thank you, and I'll see you next time.