← Back to search

150 - Understanding Agents: Communication Channels

Prompt und Antwort · 2026-04-17 · 28 min
relevance 97 4498 words Episode page ↗ Audio ↗
Show full episode description
Willkommen zur 11. Folge der Serie "Agenten verstehen" im KI Gilde Podcast. In dieser Episode dreht sich alles um die Kommunikation mit autonomen KI-Agenten (wie dem Hermes-Agenten) und deren Anbindung an Alltags-Messenger wie WhatsApp, Slack oder Telegram . Wir beleuchten die wichtigsten technischen Hürden und eleganten Lösungen moderner Agenten-Systeme: Gateway-Architekturen: Wie eine "universelle Telefonzentrale" mit Simultübersetzer Agenten plattformübergreifend nutzbar macht und den Kontext über verschiedene Apps hinweg behält. Asynchrone Verarbeitung & WhatsApp: Wie das gnadenlose 20-Sekunden-Limit von Meta umgangen wird und Agenten komplexe Aufgaben im Hintergrund erledigen, ohne deinen Chat zu blockieren. Paralleles Arbeiten: Die clevere Delegierung von Teilaufgaben an stark spezialisierte, isolierte Sub-Agenten . Maximale Sicherheit: Warum KI-Agenten zwingend in gekapselten lokalen Docker-Sandboxes arbeiten müssen, um dein Hauptsystem durch rückstandslos löschbare virtuelle Maschinen zu schützen. Rekursives Lernen: Der Ausblick auf Agenten, die aus eigenen Analysen lernen, sich kontinuierlich selbst neue Werkzeuge programmieren und so unsere künftige Rolle als reine Auftraggeber in Frage stellen.
✨ Episode Outline — click any point to jump to it in the episode
Problem solved
How agents like Hermes connect to chat platforms and stay responsive while working long tasks.
Benefits
  • Gateway architecture normalizes every platform's messages
  • Cross-channel identity keeps one continuous context
  • Voice messages auto-transcribed into standard tasks
  • Asynchronous queues keep channels unblocked during long jobs
  • Reasoning-in-a-loop self-corrects without asking each step
Use cases
  • Start an analysis on Slack and resume it via Signal on the train
  • Send a voice message via Telegram, gateway transcribes it to text
  • Queue 'download 50 annual reports' and still send new tasks meanwhile
  • Run agent on WhatsApp via unofficial bridge with a disposable VoIP number
KPIs / results
  • Meta webhook demands a response within 20 seconds
  • Supports Telegram, Discord, Slack, Signal, Matrix, email, WhatsApp
  • 11th episode of the Understanding Agents series
Tools / build
  • Hermes Agent
  • Gateway architecture
  • WhatsApp web bridge
  • Message queues / async task processing
0:00 / 0:00
🌐 This transcript was automatically translated to English from the original.
This podcast is a project of the AI ​​Guild. The content and voices are generated with AI and are used for information and demonstration. Have fun listening! Welcome to the AI ​​Guild Podcast. Today we arrived at the 11th episode of our Friday series Understanding Agents. Today we’re taking a close look at how we actually communicate with our AI agents. Correct. We now have various agents available, such as the Hermes agent. And the really big difference to a classic chat is the choice of communication channels. Exactly. And today we'll discuss which channels are now standard and, above all, how a connection to external networks works, such as WhatsApp. This is a huge topic. And a very important question is whether you can send a task via a channel when the agent is currently working on something else. Or is the channel then blocked? We'll clarify all of this in detail. We also look at how autonomously and in parallel such an agent can work in an isolated environment, for example in Docker or on a dedicated computer. But before we get to the technology, imagine you've hired a brilliant new employee. Okay, I'm imagining it. You ask him to do an in-depth market analysis for you. He nods, turns around, starts working and after exactly 20 seconds he is summarily fired from your internal communications network. Simply because he didn't answer quickly enough. Of course that sounds completely absurd. But that's exactly the situation that happens when you try to connect modern agents to everyday chat platforms. Yes, because this employee I described doesn't fail because of his intelligence. Just can't handle it yet. We really have to completely abandon the idea of ​​how we have previously interacted with large language models. Okay, let's break this down. A classic language model is basically like an extremely clever lexicon. Yes, that's exactly it. You look something up, it generates a text for you and then it immediately falls asleep again, so to speak. It's just waiting. If you want the next step, you have to make the next entry. And if the model makes a mistake, it stops. Correct. It virtually shrugs its shoulders and waits for you to tell it how to correct this mistake. In system architecture we call this the stateless conversation mode. So a mode where people have full control over the process chain. Exactly. You are in control, but you also have to laboriously dictate every single step. But with a real autonomous agent, like the Hermes agent, we are talking about something completely different. It's about targeted action. This means I give him an overarching macro goal in the morning. So I say no more, look for the first competitor on a search engine and summarize the first paragraph. No, not at all. You phrase it much more freely. You say instead, research the pricing models of our three largest competitors in the European market. Compare the hidden items and create a well-founded table. And that's exactly the point at which this dictionary analogy completely collapses. After all, I don't tell my office employees how to open their web browser or what search term to type. Yes, that would be absurd too. You expect him to find the way to the goal himself. And technically, the agent does this through a concept called reasoning in a loop. Reasoning in a loop. OK. How exactly does this work in practice? So, the agent takes a huge macro goal and breaks it down into tiny, manageable sub-tasks completely independently. He then begins to complete these tasks step by step. He accesses a website, reads the text, compares it with his target. And what happens if an error occurs now? That's the crucial thing. If a website is not accessible or the information is behind a payment barrier, then it doesn't just stop. He doesn't come back to you crying. So he doesn't immediately ask me what he should do now. No. He analyzes the failure himself, he corrects his own strategy in real time and tries an alternative path. He really only turns to you when an absolutely essential piece of information is missing. Or if a security policy requires your explicit approval. Understand. Because this agent acts like an independent digital employee, it is actually completely absurd to lock it in an isolated browser window. Exactly. If he works independently, we want to reach him exactly where we spend half the day anyway. Even. We're talking about channels like Telegram, Discord, Slack, Signal, Matrix, good old email and of course WhatsApp. Basically, we want to be able to just send him a quick voice message in the morning while we're making coffee. That's the dream, yes. But the technical hurdle to get it exactly there is really immense. Imagine you are the developer of such a system. OK. If you had to write completely unique code for each of these platforms, taking into account all the quirks of Discord, the data structures of email, and the encryption of Signal, you would have created an unmaintainable nightmare. Because it's constantly changing, right? Exactly. Every time Telegram rolls out an update, your agent would completely collapse. That's why modern systems rely on a so-called gateway architecture. A gateway. In this case, it acts like a gigantic universal telephone exchange, right? It's right in the middle between me as the user and the agent's brain. A switchboard with a built-in simultaneous translator, I would say. The gateway accepts all messages from outside, regardless of their origin or format. It strips away all the platform-specific baggage. So when I send a voice message via Telegram, the gateway picks up this audio file. Correct. In the background, it forwards them to specialized speech recognition models, transcribes what is spoken into pure text and formats this text into a standardized system protocol. And only this clean task then goes to the agent. Exactly. In the end, the agent himself has no idea whether you were speaking or typing or which app you came from. What is fascinating here is the actual added value that is created by the memory of this headquarters. You mean because the gateway sits in the middle, it can capture the status of my conversation across platforms? Exactly that. The gateway manages the user’s identity across all channels. This means that in the morning on the way to work I can initiate a complex data analysis via Slack and in the afternoon, when I'm on the train, I simply write to the agent via Signal and ask what's next with this morning's analysis. Yes, exactly this continuity is the absolute key. It brings together the threads of Slack, Signal and Telegram into a single database. This is presented to the agent as a unified context. To the agent, you are always the same entity, no matter which door you enter the building through. That sounds incredibly elegantly solved. But no matter how well the gateway solves the translation problem, it doesn't solve the landlord's problem, I would say. You allude to closed systems. Correct. When we try to integrate this architecture into the Meta system, specifically into WhatsApp, we suddenly run into a solid concrete wall. Meta is known for maintaining an extremely closed ecosystem. Completely different from the open interfaces of Discord, for example. Oh yes. WhatsApp is truly the ultimate stress test for any agent developer. We are essentially moving in two completely separate worlds. Which are these? The first world is the official architecture over the cloud from Meta. This is the prescribed path that corporations must take. This architecture is based on event-driven signals, so-called webhooks. Webhooks. So like a digital bouncer that lets you know when something happens? Something like that, yes. The agent's system is actually idle. When you as a user write a message, Meta fires a wake-up signal to your server. Sounds efficient at first. It is, but now comes the massive problem. Meta dictates a merciless time frame. Your server has a maximum of 20 seconds to respond to this wake-up signal with a ready response. 20 seconds? That might be enough for a simple chatbot that somehow spits out a greeting phrase. But we're talking about an autonomous agent here? Exactly. An agent who searches the Internet, who draws conclusions and corrects errors. It takes minutes, maybe even half an hour for a complex market analysis. What happens when these 20 seconds have simply expired and the agent is still calculating? Then Meta registers a failure. The system assumes that your server has crashed. It will immediately terminate the connection and classify your platform as faulty. Cool. This means that the agent continues to work in the background, but I never receive the answer. Even worse. Meta may send the message over and over again because it thinks it hasn't been received. This often leads to your server completely collapsing under the load of constant restarts. This official route is effectively unusable for autonomous agents. And how do you solve that then? You can't just ignore WhatsApp. For exactly this reason, developers of systems like the Hermes Agent often resort to unofficial bridges. Wait, an unofficial bridge? But that sounds to me like a gigantic liability risk. That's what it is if you're not careful. I'm not going to tie my company's expensive system to an unofficial solution that probably violates all the terms of use. But isn't such an unofficial bridge against the rules? What about the risk of being banned? My number is blocked faster than the agent can say hello. Your skepticism is absolutely justified. I'll briefly explain to you how this technically works. Such bridges emulate a completely normal human session via WhatsApp's web interface. So the system acts as if it were a human at the browser. Exactly. You scan a QR code in your command line, just like you would in the browser on your laptop. It requires no complicated corporate account, no credit cards, no bureaucratic process. But the risk of a permanent ban from Meta remains astronomically high, doesn't it? Yes, it is if you don't design the architecture with extreme caution. So how do the developers specifically protect the system from being exposed? Through two very fundamental safety precautions. First, strict network isolation. Nobody in their right mind uses their private main number or the official company number for such agent bridges. So you get a disposable number? Exactly. You rent isolated, dedicated numbers, which are often generated via Voice over IP services. This is the first protection case. And the second? Second, and actually more critical, is the agent's behavioral pattern. The agent must be so extremely restricted by software that it only behaves in a conversational manner. Conversational means he only answers when asked. Correct. Under no circumstances should he send proactive messages to people who have not previously interacted with him. No mass messages, no unsolicited status updates. Because otherwise Meta would immediately recognize it as spam. Exactly. As soon as an agent initiates unsolicited conversations, Meta's algorithms immediately recognize this machine pattern. The spam filter then strikes mercilessly and the number is blocked immediately and irreversibly. Okay, that makes sense. But if we have now overcome this hurdle and the agent is actually running safely on the channel, then we land directly on the most pressing question of our consideration today. The blocking. Correct. Imagine I write a huge task to the agent via this WhatsApp connection. I say please download the annual reports of these 50 companies, read them in their entirety and extract all sustainability statements. That takes hours. Definitely, yes. What happens to my chat during this time? Is the window simply blocked for me? Can I add a second, urgent task to the agent in the meantime or is the system stuck in an eternal loading loop? This is a super important question. In a primitive, linearly programmed system the channel would now be completely dead. You would really have to wait until the report is ready. That would be a catastrophe for the workflow. Absolutely. But intelligent architectures avoid this bottleneck through asynchronous task processing and so-called message queues. The secret is simply that the system rigorously decouples the acceptance of your command from the actual execution of the work. Ah, this is where things get really interesting. It's a bit like being in an excellent restaurant. What do you mean? Well, the waiter, who is our gateway in this case, comes to your table and takes the order for an extremely elaborate seven-course menu. But he doesn't go to the kitchen himself afterwards, he stands at the stove and spreads the meat. True. In your field of vision. The communication channel, i.e. the table, is never blocked. This is a perfect illustration. Technically, this is exactly what happens. The gateway's main process saves your task in a database at lightning speed and immediately sends you a tiny, asynchronous confirmation of receipt to a phone. Something like, understood the task, processing has started. Exactly. And at that exact moment, the communication link between your phone and the server is closed cleanly. Meta is happy, the 20 seconds are far from exceeded and your chat is immediately free again. And the actual work? Deep in the background, a completely isolated work process grabs this piece of paper from the kitchen, i.e. from the database, and starts working. This is brilliant. But how do I stop him when he basically cooks the wrong dish? What do you mean specifically? Suppose I realize five minutes later that I wrote sustainability in my text, but actually meant profitability. The agent is now in the background analyzing 50 false documents. If I now text him in the chat, Stop, look for profitability, how does the system manage this conflict in real time? This leads us to the absolutely supreme discipline of system architecture. The system usually offers two different modes for this. The queue mode and the active interruption. Okay, let's start with the queue. Gentle queue mode is technically quite trivial. The agent continues to work calmly on sustainability. Your new message with the stop command is simply passed to the kitchen as the next piece of paper. This means that he only reads the piece of paper when he has finished with the 50 documents. Exactly. Once it's finished after hours, it reads your new command, discards the old result and starts all over again. This is extremely safe for the system because nothing crashes. But that costs me an incredible amount of time and computing power. That's exactly what I want to prevent. If I see him slipping into a complete hallucination, I want to be able to pull the emergency brake immediately. And that's exactly what interrupt mode is for. Technically, however, this mode is a real balancing act. If you send stop, the gateway must fire a hard abort signal to the running work process. Sounds dangerous. That's it too. The massive danger here is that if you simply kill the process, the agent will lose all of its short-term memory. In the second he forgets which of the 50 websites he has already read. Oh, that's bad. And how do you prevent that? To prevent this, the system must freeze the so-called execution graph. Execution graph? What is that exactly? This is basically the map of his thoughts and steps so far. The system stops the calculations at a precisely defined safe point. It takes all the variables that the agent is currently holding in memory, freezes them, and writes this millisecond-precise snapshot to disk. Craziness! So he makes a complete backup of his own brain on the fly. Exactly. It preserves its current state. Then the core system reads your new instructions regarding profitability, injects this new parameter directly into the frozen snapshot and restarts the agent process. And then he remembers what he did before. Yes! The agent wakes up, has retained his memory of the 20 documents he has already read, but from that second he knows that he has to completely change his focus to profitability. This is brilliant. This saves immense amounts of computing resources and, above all, prevents us from having to pay twice for expensive calls to a programming interface, i.e. external server queries. Absolutely. This is a huge economic factor. So if we can now safely stop and start the agent and the channel always remains free anyway, the question of scaling inevitably arises. You mean the amount of tasks? Exactly. If he's working in the background, why would he only do one thing? How autonomously and, above all, how parallel can such a system really work? Can I give him 100 commands at the same time? Whether the system can do 100 things at the same time depends largely on where the physical bottleneck lies. We have to differentiate between tasks that wait for external answers and those that really use up local computing power. Can you give an example? Clear. If your agent queries 50 websites, the system is hardly under load locally. It sends 50 requests out into the world via a programming interface and simply waits for external servers to respond. A well-written system can handle this for hundreds of processes almost simultaneously because the waiting itself does not cost any performance. OK. And the other case? The situation is different if the agent on your machine has to mathematically analyze gigantic amounts of data, for example when calculating huge tables. Then the architecture must be able to physically divide the processes between the different cores of your processor. So that's the hard limitation of the hardware. But what does it look like architecturally? The Hermes agent uses a so-called subagent architecture for such complex cases. So he doesn't handle complex things as a monolithic block, but delegates the work to himself. Right? Correct. This is the principle of isolation through delegation. If you try to load a single large agent into his working memory with ten completely different problems at the same time, he will lose track incredibly quickly. His context window is practically overflowing. Exactly. He then mixes information and begins to hallucinate. To get around this, the main agent creates completely isolated sub-agents for each individual sub-task. So he's basically cloning himself? He clones himself, yes. But he creates severely dumbed-down clones that come with a massive tool limitation. Dumbed-down clones? How should I imagine this? Suppose you have a gigantic document directory and the agent is supposed to find three specific logic errors in the program code. The main agent then creates three subagents. It assigns each of these clones its own tiny folder on the hard drive. So everyone gets their own little box. Exact? And now comes the kicker. Each sub-agent only gets exactly the tools that he absolutely needs for his small goal. They don't have access to the big, long chat history between you and the main agent. You don't even know the meta level. So they really just have total tunnel vision on their specific task. Correct. These three sub-agents now rummage through the files in their respective folders completely in parallel. And the crucial point is probably that after the work is done, they don't send all their data garbage, all their attempts, error messages and drafts back to the main agent. Surely you're just submitting a highly condensed summary, right? Something like bug number one found and fixed, here is the changed line. That's exactly how it works. This protects the focus and the work spokesman of the main agent extremely. Problem. But when we combine this with the bigger picture, we have to issue a huge warning. To what extent? Parallelism always sounds wonderful in theory, but in practice it is extremely dangerous. When you have tasks that are completely unrelated to each other, it's a dream. But what happens when these isolated sub-agents suddenly become dependent on each other? Yes, that's exactly the point. If subagent A needs subagent B's result to even continue, then you suddenly have a synchronization nightmare. Because one then has to wait for the other. Yes, the agents block each other. They wait for each other. And a tiny error in Agent B's data causes Agent A to crash completely. Real, efficient parallelism requires an outstanding system design that razor-sharply separates dependencies in advance. This leads me straight to the elephant in the room we have here. You just mentioned in passing that these sub-agents work completely autonomously on my computer. They write program code, they change files, they search for solutions on the Internet. Yes, they do. That sounds like the ultimate security risk to me. We allow an artificial intelligence that is based on probabilities and can hallucinate at any time to work freely on our operating system. This is a very legitimate objection. In the developer scene this is cynically called the YOLO mode. You only live once. Oh God. And developers who run unprotected agents directly on their main operating system are clearly playing Russian roulette. What's the worst that can happen? Well, an agent with the simple task of cleaning up disk space could, based on a logical misconception, generate the command line command that irretrievably formats your entire file system. Everything deleted just like that. Just like that. Even worse, it could inadvertently read a deeply hidden file containing your most sensitive, unencrypted passwords and transmit this data to an external server the next time you search. Blind trust is absolutely out of place here. But if I can't trust him, how do we tame the system without having to manually tell him every single thing to do? You want to keep the comfort. Exactly. If I have to proofread and approve every line of code that it wants to execute, I lose the entire speed advantage. Then I actually no longer need an autonomous agent at all. The technical answer to exactly this dilemma is local Docker sandboxes. A sandbox, i.e. a sandbox. Yes, a sandbox is, just as the German word sandkasten suggests, a strictly limited playground. When the agent starts, Docker uses tiny, extremely lightweight virtual machines in the background. So the system simulates a completely separate computer within the computer. Exactly. These virtual machines create a hard, insurmountable boundary at the hardware level of your computer. So the agent is assigned a room that is completely decoupled from my actual computer. Basically walls made of steel that he can't see through. He is completely isolated. The operating system the agent wakes up in shares absolutely nothing with your main system. The agent really has no idea your computer even exists. How does he even get his job then? From the outside, the gateway only passes exactly one folder through a tiny slot into this sandbox that the agent is supposed to process. So he doesn't see all my private documents? No. The agent doesn't see your company's network shares, it doesn't see any hidden system files, it has absolutely no administrator rights over your life, it's the absolute king in its little box and has maximum freedom of action to try, fail and run code on the command line. But the box itself is absolutely tight. This is brilliant. And what happens when it's finished? That's the most elegant aspect of it. As soon as the agent reports that the task is completed, this entire virtual machine is completely destroyed without leaving any residue within milliseconds. Everything is thrown away? Everything. There is no cash, no temporary file, nothing left. For the next task, the agent gets a brand new, pristine sandbox. In this way, you give it maximum autonomy without endangering your actual system for even a single second. This really calms my paranoia a lot. Let's summarize the broad lines of our consideration today. We have seen that autonomous agents like Hermes have moved beyond constantly waiting for human input. They really act like independent employees. That's what they do. They integrate unnoticed into our everyday communication channels through gateways. Whether that is Signal, Slack or, indirectly, WhatsApp. They don't block us from working thanks to asynchronous systems and queues. They delegate complex problems to highly constrained sub-agents while working completely securely in the encapsulated sandboxes of these virtual machines. So what does this all mean for us in the end? A whole lot. We are at a turning point where the invisible infrastructure around artificial intelligence, i.e. the communication channels and security barriers, has become just as important as the intelligence of the models themselves. Let us think one step further into the future. Today we analyzed the architecture of the Hermes agent in depth. A prominent feature of this particular system is its cyclical, recursive learning process. Recursive learning process? What does that mean for us? When Hermes has solved a highly complex task in the sandbox, he stops briefly. He then goes into a reflection phase, he analyzes his own logic steps, extracts the knowledge he has gained and, completely autonomously, programs himself a new, reusable tool in the form of program code. He saves this tool permanently. So he continually teaches himself new skills. Exactly. If these agents now act day after day through our everyday channels and program tools in the background that are becoming increasingly inscrutable and complex - tools whose intermediate steps we humans will soon no longer be able to cognitively understand, what will actually happen to our role? That is a frightening idea. In a few years, will we still be the rational clients of these systems that really control what is happening? Or will we inevitably degrade into purely passive recipients of finished results, without yet understanding the machine's path? That is the question we must ask ourselves. A really massive idea that will definitely keep us busy for a long time. That was our detailed analysis for this week. Thanks for listening and see you next time.