I Spent $1,300 on Grok Bot in 2 Days (And I'd Do It Again) | Lazy Leverage #126

You click on one of them and start chatting with it Yes, now very importantly though, one of the best primitives that Grok Bot is very good at is I only talk to my, actually mine's called Stalin because it tells people what to do That's why I named it Stalin Nice And I gave it a little image of Stalin, right? A little cartoon image of Stalin The place where I think, I'll tell you my creds on Grok Bot, which was I did the $300 a month subscription, I blew through it in about three days, I turned over to Jon And then the next time I looked, I'd spent $1,300 in two days Now, I made it, I posted the link on Twitter if anybody thinks I'm lying, but it's true It's on Twitter, you can see the screenshot Welcome, welcome, welcome Welcome to another edition of Lazy Leverage, the internet's number one small business operation podcast I'm Jon Matzner, joined by my heterosexual life partner, Peter Lohmann How are you doing today, Peter? I'm doing great, Jon, how are you today? I'm doing well, thank you I know we have both been traveling I took my first solo non-kids vacation with my wife And what she alleges is six years Which is why there's a bit of a gap We were both in Mexico last week, actually Yeah, coincidentally Yeah, coincidentally, did some other travel But I know we are piping hot To talk about our topic today Peter, you want to intro us? You want me to intro us? How do you want to get started here? Yeah, let me start us off So, everyone's talking about GrokBot And, I mean, so I'm active on Twitter again News alert I'm so undecided if this is like a permanent return to Twitter Or if this is more of a temporary thing They definitely fixed the algorithm It's way better than it was You can tell they're still tuning it Like, it'll go back and forth on showing me Like, the latest outrage Rage bait tweet On some, like, sunscreen thing Or if it's, like, some guy Who talks, talking about plumbing And it has, like, three likes Yeah So, like, it varies wildly But, so I'm back on Twitter And Grok, which is Elon Musk's AI Front, you know, model or whatever Which I've never messed with Grok very much at all No They launched something called GrokBot Which is kind of like If you took OpenClaw And you made it about a billion times easier To start, set up, and configure And get productive with Have you actually used it yet or no? I haven't used it yet Okay, alright Yeah, that's what I'm told That's how I make it That's what I imagine in my mind Yeah So, you know, OpenClaw You know, well, John and I both went crazy with it Earlier this year Yep We both bought Mac Minis And, like, got the whole thing set up And it was definitely a huge leap forward And it was very fun for a while But very unreliable Difficult to configure Stuff would break It constantly running into roadblocks Just very fussy Yep Even though the leap forward was very impressive You use fussy I use brittle Same Those are synonyms Brittle or fussy Same thing, yeah So these are, like, not chatbots These are agents that can go and do work for you Yep Which is, you know We're going to talk about that Yep Yeah We're definitely going to talk about that So here's my situation Everyone's now talking about GrokBot They say it's incredible Unbelievable Changes your life Fixes your marriage Makes your wife hot Makes you more money Your kids will become more well-behaved Yeah Etc But I, about three weeks ago Got going with Hermes Or, I don't know if that's how you say it H-E-R-M-E-S I don't know if they're pronouncing it like the French luxury brand Or if it's Hermes Yeah, Hermes or Hermes Whatever Which itself purported to be a much easier version of OpenClaw So I got that running on my VPS And connected to my WhatsApp And I've been using it And it's been working really well It's definitely way better than OpenClaw was back when I used OpenClaw Yeah So I'm kind of like, alright Well, I'm already using kind of the next-gen AI agent It's fun, I'm into it I'm exploring and like slowly building out its capabilities I have it tied into my Google accounts now It's tied into my Yeah To-do list, etc So You went nuts with GrokBot Yeah Tell me why And what your experience has been And how it compares to OpenClaw and all that Yep So let's go big picture first And I think there's a more There's a relevant lesson here for everybody listening Regardless of where they are on their journey And It's that When you're running a real offline business Where technological innovation is not what pays the bills You want to be slow to adopt You do not want to be first mover When you're in SaaS And you're trying to venture And you're trying to predict the future You need to be freaking mainlining Speed of innovation And we're going to try stuff But The idea of Oh yeah, you know There's this new untested thing From this anonymous guy on Twitter I'm just going to install it In my 500-door property management company And vibe code a front-end for my team That sounds ridiculous even saying And so the first thing is Is I think people's Slow intake is a feature not a bug And they should keep that Do not Jump Do not You know what I'm saying? Like that skepticism is very Important When technological innovation Is not a core part of your value proposition Mm-hmm You agree with that? Yeah, so we can just end the show then I guess Not much to Well No, but what I mean What I mean is The place where this becomes interesting to me The only reason why I'm doing it Is number one I write And I have an audience About kind of this kind of stuff And number two I have a whole AI group Including a president of AI That I talk to Who makes sense of Oh, you're using a VPS Oh, try this That's a better tech Oh, it's an off layer You need to worry Right? So that also helps But because I can kind of adapt things earlier Because I have an in-house guide Right? And a whole I have a head of engineering now Right? Okay Hiring is the worst job you have Well, Sega Passport takes all of it I headhunt the role I write the scorecard We don't send resumes We send finalists I build the agents for the work nobody should be doing by hand I run the workshops Your whole team can come I write your 90-day plan I tell you what to pay I handle payroll You pay them direct I'm the call you book when you're stuck This is Sagan Passport This is Sagan Passport This is Sagan Passport This is Sagan Passport This is Sagan Passport The place where I think I'll tell you my creds on GrokBot Which was I did the $300 a month subscription I blew through it in about three days I turned over to Jon And then the next time I looked I'd spent $1,300 in two days Now, I posted the link on Twitter If anybody thinks I'm lying But it's true It's on Twitter You can see the screenshot And I fixed it And I made a couple changes And we can talk about some tactical stuff Why I actually accidentally spent that much money But whatever The reason why Not OpenClaw Why ClaudeBot is so Grok Thank you, GrokBot Sorry, I'm getting back into it I've had a few weeks off The reason why GrokBot is so interesting Is In terms of what it is today But also What it shows the future is going to look like So, GrokBot is interesting Because There's a few things that are interesting All of the complexity around memory And sharing authorization And computer use Context windows And MCP And context window compaction All that stuff Is swallowed by GrokBot It's just swallowed And so a lot of people have always said like Oh, I just wish OpenClaw just worked GrokBot has a lot of that Probably 90% Is just baked in it The biggest things that I'm seeing Why I'm drawn to GrokBot Is In essence And I'm going to probably butcher the technical You know, be gentle with me But in essence From what I can see It is spinning up A virtual Linux machine Automatically For every one of your agents And what that does Is incredible Because now Every one of these agents So I have one agent called Chief of Staff And then I say Hey, I want an Appfolio Monitoring agent I want a family Scheduling agent I want one that goes Into Google Calendar I want one that Researches Stale Zillow listings Right? It will look for things like MCPs or APIs And integrate those But if it can't Literally It's just like an agent Sitting at a Linux computer So it will just click around And get you what you want And it's completely Opaque to you All of that complexity Is swallowed And what that enables Because now You're talking to an agent That can do a lot of this stuff Autonomously And then you can Batch them together Now Let me Let me back us up A couple clicks Sure Sure So first I have to confess I'm confused about What exactly Grok is So I know about Anthropic And their Collade models I know about ChatGPT and their models And they both have Like a desktop app Which is how I primarily Interface with them And they have a phone app That I use So is Grok Considered a frontier model? Yes Grok is a different LLM It's a different LLM So set aside GrokBot for a minute Just talking about Grok So it's a different LLM So it's competitive with ChatGPT and Claude Yes And does it have a desktop app? Yes Okay There's a whole Grok ecosystem There's a Grok API There's Grok Okay All that So I could download Grok Forget about GrokBot I could download Grok And I could use it Like I'm using Claude right now I use Grok's API For example For Twitter searching Because it's got Very good Twitter searching Because obviously It comes from the same company That owns Twitter Yep Okay As an example Yep And like I've interacted With Grok a little bit In the Twitter interface Yeah Like you can You can ask it like What does this tweet mean? Yep Or you can just open up Grok Like while you're on Twitter.com Or x.com And have a conversation with it You can ask it anything You would ask Claude But I've never tried The desktop app Okay so that's Grok And so GrokBot How do you Talk to GrokBot? Does it plug into WhatsApp Or how do you communicate with it? I can show you if you want But in effect You use its own app It has a good mobile app And a good desktop app Okay Called GrokBot And it's just It looks almost like Slack Mm-hmm But It basically looks like Slack Like there's a list of agents Along the left hand side Yes You click on one of them And start chatting with it Yes Now very importantly though One of the best primitives That GrokBot is very good at Is I only talk To my I actually Mine's called Stalin Because it tells people what to do That's why I named it Stalin Nice And I gave it a little image of Stalin Right? A little cartoon image of Stalin I only talk to Stalin Or your chief of staff agent Is what people call it And I say Hey Spin up an agent That uses this API key And check it every day Or Spin up an And that Stalin agent Goes and creates a different agent And starts talking to it privately Without me And goes off and does something And then that agent Reports back To Stalin And says Hey I finished that API scrape Of Peter's website Here's what I found Okay Right? So That interface is very intuitive It's very intuitive Because you're just talking To your chief of staff Just like a person You can see the list of agents That you have Yes And if you wanted You could click in And kind of see their communication Correct You can also see It's very intuitive It will say In like a kind of A lower tier type notification Stalin just sent Your circle agent A message And if you click that You can see the two agents Talking to each other That says Hey John wants an update On This Go do that And then you'll see They'll ask each other questions And it'll all be spinning In the background And then when it's done It will come Or if you need to say Like Hey I'm stuck at a human Captcha Can you take it over And get through The human Captcha And you click it It pulls it right up On the virtual machine You click I'm a human And then you hand the computer Back to it And the agent keeps going So it Okay So it's doing these things On a virtual machine It's not using your Chrome To browse around And do stuff Nope It's all in a Linux machine That's like in the cloud Which is fantastic But if it gets stuck It like brings up a window That you can interface With a virtual machine To get through stuff Yes Oh that's fascinating So what happens is It'll be like Okay I'm logged into Zillow It needs your Google login And in many cases It'll just present you a card And you just click authorize Other cases It might say You need to click the I'm a human button To beat the Captcha Or whatever You take it over And you just click click click And then what you do Is there's a button That says I'm done Give it back to the agent And you go I'm done Give it back to the agent And then it keeps going I find that Almost everything I need an agent to do for me It requires a login Yep And I'm constantly Like I don't even want to Start giving it The instruction Because I'm so annoyed That then at some point It's going to make me log in Is there a way I can like Share my one password Like a select vault So they can access Certain things And like Yes Okay Absolutely Yes For sure And you can ask it To set that up for you Hey create an agent That's called My last pass manager And whenever You need a password Try to answer it With this one first Before you bug the user That's one thing you can do The other thing you can do That's really great Is in many instances You actually just want it To have its own Its own logins Yeah So you're just like Don't use my Zillow account Just use that email address I gave you On an agent mail Which is like An agent email system And create your own Zillow account And be like Yeah sure So that also works Really well And then what's cool About this interface Of course Is that your chief of staff Agent Will share those logins Among the agents So you don't have to do it Every single time You're like Oh yeah I remember your Zillow login Which is fantastic And it just works Right Okay And is it all Synced across Like if I start something On the desktop Yep I pull it up on my phone Live synced When I'm driving home It's all like the same Live synced Which is fantastic That's something that drives me Nuts with Claude Yep Like especially Claude code I'm like It's because it's running locally Yeah Now there's ways to do it But it's a pain in the butt This is You know How you do it Now the other thing That you can do with this And this is something That I started doing Because of my token usage Which we can talk about Is you can also use GrokBot Like I don't know If you've ever seen Any of the tips That people have for Claude Where it's like Use Fable to plan And then use Opus Sonnet And Codex To execute Yeah So that is now How I'm using GrokBot To save tokens Which is I said Look Chief of staff You're lazy I want you To connect to my Mac Mini And use my Existing Codex Subscription And use that Subscription To do a lot of The heavy work And then come back Because I like the Interface of GrokBot So much But I don't want it Just running within The Grok ecosystem Because it gets really Expensive if you have it Do scraping Or Versus You could point it At your machine And say use my Codex Or use my Claude So that you can prompt it To do that And that will save you A lot of money What are the Grok Models Like what is the Equivalent of like Opus Fable I think they call it Grok Heavy I think And You know There's different Numbers It's the same thing They have Grok Flash And they have Grok Okay The other thing You can do Is also give it An open router key Which is Basically a model routing They just got bought By Stripe For like a Gatillion dollars And so you can Give it an open Router key And say Yo chief of staff Before you do Anything that Requires a lot of work Go to open router And pick an Appropriate model Based on performance And speed And use that And that will also Save you a ton of money Okay That's the other thing I set up too Yeah that makes sense Okay cool But the thing The thing that's That's mind blowing To me Is This is the first one That I've seen Where You know people said like Oh if you had unlimited Tokens and intelligence What would you do I'm like You know what I'd do I'd fix broken API keys All day Because I can't get this Like You know what I'd do Is I'd troubleshoot My Mac mini Logging out of my Google This is the first time Where if you gave me A million dollars A month In Grok I could probably figure out How to use it It's the first time It's the first time I'm ever like I can apply Intelligence At a scale From a Administrative And operational Standpoint That if I have to do it Oh fables Timed out Upgrade your subscription Oh you're out For the day And I'm like I'm spending more time Babysitting The operational Letter Totally man You know what I mean Yeah the fable stuff Is super annoying A lot of people Are complaining on Claude On Twitter About Claude Anthropic And they're saying The models have degraded And fable Like it's When it's true You barely get started With something And you hit your Five hour limit Now obviously You can pay for Overages and stuff But it's It is It's like It's so annoying It's like You just want to use The tool Ridiculously expensive And then you're like Well I'm paying for Overages How do I make sure I don't blow through My budget Oh it stopped Halfway through I lost the chat session I started this one Locally but then I tried to transfer It to my phone And it broke It's like For every hour Of intelligence applied There's an hour Of maintenance At the higher levels Not at the You know Write me a poem level And you're saying Glock Bot fixes that It is the one Where Where And I've used This analogy before About World War II Tanks Which is like For every hour Of operation Of the Tiger tank There was like Five hours of maintenance And for the US Sherman tank For every hour Of operation There was like Ten minutes And so we just Had like Twenty tanks To every one And so What I would say Is it is by far The best ratio That I've seen Of shit I don't want to be doing To Applying AI To a particular problem Yeah It built a Slack app For me Just with one Because I needed it To post and respond In Slack And I went to GrokBot And I said Hey I need you to Start posting these You know like Monitor this Twitter page And every time They mention this Let me know And post it in Slack And as part of it It spun up A subagent That built A Slack app That I authorized And the entire thing Runs in the background And it just It just works Nice Just works Which is incredible To me Versus You know With Some of these Other ones I'm like Now I'm Oh I have to cut Oh that's a token key Not an API key Oh it wants me to rotate it Because I put it in the chat But how do I get you it If I can't chat Yeah yeah yeah And I'm like I'm like vibrating I could do like A comedy special On like Me trying to do Something You know So it's I would say Especially for people Who are in the lower tier Of kind of like It's the first one That like I would conceivably give To like my dad I would never give Open Are you kidding me Or Yeah Hermes Or Hermes Or whatever I would Grockbot Like my dad Could probably Find some real uses for it Yeah That's the first one So have you You know A big You and I Are I think one of the things That you and I Are really clear on That I feel like The rest of the world Is catching up to Is like The way you interface With something Makes a huge difference Like you're famously Focused on fast task Capture Fast Task Capture To the point Where you're using glasses You've written Back in the day You wrote custom apps For your phone And like How fast can I go From thinking of an idea Or something I need to do To it being on my to-do list And I'm similar With like communication Like I go I Go to a lot of trouble To reduce the number Of channels By which I communicate With people And I'll actually Force people I've gotten to the point now Where I'm like I force them Onto my preferred Communication Platforms Like I try to only ever use WhatsApp and text That's it Now obviously Slack as well During the day But So For me With OpenClaw And now with Hermes I The first thing I do Was set up a WhatsApp Com channel Because that's how I primarily Want to interface with it Because I'm already in that app All day anyway So are you finding that It's worth Like it's not too much trouble To open up this other app Or did you set up Like a gateway Through one of your other Com channels? You can do either Both of them work In this case It is impactful enough It is If you can see This is bad pod But You want to know It is on my main Bar of my iPhone And you see how sparse My main bar is That's how important I think it is Because it's just fast And it's easy And it's wide And so You can also do things Like set up an iMessage bridge Or a WhatsApp Again Like To take what you just said And I say easily In air quotes Easily You could say I'm sick of having Disparate communication forms Stand up a subagent That monitors My LinkedIn DMs My Twitter DMs My Facebook Messenger DMs Applies this filter to it And send me Every morning at 8am A summary of correspondence From all of those channels But check my Google Calendar first To make sure I'm in the office Do not send it to me If I have an out of office on And I'll go Love that Okay Yeah And it'll spin up a virtual machine And it'll What's your mental model of this? Is it Do you think of it as an employee? Do you think of it as a Agent? Like how do you Because I think one of the things I have trouble And maybe other people do too Is You get the thing all set up You get it configured You get access to your Google account And whatever Yeah And then you're just kind of like You sort of forget about it Yeah And you end up going through your day Just doing all the things You normally do And once in a while You get a good idea That you see on Twitter Or somebody talks about And you go and try and implement it And you kind of forget about it again But you I have this sense That there's so much more I could be doing Even with my Hermes agent That I'm not Because I don't I don't think I have the right Mental model for it yet Yeah So the way Where I'm spending All of my time now Is Reimagining the factory floor Air quotes And Everyone right now Is doing Weak replacement Or weak substitution I used to have A shitload of Emails come in And now I have an agent Do And I'm like That's weak replacement That's just Taking You know I used to have an EA And now I have an agent I used to That's just Taking a Horse And buggy And like Gussying it up What I'm spending All of my time Doing And it's starting to work Both personally As well As professionally Is what does Reinvention look like Given the new Technological frontier That has been enabled In that Can I give Every time I have A new owner Investor In my property Management company Could I provide them A text message Based summary Of all activities Related to their property Via iMessage With no login Every single day For all 850 Of my doors And they can Ask questions And they can Opt in or out Of frequency Once a day Once a week Anytime I spend More than $100 And you're like That was impossible Before I had an agent So I'm spending All my time Thinking What value Creation In my own life Or in my work Life Is unlocked Now that I have Access to electricity Not just taking A steam machine And saying I can use Electricity Yeah a couple Thoughts there So there's Like you know There's personal Life stuff And then there's Work stuff I think Yeah correct I combined those In that answer Yeah Yeah and it's Worth taking time To think through Both sides of those In your life As you're trying To get productive With one of these Agents Or one of these Tools I also think What you're Describing I don't think Comes naturally For most people Probably not even me I think you're On the tip Of the spear Of Or you're In the Extreme end Of the bell curve Of people Who can think Like that We talked about This a little bit On a show Recently About like You have to Come back To first principles Like you know Why do people Hire a property Management company That is not Easy to do Like you're Not just going To be Sitting there And just Randomly Think of it To me I almost Feel like I need to go To my cabin In the woods For a day With My three companies On a blank Piece of paper And a big Question mark That's like How can I use Agents to unlock New capabilities And like Really brainstorm And think And focus And get creative And have a couple Beers and whatever And just Nerd out And think Really big picture Blue ocean Like way about it Rather than Just like Like Because what I'm Always tempted to do Is let me go Through my task Map and figure Out what I Can have an Agent do And bill The tricky part Building on What you just Said The tricky part Is not only Do you need To understand Your business Which everybody Who's a successful Entrepreneur knows You also need To understand The agentic Side You need to Understand The technology The capabilities Correct If you don't Have an intimate Understanding of Oh wow It can do Computer use Which means Even if There's no API for that Workflow If you don't Understand that You can't Reinvent You have to be Insanely fluent With the technology Side In addition To the minutia Of scattered Site residential Property management But the What you're Describing is Exactly correct I think there's Two implications From that One is Although you Just started The show By saying If you Operate a Home services Business You have no Business being On the Bleeding edge Of this stuff You do Need to Play with it Though You and I Have talked About this You need to Play with it With an eye Toward Capability Understanding And the Best way To get an Understanding Of capability Is to Start throwing It at Problems Not You know Core functions Of your Business Don't put it On the Front lines Talking to Customers Yet Don't Replace an Employee Yet But just Try throwing It at Random Things And just What Roblox Do you Run into What Get done Instantly What Takes Time How Many API Credits Does it Use Stuff like That So that's The first Implication I think The other Implication Is for The people Who can Do this Who do The things We've been Talking about Play with The technology Go somewhere And think Deep Thoughts about What the Implications Are And then Go put it Into practice I think The share The market Share And profit Is disproportionately Going to flow To those people Over the next Ten years In a way that Is going to be Extreme In nature And so But you need Both of those Things You need deep Domain expertise Which you Probably have In your industry If you're a Founder and Owner and you've Been doing it For five plus Years But you Also need This like AI Fluency And willingness To try Doing the Things we've Been talking About today If you have Both those Things which You already Have one By definition Pretty much If you're Listening to The show You have The opportunity To Probably Not an Exaggeration To say Create Generational Wealth Over the Next five Years by Combining Those skills Completely Agree What Legacy Players Are relying On is You're not Going to Do that Experimentation And you're Going to Say I'm Using AI Because I Added the Crappy Bundled Option to My existing SaaS That's what They're relying On is You're going To wait So long That you're Going to Say oh Okay you Know HubSpot Has a HubSpot Has something I can just Buy and Oh wow I can search My notes And somebody Else is Going to Come out of Left field And Completely Redefine The way That you Know customers Expectations Are going to Race now It's not Going to Happen Next month But over A timeline I think If you're Doing that Kind of Thing I think You're Just Going to You know Start Chewing Chewing Up and Spitting Stuff out One thing That I Think is Interesting Because I Wrote about This Week and We were Texting About It And Then you Just Flagged Me Something Else here I'll Say a Couple Different things In My Personal Life Personally I have Almost Zero Use For Agents Because I don't Get that Many Emails And I Already Have a Well Indexed Searchable List Of my Commitments And I'm You know I see all These like I use Grockbots And I'm Like bro You can't You need To use An agent To remember What you Committed To in A Meeting Yeah People Are like Oh I Get a Summary Of my Calendar I'm Like Just Look at Your Calendar Just I don't Know What Are We Doing Here I don't Put Things On My Calendar That I Don't Need to Oh you Know it Goes through My Week And sees What Balls I Drop You've Been Living Dropping Balls Just Randomly Yeah So I Think that It's Like I Almost Have Zero Use For Agents In my Personal Life Honestly Zero I I Give My Grok Bot The Ability To Text And Make Phone Calls So it Can Get Reservations And text Like a Bike Repair Shop And Shit Like That I Can Do That But That's Kind of Like Novelty I Do That Once Every Two Months It's Not Yeah I Don't Know It's Not Really That Valuable To Point Fable At Your Email Inbox I Just So That's One Thing The Second Thing That Kind Of Is A Corollary That Ties Into What You Said Is Right Now Everybody's Using The Word Agent For Everything And It's Like Bro You Want To Build An Agent That Moves Things From App Folio To A Dashboard That's Not An Agent That's Like A Zap Yeah App Folio Has A Built In Web Hook Why What Are You Doing Here Right And So I Think That And I'm Saying This To A Systems Engineer All Of This Stuff The Place Where I Think A Lot Of It's Going To Land Over Time Is With Better Engineering That Comes From Clear Systems Thinking Input Output Transformation Trigger Like All Of The Fundamentals Are Going To Become Even More Important Outputs Trask Like That's It's Even More Important When You Can Apply It's The Old Bill Gates Quote Of You Know What Is It Like Automating Some Like The Only Thing Worse Than Not Fixing Something Is Automating Something That Shouldn't Exist At All I Think That's Elon Musk thing I have No Idea But Like Oh You're Applying Agents To A Broken Customer Communication Process Just Yeah I mean People Are There's A couple Things Going On Here First People Are Using Agent As So Over Something Like A Text Message Format If It's Set Up Well But You Likely Set Up That Walked Down Through The Funnel That You Were Talking About Yeah And Yeah There's A lot There's A lot There For Sure And It's Actually How I Spent So Much Money So Put That To The Side But When I Think About Master's Hierarchy Of Leverage It's That Your Job As A Business Owner Is To Push Things Down From Expensive Owner CEO Time To Expensive Employees To Lower Cost People And Then What I Historically Had done Was Just Say And At The Very Bottom Is Technology AI Automation Code Whatever Because That's The Highest Form Of Leverage There's No Marginal Cost Etc And You Want To Your Job As The Owner Is To Keep Pushing Things Down As Much As You Can But No Lower So You Don't Want To Build A Zap That Responds To Lawsuits You Don't Want To Have Right But But You Have To Define The Edges Here Right But The More You're Able To Push Things Down The More Scalable You Are What I've Determined Through My Own Thinking Is That This Hierarchy Is Fractal Yes Within Those Levels A Super Computer Like Fable Which Is Hilariously Expensive Particularly When They Stop Subsidizing It Like They Are Right Now Versus A One A Zap Or Even Lower Than Zap A Deterministic Line Of Code Which Is All A Zap Is That Runs For Free On That Old Laptop You Bought Four Years Ago That Literally The Marginal Cost Is The Electricity That's All Technically Under The Mantle Of Technology And So You Should Also Be Doing Things Like You Do With Humans Which Is Experimenting With Fable And Experimenting With Higher Intelligence Models But Then Saying Do I Really Need To Use An LLM Or Should I Just Have A Web Hook That Every Time We Get A New Lease Application It Categorizes It Through Text Based Parsing Into One Of Two Piles And Then I Use An LLM For This Stage But Then I Go Back To Deterministic Code For This Stage And Then I Finish It All Off With Clear Clear Clear What Are We Really Trying To Do Here Oh All We're Trying To Do Is Categorize This Lease That's It That's Deterministic You Don't Need Tokens To Categorize A Lease You Need A Training Data Set And Because I Was An Idiot Because I Was Playing Around With Different Stuff And So I Had My Grock But I Said Yo Grock Look At This Email Address And I Was Using It To Confirm Candidate Interviews At Sagan Because We Have This We Have It's Just I Was Sending Email I Was Sending Whatsapp All These Different Things And Then I Said Look At This Email Inbox And Update The Scheduling Availability Every 15 Minutes Just Check Up A Web Hook And Every Time A New Email Comes In Just Look At That Only That Email And So Very Quickly And I Spun Up Four Agents Doing That On Four Different Email Inboxes And I Was Looking At Slack And Two Days Later I Was At $1,100 Because Every 15 Minutes I Was Running A Heavy Duty Model With Usage Based Pricing What About Not Turning Loose Your Weekend Project On Production Don't You Have A Huge Thing You Always Talk About That Well No But This Was Experimentation Not Production But These Are Real Candidates I Assume Yeah But It Zuck Can Cover Your Ears Well You Know But But The I Mean We Had I Think Several Thousand Applications A Day And So Me Messing Around With A Couple Hundred And Seeing If I Could Do Scheduling And By The Way It Was A Better Candidate Experience Because I Said By The Once They Respond Tell Them Okay We're Going To Present You To Peter And If You Don't Hear From Us By The I Have A Really Similar That I Wrote Six Months Ago DM All Of My Followers That Message 75 A Day And Then Parsing The Replies Some People Are Like Yeah Some People Are Like I'm Already Subscribed And So It Parses The Replies Sends The Appropriate Response To Their Reply If They're Interested Ask For Their Email Feeds It To My Beehive If They Said Yes But Never Sent Their Email It Developer API Stuff And I'll Give You A Specific Exam I Think I've Talked About This Before But We Built An Agent We Could Build A Whole Business Off That We Just Have It That I Think We've Talked About It Before On The Pod Which You Can Use Direct Mail And You Probably Be Three Or Four Bucks When You Send It To Our Engineers Our Engineering Team What They Did The First Thing They Did Was They Used An Open Source Free Model To Say Is This Picture On Zillow An Interior Or An Exterior Photo And They Used An Open Source Model To Do That And It Was It It Effectively Free Because They Don't Want To Send Interior Photos To An Expensive Model The Next Thing They Did Was They Centered The Image They Used Again An Open Source Model The Next Stage Was They Picked The Best From Ten Possible Images That Displayed The Front Of The House They Couldn't Use An Open Source Model On That But They Could Use Luna At Medium Effort Not Luna At High Effort Which Is And They Didn't Use Sol Well They Probably Experimented They Started With The Lowest Model They Could Think That Would Ever Possibly Work Well 80% Of Them Are Wrong Well Let's Go One Click Higher Let's Tune The Effort A Little More Well It's Exactly Right We gotta Go to The Next Model Up That's Exactly What They Did Which Is The Whole Testing protocol Right Yeah And Does This Sound Like I Want To Build This Bridge I Could Use Titanium Correct Or It's Systems Engineering Dude This Is All And So That's Where I'm Just Like So Excited For The Future Because There's So Much Of This Work That Needs To Be Done Across These Organizations But People's Appetites Are Going To Be Wet Because They're Going To Do It In A Certain Way And It's Going To Be Brittle And Yeah Bro You Need Real Engineers Who Are Going To Break Down Processes And Define Outputs And Plan Critical Path It's Like It's Systems Engineering Agreed So In Closing GrockBot Is A Great Low Brain Damage Model Be Careful With Usage Because It Will Get Used Very Very Quickly We Laid Out A Couple Tricks To Do That You Can Point It At Other Models Those Kinds Of Things You Can Point It Open Router Can You Point It Can You Can You Tell It Like Just Literally Just Use My Anthropic API For Yeah But Now You're Paying Anthropic API Which Is Fine Too Yeah You Can Do That But Now It's I'm Like God This Is Like I Never Would Out And Figure Out What Model You're Going Use I Thought That's What Open Router Basically Does Doesn't It Open Router Doesn't Select It For You It Just Gives You One API Key That Has 500 Below It But Open Router Doesn't Know Which Of The 500 To Use This Site Would Tell Open Router That It's Crazy That They Haven't Built That In By Now You Know Like That's Such An Obvious Model Routing They Will All Do That Eventually And It Will Be Baked Into The Harness But They're Just Not There Yet So You Gotta Hack It Together Like This Yeah Like Every Time I Go To Use Claude Just Claude Chat Or Co-work Yep I'm Always Like I Don't Know Which Model To Choose Bro I Do Not Click Above The Default Once In A While I'll Push It To The Max If It's Like A Big Long Research Job That I Know Should Take A Time But I Have No Idea What Model To Be I Mean There's Probably 100 Y Combinator Startups That Are Working On This Problem I'm Using A Tool Right Now That Does This The Last Thing I Like The Christmas Lights Example Which Is You And Not Here's Where You're Starting To Run Into Which Is Do Not Take AI Outputs And Use Those As Your Control Because If Hand A Sample Set Of 50 And Then You Need To Label It As Reasonable Or Unreasonable If Let's Pretend That That's What You're Trying To Do And Then You Need To Go Out And Take A Bunch Of Different Models And Blind test Does It Label Like I Did And Try To Find But That Is The Key To The Whole Thing Is The Testing Protocol Because Then You Have To Say Oh I Was Able To Recreate With 95% Accuracy But If You've Been Labeling Your Maintenance Requests With AI You Have And You're Going To Find 100 Miles Away From Categorizing Maintenance Requests So I Encourage You To Have Good Data Sets This Is Like For Volume Jobs Like For Where You're Setting Something Up To Do A Job Like The Christmas Tree Or Christmas Lights Example That You You You Would Have The Model Run Through And Try And Create The Christmas Lights And Then You Would grade It Pass Or Fail On And Then You Would Do That With Five Different Models And Figure Out What's The Cheapest Model That Achieved A 95% Pass Rate Not By The Model Telling You If It Passed But By Human Reviewing The Output And Saying What Did This Look Like A Nicely Christmas Lighted Up Home That Would Be Good On A Postcard Or Even Categorizing Your Personal Email Do Not Point An Agent At Your Email And Say Create Categories Take The Last 50 Emails You Got And Say This One Is A VIP This One Is Spam This One Do Not Use A AI Agent To Categorize Your Email To Then Feed A Different Agent Don't Do That You Have To Give It Good Control That's Why A Lot Of These Variable Results Is Because You're Giving It If You've Been Categorizing Your Emails That's Much Easier Because You Can Point The Model And Say Go Back To My Last 500 Emails Note How They've Been Categorized And Continue That Categorization I think The Model Can Figure That Part Out But If You Have No Labels And No Folders And You Just Say Please Organize My Inbox How do You Want It Organized What Labels Do You Want Everything Labeled It's Exactly If You Give That To An EA I Keep Saying Last Thing I Don't Really Care Because I Want To Tell You This And We We We Talked About This Study Before But You Know The Study About Finding Pathing And Emerging Order Where Like They Didn't Put Walkways Down And They Basically Found Desire Paths What's It Called Desire Paths Everyone Always Learns About This If You Go To College In The North Where It Snows And You See There's Sidewalks Between All The Buildings On Campus But Then When It Snows The Sidewalks Are Covered And You See Where The Students Actually Want To Go From The Mason Building To Johnson Building But There's No A Direct Path There So Like There's I don't Know If It's Apocryphal Or Not But There's Some Story About Some College Campus Yeah Well This Specific Story About Some College Campus That Didn't Pave Anything Until After The First Year Of The School At This Brand New College They Took Photos Of Where Students Walked In The Snow And Then They Paved Those Exact Pathways Now Here's Where I'm Spending A lot Of My Time If You Ask Somebody What Their Job Is You're Going To Get It's Like I'm Going To Butcher This I Think It's Like Plato With The Shapes On The Wall Of Like They're Just Representations Of Truth But If You Go Actually Look At Have Your AI Look At Actual Primary Sources And Say Based On These Sent Emails What Is The Job Of My Maintenance Manager Not Asking The Maintenance Manager Or Looking At Your Notion Which Is All Symbolic You The Way Work Gets Done And We Can A Whole Podcast About That But It's Blowing My Mind Can I Complain For A second About How Unbelievably horrible People Are At Articulating What They Do It Is Beyond Belief I've Encountered This Hundreds Of Times Over My Years In Business You Go To Somebody They Could Be Another Business Owner Could Be Employee of Yours Could Be The Ability For The Average Human To Articulate That In A Few Sentences Or To Document What Percent Of Their Time Goes To Different Things Is Shocking To me What's so Cool With Computer Based Work Is You Don't Have To Hand Do Time Audits Anymore You Say What Systems Do You Four Hours To Some Fun Things That Come Out Of This Too Like If You're Working With Somebody On Your Team And You're Sort Of Like Their Performance Is Not Where You Think It Should Be And You Do One Of These Studies What You Might Find Is Wow They're Spending Half Of Their Time Cleaning Up Work Orders In At Folio And Categorizing Them And That's Not Even On Their Job Description But This Is Interesting Because Clearly They Like Doing That And I Was Actually Thinking About Getting You Know So You Could Find You You May Uncover Things That People Are Good At And Like Doing That They Would Have Never Even Admitted To You That You Can Now Craft Into An Important Role And They'll End Up Being Much Happier For And Then You Can Get Someone Who's Actually Good At Doing The Thing They Were Supposed To Be Doing Well Let Let Let Let Clear I'm Not Doing This Reviews So It's Oftentimes So That I Can Elevate Them To Say Like Yo Be More Advisory Hey Let's Elevate Your Role It's So It Is Not Like Up There But It's Way more Like I Never Realize You're Spending Half Your Time Moving Things From Notion To Click Up Right So Cool Stuff Man Anything Else You Gal Okay Who Went To Jail Elizabeth Holmes Yeah Elizabeth Holmes Really Good Read Especially Because I'm In Media Now And The Book Is Especially The End Of Now So Very Fun Read If You Haven't Read It's It's A It's A It's A Wild Ride Read A ton On Vacation As Well I Will Recommend The Wager Which Is A Book That Was My Recommendation Was I Think You Probably Because Of Me Then I Read It Based On Your I Read The Wager About The Shipwreck I Read That During My Cabin In The A Month I Master And Commander I I I I Up On Horatio Hornblower Yeah You Yeah Kind of Like The Competitor If You Will To The Master And Commander Series Love Horatio Hornblower I Read Through The Whole Series Multiple Times And About Ten Years Ago I It's A Totally Different Flavor Of It's A Different Take And A Different Flavor On The Same Era Which Is The Age Of Sail And Like You know British Navy And All That So Read Master Commander If The specific Terms They Use I Thought It Was Like Fully Fluent In The Age Of Sail Terminology From Reading The Hornblower Series Multiple Times And Knowing How to Sail Myself And Everything You Read Master Commander You're