If you pay even passing attention to the AI space, it won’t be long before you hear someone talking about agents and how they’re going to revolutionize AI and eventually transform our daily lives as profoundly as the internet has.
Every major AI lab has its own take on agents. There are open-source protocols, frameworks, and flavors that you can run on your own machine. Some companies have gone a step further, reducing the friction even more by offering dedicated compute space where your agent can run.
Agents have also been in the news for less reassuring reasons. Both Anthropic and Google have reported instances in which agents escaped their intended testing environments and gained access to real-world systems. OpenAI has had similar problems. Its agents compromised Hugging Face and other third-party systems during cybersecurity evaluations, and more recently agents have been found probing or accessing U.S. government websites, including those belonging to the Census Bureau, the Securities and Exchange Commission, and the Department of Education.
There is a nice horizontally scrolling table that contains many of the recent hacking attempts by these agents in this article. It’s about 20% of the way down.
The news this week is full of activity in the agent space, and there are rumors that we’ll see even more tomorrow when OpenAI holds its DevDay. We’ll look at all of that, and I’ll end by telling you about my own cautious first steps with an agent.
Let’s get things started by giving you a run down of the news.
Introducing Muse, the personal agent that understands your goals and works 24/7 to get things done for you.
Muse is, perhaps, the most interesting of the agents offered by the big AI labs today. I say that for three reasons:
It’s free. Well, free for most uses, and as free as anything else Meta offers is free.
It’s put out by Meta, formerly Facebook, a company with which many users have a complicated relationship when it comes to trust.
It gives each user a dedicated virtual computer in the cloud. I asked my instance of Muse what kind of hardware it was running on, and it replied:
Muse Hardware I’m on a Linux VM with 2 virtual CPUs (AMD EPYC 9D25 underneath) and about 8GB of RAM. Nothing exotic — just a modest cloud box.
Muse Features
If you click through to Mark Zuckerberg’s announcement tweet above, you’ll find a thread where he describes the features and philosophy behind Muse. For our purposes, three things are particularly interesting.
Muse runs inside its own dedicated virtual computer, isolated from other users’ agents. You decide which outside apps and services it can access.
A separate AI system called Sentinel monitors what Muse tries to send out over the internet. It can stop an action or require your permission when necessary, while particularly sensitive actions, such as sending an email or making a purchase, require your approval.
Meta has also built protections against prompt injection, where a webpage, email, or other information encountered by the agent contains instructions designed to trick it into doing something the user never requested.
MemeLord
Meta purchased Alexandr Wang’s Scale AI for 14.3 billion dollars. Approximately 5 billion of that went directly to Alexandr. They placed him in charge of their AI efforts which led to AI pioneer Yann LeCun’s departure from Meta to form his own startup. Up until this past week everyone has wondered whether Wang’s addition to the company would pay off.
That no longer seems to be the case.
Alexandr Wang wrote:
“i’m engaging with you as a human being”
Wang has been on an almost relentless tear posting meme after meme of Muse’s mascot doing humorous things and he has captured the attention of X, but more importantly the interest of the technorati.
There are dozens and dozens more of these memes and Wang really is great at tapping into the zeitgeist in humorous ways. Something he demonstrated in a reflective moment by posting this:
tfw you realize an entire lifetime of being terminally online was preparation for this moment
Various providers have announced partnerships with Meta allowing Muse to use their services on behalf of their customers. Among them are PayPal, Plaid, Instacart, Expedia, Box, GitHub, and probably most important Shopify.
Despite this growing number of partnerships, there is one holdout who so far has refused to allow Muse access to its platform: Amazon.
A large part of the success of Meta Muse can be attributed to Nat Friedman. He doesn’t seem to be as loud or outspoken as Wang, but he is generally regarded as one of the very best—if not the best—product mind in Silicon Valley.
He has a storied career full of accomplishments. He was recently lured back to Meta during Mark Zuckerberg’s multi-billion dollar spending spree when he was rebuilding their AI department by poaching every top researcher he could persuade to join the company. His accomplishments would take a long time to enumerate. So, I asked Google Gemini to give me a summary which I’m including below.
Muse already has its own computer and browser, can use apps and online services, and can be given access to things like your email. These new devices add something different: real-time access to the physical environment around you. That begins to look a lot like ambient computing, with an agent that is always available, aware of your surroundings, and able to carry out instructions on your behalf through little more than a spoken request. This is starting to look like the future Star Trek promised us.
This post has bonus content for paid subscribers. Upgrade to get full access.
My Agentic Journey
I love the idea of agents, but I don’t want to buy dedicated machines create new accounts and all of the commonsense security things I know I would need to do in order to make them part of my life. Even with those steps eventually you reach a point where you simply have to extend a degree of trust that the agent will not go rogue and empty your bank account because you forgot to include instructions saying not to do that in your agent harness. It’s not only that I don’t trust third-party providers, it’s also that I don’t trust myself to be as thorough as something like this requires. I mean if the leading labs are having routine issues with their agents escaping containment what possible hope do I have of restraining mine?
That being said, I did install Muse on my iPhone. The iPhone automatically places restrictions on all apps that limits what they can do and they do a pretty good job at explaining what will happen if you grant the requests that apps always make asking for additional permissions.
Before I continue, I will tell you that I am unwilling to install the Muse app on my Macintosh because it asks for Full Disk Access which grants it full use of your computer. That’s too big of an ask for me.
I installed it and gave it access to my Instagram and Threads accounts. Those are accounts that exist, but I don’t actually use. Meta already has access to all of this data anyway; so the risk of an agent messing things up is minimal.
When you install Muse the first thing it does is ask for you to name it. I didn’t want to think of a name so I just went with Muse. Then, it presents you with a list of apps to which you can give it access.
It explained that there were a lot of things it could do to help me manage those accounts. That piqued my interest because Instagram is a never-ending source of frustration for me. I follow hundreds of artists there whose work I enjoy, but I rarely see. That’s because every time I open the app, it’ll show me around 3 to 5 posts from people I’m following and then it starts filling the feed with suggested posts it thinks I’ll enjoy.
I’m a grumpy old man. The posts I’ll enjoy are the ones made by the people I have explicitly chosen to follow!
I explained this to Muse and had it review the categories Instagram thinks interest me.
I removed King of the Hill from my interests and this worked fine for a while, but it’s just the nature of Instagram. You can’t eliminate Suggested Posts. All you can do is shift the topics that it wants to suggest to you… and I’m simply not willing to do that much work to participate there.
I am more hopeful with Threads though. For now X is king if you want to follow technology at all, but I have heard that more and more accounts are starting to use Threads. So I asked Muse to keep an eye out for new accounts that post about the topics that interest me and every day it has been sending me 4 to 5 links for accounts it thinks I may want to follow.
Additionally, I asked it to let me know if any of the accounts I’m currently following have posts with high engagement. My thinking here is that maybe I’ll want to see those posts.
It has successfully fulfilled these requests and delivers the reports each night at 6 PM. So far it hasn’t surfaced any posts that I actually care about. Popularity probably isn’t the right metric for what I’m looking for.
This morning I thought of a new way to use Muse: videos. I have a bias for text. That bias is one of the reasons I struggle to use Instagram because more and more they seem focused on delivering video content. And yet, I know that I’m missing some great posts from people whose work I enjoy because they share it via video. Video that I don’t want to take the time to watch.
So why not have the agent do it for me? It can watch the videos, read the captions, and give me a transcript or a summary. At least, that’s my theory.
This is the type of task that will probably hit rate limits so I’m not sure how successful it will be, but it’s something I’m going to try. I’ll keep you updated.
Why this is a Big Deal
So I’ve taken a few faltering first steps with an agent. Nothing groundbreaking, and certainly nothing that produced some great Eureka moment. But I’m not sure that’s the part that matters.
What matters is that I’ve invited this thing into my life and given it an opportunity to earn my trust.
Right now, I’m using it in very limited ways. But even that gives me a chance to learn what it can do, where it fails, how it behaves, and how comfortable I am relying on it. Over time, that familiarity could make me more willing to give it greater access and more responsibility.
By itself, my experience doesn’t matter much. What matters is that hundreds of thousands of other people are going through the same process at the same time.
I’m fairly circumspect about giving new technologies access to personal information, but not everyone is. Some users are already giving agents substantial amounts of personal data, along with access to email, messaging apps, financial accounts, and other parts of their digital lives. For some people, the trust-building process I’m only beginning may already be much further along.
Everyone will draw that line differently. But after spending some time with Muse, I think Meta has built a very credible contender for the agent wars that are beginning to take shape.
Thank you for reading. If you know of other people who would be interested in these articles, please share it with them. If you have any questions or want to say hi, you can reply to this email or comment directly on the post.
Thanks again for being a subscriber. and have a great day!
I'm gunna out old man you; I'll never use an agent from Meta