AI Escapes
OpenAI Attacks Hugging Face
AI Humor
Humans Got Nerfed
Justine Moore wrote:
If models spoke about humans the way we speak about them…
Open Source All the Things!
Julian Schrittwieser wrote:
I’m so excited that Jensen Huang is a believer in open source now, looking forward to the CUDA and GPU driver open source release!
This is funny because Jensen Huang, CEO of Nvidia, recently joined Twitter and his very first tweet was a defense of open source AI models. He encouraged industry wide support for these models. Julian is calling him out because the CUDA framework and GPU drivers that Nvidia controls are very much proprietary and a key reason why Nvidia has such dominance in the GPU market.
Don’t Disrupt Us
bubble boi wrote:
Anthropic: “We are disrupting thousands of industries by being able to replicate their products in just a few minutes using AI making all of their hard work pointless.”
Also Anthropic: “Chinese AI labs are replicating and ripping off our product making all of our hard work pointless.”
News
Om Patel wrote:
UPDATE, ITS WORSE THAN THE CHATS
the same thing is happening with shared artifacts. every app, doc, dashboard and tool people published from claude is also sitting indexed and searchable
people have already pulled up internal company dashboards, full project plans with client details, and someones entire clinical trial summary with real medical data in it
but think about what people actually build in artifacts:
> financial models, cap tables, revenue spreadsheets
> payroll and salary breakdowns with employee names attached
> customer lists and crm exports
> internal wikis, roadmaps and unreleased product plans
> legal docs, contracts and agreements being drafted
> personal health and medication trackers
> tax documents and bank statement breakdowns
> and every dashboard someone pasted an api key or env variable into to get it working
when you publish an artifact your only two options are "only me" or "anyone with the link"
nobody reading that thinks it means "and google will list it publicly." its the same as an unlisted youtube video showing up in search results
and its already past the point of just being google. third party sites have started scraping and organizing these into browsable collections, so deleting it from search doesnt mean its gone
if you have ever published an artifact, go check your published artifacts and unpublish anything you dont want public
link and not indexed were always two different promises. people only ever agreed to the first one.
The story originally broke because it was discovered that Google had been indexing everytime someone created a link to share a chat session with Claude meaning that all of those seemingly private conversations were much less private than thought.
Things got exponentially worse when it was also discovered that any artifacts that were shared in the same way were also indexed and searchable. Artifacts are the much larger risk because they may contain private or proprietary information that may carry legal penalties for accidental or careless disclosure.
China Appears to Have Solved DeepUV
Chubby, referencing a report from The Information, wrote:
Holy China did it: a homegrown alternative to ASML’s immersion DUV lithography machines is now entering production.
Lithography was considered one of the hardest parts of the semiconductor supply chain for China to reproduce. ASML dominates advanced DUV equipment and remains the world’s only supplier of EUV systems.
Western export controls were built around this bottleneck: without continued access to ASML’s machines and servicing, Chinese chipmakers were expected to fall further behind.
Now, a state-backed Shanghai company reportedly plans to build five domestic DUV systems this year and around 20 in 2027. The first machines will go to SMIC, Hua Hong and CXMT.
lithography was one of the hardest gaps in China’s semiconductor supply chain. And they figured out a solution. This is insane.
This matters because DeepUV Lithography was the last meaningful obstacle preventing China from serious competition with US-based chip companies. The DeepUV machines contain over 500k moving parts and are notoriously difficult to build and maintain. Their development required advances in many different areas least of which was optics because the lenses required for deep ultraviolet lithography have to be so precise. It was widely believed that the Dutch company ASML had a monopoly on this technology because of how exacting the standards are. If these reports are true and the Chinese can produce these machines at scale, it will change the entire tech landscape.
Hugging Face Hacked
This happened last week, but additional details have come out since it was first released and I wanted to write a bit of an explainer for it to help people understand the full scope of what went down.
OpenAI wrote:
We recognize there are a lot of questions and speculative details circulating related to the Hugging Face incident.
This is an unprecedented incident, and we think it marks an important moment for AI safety.
We are still conducting a thorough review along with external advisors and with oversight from our Safety and Security Committee.
Once the review is complete, we plan to publish a technical report of our learnings in the coming weeks.
Hugging Face was attacked by an unknown AI agent. Eventually it was discovered that an OpenAI experiment had escaped its sandboxed environment1 and reached the open internet.
OpenAI was testing an unreleased model to evaluate how well it would be able to complete the ExploitGym evaluation. In an effort to fully evaluate it’s capabilities the safety protocols were removed and the model was given the goal to beat the eval.
ExploitGym tests whether an AI model is able to use a known software weakness to create a practical method for taking control of a computer system. Typically these tests are run against various systems like the Linux kernel.
The expectation at OpenAI seems to have been that the model would identify some software flaw and then seek to exploit it. The model did not do this. It didn’t even try to do this. It decided that the best way to pass this test would be to go find the answers to the evaluation.
In an effort to do this, it identified weaknesses within OpenAI’s sandbox environment, used those weaknesses to gain access to the open web where it quickly determined that the answer key for ExploitGym was stored on Hugging Face’s servers.
It gained access to Hugging Face’s systems and performed more than 17,000 automated actions before the activity was detected. Hugging Face tried to use American frontier models (it’s believed that they tried with both OpenAI and Anthropic) but those models would not help because the requests involved actual commands related to real world hacking. In desperation they resorted to a locally hosted version of GLM 5.7 (a Chinese model) which did not have all of the safety guardrails present on the American LLMs. That model was able to help them put an end to the attacks.
Hugging Face has since closed the vulnerabilities and replaced the compromised credentials. OpenAI has restored the safety guardrails that had been removed from the model for the test and is helping Hugging Face resolve any lingering issues.
Regardless of what this model may or may not score on the ExploitGym eval, I would say that it has successfully passed the test.
This post has bonus content for paid subscribers. Upgrade to get full access.
Industry Shaking Purchase
Polymarket Money wrote:
BREAKING: Stripe is in talks to acquire AI-model marketplace OpenRouter for close to $10,000,000,000.00.
This comes from a report in the Wall Street Journal which claims that Stripe is in advanced negotiations to purchase OpenRouter. The deal may still fall through or be stopped by regulatory oversight. If it does go through it will definitely shake things up.
OpenRouter provides access to over 400 different AI models. Stripe provides the financial infrastructure for most of the internet. Combining the two could potentially redefine how AI businesses are built.
More than that though, it has the potential to change Stripe from being primarily focused on payments to becoming something more akin to an AI-infrastructure company.
OpenRouter already has information about which models developers most frequently select, which ones are the fastest, and which ones fail most often as well as which ones provide best results for their price. Stripe has detailed profiles about how much customers are willing to pay for various services. Those two factors have the potential to combine to create a very detailed picture of the economics within this industry.
This is something to keep your eye on.
Thank you for reading. If you know of other people who would be interested in these articles, please share it with them. If you have any questions or want to say hi, you can reply to this email or comment directly on the post.
Thanks again for being a subscriber. and have a great day!
A sandbox is a contained environment where software can be tested without being allowed to affect outside systems. It’s the digital equivalent of conducting a potentially dangerous experiment inside a closed lab that’s sealed off from everything else.
when my kids were little I bought them a sandbox. It was a red crab. while it did have a cover so cats didn't turn it into a litter box, it was exceedingly easy to get out of.