I currently do not automatically detect the agents themselves, and most of the bot traffic is ignored, but for the traffic that is not ignored it usually shows as a 0s session (so you can filter for all sessions that have a 0s replay, which are usually bots).
I am still torn whether I should actually track bots/AI traffic or simply drop it, maybe I will add a toggle in the setting, maybe it's at least interesting to see how much spam the website gets and coming from where.
Oh, and as for the agents themselves browsing the website, if the agent uses an actual browser with JS enabled (like headless puppeteer), then you would actually be able to see how the agent browsed the website.
That's a good start I'd say, but I agree with you that detection is not trivial. I wonder if there's enough value in distinguishing between AI agents (with full browser) and humans. What use cases would it enable?
For the distinguishing part, it's hard to tell if it can be done anyway, as the agent browsers are still new and constantly changing, and it's up to them if they will correctly identify themselves or not (same as with crawler/bots, the main indication is still the source IP address).
There could be use cases, like tracking if your content was stolen/parsed by an AI, maybe future pay-per-request for LLMs, etc.
Tell me a better alternative that allows me to run, say, 'markdown lint', an npm package, on the current directory without giving access to the full system on Mac OS?
I understand the concern. However, you can customize the profile (e.g., allowlist) to only allow network access to required domains. Also, looks like your sandboxing solution is Docker based, which uses VMs on a Mac machine, but will not use VMs on a Linux machine (weak security).
dockerd is a massive root-privileged daemon just sitting there, waiting for its moment. For local dev it’s often just unnecessary attack surface - one subtle kernel bug or namespace flaw, and it’s "hello, container escape". bwrap is much more honest in that regard: it’s just a syscall with no background processes and zero required privileges. If an agent tries to break out, it has to hit the kernel head-on instead of hunting for holes in a bloated docker API
Modal engineer here. This isn’t correct. You can DIY this but certainly not by wrapping EC2 which is using the Nitro hypervisor and is not optimized for startup time.
Nearly all players in this space use Gvisor or Firecracker.
Do you know Eric Zhang by chance? I went to school with him and saw that he was at Modal sometime back. Potentially the smartest person I’ve ever met… and a very impressive technical mind.
Super impressed with what you’ve all done at Modal!
You can and can’t, at least in AWS. For instance, you can’t launch a EC2 to a point you can ssh in less than 8-10 seconds (and it takes a while to get EBS to sync the entire disk from s3).
Many a time I have tried to figure a self scaling EC2 based CI system but could never get everything scaled and warm in less than 45 seconds, which is sucky when you’re waiting on a job to launch. These microvm as a service thingys do solve a problem.
(You could use lambda, but that’s limited in other ways).
I will ask what I've asked before: how to know what resources to make available to agents and what policies to enforce? The agent behavior is not predefined; it may need access to a number of files & web domains.
For example, you said:
> I don't expose entire /etc, just the bare minimum
How is "bare minimum" defined?
> Inspecting the log you can spot which files are needed and bind them as needed.
This requires manual inspection.
Article author here. I used trial and error - manual inspection it is.
This took me a few minutes but I feel more in control of what's being exposed and how. The AI recommended just exposing the entire /etc for example. It's probably okay in my case, but I wanted to go more precise.
On the network access part, I let it fully loose (no restrictions, it can access anything). I might want to tighten that in the future (or at least disallow 192.168/16 and 10/8), for now I'm not very concerned.
So there's levels of how tight you want to set it.
In the last one year, we have seen several sandboxing wrappers around containers/VMs and they all target one use case AI agent code execution. Why? perhaps because devs are good at building (wrappers around VMs) and chase the AI hype. But how are these different and what value do they offer over VMs? Sounds like a tarpit idea, tbh.
Here's my list of code execution sandboxing agents launched in the last year alone: E2B, AIO Sandbox, Sandboxer, AgentSphere, Yolobox, Exe.dev, yolo-cage, SkillFS, ERA Jazzberry Computer, Vibekit, Daytona, Modal, Cognitora, YepCode, Run Compute, CLI Fence, Landrun, Sprites, pctx-sandbox, pctx Sandbox, Agent SDK, Lima-devbox, OpenServ, Browser Agent Playground, Flintlock Agent, Quickstart, Bouvet Sandbox, Arrakis, Cellmate (ceLLMate), AgentFence, Tasker, DenoSandbox, Capsule (WASM-based), Volant, Nono, NetFence
I'm not saying sandboxes are not needed, I'm saying VMs/containers already provide the core tech and it's easy to DIY a sandbox. Would love to understand what value E2B offers over VMs?
That's right. But they (E2B) rely on the underneath Cloud infra to achieve high scalability. Personally, I'm still not sure about the value they add on top of Cloud hosted VMs. GCP/AWS already offer huge discounts to startups, which should be enough for VM-based sandboxing of agents in the MVP phase.
Well, this is the hard part, but the idea is that if you're working with both untrusted inputs and private data/resources, then your agent is susceptible to the "lethal trifecta"[0], and you should be extremely limiting in its ability to have external network access. I would suggest starting with nothing beyond the single AI provider you're using, and only add additional domains if you are certain you trust them and can't do without them.
What about VMs? They offer strong isolation, as they don't share kernels, and have long been a foundational piece for multi-tenant computing. Then, why would we put an extra layer on top and rebrand it as an AI agent sandboxing solution? I'm genuinely curious what pushes everyone to build their own and launch here Is it one of those tarpit ideas: driven by own need and easy to build?
reply