One promising direction is building abstraction layers to sandbox individual tools, even those that don't have an API already. For example, you could build/vibe code a daemon that takes RPC calls to open Amazon in a browser, search for an item, and add it to your cart. You could even let that be partially "agentic" (e.g. an LLM takes in a list of search results, and selects the one to add to cart).
If you let OpenClaw access the daemon, sure it could still get prompt injected to add a bunch of things to your cart, but if the daemon is properly segmented from the OpenClaw user, you should be pretty safe from getting prompt injected to purchase something.
I like simonw's definition: "An LLM agent runs tools in a loop to achieve a goal."
I guess agent isn't the best term here since the LLM wouldn't be driving the logic in the daemon. Using an LLM to select which item to add to the cart would mimic the behavior of full agentic loop without the risk of it going off the rails and completing the purchase.
So if I understand correctly, in an agent, the LLM is in charge, but it can send part of the work off to other tools. And the problem here is that we're trying to have something in charge over the LLM, which is the reverse of the "agent" setup. Do I have that right?
Yeah, OpenClaw agents have a full set of tools to interact with a browser in arbitrary ways. My idea was to instead give it a tool for a browser wrapper with a limited API surface. And that tool could use LLMs internally in specific contexts.
If you let OpenClaw access the daemon, sure it could still get prompt injected to add a bunch of things to your cart, but if the daemon is properly segmented from the OpenClaw user, you should be pretty safe from getting prompt injected to purchase something.