Sketch of an LLM agent that is a Pet Dæmon plugin

[kriskowal]

This is a not-quite-working sketch that connects the Pet Dæmon API to the Ollama API, shuttling messages between users and the LLM. It is not in the right shape, doesn’t do the right work, and certainly isn’t executing proposed code, but more on that later.

This presumes you have a working version of Ollama running locally. I’ve sketched a way to communicate the OLLAMA_HOST environment variable, but the Pet Dæmon doesn’t thread environment variables yet. That’s a relatively small change to Pet Dæmon and the Endo CLI, like supporting -D OLLAMA_HOST=http://llamabox.local:8888.

You also need the Qwen3 model, so ollama pull qwen3 to download that and make it available to the Ollama API.

You need endo in your path, which I set up with a symlink to packages/cli/bin/endo

At the repository root, yarn, yarn build.

In packages/llamadrome,

endo run --UNCONFINED setup.js --powers HOST is one of the commands that is sufficiently spooky that you read setup.js very carefully. This sets up a llamadrome handle you can chat with. It should send you a message when it starts.

endo inbox will show you that @llamadrome said it’s ready. endo inbox -f will follow the messages as they come in, until you ^C.

endo send llamadrome "What is the meaning of life, the universe, everything" will put a message in the LLM-agent inbox.

LLM-agent will grab the message, add it to a transcript, and push that down to Ollama over HTTP. When a response comes in, it makes a meager attempt to send the response back to its host (you).

So, you should get the message back on your endo inbox when it’s done.

Once this is working, the next step is to reframe the system prompt as a tool. That’s just more information up front in the Ollama Chat request

https://docs.ollama.com/api/chat

When you send a chat request to Ollama with a transcript, the transcript contains messages for various roles. The system role tells the LLM what it is. You inform the LLM about tools and the LLM may respond with a tool request instead of a user response. So, we would introduce an evaluate-in-sandbox tool and move the information I’ve provided in the tentative system prompt there.

Then, the system prompt is free to provide different information about what the LLM is in relation to the user, to inform how it answers. We have options for how to do that.

The powers that the LLM receives are missing an evaluate method, which would be implemented a lot like send, in that it would generate another kind of pet dæmon message, asking permission to run some code in a particular worker. That’s more pet dæmon work, but mostly just extending existing design patterns. This will require us to find a way to avoid asking the user a Yes or No question. This should be more like “Execute” vs “Deny”. We have some design options.

We also have the option of presenting more tools so the LLM can interact with the pet dæmon powers in other ways, but code should suffice.

Sending a message to the LLM can also potentially carry @powers so we should probably train the LLM to interact with the messages in their inbox with either code or more tools.

I think once we get this working, it will be pretty straight-forward to iterate.

This will work with the existing web UI under package/cli/demo/cat.js, but that’s a bit of a chore to set up that I want to simplify in the scope of polishing the final demo for the Foresight grant. #3019