← All posts

What Agent! Can Really Do: Any App, Its Own Code, Goals That Finish, and Games

Agent! now works with 27 LLM providers. This post is the short tour of what it actually does with them. It automates any Mac app on the fly through AppleScript and Accessibility, writes its own reusable Swift scripts, keeps working on a /auto goal cycle after cycle until it is done, and builds games, from Go and Chess to the new Umbravault dungeon crawler and a fighting game drawn pixel by pixel from code.

Umbravault: a first-person view down a torch-lit corridor of the Sunken Crypt.
Umbravault, the newest game built with Agent!: an online dungeon crawler you can play at umbravault.xcf.ai.

Agent! now works with 27 LLM providers: Claude, Codex, OpenAI, Gemini, Grok, Mistral, Mistral Vibe, DeepSeek, Hugging Face, Z.ai, BigModel, Alibaba DashScope, Qwen, Qwen Code, Meta Muse, Muse Code, MiniMax, OpenRouter, Requesty, TheRouter, A2Agent, OrcaRouter, Fluxion AI, Ollama, local Ollama, vLLM and LM Studio. Apple Intelligence helps on the side, on your Mac.

A long list of providers only matters if the agent does something useful with them. So here is the short tour of what Agent! really does.

It automates any app, on the fly

AppleScript and Accessibility automation are built into Agent!. You don't install a plugin for each app, and Agent! doesn't need to know the app ahead of time.

When an app has a scripting dictionary, Agent! looks it up first (lookup_sdef) and writes AppleScript using only the commands that app really has. When it doesn't, Agent! uses the Accessibility API. It finds buttons, fields, menus and table rows by name, then clicks, types, picks options and reads the screen back as text. No screen coordinates, no guessing from screenshots.

That's how "play my Workout playlist", "take a photo with Photo Booth" or "send Mom an iMessage" work. It's also how Agent! tested one of the games below. It launched the Go app, used Accessibility to set up a 19×19 board, and read the move count back to check that the game had really been played.

It writes its own reusable code

Some jobs are bigger than one AppleScript line. For those, Agent! writes an AgentScript: a plain Swift file that it compiles at runtime and loads into the app, with the app's own permissions (Accessibility, Automation, Calendar, Contacts, Mail, Photos and so on).

The scripts stay in your library, ~/Documents/AgentScript/agents/. The next time a task needs the same thing, Agent! runs the script it already wrote, or edits it, or combines two scripts into one. About 35 examples ship with the app. If a script gets deleted, it goes to a Trash you can restore it from.

So over time Agent! builds up its own set of tools, made for the way you use your Mac.

/auto: a goal it keeps working on until it's done

Type /auto and a goal, then walk away. Agent! works in cycles:

  1. It reads the progress log from the cycles before, plus git status and the project index.
  2. It picks the single most valuable next step.
  3. It does that step completely: edit, build, fix, commit.
  4. It writes down what it did and what's left, and the next cycle starts.

There's no limit on how many cycles it runs. Sessions survive an app restart, and every tab can run its own /auto session on the same project. It stops when the goal is reached and checked with real evidence, when it's honestly blocked and needs you, or when you press Stop All.

This post is an example. It was written during an /auto session whose goal was to update the provider count everywhere and tell this story on the website, in the README and here on the blog. Each of those was a separate cycle with its own commit.

It builds games, including big ones

Games are a good test for an agent. They're large, they have many moving parts, and you can tell right away when something is broken. Here's what Agent! has built:

  • Umbravault (new): an online, first-person, step-and-turn dungeon crawler for 1 to 8 players. It has six classes, five floor themes, and a boss every third floor. It runs on Three.js in the browser and on Cloudflare Workers with Durable Objects on the server. Source on GitHub.
  • Batman v Superman: a street-fighter style 3D fighting game where every pixel is drawn from code. It has no model files, no image textures and no audio samples. The heroes, the city and the music are all generated. Read the player's guide.
  • 3D racers: GoKart.games in Godot 4, the AgentiLoop Racing F1 simulator with 18 real circuits, a three.js CrazyTaxi, and MarioKart64JS.
  • Go: a native SwiftUI Go app for the Mac. It has 9×9, 13×13 and 19×19 boards, captures, ko, handicaps, Japanese or Chinese scoring and a computer opponent.
  • Chess: Agent! plays chess right in its output panel. It also built AgenticChess, a SwiftUI chess app where Apple Intelligence is the opponent.
The Go app: a 19 by 19 wooden board with black and white stones after 30 moves, and the game panel on the right.
The Go app after 30 moves on a 19×19 board. Agent! played the game itself to take this screenshot.

You can see all of them in the games table on the home page.

Try it

Agent! is free for personal use, on macOS 14.6 or later, Apple Silicon or Intel. Pick any of the 27 providers, or run fully local. Then give it something big with /auto and see how far it gets.