Macchinina

The problem

I wanted to give an AI agent a physical body. The goal was a robot I could drive by hand from a web page, that an LLM (Claude Code) could also pilot, and that had a little home autonomy: knowing which room it is in and moving towards objects I name aloud. It is a personal experiment, with no client.

What I built

• Two halves talking over MQTT: a Mac running Docker Compose (broker, brain, Flask web controller with video and two-way audio) and a Raspberry Pi 5 running systemd services.

• Perception on the Pi: YOLO11n object detection about once per second, room recognition with DINOv2 embeddings and k-NN over reference photos, ultrasonic distance and a WiFi signal fingerprint as an extra signal.

• A reproducible mirror of the Pi setup in the repo (deploy, provisioning and check scripts), and tests that run against a fake robot, never real motors.

How the agent works

Claude connects through an MCP server that exposes tools to drive, follow a short path, approach a named object, stop, look through the camera, read perception, listen and speak. Each tool has hard limits enforced on the Pi and returns a structured outcome (arrived, lost, not found, blocked, timeout, operator). For named objects, a resident process on the Mac runs OWLv2 for open-vocabulary detection and steers the robot in a closed loop.

Guardrails live on the robot, not in the model:

• A command whitelist on the Pi, so LLM output is never forwarded raw.

• Every AI command is valid for under a second and must be renewed at about 10 Hz, so if the Mac or the network drops the robot stops by itself. A watchdog and an obstacle stop under 20 cm back this up.

• Human in the loop: the operator always has priority over the agent, and autonomy (off, dry run, on) is never active at startup and expires without a heartbeat from the UI.

Interesting decisions

• Split by latency: deterministic reflexes and motor control on the Pi, slow deliberation on the Mac.

• Stop-and-go driving: the camera blurs while moving, so the pilot looks while stopped, makes one move, then looks again.

• Camera and microphone are off by default and switched on with a keepalive.