AI Pets
Back to the blog

Published · By AI Pets · 5 minute read

Open-source firmware · Apache 2.0

Build your own AI Pet — from your browser to an ESP32

Make a little character. Talk with it in your browser. Then give it a small screen of its own, and talk to it out loud.

Create your pet
Luna, a small green AI Pet, beside a round screen showing the AI Pets control deck
Luna, and the control deck you see when you swipe down on the device. Every device screen in this post is a render from the firmware’s own preview tool, not a photo.

An AI Pet is a small animated companion you make yourself. It has a face, a way of moving and a personality you write. It talks with you in your browser, and it can live on a little touch screen on your desk.

Create an account at aipets.com and you can do all of it. The device firmware is open source, so you can also build it yourself, change it, and connect it to your own backend.

Start with a character

Every AI Pet starts as an .aipetframes pack: 240 × 240 animation frames for resting, listening, thinking and speaking, reactions to a touch or a shake, and a few moments that belong to that character alone.

You make one with the AI Pets Sprite Skill for Codex, which is open source under MIT. Describe a character, choose the face you love, and Codex builds its animations, checks them and packs them with a preview you can try in your browser. Our guide to making an AI Pet with Codex walks through it.

Then upload the pack under Your pets. You can play every animation, name your pet and give it a short personality without any API key.

Talk with it in your browser

Conversations run on your own provider accounts. A new pet uses Cartesia to hear you and to speak, and a model on OpenRouter to write its replies, so add both keys in Settings. If you prefer one key, a pet can also run on OpenRouter alone.

Then tap your pet and type a message, or record one, and it answers out loud. Your providers bill you for what you use, and opening a page never starts a paid conversation.

Give it a little body

The device is an ESP32-S3 touch board with microphones, a speaker codec and a motion sensor. The firmware supports three boards from Waveshare:

  • ESP32-S3-Touch-LCD-1.85B, a 1.85-inch round screen (360 × 360). This is our reference board, the Pocket.
  • ESP32-S3-Touch-AMOLED-2.06, a 2.06-inch AMOLED screen (410 × 502).
  • ESP32-S3-Touch-LCD-1.54, a 1.54-inch screen (240 × 240).

Another ESP32-S3 board with 16 MB of flash and 8 MB of PSRAM can be added by following the porting guide.

The AI Pets start-up screen on a round screen, a tall rectangular screen and a small square screen
The start-up screen on each board, rendered at its native resolution with the firmware’s LVGL preview tool and shown at the same scale. Left to right: the round 1.85B, the AMOLED 2.06 and the LCD 1.54.

Three ways to start

  1. Install from aipets.com. Connect the board by USB, sign in at aipets.com in Chrome or Edge on a Windows or Mac computer, and open Devices. Install or update firmware installs the build for the board you connect. Before writing anything, the page checks which board it is and lists exactly what it will write. If nothing on a brand-new board shows its model, the page asks you. A board that isn’t running AI Pets yet, such as one still showing Waveshare’s demo, gets Erase and install.
  2. Build it yourself. Get the firmware source, build it with ESP-IDF 5.5.3 on your computer or in Docker, and flash it with esptool. Each GitHub release also has ready-made images for every board.
  3. Connect it to your own backend. Build the firmware with your backend’s address and release key. The backend guide explains what each setting covers. A self-hosted AI Pets backend for Docker, also under Apache 2.0, is coming soon.

Say hello

On first start, the device asks for Wi-Fi. Pick your network and type the password on the screen, or plug it in and use Set up Wi-Fi on the Devices page to type with your computer’s keyboard.

Then it shows a link code and AIPETS.COM/LINK. Sign in at aipets.com/link and type the code. The device asks whether to link to your account, next to a three-digit number, and aipets.com shows a number too. If they match, tap LINK. If they don’t, tap NOT ME: someone else typed your code, and the device shows a new one.

Two round screens: a link code with AIPETS.COM/LINK, then a question with the number 472
Linking on the round board. The code, account name and number are sample values from the preview tool.

A little company on your desk

Open your pet on aipets.com, choose your device and install it. The pet travels over Wi-Fi, and the device checks every byte before switching to it. It holds up to three pets: swipe left or right to switch.

Tap to talk. The device records your question and plays your pet’s spoken answer. Speech recognition, your pet’s personality, memory and voice run on the backend, not on the ESP32.

Press and hold to pet it. Your pet then tells you what is happening in its own world, and you answer by voice. Give the device a shake for a reaction.

Swipe down for the control deck: Wi-Fi, volume, brightness, linking and a system log that shows what the device is doing. Swipe up to go back to your pet.

Yours to take apart

The firmware is open source under the Apache License 2.0, copyright Gauss Labs. Its repository holds the application, the player that animates your pet, board support, host tests that run without a board, and the device protocol it speaks.

Fonts, sounds and libraries in it keep their own licenses. The AI Pets name, logo and official characters are not part of the license, so make a pet of your own. The Codex sprite skill and Luna for Claude are open source as well, under MIT.

From an idea to a friend on your desk

A face you chose. A voice you picked. A little screen that answers when you tap it.

Create an account, install the Codex skill, or open the firmware on GitHub.