Talking Photos: AI video from a single photo
A talking-pet clip normally takes animation skills, a voice recording and a video editor. Talking Photos does it in one browser flow — and marks every video it exports as AI-made.
The problem
Short videos where a cat, a dog or a coffee maker talks are popular, but making one means animating a face, recording or generating voices, and editing a vertical video with captions. Most people stop at the first step.
What was built
- A seven-step editor — characters, dialogue, voices, layout, montage, preview, download — that guests can start without an account; the draft moves into the account after sign-in.
- Four ways to animate a photo: free cartoon eyes and mouth, a Live photo mode that deforms the real photo, a Neural mode built on LivePortrait, and a realistic GPU Studio mode for people that is in testing.
- AI voices: 10 free Supertonic voices and 30 expressive Gemini voices with per-line delivery such as joyful, whisper or sarcastic.
- An AI assistant that writes the dialogue from the characters’ personas, an “Arrange with AI” montage and an autopilot that goes from one idea to a rendered video.
- 1080p or 720p MP4 export in 9:16 or 1:1 with burned-in captions, a free plan, and PRO subscriptions and token packs through Dodo Payments.
- The interface in 14 languages.
The GPU service
The realistic talking head runs as a separate Python service on rented GPUs. It never accepts inbound traffic: workers call the API to claim a job, fetch its inputs, report progress and upload the result.
- Two interchangeable serverless backends, Vast.ai and RunPod, chosen with one setting and running the same Docker image. Both scale to zero, so idle time costs nothing.
- The API wakes the provider once per queued clip; jobs are claimed with leases that heartbeats extend, so a crashed or scaled-down worker loses nothing — the lease expires and the job is dispatched again.
- Ditto lip-sync runs as TensorRT engines, combined with LivePortrait-format motion recorded from real people for head pose, blinks and gaze, and a Mask R-CNN silhouette that drives subtle upper-body motion.
- Moving Mask R-CNN to TensorRT and dropping CUDA PyTorch cut the image from 11.3 GB to 4.64 GB unpacked with near-identical output, and cold starts on Vast.ai from 4.5–22 minutes to about 3.75.
AI in the browser and on the server
- Face models run on the visitor’s device: MediaPipe face landmarks, cat and dog landmark models, and LivePortrait and Depth Anything V2 through ONNX Runtime Web, using WebGPU where available.
- One Rust animation engine is compiled to WebAssembly for the live preview and runs natively in the export renderer, with golden-frame tests that keep the two identical.
- Supertonic-3 speech runs on the CPU worker through ONNX Runtime with pinned, checksum-verified weights; Gemini voices come from Google Cloud Vertex AI; DeepSeek or Gemini writes dialogue.
- A job queue built only on PostgreSQL — row locking, lease fencing, LISTEN/NOTIFY — streams progress to the browser over server-sent events, without Redis or a separate queue.
Every video is marked as AI
Each export carries a visible AI badge, signed and time-stamped C2PA Content Credentials and an inaudible audio watermark. Anyone can check a video on the public verify page.
Running it
- Spending guards for paid AI: cost reservations, a monthly budget cap and a daily cap on Gemini voice minutes.
- A health endpoint and built-in alerts for a stopped worker, failing or stuck jobs, an unknown voice and storage problems, plus outside uptime checks.
- Daily database and media backups to separate storage, with a weekly restore check.
Evidence
From the project’s own measurements, October 2026: the GPU worker renders about 1.7 seconds per second of video on an RTX 3060, and a renderer optimisation cut a 30-frame 1080×1920 export from 52.03 seconds to 2.73.
Status and limits
The web app is live, and the Android app is on Google Play (checked 23 Oct 2026). The realistic Studio mode is in testing and limited to selected accounts. The app was released in October 2026, so no user numbers are claimed here.
- Talking Photos — Turns a photo of a pet, person or object into a short voiced, lip-synced video for TikTok, Shorts and Reels.
- Audience
- People and creators making short vertical videos for TikTok, YouTube Shorts and Instagram Reels
- Stack
- Rust (axum, sqlx) + PostgreSQL; Next.js 16 and React 19; Python GPU worker with TensorRT on Vast.ai or RunPod serverless; ONNX Runtime, MediaPipe and WebGPU in the browser; MinIO; Docker Compose and Caddy on Hetzner
- AI models
- Ditto talking head, LivePortrait motion, Mask R-CNN, Supertonic-3 and Gemini text-to-speech, DeepSeek or Gemini for dialogue, MediaPipe face landmarks, Depth Anything V2
Custom web applications are a studio service — see the service