The R&D Lab
Where new senses are born.
Every serious product in this platform can see, hear, or act because something in the lab learned to first. This is the playful end of the portfolio — voice assistants, avatars, game coaches, learning agents — and it's also the proving ground for every way of seeing, hearing, and acting the rest of the company inherits.
"Hey Jarvis" — and it's listening
The voice assistant is complete and working: a wake word opens a real conversation — not a one-shot command — that stays open for follow-ups until you go quiet, all running through the platform's supervised action layer.
- Wake word → on-device speech recognition → the supervised action layer → a natural computer voice, end to end on your own hardware
- Conversational mode: wake once, then talk — follow-ups need no re-wake, and the session sleeps on silence
- Real desktop actions proven by voice: "turn Discord down to 30 percent" understood, carried out, and verified
- Suggestions are spoken but never carried out automatically — sensitive actions wait for the identity check
🧬Avatar + identity check
The face of the assistant and the lock on its hands. A 3D talking-head avatar with lip-sync floats on the desktop, while an identity check — your voice plus your face — sits between any sensitive suggestion and the moment it happens. Locked by default and verified: no match, no action. It turns "a human approved" into "the authorized human, verified live, approved."
The game coach that watches and listens
Built for a kid's Fortnite sessions: a real-time squad coach that sees the screen, hears the party chat, and calls out what matters — watching and listening only, never touching the controls.
👁️Screen vision
Computer vision on a local graphics card reads the screen every ~1.5 seconds, about 0.3 seconds per look, with a cloud backup — timed so callouts stay current, not stale.
🎧Hears the game
The coach hears the game's own sound: party voice chat is transcribed live by on-device speech recognition, so it reacts to "enemy north" like a teammate — and knows who said it by their voice.
🗣️Teammate manners
Spoken callouts under ten words, a pause between them, and a hard bias toward silence in lobbies — a good squadmate mostly stays quiet.
An agent that learns the game by watching
The ambitious cousin of the coach: a game agent that learns by watching how a human plays. Every session banks matched-up video and controller input — what was on screen, and what the human actually did — into a growing training library.
- Computer vision that can be told what to look for auto-labels enemies across thousands of frames — including the catch that game characters aren't always human-shaped
- A continuous learning loop retrains after every finished game and logs the results, so improvement is measured game over game
- A human-assisted labeling tool for clean training data — because the honest finding was that label quality, not data volume, set the accuracy ceiling
- Honest limits, stated up front: an agent that learns by watching can only get as good as the player it watched, and simple reaction-time physics bounds what a see-then-act loop can do
Why it matters beyond games
Playful on the surface, structural underneath
None of this ships as a product, and that's by design. The lab is where ways of seeing, self-improving training loops, wake-word ergonomics, and verified human-approval controls get invented cheaply — on games and desktop assistants — before the serious products bet on them. The weather platform's prediction-grading loop, the AI's sense wiring, and the voice-and-face identity check all trace back here.
The lab's one rule: watching and listening are free, acting is controlled. Everything here can see and hear on your own hardware, but anything that acts goes through the same human-approval and identity-check discipline as the rest of the platform.
On-device speech recognitionNatural computer voiceLocal computer visionDesktop overlayVoice + face verification