An AI Companion That Actually Plays With You: Inside Varkos

Most AI NPC demos are just chatbots wearing a game costume. They talk well but act poorly, and the latency is masked by cutting away between player and AI. Pantelis Kalogiros wanted something different: a companion that fights, loots, follows multi-step instructions, and does it fast enough for VR.

The result is Varkos, a demon reincarnated as a dog in Skyrim. He's not a dialogue menu — the microphone is always on, and you just talk to him. He can wait for a signal arrow, then fetch a potion. He can search the world state for a ceremonial sword and bring you a different one when he can't find it. He plays hide-and-seek as a persistent goal, not a single API call.

Architecture: Local-First, Hybrid Brains

The system runs on a Windows PC with an M4 MacBook handling audio and the 'brain.' The key insight is that a single big LLM is too slow and expensive for real-time control. Instead, Kalogiros uses a hybrid approach: behavioral graphs and traditional NLP for the fast, reactive paths, and a large model only for slow, offline personality evolution.

Audio is processed with an optimized Qwen3-ASR 1.7B model, stitched into rolling partials to handle streaming. VAD methods like Silero detect turn boundaries, and lexical analysis decides if you're done talking. The goal is 40-80ms audio processing. Speech generation uses PocketTTS-Raven, an optimized TTS engine.

Personality That Evolves

Varkos starts as a proud, sarcastic demon trapped in a dog's body. As you travel together, significant interactions become evidence for gradual personality changes. He might become more domesticated, start bringing you toys, or learn to love cabbage. These changes are versioned and reversible, and they affect his emotional state, which in turn influences combat behavior.

Beyond Skyrim: Void Mode

Varkos is designed to follow you across games. When the game closes, he enters 'void mode' — he can't see or feel anything, and his reaction depends on how you've treated him. This also serves as a bridge between games: one moment he's fighting a dragon, the next he's beside you in Microsoft Flight Simulator, trying to chase the sun.

The project is a work in progress, but the Qwen3-ASR harness is promised as open source soon. If you're building AI companions for games, this is a blueprint worth studying.