Multi-Agent Multimodal AI
VisionSpec
A tri-agent system that translates screen recordings into functional frontend implementations by coordinating visual understanding, architecture planning and code generation.
LangGraphPythonMultimodal AIVLMTool Calling
VisionSpec explores how multimodal foundation models and coordinated AI agents can reason across visual, spatial and temporal information in interface recordings.
Architecture
SCREEN RECORDING ↓ WATCHER ↓ ARCHITECT ↓ BUILDER ↓ FRONTEND IMPLEMENTATION
WATCHER Analyzes the recorded interface and extracts visual and interaction context.
ARCHITECT Transforms observations into an implementation plan and structured representation.
BUILDER Uses the architecture and extracted information to generate the frontend implementation.
Tech Stack
- Python
- LangGraph
- Multimodal AI
- Vision-language models
- LLM tool calling
- Agent state management
- Gemini 3.1 Pro