· Infrastructure
Inside FlytBase's Workflow for Testing and Deploying Emerging AI Models
Most teams talk about AI through the lens of ChatGPT, Claude, or Gemini. But what happens when the model you actually want to use doesn’t have an API, doesn’t run locally, and barely has documentation?
That’s the problem tackled in a recent AI in Action session by Abhay Thakur, Principal Software Engineer at FlytBase. As a physical AI company working heavily with drone data, the team processes massive amounts of images and video rather than plain text — so vision-language models, embedding models, and multimodal systems matter far more than generic chatbots, and unlike mainstream LLM APIs, these systems require a completely different deployment mindset.
The Gap Nobody Talks About
Finding interesting models is easy. Actually running them is not. Once you move beyond mainstream models on Hugging Face, things get messy fast — many niche or newly released models have no inference providers, don’t support standard pipelines, require custom handlers, break on local machines, and need specialized GPU setups. That’s especially true for multimodal systems involving images, videos, and embeddings.
The session focused on experimenting with Qwen-based embedding models capable of understanding both text and visual inputs together — not just generating embeddings, but semantic understanding for questions like “show me where an accident happened” or “retrieve events involving both vehicles and pedestrians.” These require multimodal retrieval systems reasoning across images, video frames, and metadata simultaneously.
Why Running Models Locally Still Hurts
At FlytBase, most developers work on Macs. Loading large multimodal models locally often becomes impractical — limited GPU support, exploding memory requirements, unstable inference, setup that takes hours or days. And in the AI world, “a day or two” is already too slow.
Hugging Face Spaces Became the Shortcut
For many developers, Hugging Face is simply a model repository. But Spaces unlock something more powerful — lightweight, containerized runtimes where developers can deploy custom AI interfaces rapidly without building full infrastructure stacks from scratch. The FlytBase team used Spaces to wrap a multimodal embedding model with a custom handler, Gradio interfaces, GPU-backed inference, API endpoints, and video/image processing pipelines.
Building a Working Interface in Hours Instead of Days
The deployment boiled down to just a few moving parts: an app.py Gradio initializer, a handler file for inference logic, a configuration README, and GPU runtime selection — enough to create a fully interactive interface processing text, images, video samples, and embedding generation, with APIs exposed automatically.
The Real Skill Is Choosing the Right Deployment Path
- For quick testing: Hugging Face Inference APIs, OpenRouter
- For niche or unsupported models: Hugging Face Spaces
- For cheap GPU access: RunPod, Vast.ai with vLLM
Why OpenRouter Is Becoming More Important
Initially treated as a testing playground, OpenRouter is increasingly becoming an orchestration layer for cost optimization — routing small tasks to cheaper models, reserving expensive models for hard problems, maintaining one unified API interface, and building failover systems easily. The future probably won’t revolve around a single dominant model. It’ll revolve around systems intelligently routing workloads across many models based on cost, latency, and capability.
AI Engineering Is Becoming Infrastructure Engineering
The challenge is no longer just “which model is smartest?” Now the questions are: where should this model run, how fast can we test it, can it scale affordably, what infrastructure tradeoffs exist, how do we expose it safely, can we switch models later? That’s a very different mindset from basic prompt engineering — and teams that learn these infrastructure workflows early will likely move much faster than teams relying entirely on closed APIs.
“There’s an enormous gap between discovering an interesting AI model and turning it into a working product feature.”
Most developers stop at discovery. The teams pulling ahead are the ones learning how to bridge that gap quickly — sometimes using Hugging Face Spaces instead of local deployments, sometimes routing workloads through OpenRouter, sometimes renting GPUs and deploying through vLLM. The tools matter less than the mindset.