Should online vector search be split from offline batch jobs?

A common but rarely discussed pattern with vector search: one large embedding collection ends up serving two very different kinds of workloads. The first is online retrieval, where users are waiting and latency directly matters — if p95 or p99 response times slow down, the product feels broken, so dedicated, always-on compute makes sense here. The second workload is offline and batch-like: mining hard negatives before a training run, deduplicating a large embedding collection, inspecting clusters or drift, running offline evaluations, or exploring a dataset for a few hours before going quiet for a week.

This second type doesn't care whether a query takes 50ms; what matters is that the job finishes, costs stay bounded, and it doesn't interfere with production retrieval. The core tension raised is whether it makes sense to keep serving-style compute warm all month just to handle this occasional analytical workload, when the two workloads actually have very different requirements that tend to get blurred together.