Hybrid Search Recommendation API
A low-latency retrieval pipeline combining lexical matching, semantic embeddings, and downstream user segmentation.
The Data Problem
The Operating Context
"A traditional keyword system was failing users on typos and conceptual queries (e.g., 'breakfast' vs 'cereal'). We needed a reliable source-to-retrieval flow that protected latency without requiring cloud LLMs."
Designed to reduce the 'zero results' problem in large retail catalogs, the workflow needed reliable query preparation, retrieval indexing, and ranking before the downstream search layer could understand intent beyond exact keyword overlap.
- The Reliability Trade-offs: Each workflow balanced data preparation effort, downstream usefulness, and measurable reliability under practical constraints.
- Constraint 01: VM/CPU Training Limits: Constrained upstream data preparation and retrieval components to efficient Bi-Encoders instead of massive LLMs.
- Constraint 02: Decoupled Deployment: Kept runtime artifacts separate from heavy vector databases so the serving path could consume prepared indexes without choking.
The Data Flow
Pipeline Rationale
Data Flow
Built a staged workflow that prepares queries, runs Lexical and Semantic retrieval, and combines results in a decoupled FastAPI ecosystem; representation learning and query understanding remain downstream components.
Downstream Outcome
End-to-end retrieval engineering: bridging typo recovery, cosine-similarity embeddings, and cohort-level personalization within a sub-100ms inference path.
Lexical stage
BM25 matching paired with SymSpell recovery to transform noisy queries into precise candidates.
Semantic stage
IndoBERT encoders compressed into an in-memory Meta FAISS index for downstream similarity retrieval.
Validation Results
Validation evidence: Stage 1 retrieval achieved NDCG@10 of 0.798 for candidate selection.
Downstream ranking evidence: Stage 2 personalization reached combined MRR of 0.373 and Recall@5 of 0.558.
Pipeline latency evidence: Maintained an average latency of 98.9ms across the hybrid pipeline.
Retrieval Delivery & Reflections
Pipeline boundaries: Training and inference are decoupled into two isolated FastAPI layers so prepared vector indexes do not choke the API runtime.
CPU-aware processing: CPU-only deployment led to optimized Bi-Encoders and FAISS HNSW over heavy Cross-Encoders to maintain sub-100ms speeds.
Artifact refresh: Offline jobs handle embedding updates and push lightweight runtime checkpoints so the serving API consumes refreshed data without downtime.