# Query Understanding Service Low-Level Design: Intent Classification, Entity Extraction, and Query Rewriting

Source: https://www.techinterview.org/post/3233470347/lld-query-understanding/
Updated: 2026-04-18 · techinterview.org

## Query Understanding Service Overview

The Query Understanding Service (QUS) sits between the raw user input and the retrieval layer. It enriches a plain-text query with structured annotations — intent labels, named entities, corrected spelling, and a semantically rewritten form — so that downstream retrieval and ranking components can act on richer signals than keyword tokens alone.

## Requirements

### Functional Requirements

- Classify query intent into a taxonomy (navigational, informational, transactional, local).

- Extract named entities: people, organizations, locations, products, and domain-specific concepts.

- Detect and correct spelling errors using a noisy-channel model and domain vocabulary.

- Rewrite ambiguous or underspecified queries into canonical forms using a seq2seq model.

- Return enriched query annotation within 30 ms at p95.

### Non-Functional Requirements

- Throughput: 50,000 queries per second, horizontally scalable.

- Model updates deployable without service restart via hot-swap.

- Annotation confidence scores included so callers can apply thresholds.

## Data Model

The **QueryAnnotation** object returned by the service contains:

- `raw_query` — original user input.

- `corrected_query` — post-spelling-correction form.

- `rewritten_queries[]` — one or more semantically equivalent rewrites with confidence scores.

- `intent` — top intent label and probability distribution over the full taxonomy.

- `entities[]` — list of `{span, entity_type, canonical_id, confidence}` tuples.

- `language` — detected language code (BCP-47).

An offline **Query Log** table stores raw queries, resulting annotations, and downstream engagement labels for periodic model retraining. Keys: `query_id UUID`, `session_id`, `timestamp`, `annotation JSON`, `click_signal BOOL`.

## Core Algorithms

### Intent Classification

A fine-tuned BERT-class transformer trained on human-labeled query-intent pairs produces a softmax over four top-level intent buckets and up to 200 fine-grained sub-intents. The model is quantized (INT8) and served via ONNX Runtime to meet the 30 ms budget. A lightweight rule layer intercepts high-confidence patterns (e.g., URL-like strings map directly to navigational intent) before the neural model runs.

### Named Entity Recognition

A sequence-labeling NER model (BERT fine-tuned with BIO tagging) identifies entity spans. Recognized spans are linked to a knowledge-graph canonical ID via a fuzzy prefix trie over entity surface forms. Ambiguous spans are resolved using surrounding context tokens.

### Spelling Correction

A noisy-channel model scores candidate corrections as `P(intended | observed) ∝ P(observed | intended) × P(intended)`. Edit distance candidates are generated using a weighted Damerau-Levenshtein automaton. The language model prior is a trigram model trained on query logs, compressed with KenLM for sub-millisecond lookup.

### Semantic Query Rewriting

A seq2seq T5-small model trained on (query, expanded query) pairs generates alternative phrasings. Rewrites are ranked by a cross-encoder relevance model against the corrected query and the top-three are surfaced with confidence scores. Downstream retrieval may issue parallel searches across original and rewritten forms and merge results.

## API Design

- **AnnotateQuery(QueryRequest) → QueryAnnotation** — main synchronous call; returns full annotation object.

- **BatchAnnotate(BatchQueryRequest) → BatchQueryAnnotation** — used by offline evaluation and training pipelines.

- **GetModelVersion() → ModelVersionInfo** — returns checksums and training metadata for each deployed sub-model; used by monitoring dashboards.

The service is exposed as a gRPC endpoint with a JSON/HTTP transcoding layer for tooling compatibility. Callers pass an optional `annotation_mask` field to request only a subset of annotations and reduce latency.

## Scalability and Fault Tolerance

Each QUS instance is stateless; horizontal scaling is achieved by adding replicas behind a load balancer. Models are loaded once at startup into shared memory, keeping per-request heap allocations near zero. A model registry service (backed by object storage) publishes new model artifacts; instances poll for updates every 60 seconds and perform a non-blocking hot-swap. If the swap fails, the previous model remains active and an alert fires.

Circuit breakers protect each sub-component. If the NER model exceeds its latency budget, the service returns a partial annotation (intent and correction only) rather than timing out the full request. SLO: full annotation p95 ≤ 30 ms, partial annotation p99 ≤ 15 ms.

## Monitoring

- Track intent distribution drift over 1-hour rolling windows; alert if any bucket shifts by more than 10% relative.

- Monitor correction acceptance rate (fraction of corrected queries that receive user engagement) as a proxy for correction quality.

- Log annotation confidence histograms to detect model degradation between retraining cycles.

See also: [Netflix Interview Guide 2026: Streaming Architecture, Recommendation Systems, and Engineering Excellence](https://www.techinterview.org/post/3233460274/netflix-interview-guide-2026-streaming-architecture-recommendation-systems-and-engineering-excellence/)

See also: [Scale AI Interview Guide 2026: Data Infrastructure, RLHF Pipelines, and ML Engineering](https://www.techinterview.org/post/3233460275/scale-ai-interview-guide-2026-data-infrastructure-rlhf-pipelines-and-ml-engineering/)

See also: [LinkedIn Interview Guide 2026: Social Graph Engineering, Feed Ranking, and Professional Network Scale](https://www.techinterview.org/post/3233460278/linkedin-interview-guide-2026-social-graph-engineering-feed-ranking-and-professional-network-scale/)

See also: [Meta Interview Guide 2026: Facebook, Instagram, WhatsApp Engineering](https://www.techinterview.org/post/3233460272/meta-interview-guide-2026-facebook-instagram-whatsapp-engineering/)