Spam Detection System Low-Level Design: Velocity Checks, Graph-Based Detection, and Classifier Ensemble

Spam Detection System Overview

Spam detection combines fast rule-based velocity checks, text classification, and graph-based account analysis. No single signal is sufficient — spammers adapt to any single detector. The system scores submissions using an ensemble of signals and enforces in real time at the point of posting or messaging.

Spam Signal Categories

Velocity signals: Too many posts or direct messages within a short time window.
Content signals: Repeated identical messages, known spam URLs, promotional patterns.
Behavioral signals: Account created recently, no profile photo, mass follow/unfollow cycles.
Graph signals: Connected to known spam accounts in the follow or messaging graph.

Rule-Based Velocity Checks

Velocity rules execute in under 1ms using Redis counters with sliding windows:

Posts in the last hour per account.
DMs sent today per account.
New follows in the last 10 minutes.

When a counter exceeds a threshold, the action is immediately rejected or rate-limited. Redis INCR with TTL-based expiry implements sliding window counters efficiently. These rules catch the most obvious bot behavior before any ML inference runs.

Text Classifier

A BERT-based classifier fine-tuned on labeled spam vs legitimate content identifies non-obvious spam:

Detects spam URLs not yet in blocklists.
Identifies obfuscated spam text (character substitution, spacing tricks).
Trained on historical labeled data augmented with user reports.

The model outputs a spam probability score. Inference runs in ~10ms on GPU, or ~50ms on CPU with quantized model.

Graph-Based Detection

Spam accounts do not operate in isolation — they follow each other, message each other, and are created by the same operators. The account graph exposes this:

Build a graph: nodes are accounts, edges are follow/message relationships.
Apply community detection (Louvain algorithm) to identify tightly connected clusters.
If any node in a cluster is a confirmed spam account, flag the entire cluster for review.
Graph analysis runs as an offline batch job (hourly or daily), feeding a risk score into the serving feature store.

Ensemble Scoring

Signals are combined into a single spam probability score:

spam_score = w1 * velocity_score
           + w2 * text_classifier_score
           + w3 * graph_risk_score
           + w4 * account_age_signal

Weights are calibrated on labeled data. The combined score is compared against enforcement thresholds: auto-reject, queue for review, or allow.

Real-Time Enforcement

The spam score is computed synchronously at post/message submission time:

Velocity check (Redis, <1ms).
Text classifier inference (~10-50ms).
Graph risk score lookup from feature store (Redis, <2ms).
Ensemble score computed.
Action: reject with error, silently drop (shadow ban variant), or allow.

Feedback Loop from User Reports

User reports are the primary source of new labeled training data:

Reported content is reviewed and labeled as spam or not spam.
Labels feed into weekly or monthly classifier retraining.
Rapid-response pipeline: new spam URL domains can be added to blocklist within minutes of first report without model retrain.

IP and Device Fingerprint Signals

Multiple accounts originating from the same IP or device fingerprint are a strong spam signal:

IP-to-account mapping stored in Redis: if one account on an IP is confirmed spam, other accounts from the same IP receive elevated risk scores.
Device fingerprint (browser, OS, screen resolution, timezone) used similarly for web-based spam.

Hash-Based Deduplication

Identical or near-identical spam messages are detected before classifier inference:

Exact match: SHA-256 hash of message content, checked against a bloom filter of known spam hashes.
Near-duplicate: SimHash (locality-sensitive hashing) detects messages that differ only in minor character substitutions, whitespace, or URLs.

False Positive Mitigation

Legitimate users must not be incorrectly blocked. Risk-adjustment factors:

Verified accounts (phone-verified, payment method on file) receive a lower base spam score.
Accounts with long positive history get more benefit of the doubt before enforcement.
Appeals process available for incorrectly actioned accounts, with rapid human review SLA.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “How do you implement velocity checks in a spam detection system, and what data structures support them?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Velocity checks count how many times an entity (user, IP, device fingerprint, email domain) performs an action within a sliding time window. The canonical implementation uses a Redis sorted set where each event is stored as a member with its timestamp as the score. To check velocity, you ZRANGEBYSCORE the last N seconds and count results; to prune, you remove entries older than the window. For high-throughput scenarios, a Count-Min Sketch provides approximate counts with configurable error bounds using O(1) space per entity. Typical thresholds: more than 50 messages/hour from a new account, more than 10 failed login attempts/minute from an IP, or more than 100 email sends/day from a domain. Thresholds are tiered by account age and historical reputation to reduce false positives on legitimate power users.”
}
},
{
“@type”: “Question”,
“name”: “How does graph-based spam detection work, and what graph algorithms are most effective?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Graph-based detection models the social or communication network as a graph where nodes are accounts and edges represent interactions (messages, follows, transactions). Spam rings appear as dense subgraphs with high internal connectivity and low external connectivity. Effective algorithms include: Label Propagation, which iterates spam labels from known bad nodes through the graph and is scalable to billions of edges; Louvain or METIS for community detection to surface suspicious clusters; and Belief Propagation on factor graphs, which is used in production at Facebook and models the joint probability of spam across connected accounts. Features derived from graph structure — such as in/out degree ratio, clustering coefficient, and betweenness centrality — are fed as features to a downstream classifier. The graph is typically stored in a distributed graph engine (Apache Giraph, GraphX, or a specialized system like LinkedIn's Expander).”
}
},
{
“@type”: “Question”,
“name”: “How do you design a classifier ensemble for spam detection that balances precision and recall?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A classifier ensemble combines multiple models — a fast rule-based filter, a gradient-boosted tree (XGBoost or LightGBM) on hand-crafted features, and a neural text classifier (fine-tuned BERT or FastText) — using a stacking or voting approach. The rule-based layer handles obvious patterns (known bad domains, blocklisted phrases) with near-zero latency. The GBDT model scores on behavioral and account features. The neural model analyzes message content for semantic spam patterns. A meta-learner (logistic regression) combines scores from all three. Threshold tuning is critical: spam detection typically operates at high recall (> 95%) with precision as a secondary constraint, since false negatives (missed spam) cause more user harm than false positives (blocked legitimate messages). Calibrated probability outputs enable threshold adjustment without retraining. The ensemble is retrained weekly on fresh labeled data to adapt to evolving spam tactics.”
}
},
{
“@type”: “Question”,
“name”: “How do you architect a spam detection system to process millions of messages per second with low latency?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The architecture uses a tiered, asynchronous pipeline. A synchronous inline check (< 10ms) runs velocity checks via Redis and a fast rule engine before the message is accepted. Accepted messages are published to a Kafka topic and processed by a Flink streaming job that computes richer features (graph signals, content embeddings) and scores the message with the full ensemble model within 1-2 seconds. If the score exceeds a threshold, the message is flagged and a compensating action is triggered (suppress delivery, queue for human review). Model serving uses batched inference with GPU acceleration to handle throughput spikes. Feature stores are sharded by user ID and replicated for read availability. The system is designed for eventual consistency: some spam may be delivered before detection, but downstream enforcement (retroactive removal, account suspension) corrects this within the SLA window."
}
}
]
}