How to design a web crawler that stays polite at a billion pages
7 min read A walkthrough of the web crawler design question, from the URL frontier and politeness to deduplication and the follow-ups that trip candidates up. Read article
7 min read A walkthrough of the web crawler design question, from the URL frontier and politeness to deduplication and the follow-ups that trip candidates up. Read article
9 min read Plain definitions for the system design vocabulary that shows up in interviews, from p99 and quorum to backpressure and idempotency. Read article
7 min read A reference grid of the thirty components that turn up in nearly every system design interview, what each one does, and where it breaks. Read article
What's missing, wrong, or worth adding? One line helps.
Sent — thank you. We read every one.