Design Problems/Twitter Search / Full-Text Search Engine

Twitter Search / Full-Text Search Engine

Medium
SearchElasticsearchIndexing

High-Level Architecture

Rendering diagram...

Design a full-text search system similar to Twitter Search that allows users to search through billions of tweets/posts in real-time. The system should support keyword search, phrase matching, hashtag search, and provide relevant, ranked results with low latency.

Clarifying Questions

  • How many tweets/documents are searchable? (Assume 500M new tweets/day, 200B total)
  • What is the acceptable search latency? (< 500ms p99)
  • Should we support real-time indexing? (Yes, tweets should be searchable within seconds)
  • What ranking factors? (Recency, relevance, engagement, user authority)
  • Do we need autocomplete/suggestions? (Yes, as stretch goal)
  • Geographic filtering needed? (Yes)