What is Twitter Search?

Twitter users can update their status whenever they like. Each status (called tweet) consists of plain text and our goal is to design a system that allows searching over all the user tweets.

Requirements and Goals of the System

We need to design a system that can efficiently store and query tweets:

• Let’s assume Twitter has 1.5 billion total users with 800 million daily active users;
• On average Twitter gets 400 million tweets every day;
• The average size of a tweet is 300 bytes;
• Let’s assume there will be 500M searches every day;
• The search query will consist of multiple words combined with AND/OR;

What is a Rate Limiter?

Imagine we have a service which is receiving a huge number of requests, but it can only serve a limited number of requests per second. To handle this problem we would need some kind of throttling or rate limiting mechanism that would allow only a certain number of requests so our service can respond to all of them. A rate limiter, at a high level, limits the number of events an entity (user, device, IP, etc.) can perform in a particular time window. For example:

• A user can send only one message per second;
• A user is allowed only three failed credit card transactions per day;
• A single IP can only create twenty accounts per day;

In general, a rate limiter caps how many requests a sender can issue in a specific time window. It then blocks requests once the cap is reached.

What is Typeahead Suggestion?

Typeahead suggestions enable users to search for known and frequently searched terms. As the user types into the search box, it tries to predict the query based on the characters the user has entered and gives a list of suggestions to complete the query. Typeahead suggestions help the user to articulate their search queries better. It’s not about speeding up the search process but rather about guiding the users and lending them a helping hand in constructing their search query.

Requirements and Goals of the System

• Functional Requirements: As the user types in their query, our service should suggest top 10 terms starting with whatever the user has typed;
• Non-Function Requirements: The suggestions should appear in real-time. The user should be able to see the suggestions within 200ms;

Why Youtube?

Youtube is one of the most popular video sharing websites in the world. Users of the service can upload, view, share, rate, and report videos as well as add comments on videos.

Requirements and Goals of the System

For the sake of this exercise, we plan to design a simpler version of Youtube with following requirements:
Functional Requirements:

1. Users should be able to upload videos;
2. Users should be able to share and view videos;
3. Users should be able to perform searches based on video titles;
4. Our services should be able to record stats of videos, e.g., likes/dislikes, total number of views, etc;
5. Users should be able to add and view comments on videos;

Non-Functional Requirements:

1. The system should be highly reliable, any video uploaded should not be lost;
2. The system should be highly available. Consistency can take a hit (in the interest of availability); if a user doesn’t see a video for a while, it should be fine;
3. Users should have a real-time experience while watching videos and should not feel any lag;

Not in Scope: Video recommendations, most popular videos, channels, subscriptions, watch later, favorites, etc.

What is Twitter?

Twitter is an online social networking service where users post and read short 140-character messages called “tweets”. Registered users can post and read tweets, but those who are not registered can only read them. Users access Twitter through their website interface, SMS, or mobile app.

Requirements and Goals of the System

We will be designing a simpler version of Twitter with the following requirements:
Functional Requirements:

1. Users should be able to post new tweets;
2. A user should be able to follow other users;
3. Users should be able to mark tweets as favorites;
4. The service should be able to create and display a user’s timeline consisting of top tweets from all the people the user follows;
5. Tweets can contain photos and videos;

Non-Functional Requirements:

1. Our service needs to be highly available;
2. Acceptable latency of the system is 200ms for timeline generation;
3. Consistency can take a hit (in the interest of availability); if a user doesn’t see a tweet for a while, it should be fine;

Extended Requirements:

1. Searching for tweets;
2. Replying to a tweet;
3. Trending topics – current hot topics/searches;
4. Tagging other users;
5. Tweet Notification;
6. Who to follow? Suggestions?
7. Moments;