Keep in mind that everything is a trade-off.

Next, we’ll look at high-level trade-offs:

• Performance vs. Scalability;
• Latency vs. Throughput;
• Consistency vs. Availability;

性能与可扩展性

如果服务性能的增长与资源的增加是成比例的，服务就是可扩展的。通常，提高性能意味着服务于更多的工作单元，另一方面，当数据集增长时，同样也可以处理更大的工作单位。

另一个角度来看待性能与可扩展性:

• 如果你的系统有性能问题，对于单个用户来说是缓慢的；
• 如果你的系统有可扩展性问题，单个用户较快但在高负载下会变慢；

延迟与吞吐量

延迟是执行操作或运算结果所花费的时间。吞吐量是单位时间内（执行）此类操作或运算的数量。通常，你应该以可接受级延迟下最大化吞吐量为目标。

一致性与可用性

CAP 理论

在一个分布式计算系统中，只能同时满足下列的两点:

• 一致性：每次访问都能获得最新数据但可能会收到错误响应；
• 可用性：每次访问都能收到非错响应，但不保证获取到最新数据；
• 分区容错性：在任意分区网络故障的情况下系统仍能继续运行；

网络并不可靠，所以你应要支持分区容错性，并需要在软件一致性和可用性间做出取舍。

CP：一致性和分区容错性

等待分区节点的响应可能会导致延时错误。如果你的业务需求需要原子读写，CP 是一个不错的选择。

AP：可用性和分区容错性

响应节点上可用数据的最近版本可能并不是最新的。当分区解析完后，写入（操作）可能需要一些时间来传播。如果业务需求允许最终一致性，或当有外部故障时要求系统继续运行，AP 是一个不错的选择。

一致性模式

有同一份数据的多份副本，我们面临着怎样同步它们的选择，以便让客户端有一致的显示数据。回想 CAP 理论中的一致性定义 – 每次访问都能获得最新数据但可能会收到错误响应。

弱一致性

在写入之后，访问可能看到，也可能看不到（写入数据）。尽力优化它，让其能访问最新数据。这种方式可以在 Memcached 等系统中看到。弱一致性在 VoIP，视频聊天和实时多人游戏等真实用例中表现不错。例如：如果你在通话中丢失信号几秒钟时间，当重新连接时，你是听不到这几秒钟所说的话的。

强一致性

在写入后，访问立即可见，数据被同步复制。文件系统和 RDBMS 使用的是此种方式。强一致性在需要记录的系统中运作良好。

最终一致性

在写入后，访问最终能看到写入数据，数据被异步复制。DNS 和 Email 等系统使用的是此种方式。最终一致性在 HA 系统中效果不错。

可用性模式

有两种支持高可用性的模式：failover 和 replication。

故障转移

1. Active-Passive
   关于工作到备用的故障转移流程是：工作服务器发送周期信号给待机中的备用服务器。如果周期信号中断，备用服务器切换成工作服务器的 IP 地址并恢复服务。宕机时间取决于备用服务器处于“热”待机状态，还是需要从“冷”待机状态进行启动。只有工作服务器处理流量。工作到备用的故障转移也被称为主从切换；
2. Active-Active
   在双工作切换中，双方都在管控流量，在它们之间分散负载。如果是外网服务器，DNS 将需要对两方都了解；如果是内网服务器，应用程序逻辑将需要对两方都了解。双工作切换也可以称为主主切换；

故障转移的缺陷：

• 故障转移需要添加额外硬件并增加复杂性；
• 如果新写入数据在能被复制到备用系统之前，工作系统出现了故障，则有可能会丢失数据；

复制

主从复制和主主复制

How many 9s are enough?

Availability is often quantified by uptime (or downtime) as a percentage of time the service is available. Availability is generally measured in number of 9s – a service with 99.99% availability is described as having four 9s.

99.9% availability – three 9s:

99.99% availability – four 9s:

In sequence vs. In parallel

If a service consists of multiple components prone to failure, the service’s overall availability depends on whether the components are in sequence or in parallel:

• In sequence
  Overall availability decreases when two components with availability < 100% are in sequence:
  If both Foo and Bar each has 99.9% availability, their total availability in sequence would be 99.8%.

  Availability(Total) = Availability(Foo) * Availability(Bar)

• In parallel
  Overall availability increases when two components with availability < 100% are in parallel:
  If both Foo and Bar each has 99.9% availability, their total availability in parallel would be 99.9999%.

  Availability(Total) = 1 - (1 - Availability(Foo)) * (1 - Availability(Bar))

Just recently I was asked what it would take to make a web service massively scalable. My answer was lengthy and maybe it is also for other people interesting. So I share it with you here in my blog and split it into parts to make it easier to read. New parts are released on a regular basis. Have fun and your comments are always welcomed!

Clones

Public servers of a scalable web service are hidden behind a load balancer. This load balancer evenly distributes load (requests from your users) onto your group/cluster of application servers. That means that if, for example, user Steve interacts with your service, he may be served at his first request by server 2, then with his second request by server 9 and then maybe again by server 2 on his third request.

Steve should always get the same results of his request back, independent what server he “landed on”. That leads to the first golden rule for scalability: every server contains exactly the same codebase and does not store any user-related data, like sessions or profile pictures, on local disc or memory.

Sessions need to be stored in a centralized data store which is accessible to all your application servers. It can be an external database or an external persistent cache, like Redis. An external persistent cache will have better performance than an external database. By external I mean that the data store does not reside on the application servers. Instead, it is somewhere in or near the data center of your application servers.

CS75 Lecture 9: Scalability

Vertical Scaling

Get more RAM, processors, disks for one machine, but you will exhaust the financial resources/state.

Horizontal Scaling

Plural number of machines, use multiple servers to build the topology.

The successful job interviewer

What makes you suitable for this job, do you think?

Useful Tips

When interviewing for a job, there are 4 main areas to ask about:

• Work history.
• Personality.
• Suitability for the company and the specific job.
• Goals and ambitions.

Note that it is not appropriate to ask the candidate questions about race, age, marital status, or sexual preferences. These questions are potentially discriminatory.

Presenting a product or service

I want to tell you today about…

Useful Tips

When presenting a product or service, an effective way to convince your audience is to follow the FAB approach:

• Feature: Highlight the features of the product or service you want the audience to focus on.
• Advantage: Show how these features make the product or service better than its predecessors or competitors.
• Benefit: Show how this product or service will improve the user’s life.

This approach can be grouped into a 3-step presentation structure:

1. Introduction: Summarize what you are going to tell the audience.
2. Main body: Tell them.
3. Conclusion: Tell them what they should do next.