Building Flexible High-Performance Key-Value Systems

Add to your list(s) Download to your calendar using vCal

• Amar Phanishayee, Carnegie Mellon University
• Wednesday 10 April 2013, 10:00-11:00
• Small lecture theatre, Microsoft Research Ltd, 7 J J Thomson Avenue (Off Madingley Road), Cambridge.

If you have a question about this talk, please contact Microsoft Research Cambridge Talks Admins.

This talk has been canceled/deleted

This talk will focus on the design of a new replication mechanism, called Ouroboros, for the construction of adaptable, fault-tolerant, and high-performance key-value (KV) systems. Ouroboros unifies chain replication and consistent hashing to provide provably strong data consistency while maintaining consistent hashing’s efficient partitioning. Key to its usability is its ability to support non-blocking node addition and removal, ensuring that Ouroboros-based clusters continue to provide high-performance during the normal failures that affect highly scalable storage systems. In the context of FAWN -KV, a distributed KV system I built for for Flash-based clusters, I will show how the protocols in Ouroboros help overcome slow random writes on Flash devices and offer high-performance under churn.

Distributed KV systems are a critical part of the infrastructure at many large sites such as Amazon, LiveJournal, Facebook, and Twitter. Unfortunately, the ecosystem of these KV stores is a mess—-no one existing system meets the needs of all applications. Systems designers worry about running multiple stores from different codebases, vendors, and so on, each optimized for certain application requirements and hardware configuration. In contrast, I will show how a single KV system, Flex-KV, extends Ouroboros to flexibly meet the needs of a variety of applications. Flex-KV can support DRAM , Flash, and disk-based storage, can act as an unreliable cache or a durable store, and can offer strong or weak data consistency. The value of such a system goes beyond ease-of-use: While exploring these dimensions of durability, consistency, and availability, we find new choices for system designs, such as a cache-consistent memcached, that offer some applications a better balance of performance and cost than was previously available.

This talk is part of the Microsoft Research Cambridge, public talks series.

This talk is included in these lists:

This talk is not included in any other list

Note that ex-directory lists are not shown.