University of Cambridge > Talks.cam > Microsoft Research Cambridge, public talks > Building Flexible High-Performance Key-Value Systems

Building Flexible High-Performance Key-Value Systems

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Microsoft Research Cambridge Talks Admins.

This event may be recorded and made available internally or externally via http://research.microsoft.com. Microsoft will own the copyright of any recordings made. If you do not wish to have your image/voice recorded please consider this before attending

This talk will focus on the design of a new replication mechanism, called Ouroboros, for the construction of adaptable, fault-tolerant, and high-performance key-value (KV) systems. Ouroboros unifies chain replication and consistent hashing to provide provably strong data consistency while maintaining consistent hashing’s efficient partitioning. Key to its usability is its ability to support non-blocking node addition and removal, ensuring that Ouroboros-based clusters continue to provide high-performance during the normal failures that affect highly scalable storage systems. In the context of FAWN -KV, a distributed KV system I built for for Flash-based clusters, I will show how the protocols in Ouroboros help overcome slow random writes on Flash devices and offer high-performance under churn.

Distributed KV systems are a critical part of the infrastructure at many large sites such as Amazon, LiveJournal, Facebook, and Twitter. Unfortunately, the ecosystem of these KV stores is a mess—-no one existing system meets the needs of all applications. Systems designers worry about running multiple stores from different codebases, vendors, and so on, each optimized for certain application requirements and hardware configuration. In contrast, I will show how a single KV system, Flex-KV, extends Ouroboros to flexibly meet the needs of a variety of applications. Flex-KV can support DRAM , Flash, and disk-based storage, can act as an unreliable cache or a durable store, and can offer strong or weak data consistency. The value of such a system goes beyond ease-of-use: While exploring these dimensions of durability, consistency, and availability, we find new choices for system designs, such as a cache-consistent memcached, that offer some applications a better balance of performance and cost than was previously available.

This talk is part of the Microsoft Research Cambridge, public talks series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity