Analyzing the tradeoff between throughput and latency in multicore scalable in-memory database systems

In this paper, we present a tradeoff between throughput and latency in multicore scalable in-memory database systems by showing the results of a performance evaluation and analysis of Masstree, a state-of-the-art multicore scalable data structure that forms the foundation of a variety of multicore scalable database systems. The key technique to make Masstree scalable is an advanced concurrency control technique. Such a technique reduces cache line contention between cores and contributes to high throughput and scalability. However, surprisingly, the worst case latency of the Masstree-based key-value storage system was more than 10 times larger than the score of the memcached system. To detect the main source of the high latency spikes, we analyzed the concurrency control techniques of Masstree. As a result, we found that read-copy update (RCU), an important technique that enables scalability in Masstree, becomes the dominant factor in the high latency spikes. We present a straightforward approach to resolve the latency spikes. In addition, we also show the limitation of the straightforward approach and possible future directions of essential solutions.