Content area
While much of the research on transaction processing has focused on improving over- all performance in terms of throughput and mean latency, surprisingly less attention has been given to performance predictability: how often individual transactions exhibit execution latency far from the mean. Performance predictability is increasingly important when transactions lie on the critical path of latency-sensitive applications, enterprise software, or interactive web services.
This dissertation proposes a systematic approach to solving performance predictability issues in transactional database systems. We propose the first profiler (to the best of our knowledge) to diagnose these issues and identify sources of unpredictability. Based on our findings from applying this tool to MySQL, we introduce techniques to mitigate the sources of unpredictability. Specifically, this dissertation makes three main contributions:
- Variance Profiler (VProfiler): In contrast to most software profiling tools that quantify average performance, we propose a profiler called VProfiler that, given the source code of a software system and programmer annotations indicating the start and end of semantic intervals of interest, is able to identify the dominant sources of latency variance in a semantic context.
- Studying Causes of Unpredictability in Existing Database Systems: We conduct the first quantitative study of major sources of variance in MySQL, Postgres (two of the largest and most popular open-source products on the market), and VoltDB (a non-conventional database). Based on our findings, we investigate alternative algorithms, implementations, and tuning strategies to reduce latency variance without compromising mean latency or throughput. For instance, we realize that locking in various components is the major source of variance in traditional database systems, and thus propose techniques, including our VATS scheduling algorithm and parallel logging, to mitigate this.
- Contention-Aware Transaction Scheduling: Nearly all existing database systems use a First-In-First-Out strategy for deciding which transaction should be granted the lock on a row when it becomes free. Inspired by our VATS algorithm, we further study the effect of lock scheduling in database systems, and propose contention-aware scheduling algorithms (LDSF and bLDSF) that reduce average latency, and thereby improve the overall predictability of database systems.
VProfiler enables us to identify the root causes of performance variance in existing database systems. By introducing algorithms targeting those root causes, we not only improve performance predictability by up to 5.6x, but also improve average performance by up to 6x. Most notably, our VATS algorithm has been merged into MariaDB and our LDSF algorithm has been made the default scheduling algorithm in Oracle MySQL and Percona, starting from version 8.0.3.