Content area
Hardware-based cache coherence is employed nearly universally in modern shared-memory multicore processors due to its power, performance, and area efficiency. During the development of shared-memory processors, researchers proposed programmable controllers to support both cache coherence and message passing designs. Despite the flexibility afforded by programmability for post-design protocol changes or runtime selection of communication paradigms, such systems were never widely adopted. Hardware-based fixed-function designs provided excellent performance and area efficiency, as well as an easy to use shared-memory programming model that alleviated programmers of explicit cache and data-movement management. Combined with the rapid advancement of integrated circuit design technologies, single-chip shared-memory multicore processors with hardware-based cache coherence became the dominant design.
However, in the decades since the first shared-memory multicore processors emerged, the computing landscape has changed dramatically. Transistor density and power scaling have slowed or collapsed while the diversity and application of computing systems has increased significantly. It is no longer clear whether design decisions adopted by and retained from early multiprocessor designs are correct today.
This dissertation revisits the topic of programmable cache coherence engines in the context of modern shared-memory multicore processors. First, the open-source BedRock cache coherence protocol is described. BedRock employs the canonical MOESIF coherence states and reduces implementation burden by eliminating transient coherence states from the protocol. The protocol's design complexity, concurrency, and verification effort are analyzed and compared to a canonical directory-based invalidate coherence protocol. Second, the architecture and microarchitecture of three separate cache coherence directories implementing the BedRock protocol within the BlackParrot 64-bit RISC-V multicore processor, collectively called BlackParrot-BedRock (BP-BedRock), are described. A fixed-function coherence directory engine implementation provides a baseline design for performance and area comparisons. A microcode-programmable coherence directory implementation demonstrates the feasibility of implementing a programmable coherence engine capable of maintaining sufficient protocol processing performance. A hybrid fixed-function and programmable coherence directory blends the protocol processing performance of the fixed-function design with the programmable flexibility of the microcode-programmable design. Commentary and analysis are provided to illuminate the practical architectural and microarchitectural design and implementation challenges of cache coherence systems, both with and without programmability. All three designs are available open-source, providing researchers with an easy-to-use platform for further investigation. Collectively, the BedRock coherence protocol and its three BP-BedRock implementations demonstrate the feasibility and challenges of including programmable logic within the coherence system of modern shared-memory multicore processors, paving the way for future research into the application- and system-level benefits of programmable coherence engines.