Content area
Distributed-memory parallel applications are the backbone of large-scale data analytics and modern computing, forming foundations that enable the efficient distribution and execution of large-scale computations across heterogeneous and distributed systems. Classical primitives for data analytics, including Bulk Synchronous Parallel (BSP) models, focused on moving data to compute. Although these primitives were good enough for past High Performance Computing (HPC) applications, they are highly inefficient for the fine-grained asynchrony and data distribution required by modern large-scale data analytics applications, motivating the need for new architectural approaches for data analytics. Alleviating these challenges inherent in distributed computing environments, the underlying FineGrained Asynchronous Bulk Synchronous Parallel (FABS) model in the Actor-based programming system HClib-Actor proposes moving compute to data via asynchronous active messages. This Actor-based approach presents a lightweight, asynchronous computation model that utilizes fine-grained asynchronous actor messages to express point-to-point remote operations, allowing fine-grained, distributed, asynchronous, and scalable executions across systems. We realize the efficacy of the actor-based approach through exploring multiple perspectives and facets of distributed computing, including algorithm design, runtime systems, and system-level optimizations. The Algorithm perspective (”Crafting Scalable Distributed Applications”) elucidates novel algorithms tailored for scalable, distributed applications in a wide variety of application domains. Through theoretical analysis and empirical evaluations, we demonstrate the superior scalability and efficiency in the asynchronous actor-based model compared to traditional parallel computing paradigms. The Runtime perspective (”Empowering Distributed Systems with Actors”) delves into enhancements made to HClib-Actor, including extending termination protocols, introducing light-weight global termination schemes, and cloud environment deployment. It details optimizations incorporated into HClib-Actor, showing insights that underscore the critical role of runtime systems in facilitating scalable distributed computing. Finally, the System-Level perspective (”Architecture-Aware Optimization and Tuning”) signifies the importance of aligning distributed-memory parallel applications with underlying hardware architecture characteristics for achieving peak performance by delving into hardware-specific optimizations and architectural considerations on diverse hardware platforms. We elucidate the importance of architecture-aware bindings and software-level buffer sizes on distributed computing efficiency and bandwidth. Through the realization of the three perspectives and facets of distributed computing, we offer a comprehensive scalable asynchronous actor-based approach for distributed-memory parallel applications. The synthesis of these perspectives, demonstrated through the systematic transformation methodology as an implementation guideline, provides researchers with a practical framework for adopting actor-based approaches in their own applications that are based on traditional models. By integrating these insights, we provide valuable contributions to the advancement of distributed computing, forming foundations for more efficient and scalable distributed and parallel applications in diverse computing landscapes.