GDI: Scaling Online Transactional and Analytical Graph Workloads to Hundreds of Thousands of Cores
115 بار بازدید -
8 ماه پیش
-
Paper Title: The Graph Database
Paper Title: The Graph Database Interface: Scaling Online Transactional and Analytical Graph Workloads to Hundreds of Thousands of Cores
Venue: International Conference for High Performance Computing, Networking, Storage, and Analysis (#SC23), 'Graph Frameworks and Databases' Session
Best Paper Candidate
Speaker: Maciej Besta
Authors: Maciej Besta, Robert Gerstenberger, Marc Fischer, Michał Podstawski, Nils Blach, Berke Egeli, Georgy Mitenkov, Wojciech Chlapek, Marek Michalewicz, Hubert Niewiadomski, Jürgen Müller, Torsten Hoefler
Abstract:
Graph databases (GDBs) are crucial in academic and industry applications. The key challenges in developing GDBs are achieving high performance, scalability, programmability, and portability. To tackle these challenges, we harness established practices from the HPC landscape to build a system that outperforms all past GDBs presented in the literature by orders of magnitude, for both OLTP and OLAP workloads. For this, we first identify and crystallize performance-critical building blocks in the GDB design, and abstract them into a portable and programmable API specification, called the Graph Database Interface (GDI), inspired by the best practices of MPI. We then use GDI to design a GDB for distributed-memory RDMA architectures. Our implementation harnesses one-sided RDMA communication and collective operations, and it offers architecture-independent theoretical performance guarantees. The resulting design achieves extreme scales of more than a hundred thousand cores. Our work will facilitate the development of next-generation extreme-scale graph databases.
Learn more:
SC'23 article: https://doi.org/10.1145/3581784.3607068
extended technical report: https://arxiv.org/abs/2305.11162
code and GDI specification: https://github.com/spcl/GDI-RMA
Timestamps:
00:00 Introduction: Labeled Property Graph and Graph Database Workloads
01:56 State of Challenges & Problems
04:42 Our Objectives
05:27 Graph Database Interface
08:13 GDI Standard
12:01 GDI-RMA - GDI Implementation for RDMA Systems
13:48 GDI-RMA Data Layout
15:59 GDI-RMA Example Graph Update Protocol
18:28 Evaluation: Low Latency
20:37 Evaluation: Scaling OLTP to 120,000 Cores
22:20 Evaluation: Scaling OLAP to 120,000 Cores
23:56 Conclusion
#SC23 #supercompute2023 #GDB #RDMA #LPG
Venue: International Conference for High Performance Computing, Networking, Storage, and Analysis (#SC23), 'Graph Frameworks and Databases' Session
Best Paper Candidate
Speaker: Maciej Besta
Authors: Maciej Besta, Robert Gerstenberger, Marc Fischer, Michał Podstawski, Nils Blach, Berke Egeli, Georgy Mitenkov, Wojciech Chlapek, Marek Michalewicz, Hubert Niewiadomski, Jürgen Müller, Torsten Hoefler
Abstract:
Graph databases (GDBs) are crucial in academic and industry applications. The key challenges in developing GDBs are achieving high performance, scalability, programmability, and portability. To tackle these challenges, we harness established practices from the HPC landscape to build a system that outperforms all past GDBs presented in the literature by orders of magnitude, for both OLTP and OLAP workloads. For this, we first identify and crystallize performance-critical building blocks in the GDB design, and abstract them into a portable and programmable API specification, called the Graph Database Interface (GDI), inspired by the best practices of MPI. We then use GDI to design a GDB for distributed-memory RDMA architectures. Our implementation harnesses one-sided RDMA communication and collective operations, and it offers architecture-independent theoretical performance guarantees. The resulting design achieves extreme scales of more than a hundred thousand cores. Our work will facilitate the development of next-generation extreme-scale graph databases.
Learn more:
SC'23 article: https://doi.org/10.1145/3581784.3607068
extended technical report: https://arxiv.org/abs/2305.11162
code and GDI specification: https://github.com/spcl/GDI-RMA
Timestamps:
00:00 Introduction: Labeled Property Graph and Graph Database Workloads
01:56 State of Challenges & Problems
04:42 Our Objectives
05:27 Graph Database Interface
08:13 GDI Standard
12:01 GDI-RMA - GDI Implementation for RDMA Systems
13:48 GDI-RMA Data Layout
15:59 GDI-RMA Example Graph Update Protocol
18:28 Evaluation: Low Latency
20:37 Evaluation: Scaling OLTP to 120,000 Cores
22:20 Evaluation: Scaling OLAP to 120,000 Cores
23:56 Conclusion
#SC23 #supercompute2023 #GDB #RDMA #LPG
8 ماه پیش
در تاریخ 1402/09/06 منتشر شده
است.
115
بـار بازدید شده