The emergence of graph neural networks (GNNs) as a popular area of deep learning has sparked interest in their ability to learn from graph-structured data. However, when dealing with large-scale graphs, the computational and memory requirements of training GNNs can surpass the capabilities of single machines or GPUs. This has led to the exploration of distributed GNN training as a potential solution for large-scale training. To implement distributed GNN training, it is necessary to partition the input graph into smaller parts that can be distributed among multiple machines in a compute cluster. While graph partitioning has been extensively studied in the context of graph analytics and databases, its impact on GNN training performance remains largely unexplored.
This paper investigates the effectiveness of graph partitioning for distributed GNN training. The study aims to understand how various factors, such as GNN parameters, mini-batch size, graph type, features size, and scale-out factor, influence the effectiveness of graph partitioning. The experiments conducted in this study utilize two different GNN systems with vertex and edge partitioning. The findings reveal that graph partitioning is a crucial pre-processing step that can significantly reduce training time and memory usage. Moreover, the results demonstrate that the time spent on partitioning can be offset by the reduced training time of GNNs, making it a relevant optimization strategy.