A proposal has been made to address the challenges of handling graphs with evolving features or connectivities over time. This proposal introduces a series of temporal graph neural networks (TGNNs). However, previous evaluations of these TGNNs have identified several limitations related to inconsistent datasets, evaluation pipelines, workload diversity, and efficient comparison. Therefore, there is a need for an empirical study that can comprehensively compare TGNN models on a level playing field. To fulfill this need, the authors propose BenchTemp, a benchmark specifically designed to evaluate TGNN models across various workloads. BenchTemp includes a collection of benchmark datasets to ensure fair comparisons between different TGNN models. Additionally, it establishes a standardized pipeline for TGNN evaluation. With BenchTemp, the authors extensively compare representative TGNN models on different tasks (such as link prediction and node classification) and settings (transductive and inductive), taking into account both effectiveness and efficiency metrics. The authors have made BenchTemp publicly available at https://github.com/qianghuangwhu/benchtemp.
BenchTemp: An All-Purpose Benchmark for Assessing Temporal Graph Neural Networks (arXiv:2308.16385v1 [cs.LG])
