Factor Graph Neural Networks
Zhen Zhang, Mohammed Haroon Dupty, Fan Wu, Javen Qinfeng Shi, Wee Sun Lee; 24(181):1−54, 2023.
Abstract
In recent years, Graph Neural Networks (GNNs) have gained significant popularity and have achieved remarkable success in various real-world applications. Unlike Probabilistic Graphical Models (PGMs), GNNs focus on representation learning rather than computing marginals or most likely configurations. They provide flexibility in information flow rules while maintaining good performance. However, existing GNNs lack efficient methods to represent and learn higher-order relations among variables/nodes. To address this limitation, we propose Factor Graph Neural Networks (FGNNs) that effectively capture higher-order relations for inference and learning. We introduce an efficient approximate Sum-Product loopy belief propagation inference algorithm for discrete higher-order PGMs and neuralize the message passing scheme into an FGNN module. This allows for richer representations of the message update rules, enabling efficient inference and powerful end-to-end learning. Additionally, our FGNN architecture can represent both Max-Product and Sum-Product loopy belief propagation by using suitable message aggregation operators. Our experimental evaluation on synthetic and real datasets demonstrates the potential of the proposed model.
[abs]