HIGHER-ORDER MOLECULAR LEARNING: THE CELLULAR TRANSFORMER
1.This paper introduces the Cellular Transformer (CT), a topological deep learning (TDL) framework that generalizes graph transformers to operate on cell complexes, enabling the modeling of higher-order molecular structures like rings, fused motifs, and multi-bond systems.
2.A key innovation is the augmented molecular cell complex (AMCC), a richer molecular representation where atoms, bonds, and rings are treated as 0-, 1-, and 2-cells, respectively—embedding chemical topology directly into the learning architecture.
3.CT performs attention not just over nodes or edges but across multiple structural ranks (0D, 1D, 2D) using a novel pairwise and general cellular attention mechanism, capturing multiscale interactions without relying on graph rewiring, virtual nodes, or ad-hoc biases.
4.The architecture employs tensor diagrams to formalize attention flow across cochain ranks, integrating both cross-rank and intra-rank attention, guided by neighborhood matrices derived from topological relations like incidence and adjacency.
5.To encode structure, CT introduces cellular positional encodings (CPEs), extending Laplacian and random walk encodings to the cellular domain. It also proposes a novel barycentric subdivision encoding (BSPe) that enhances topological locality.
6.Extensive benchmarking on MoleculeNet and the Graph Classification Benchmark (GCB) demonstrates that CT consistently outperforms GNNs, MPNNs, and graph transformers, especially in datasets where topological motifs matter most.
7.In GCB, CT achieved the highest accuracy (75.4%) compared to other message-passing and kernel-based methods, showing the benefit of high-order attention even in originally graph-based domains.
8.On MoleculeNet, CT ranked among the top across both classification (AUC) and regression (RMSE) tasks, performing particularly well in datasets like HIV, ClinTox, and ESOL, where higher-order features are vital.
9.The method is highly generalizable: lifting molecular graphs into CCs using tools like TopoX allows CT to apply broadly, even when only graph data is available, making it backward-compatible with existing pipelines.
10.This work positions CT as a foundation for topologically informed molecular modeling, offering a scalable, interpretable, and efficient alternative to current GNN-based methods, with applications across drug discovery and materials science.
📜Paper:
openreview.net/pdf?id=GW3h79…
#MolecularModeling #TopologicalDeepLearning #GraphTransformer #DrugDiscovery #CellComplex #MoleculeNet #ICLR2025 #ChemicalML #AttentionMechanism #CellularTransformer