How GeoGNN preserves the geometry of text embeddings

A text-attributed graph connects documents, entities, or other nodes whose features come from language-model embeddings. GeoGNN studies what happens to those features as graph layers combine information from neighboring nodes.

Measuring the movement of meaning

Language encoders organize related text within a structured embedding space. Graph message passing then moves each node representation according to its neighbors. GeoGNN introduces a local principal-component-analysis measure of semantic drift, giving researchers a way to quantify how aggregation changes the geometry of these representations.

This measurement links two stages of a learning system: the encoder that places text in an embedding space and the graph network that propagates information across edges. It makes the effect of the aggregation operator visible.

Aggregating along the manifold

The proposed Geodesic Aggregation mechanism operates on a unit sphere. Logarithmic maps express neighboring representations in a tangent space, and exponential maps return the aggregated information to the sphere. GeoGNN combines this construction with spherical attention and manifold interpolation.

Attention controls how neighboring information contributes to a node. The geometry-aware update controls how that contribution changes its representation. These components give message passing an explicit relationship to the space produced by the language encoder.

An inspectable representation choice

The paper evaluates semantic drift and downstream learning across text-attributed graph datasets and several encoders. Its experiments report improved performance alongside reduced drift. The resulting analysis gives graph-model developers a concrete design variable to investigate: the geometry used when information crosses an edge.

The experiments cover the Photo, Children, Arxiv, and History datasets, spanning e-commerce and academic text graphs. Frozen encoders supply fixed node features, and graph models are compared under shared settings. This design connects the observed changes in representation with the aggregation mechanism applied to those same inputs.

Liangwei Yang and collaborators at Salesforce AI Research developed GeoGNN with independent researcher Jing Ma. Haolin Chen and Weiran Yao are co-authors. Read the research.