Heterogeneous and Relational GNNs

A standard graph neural network assumes every node is the same kind of thing and every edge means the same kind of relationship — all nodes are, say, "company," all edges mean "is correlated with." Most real financial graphs are not like that at all: a single graph might have companies, mutual funds, individual executives, and countries as nodes, connected by edges meaning "owns shares in," "employs," "is headquartered in," or "sues." Averaging a fund's neighbors the same way you'd average a company's neighbors makes no sense — a fund's neighbors are holdings, a company's neighbors might be suppliers, competitors, and lawsuits, all mixed together. Heterogeneous and relational GNNs are the fix: give each node type and each edge type its own aggregation rule.

The analogy: a company's mixed mailroom

Imagine a corporate mailroom that receives four completely different kinds of mail — invoices from suppliers, subpoenas from lawyers, dividend checks from investments, and job applications — and imagine one clerk tries to process every piece of mail with the exact same procedure regardless of type. That clerk would either mangle everything (stapling a subpoena the way you'd staple an invoice) or, more realistically, would need a different processing rule per mail type before combining what they learned into one daily summary. A heterogeneous GNN is that smarter mailroom: it sorts incoming "messages" (from neighboring nodes) by what kind of relationship delivered them, applies a relationship-specific rule to each, and only then combines the results into an updated node representation.

The mechanism: one weight matrix per relation type

A relational GNN (the classic version is R-GCN) extends ordinary message passing by giving every edge type $r$ its own learnable weight matrix $W_r$, and aggregates messages separately per relation before summing:

$$h_v^{(k+1)} = \sigma\left( \sum_{r \in R} \sum_{u \in N_r(v)} \frac{1}{c_{v,r}} W_r h_u^{(k)} + W_0 h_v^{(k)} \right)$$

In words: for each relation type $r$ (owns, supplies, employs, ...), gather the neighbors connected to $v$ specifically through that relation type, transform each of their representations with that relation's own weight matrix $W_r$, average by a normalizing constant $c_{v,r}$, sum the contributions across all relation types, add a self-loop term $W_0 h_v^{(k)}$ that preserves the node's own prior representation, and squash with a nonlinearity $\sigma$. Node type heterogeneity is handled the same way: a node's own feature transformation and its neighbors' transformations can differ by node type, so a "fund" node and a "company" node are never forced through an identical function.

Worked example 1: R-GCN update for one node by hand

Take a company node $v$ with two relation types feeding it: "owned by" (one neighbor, fund $F$, current representation $h_F = 2$) and "supplied by" (two neighbors, suppliers $S_1, S_2$, with $h_{S_1}=1$, $h_{S_2}=3$). Suppose $W_{\text{owned by}} = 0.5$, $W_{\text{supplied by}} = 0.4$, $W_0 = 1$, $h_v^{(k)} = 1$, and normalizing constants are just the neighbor counts per relation ($c=1$ for "owned by," $c=2$ for "supplied by"). The pre-activation sum is: owned-by contribution $= \tfrac{1}{1}(0.5 \times 2) = 1.0$; supplied-by contribution $= \tfrac{1}{2}(0.4 \times 1 + 0.4 \times 3) = \tfrac{1}{2}(0.4+1.2) = 0.8$; self-loop $= 1 \times 1 = 1.0$. Total $= 1.0 + 0.8 + 1.0 = 2.8$, then apply a nonlinearity (say ReLU, which leaves a positive value unchanged): $h_v^{(k+1)} = 2.8$. Note the "owned by" relation and "supplied by" relation each got their own weight before being combined — a plain GCN would have mixed fund and supplier signals through one identical weight.

Worked example 2: why one shared weight would fail here

Suppose instead a plain (non-relational) GCN forced a single shared weight $W = 0.45$ (roughly the average of $0.5$ and $0.4$) across both relation types, ignoring which edge is which. The update becomes: owned-by $= \tfrac{1}{1}(0.45\times2)=0.9$; supplied-by $=\tfrac12(0.45\times1+0.45\times3)=\tfrac12(0.45+1.35)=0.9$; self-loop $=1.0$; total $=2.8$ — in this particular toy case the numbers happen to land close, but that is an artifact of the chosen values, not a general property. The real problem shows up during training: with one shared weight, the model has no way to learn that "being owned by a fund" and "being supplied by a vendor" should influence the company's representation in structurally different ways — gradient signal from ownership relationships and supply relationships gets blended into updating the exact same parameters, so the model cannot specialize either relation's effect.

Each relation type in a heterogeneous GNN applies its own transformation matrix to incoming messages, similar to how the matrix above stretches and rotates space differently depending on its entries — a relational GNN just uses a different one of these per edge type instead of one for the whole graph.

What this means in practice

Financial knowledge graphs — combining companies, funds, executives, filings, and regulatory actions — are almost always heterogeneous, and R-GCN-style architectures (or heterogeneous attention variants) are the standard tool for tasks like predicting fraud risk from a mixed entity-relationship graph, scoring counterparty risk across ownership and lending relationships simultaneously, or building a single embedding space where "similar company" queries can be answered even though companies are only indirectly connected through funds, people, and filings. The practical cost is more parameters — one weight matrix per relation type instead of one total — which for graphs with many relation types (dozens in a rich knowledge graph) can overfit; basis decomposition or block-diagonal weight sharing across relations is the standard mitigation.

Practice

1. Redo worked example 1 with a third relation, "litigated by," contributing one neighbor with $h=0.5$ and $W_{\text{litigated by}}=0.3$, and recompute $h_v^{(k+1)}$.
2. Explain in one sentence why forcing all edge types through one shared weight matrix limits what the model can learn, even if it happens to produce similar numbers on one toy example.
3. Name two node types and two edge types that would appear in a graph built from 13F institutional holdings filings.