Blog 2: Vectorizing Relationships: A Deep Dive into Knowledge Graph Embeddings

Last time, we learned that a knowledge graph (KG) is a network of relationships: entities connected via labeled edges. But if we want an AI model to reason over this structure, we need a way to convert symbolic triples into something numerical, something that can go into a neural network.

Enter: Knowledge Graph Embeddings.

Why Embeddings?

Let’s say we have a triple:

\[ (\text{Albert Einstein}, \text{born_in}, \text{Ulm}) \]

We want to learn vector representations:

• \( \mathbf{h} = \text{embedding of Albert Einstein} \)
• \( \mathbf{t} = \text{embedding of Ulm} \)
• \( \mathbf{r} = \text{embedding of born_in} \)

so that some function \( f(h, r, t) \) can evaluate how likely this triple is true.

The Geometry of Meaning

Different KG embedding models define different scoring functions over these vectors. Here are the most popular ones:

1. TransE (Translation Embedding)

The simplest and most intuitive. It models relations as translations:

\[ \mathbf{h} + \mathbf{r} \approx \mathbf{t} \]

So the score function is:

\[ f(h, r, t) = -\| \mathbf{h} + \mathbf{r} - \mathbf{t} \| \]

Closer vectors = higher score = more likely true.

Pros: Fast, scalable.
Cons: Struggles with 1-to-many or many-to-many relations.

2. TransH (Hyperplane-aware)

Instead of embedding all relations in the same space, it projects head/tail onto a relation-specific hyperplane. Each relation defines a hyperplane with normal vector \( \mathbf{w}_r \), and embeddings are projected as:

\[ \mathbf{h}_\perp = \mathbf{h} - \mathbf{w}_r^\top \mathbf{h} \mathbf{w}_r \]

Then apply TransE on the projected versions.

3. RotatE

What if relations are not translations but rotations in complex space? RotatE maps all vectors into \( \mathbb{C}^n \) and applies:

\[ \mathbf{t} \approx \mathbf{h} \circ \mathbf{r} \]

where \( \circ \) is element-wise complex multiplication (rotation in complex space). This model is great for capturing symmetry, inversion, and composition.

4. ComplEx

Builds on RotatE, but allows different embeddings for entities in subject vs. object roles.

Score function:

\[ f(h, r, t) = \text{Re}(\langle \mathbf{h}, \mathbf{r}, \overline{\mathbf{t}} \rangle) \]

Uses the Hermitian dot product, allowing antisymmetric relations like "is parent of".

Training the Embeddings

The standard method is to use margin-based loss:

\[ \mathcal{L} = \sum_{(h, r, t) \in \mathcal{G}} \max(0, \gamma + f(h', r, t') - f(h, r, t)) \]

where \( (h', r, t') \) is a corrupted triple (either head or tail replaced), and \( \gamma \) is the margin.

Link Prediction and Reasoning

Once trained, we can use these embeddings to:

• Predict missing links: Given \( (h, r, ?) \), find best \( t \)
• Infer analogies: If \( \text{France} + \text{has\_capital} \rightarrow \text{Paris} \), what’s \( \text{Germany} + \text{has\_capital}? \)
• Cluster or visualize knowledge space

Table: Model Comparison

\[ \begin{array}{|c|c|c|c|} \hline \textbf{Model} & \textbf{Space} & \textbf{Captures} & \textbf{Limitation} \\ \hline \text{TransE} & \mathbb{R}^n & \text{Simple translations} & \text{Fails on 1-to-N} \\ \hline \text{TransH} & \mathbb{R}^n + \text{hyperplane} & \text{Relation-aware projection} & \text{Still struggles with symmetry} \\ \hline \text{RotatE} & \mathbb{C}^n & \text{Rotation, symmetry} & \text{Harder to train} \\ \hline \text{ComplEx} & \mathbb{C}^n & \text{Antisymmetry, inversion} & \text{More parameters} \\ \hline \end{array} \]

TL;DR

• KG Embeddings turn symbolic triples into vectors.
• Different models define different vector operations (translation, rotation).
• These embeddings power reasoning, search, and link prediction.

Coming Up Next:

Blog 3: "Graph Neural Networks for Knowledge Graphs"

We’ll explore how GNNs go beyond embeddings by letting nodes exchange messages, learning from both the structure and attributes of the graph. It’s how we move from static vectors to dynamic reasoning.