Knowledge Graph Embeddings: TransE, RotatE & Link Prediction Basics

A knowledge graph embedding is a learned representation of a graph in which every entity and every relation is assigned a vector (or a small matrix) such that the geometry of the vectors reflects the structure of the graph. If the triple (Marie Curie, bornIn, Warsaw) is true, the vectors for those three symbols should satisfy a scoring function that the model has been trained to associate with true triples. Once trained, the vectors can be used to predict missing edges, classify entities, and feed graph structure into other machine learning systems.

What is being embedded

A knowledge graph (see /concepts/what-is-a-knowledge-graph/) is a set of triples (h, r, t): a head entity, a relation, and a tail entity. Symbolically, Q7186 bornIn Q270 says nothing about how similar Warsaw is to Paris or how bornIn relates to diedIn. An embedding model learns those regularities from the pattern of edges alone.

The model maintains a lookup table: one d-dimensional vector per entity and one per relation type, where d is typically between 50 and a few hundred. A scoring function f(h, r, t) combines the three vectors into a single number that is high for triples the model believes are true. Training adjusts the vectors so that observed triples score well and corrupted triples score badly.

The embedding is of the graph’s structure, not of any text. Entity names, descriptions and literal values are ignored by the basic models, which is both a strength (the method works on any graph, in any language) and a limitation discussed below.

Why embed a knowledge graph

Link prediction and knowledge graph completion. Real knowledge graphs are incomplete. Wikidata records a place of birth for many people but not all; a product graph lists some compatible accessories but misses others. Given (Marie Curie, bornIn, ?), a trained model ranks every entity as a candidate tail, and the highest-ranked candidates are proposed as missing facts for review or insertion. This is the task most embedding research is measured on.

Entity classification and clustering. Entities with similar neighborhoods end up close together, so a nearest-neighbor search over embeddings finds entities that play the same structural role even when they share no explicit type.

Recommendation. Product, user and attribute nodes embedded together let a system score (User, mightBuy, Product) as a link prediction problem.

Graph structure as ML input. Entity vectors can be concatenated with other features for a downstream classifier, used to initialize a graph neural network, or injected into a large language model pipeline to ground entities in structured knowledge. See /build/graphrag/ and /seo/knowledge-graphs-ai-search-and-llms/.

Model families

Dozens of scoring functions have been proposed. Most fall into four families.

Translational models

TransE (Bordes et al., 2013) is the simplest and still the reference point. Each entity and relation is a vector in the same space, and a relation is modeled as a translation: for a true triple, h + r ≈ t. The score is the negative distance ‖h + r − t‖. If the vector for bornIn moves from person vectors to city vectors, then Marie Curie + bornIn lands near Warsaw and Albert Einstein + bornIn lands near Ulm.

The elegance comes with limits. A single translation vector handles one-to-many relations poorly: every person born in Warsaw is pushed toward the same point. Symmetric relations (spouse) are also awkward, since h + r ≈ t and t + r ≈ h can only both hold if r is near zero. TransH addresses this by projecting entities onto a relation-specific hyperplane before translating, and TransR gives each relation its own projection matrix. Both keep the translation intuition while adding capacity.

Rotational models

RotatE (Sun et al., 2019) places entities in complex vector space and models each relation as an element-wise rotation: t ≈ h ∘ r, where each component of r has modulus 1 and rotates the matching component of h by an angle. Rotation handles patterns translation cannot. A symmetric relation is a rotation by 180 degrees, an inverse relation is the opposite rotation, and composing two relations adds their angles, so bornIn followed by locatedInCountry can behave like a single relation.

Bilinear and tensor factorization models

This family treats the graph as a three-way tensor (entity × relation × entity) and factorizes it. RESCAL (Nickel et al., 2011) gives each relation a full d × d matrix and scores a triple as hᵀ M_r t; it is expressive but costs d² parameters per relation. DistMult restricts M_r to a diagonal, which is far cheaper but symmetric by construction: it cannot distinguish (h, r, t) from (t, r, h), so it cannot learn that bornIn is directional. ComplEx (Trouillon et al., 2016) fixes that by moving to complex numbers and taking the real part of the product with the conjugate of the tail, so asymmetric relations become representable while the parameter count stays linear in d.

Graph neural network approaches

Rather than learning a free vector per entity, message-passing models compute an entity’s representation from its neighbors. R-GCN (Relational Graph Convolutional Network) extends graph convolution with relation-specific weight matrices, so a node aggregates neighbors differently depending on the edge type. CompGCN jointly embeds entities and relations and uses composition operators from the translational and rotational families inside the aggregation step. These encoders are usually paired with a decoder such as DistMult for the final score, and they can use node features when they exist.

Family	Representative models	Intuition	Handles asymmetry	Handles composition
Translational	TransE, TransH, TransR	Relation is a displacement	Yes	Yes
Rotational	RotatE	Relation is a rotation in complex space	Yes	Yes
Bilinear / tensor	RESCAL, DistMult, ComplEx	Score is a (weighted) product of components	RESCAL and ComplEx yes, DistMult no	Partially
Graph neural network	R-GCN, CompGCN	Aggregate messages from typed neighbors	Depends on decoder	Depends on decoder

Training

Knowledge graphs contain only positive examples. Nobody records that Marie Curie was not born in Lisbon. Training therefore relies on negative sampling: for each true triple, corrupted triples are generated by replacing the head or tail with a random entity, and the model is asked to score the true triple above the corruptions. Smarter samplers draw corruptions from plausible entities (a city rather than a person for the tail of bornIn). Some frameworks instead treat every other entity as a negative, the 1-vs-all or softmax formulation.

Two loss functions dominate. The margin ranking loss, used by TransE, requires the score gap between a positive and its negative to exceed a fixed margin. Cross-entropy losses, binary over sampled negatives or softmax over all candidate tails, treat link prediction as classification and are generally preferred for bilinear and rotational models. Regularization (L2 or N3 penalties, or unit-norm constraints on entity vectors) prevents scores from growing without bound.

Results for the same model vary substantially with embedding dimension, number of negatives, learning rate and loss, which is one reason the field has moved toward shared libraries with reproducible configurations.

Evaluation

Link prediction is evaluated by ranking. For each test triple (h, r, t), the tail is removed and every entity in the graph is scored as a replacement; the rank of the true tail among all candidates is recorded. The same is done for the head. Aggregate metrics are:

Mean Reciprocal Rank (MRR): the average of 1/rank over all test queries. A value of 1 means the true entity always ranked first.
Hits@k: the fraction of test queries where the true entity appears in the top k, usually reported for k = 1, 3 and 10.
Mean Rank: the average rank, less commonly reported because it is dominated by outliers.

The filtered setting matters. If the query is (Marie Curie, awarded, ?) and the graph already contains both Nobel Prize in Physics and Nobel Prize in Chemistry, a model that ranks Physics first when the test triple is Chemistry has not made a mistake. Filtered evaluation removes all other known true triples from the candidate list before computing the rank, and is the standard convention.

Two benchmark datasets appear in nearly every paper. FB15k-237 is a subset of Freebase with inverse relations removed, after the original FB15k was found to leak test answers through trivially invertible edges. WN18RR is the analogous corrected subset of WordNet. Others include YAGO3-10, CoDEx and Wikidata5M. Reported scores are sensitive to implementation details, so comparisons should use the same library and evaluation code.

Libraries and a PyKEEN sketch

PyKEEN (Python): the most comprehensive research library, with dozens of models, samplers, losses and datasets behind a unified pipeline function.
DGL-KE: built on the Deep Graph Library, focused on training TransE, DistMult, ComplEx and RotatE at scale across GPUs or machines.
AmpliGraph: a TensorFlow-based library with a scikit-learn-style API.
PyTorch Geometric: a general graph learning library with knowledge graph embedding models alongside GNN layers such as RGCNConv.

A minimal PyKEEN run on a built-in dataset:

from pykeen.pipeline import pipeline

result = pipeline(
    dataset="FB15k237",
    model="RotatE",
    model_kwargs={"embedding_dim": 200},
    training_kwargs={"num_epochs": 100, "batch_size": 512},
    negative_sampler="basic",
    random_seed=42,
)

print(result.metric_results.get_metric("hits_at_10"))
result.save_to_directory("rotate_fb15k237")

# Score candidate tails for a query
from pykeen.predict import predict_target
preds = predict_target(
    model=result.model,
    head="/m/0d6lp",      # replace with any entity label from result.training
    relation="/people/person/place_of_birth",  # a relation label from the dataset
    triples_factory=result.training,
)
print(preds.df.head())

To embed a custom graph, replace dataset with a TriplesFactory built from a tab-separated file of head, relation and tail strings. PyKEEN handles the entity and relation indexing, negative sampling, training loop and filtered evaluation. More tooling is listed on /build/knowledge-graph-tools/.

Relation to text embeddings and vector databases

Text embeddings (from sentence transformers or commercial embedding APIs) and knowledge graph embeddings both produce vectors, and both can be stored in a vector database. They encode different things. A text embedding of “Marie Curie” captures how the phrase is used in language; two entities with similar descriptions land close together whether or not the graph connects them. A knowledge graph embedding of Q7186 captures only the pattern of edges around that node and knows nothing about the name.

The two are increasingly combined. Text embeddings of entity descriptions can initialize or regularize graph embeddings, which helps entities with few edges, and graph embeddings can sit beside text embeddings in a vector index so retrieval can use either similarity. The trade-offs between storing knowledge as triples and as vectors are covered on /concepts/knowledge-graph-vs-vector-database/.

Limitations

Open-world assumption. A missing triple is unknown, not false. Negative sampling treats random corruptions as false, and some of them will be true facts the graph has not recorded yet. A low score should be read as “unsupported,” not “refuted.”

Scaling. One vector per entity grows linearly with the graph. Wikidata has over 100 million items; at 200 dimensions in 32-bit floats that is roughly 80 gigabytes of entity parameters before any training state. Distributed training (DGL-KE), parameter sharing, or inductive GNN methods are needed at that scale.

No literals by default. Numbers, dates, strings and coordinates are dropped by the standard models. Extensions such as LiteralE incorporate them, but the mainstream benchmarks do not.

Transductive by default. A basic model can only score entities it saw during training. A new entity requires retraining or an inductive model that builds representations from features or neighbors.

Interpretability. Vectors do not explain why a link was predicted. Rule-mining approaches such as AMIE and AnyBURL produce symbolic explanations and remain competitive on the same benchmarks.

For developers

Start with a well-tuned baseline (RotatE or ComplEx in PyKEEN) before trying anything exotic; many published improvements disappear under equal hyperparameter budgets. Use the filtered setting, hold out a validation split for early stopping, and check that inverse relations are not leaking answers.

For SEOs

Modern search and answer systems represent entities as vectors as well as symbols. Learned entity representations are standard in the published literature on entity retrieval and disambiguation, and large language models represent every entity they know as points in a high-dimensional space. How Google or any other production system builds and uses those representations is not public, and no on-page technique manipulates them directly. The general principle still holds: an entity with many consistent, well-typed connections (founder, location, products, official profiles, Wikidata item) is easier for any embedding-based system to place accurately than one with few or contradictory edges. That is the advice of /seo/entity-seo/, arrived at from a different direction.

FAQ

What is a knowledge graph embedding?

A knowledge graph embedding is a set of learned vectors, one per entity and one per relation type, arranged so that a scoring function over the three vectors of a triple is high for true facts and low for false ones. The model learns the vectors from the graph’s existing edges. They are then used to predict missing links, group similar entities, and supply graph structure to other machine learning models.

What is link prediction in a knowledge graph?

Link prediction is the task of estimating which edges are missing from a knowledge graph. Given a head entity and a relation, such as (Marie Curie, bornIn, ?), a model ranks every entity as a possible tail. High-ranked candidates are proposed as new facts. It is the main benchmark task for knowledge graph embeddings and is evaluated with Mean Reciprocal Rank and Hits@k in the filtered setting.

How does TransE work?

TransE represents every entity and relation as a vector in the same space and treats a relation as a translation. For a true triple, adding the relation vector to the head vector should land close to the tail vector: h + r ≈ t. Training pushes true triples toward that condition and random corruptions away from it. It is simple and fast but struggles with one-to-many and symmetric relations.

Are knowledge graph embeddings the same as word embeddings?

No. Word and sentence embeddings are learned from text and capture how words are used in language. Knowledge graph embeddings are learned only from the graph’s edges and capture which entities are connected by which relations. An entity’s name plays no role in the basic models. The two kinds of vectors can be combined, for example by using text embeddings to initialize entities that have few edges.

Sources and further reading

Bordes et al., Translating Embeddings for Modeling Multi-relational Data (TransE), NeurIPS 2013: https://papers.nips.cc/paper/2013/hash/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html
Sun et al., RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space, ICLR 2019: https://arxiv.org/abs/1902.10197
Trouillon et al., Complex Embeddings for Simple Link Prediction (ComplEx), ICML 2016: https://arxiv.org/abs/1606.06357
Schlichtkrull et al., Modeling Relational Data with Graph Convolutional Networks (R-GCN): https://arxiv.org/abs/1703.06103
PyKEEN documentation: https://pykeen.readthedocs.io/
Hogan et al., Knowledge Graphs (Chapter 5, Inductive Knowledge): https://kgbook.org/