Entity Extraction, Named Entity Recognition & Entity Linking

Entity extraction is the process of finding the real-world things mentioned in text (people, organizations, places, products, works) and turning those mentions into structured records. It usually runs as a pipeline: named entity recognition (NER) locates and types each mention, entity linking resolves the mention to a unique identifier such as a Wikidata Q-ID, and relation extraction captures how the entities are connected. The output is a set of triples that can be loaded into a knowledge graph.

From text to knowledge graph: the pipeline

A knowledge graph is built from identifiers and relationships, while a web page or document is built from words. Entity extraction bridges the two. The standard stages are:

Mention detection. Find the spans of text that refer to an entity: “Curie”, “the University of Paris”, “Paris”.
NER typing. Assign each span a coarse type: Person, Organization, Location, Date, and so on.
Candidate generation. For each mention, retrieve the entities in a target knowledge base that it could refer to. “Curie” might yield Marie Curie, Pierre Curie, Irène Joliot-Curie, the curie unit of radioactivity, and the Curie crater on the Moon.
Disambiguation. Choose the correct candidate using context: the words around the mention, the other entities in the document, and prior probabilities about how often a surface form refers to each candidate.
Linking. Attach the chosen identifier. In Wikidata, Marie Curie is Q7186; in the Google Knowledge Graph she has a machine ID (MID) of the form /m/.... From this point the text mention is anchored to a node that already exists in a graph.
Relation extraction. Identify the relationship the sentence asserts between two linked entities, such as employedBy, bornIn or spouse, and emit it as a triple.

Steps 3 through 5 together are what most literature calls entity linking. Steps 1 and 2 together are NER. The whole chain, sometimes with coreference resolution added so that “she” in a later sentence resolves to the same entity, is entity extraction or information extraction.

A worked sentence

Take the sentence:

Curie taught at the University of Paris.

NER produces two typed mentions: “Curie” (Person) and “University of Paris” (Organization). Entity linking resolves “Curie” to Marie Curie (Wikidata Q7186) rather than Pierre Curie, because the verb “taught” and the university context fit the biography of Marie Curie, who became the first woman to hold a professorship there. “University of Paris” resolves to the historical University of Paris (Q209842). Relation extraction reads “taught at” and maps it to a predicate in the target schema, for example employedBy or schema.org’s worksFor.

The result, in Turtle:

@prefix wd:     <http://www.wikidata.org/entity/> .
@prefix schema: <https://schema.org/> .
wd:Q7186 a schema:Person ;
    schema:name "Marie Curie" ;
    schema:worksFor wd:Q209842 .
wd:Q209842 a schema:Organization ;
    schema:name "University of Paris" .

One sentence has become two typed nodes, one edge, and two labels, all keyed to identifiers that any other system using Wikidata can join against. Background on why identifiers matter more than names is on /concepts/what-is-an-entity/.

Named entity recognition in more detail

NER is a sequence labeling task: every token in the input receives a tag indicating whether it begins an entity, continues one, or is outside any entity (the BIO scheme), along with the entity type. The classic type set from the CoNLL-2003 shared task is Person, Organization, Location and Miscellaneous. OntoNotes uses eighteen types including Date, Money, Product and Work of Art. Domain-specific systems define their own: gene, protein and chemical in biomedical text; ticker and instrument in finance.

Three generations of technique are in use.

Feature-based statistical models. Conditional random fields (CRFs) and related models tag tokens using hand-designed features: capitalization, suffixes, part-of-speech tags, gazetteer membership, and the tags of neighboring tokens. They are fast, interpretable and still adequate for narrow domains with good gazetteers.

Neural sequence models. The BiLSTM-CRF architecture replaced hand-built features with learned word and character embeddings, read by a bidirectional LSTM and decoded by a CRF layer. It became the standard approach in the mid-2010s.

Transformer-based taggers. Fine-tuning a pretrained transformer such as BERT, RoBERTa or their multilingual variants with a token classification head is now the default for high accuracy. Contextual embeddings let the same word (“Washington”) receive different tags depending on whether it is used as a person or a place. spaCy’s transformer pipelines, Hugging Face token classification models and Flair all follow this pattern.

LLM-based extraction. Large language models can perform NER and relation extraction from a prompt, with no task-specific training, and can emit output directly as JSON or triples. They handle unusual entity types and long-range context well. Their drawbacks are cost, latency, non-deterministic output, and a tendency to produce plausible but unsupported entities or relations, which is why production pipelines often pair an LLM extractor with a conventional linker and schema validation. The same caution applies when LLM output feeds a graph used for retrieval, as discussed on /build/graphrag/.

Entity linking and disambiguation

Entity linking (also called named entity disambiguation or entity resolution against a knowledge base) is where a string becomes a thing. It depends on three sources of evidence.

Surface form statistics. Knowledge bases derived from Wikipedia record how often each anchor text links to each article. “Paris” links to the French capital far more often than to Paris, Texas, or Paris Hilton, which gives a strong prior.

Local context. The words around a mention are compared with the description or Wikipedia text of each candidate. “Curie” near “radioactivity” and “Sorbonne” favors Marie Curie; “Curie” near “millicurie” favors the unit.

Global coherence. Entities in the same document tend to be related. If the document also mentions Pierre Curie and polonium, the candidate Marie Curie scores higher because she is connected to both in the graph. Collective disambiguation methods optimize the whole document’s assignments jointly.

A mention that has no matching entity in the knowledge base should be linked to NIL. Handling NIL correctly is one of the harder parts of the task: a linker that always forces a match will attach a small local business to a famous namesake. NIL mentions are often clustered so that the same unknown entity across documents can later be added to the graph as a new node, a process sometimes called knowledge base population.

Systems and tools

Tool	Type	Notes
spaCy	Library (Python)	Statistical and transformer NER pipelines for many languages; includes an `EntityLinker` component that must be trained against a knowledge base the user supplies.
DBpedia Spotlight	Service and open source	Annotates text with DBpedia resources, which map to Wikipedia articles and, through `owl:sameAs`, to Wikidata. See /ecosystems/dbpedia/.
REL (Radboud Entity Linker)	Open source (Python)	Modular pipeline with Flair-based NER and a neural disambiguation model trained on Wikipedia.
TagMe	Service	Designed for short, noisy text such as tweets and search queries; returns Wikipedia page IDs.
Hugging Face Transformers	Library	Thousands of pretrained token classification models; linking must be added separately.
Google Cloud Natural Language API	Commercial API	Entity analysis returns entity type, a salience score, mentions, and for well-known entities a Wikipedia URL and a Knowledge Graph MID.
Wikifier, Babelfy, OpenTapioca	Services	Alternative linkers targeting Wikipedia, BabelNet and Wikidata respectively.

Wikidata is the most common linking target for new systems because it is multilingual, has stable Q-IDs, and is openly licensed. The Google Knowledge Graph Search API can also be used to look up MIDs by name, as described on /ecosystems/google-knowledge-graph-search-api/.

Evaluating extraction and linking

NER is scored with precision, recall and F1 over entity spans. Precision is the fraction of predicted entities that are correct; recall is the fraction of true entities that were found. Strict evaluation requires both the span boundaries and the type to match; relaxed evaluation gives partial credit for overlapping spans.

Entity linking adds accuracy over linked mentions and, in end-to-end settings, precision and recall over (span, identifier) pairs. Two details matter. First, NIL handling is scored separately, because a system can achieve high in-KB accuracy while linking everything and failing on unknown entities. Second, results depend heavily on the knowledge base version and the domain: a linker trained on news performs differently on scientific abstracts or product reviews.

Relation extraction is evaluated per relation type with precision, recall and F1, and often distinguishes between sentence-level and document-level extraction.

A short spaCy NER example

import spacy
nlp = spacy.load("en_core_web_trf")
doc = nlp("Curie taught at the University of Paris after Pierre Curie died in 1906.")
for ent in doc.ents:
    print(ent.text, ent.label_, ent.start_char, ent.end_char)

Typical output tags “Curie” and “Pierre Curie” as PERSON, “the University of Paris” as ORG and “1906” as DATE. The model must be downloaded first with python -m spacy download en_core_web_trf; the smaller en_core_web_sm runs faster on CPU with lower accuracy. To link the recognized entities to Wikidata, the next step is either to train spaCy’s EntityLinker on a custom knowledge base or to pass the spans to an external linker. A fuller build walkthrough is on /build/knowledge-graph-python/.

For developers

The hardest problems in production are rarely the model. They are the knowledge base (which identifiers to link to, how to keep them current), the schema (which relation types to extract and how they map to the ontology, see /concepts/ontologies/), and validation (rejecting triples whose subject or object type violates the schema). A practical pattern is to run NER and relation extraction with a neural or LLM-based model, link against Wikidata for public entities and a private entity registry for internal ones, validate with SHACL shapes, and store provenance (source document, character offsets, model version, confidence) on every triple so extraction errors can be traced and corrected.

For SEOs

Search engines run the same pipeline over every page they index. Google’s Cloud Natural Language API exposes a version of it: its entity analysis returns each entity found on a page, its type, a salience score between 0 and 1 indicating how central the entity is to the text, and, where the entity is in the Google Knowledge Graph, a Wikipedia URL and MID. SEOs use it to audit whether the entity a page is meant to be about is in fact the most salient one, whether the page’s main subject links to a known entity at all, and which competing entities dilute the topic.

The disambiguation stage explains two common recommendations. Consistent naming (always “University of Florida”, not sometimes “UF” and sometimes “Florida”) gives the linker a stable surface form. And sameAs links in structured data to Wikidata, Wikipedia and official profiles hand the search engine an explicit identifier instead of asking it to infer one. See /seo/sameas-schema/ and /seo/what-are-entities-in-seo/.

Common mistakes

Confusing NER with entity linking. NER says a span is a Person; linking says which person. A page can rank well on the first and fail on the second.

Ignoring NIL. Not every mention has a Wikidata entry, and forcing one produces confident errors.

Extracting relations without a target schema. Free-text predicates (“taught at”, “was a professor of”, “lectured at”) must be normalized to a fixed set of properties or the graph becomes unqueryable.

Skipping provenance. Extracted triples are hypotheses with a confidence score, and they need to be distinguishable from curated facts.

FAQ

What is the difference between NER and entity linking?

Named entity recognition finds the spans in text that mention an entity and assigns each a type such as Person or Organization. Entity linking takes those spans and resolves each to a specific identifier in a knowledge base, for example Wikidata Q7186 for Marie Curie. NER answers “is this a person,” while linking answers “which person,” and a full pipeline needs both.

What is entity disambiguation?

Entity disambiguation is the step in entity linking that chooses among candidate entities sharing the same name. “Paris” could be the French capital, a city in Texas or a person. The system uses the surrounding words, other entities in the document and how often the name usually refers to each candidate to pick one, or to decide that none match and return NIL.

How is entity extraction used in SEO?

Search engines extract entities from every page to understand what it is about and to connect it to their knowledge graph. SEOs use tools such as Google’s Natural Language API to check which entities a page surfaces, how salient the intended topic is, and whether the main entity links to a known Wikipedia or Knowledge Graph identifier. Consistent naming and sameAs markup make that linking easier.

Which tools perform entity linking to Wikidata?

Open source options include spaCy’s EntityLinker component (trained on a user-supplied knowledge base), REL, OpenTapioca and DBpedia Spotlight, whose DBpedia resources map to Wikidata through sameAs links. Google’s Cloud Natural Language API returns Wikipedia URLs and Knowledge Graph MIDs, which can be mapped to Q-IDs. Large language models can also propose Q-IDs, but their output should be verified against Wikidata.

Sources and further reading

spaCy documentation, Linguistic Features: Named Entities: https://spacy.io/usage/linguistic-features#named-entities
spaCy API, EntityLinker: https://spacy.io/api/entitylinker
Google Cloud Natural Language, Analyzing Entities: https://cloud.google.com/natural-language/docs/analyzing-entities
DBpedia Spotlight: https://www.dbpedia-spotlight.org/
Wikidata, Introduction: https://www.wikidata.org/wiki/Wikidata:Introduction
Hogan et al., Knowledge Graphs (Chapter 6, Creation and Enrichment): https://kgbook.org/