Splink

Skip to content

Initializing search

GitHub

Examples

PySpark

Athena

SQLite

API Docs

Comparisons Library

Other

In-build datasets

Splink Settings

User Guide

Linkage Models in Splink

Data Preparation

Blocking

Comparing Records

Training

Evaluation

Clusters

Performance

DuckDB Performance

Charts Gallery

Blocking

Similarity analysis

Model Training

Clustering

Model Evaluation

LLM prompts

Contributing

How Splink works

Charts

User-Defined Functions

Dependency Compatibility Policy

Blog

Fast, accurate and scalable probabilistic data linkage¶

Splink is a Python package for probabilistic record linkage (entity resolution) that allows you to deduplicate and link records from datasets without unique identifiers.

Get Started with Splink

Key Features¶

⚡ Speed: Capable of linking a million records on a laptop in approximately one minute.

🎯 Accuracy: Full support for term frequency adjustments and user-defined fuzzy matching logic.

🌐 Scalability: Execute linkage jobs in Python (using DuckDB) or big-data backends like AWS Athena or Spark for 100+ million records.

🎓 Unsupervised Learning: No training data is required, as models can be trained using an unsupervised approach.

📊 Interactive Outputs: Provides a wide range of interactive outputs to help users understand their model and diagnose linkage problems.

Splink's core linkage algorithm is based on Fellegi-Sunter's model of record linkage, with various customizations to improve accuracy.

What does Splink do?¶

Consider the following records that lack a unique person identifier:

Splink predicts which rows link together:

and clusters these links to produce an estimated person ID:

What data does Splink work best with?¶

Before using Splink, input data should be standardised, with consistent column names and formatting (e.g., lowercased, punctuation cleaned up, etc.).

Splink performs best with input data containing multiple columns that are not highly correlated . For instance, if the entity type is persons, you may have columns for full name, date of birth, and city. If the entity type is companies, you could have columns for name, turnover, sector, and telephone number.

High correlation occurs when the value of a column is highly constrained (predictable) from the value of another column. For example, a 'city' field is almost perfectly correlated with 'postcode'. Gender is highly correlated with 'first name'. Correlation is particularly problematic if all of your input columns are highly correlated.

Splink is not designed for linking a single column containing a 'bag of words'. For example, a table with a single 'company name' column, and no other details.

Videos¶

Our PyData Global 2024 talk provides a brief introduction to Splink and is available on YouTube here.

Support¶

If after reading the documentation you still have questions, please feel free to post on our discussion forum.

Use Cases¶

Here is a list of some of our known users and their use cases:

Public Sector (UK)Public Sector (International)AcademiaOther

Office for National Statistics's Business Index (formerly the Inter Departmental Business Register), Demographic Index and the 2021 Census. See also this article and 2021 Census to PDS linkage report.

NHS England is working on developing an alternative data linkage model using splink as the core engine for a new probabilistic data linkage service. This is in order to improve linkage and linkage explainability across NHS datasets. Code now available on github.

Ministry of Defence launched their Veteran's Card system which uses Splink to verify applicants against historic records. This project was shortlisted for the Civil Service Awards

Ministry of Justice created linked datasets (combining courts, prisons and probation data) for use by researchers as part of the Data First programme

Ministry of Justice and the BOLD programme used Splink to power the North Essex Probation Delivery Unit Case Information Dashboard, which won the 2025 Civil Service Award for Excellence in Delivery.

UK Health Security Agency used Splink to link HIV testing data to national health records to evaluate the impact of emergency department opt-out bloodborne virus testing.

The Department for Education uses Splink to match records from certain data providers to existing learners and reduce the volume of clerical work required for corrections

SAIL Databank, in collaboration with Secure eResearch Platform (SeRP), uses Splink to produce linked cohorts for a wide range of population-level research applications

Lewisham Council (London) identified and auto-enrolled over 500 additional eligible families to receive Free School Meals

Leicestershire County Council use Splink to match individuals across their Education and Social Care systems. This ensures triage and front-line practitioners have a complete picture of those individuals.

Integrated Corporate Services have used Splink to match address data in historical...