GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - RobinL/fuzzymatcher: Record linking package that fuzzy matches two Python pandas dataframes using sqlite3 fts4

BranchesTags
Open more actions menu

Latest commit

 

History

144 Commits
144 Commits

Folders and files

NameName
Last commit message
Last commit date
.github/workflows
.github/workflows
 
 
docs
docs
 
 
fuzzymatcher
fuzzymatcher
 
 
tests
tests
 
 
.gitignore
.gitignore
 
 
LICENSE
LICENSE
 
 
README.rst
README.rst
 
 
examples.ipynb
examples.ipynb
 
 
requirements.txt
requirements.txt
 
 
setup.py
setup.py
 
 

Repository files navigation

fuzzymatcher

Note: fuzzymatcher is no longer actively maintained. Please see splink for a more accurate, scalable and performant solution

A Python package that allows the user to fuzzy match two pandas dataframes based on one or more common fields.

Fuzzymatches uses sqlite3's Full Text Search to find potential matches.

It then uses probabilistic record linkage to score matches.

Finally it outputs a list of the matches it has found and associated score.

Installation

pip install fuzzymatcher

Note that you will need a build of sqlite which includes FTS4. This seems to be widely included by default, but otherwise see here.

Usage

See examples.ipynb for examples of usage and the output.

You can run these examples interactively here.

Simple example

Suppose you have a table called df_left which looks like this:

id ons_name
0 Darlington
1 Monmouthshire
2 Havering
3 Knowsley
4 Charnwood
... etc.

And you want to link it to a table df_right that looks like this:

id os_name
0 Darlington (B)
1 Havering London Boro
2 Sir Fynwy - Monmouthshire
3 Knowsley District (B)
4 Charnwood District (B)
... etc.

You can write:

import fuzzymatcher
fuzzymatcher.fuzzy_left_join(df_left, df_right, left_on = "ons_name", right_on = "os_name")

And you'll get:

best_match_score ons_name os_name
0.178449 Darlington Darlington (B)
0.133371 Monmouthshire Sir Fynwy - Monmouthshire
0.102473 Havering Havering London Boro
0.155775 Knowsley Knowsley District (B)
0.155775 Charnwood Charnwood District (B)
... etc. etc.

About

Record linking package that fuzzy matches two Python pandas dataframes using sqlite3 fts4

Topics

data-matchingfuzzy-matchingprobabalistic-matchingpypi

Resources

Readme
MIT license
Activity

Stars

287 stars

Watchers

10 watching

Forks

59 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.