{"categories":[{"label":"Information Analysis","url":"https://skillfed.io/packages/category/scientific-engineering-information-analysis/3"}],"enrichment":{"capability":"Performs fast fuzzy string matching on large datasets using TF-IDF vectorization and K-Nearest Neighbours, scaling better than traditional fuzzy matchers by avoiding O(n\u00b2) complexity.","skillfed_tags":["fuzzy-matching","text-similarity","data-deduplication"],"use_cases":["Deduplicate or link company names across two large databases without manual review.","Match product names from a supplier catalog to your internal inventory.","Find similar addresses or customer names across datasets for record linkage.","Bulk fuzzy search of user-provided strings against a large reference corpus.","Identify near-duplicate entries in a dataset before data cleaning."],"what_it_does":"tfidf_matcher solves the scalability problem of fuzzy string matching by using TF-IDF vectorization and K-Nearest Neighbours instead of pairwise comparison. It takes two lists\u2014an original list you want to find matches for and a lookup list to search within\u2014and returns a pandas DataFrame with the k closest matches from the lookup list for each item in the original, along with match scores. The approach trades off some matching quality for speed, making it practical for matching hundreds or thousands of items against large reference corpora.\n\nThe package depends on scikit-learn for vectorization and KNN, and pandas for result formatting. It exposes two main functions: ngrams() for generating n-grams and matcher() for the core matching operation. The author tested it successfully on 640 company names matched against a corpus of over 700,000 names, but explicitly notes the package is not well-tested beyond that use case and may produce unstable results in other scenarios.","worth_installing":"Yes, but with caution. Install if you need fast fuzzy matching on large datasets and can tolerate the risk of an abandoned package. The MIT license and low dependency friction are favorable, and it has no known vulnerabilities. However, the last update was in April 2023, there is no active maintenance, and testing is limited to a single use case. Use only if you can verify it works for your specific data and are willing to maintain a fork if needed."},"id":"tfidf-matcher","links":{"html":"https://skillfed.io/packages/tfidf-matcher","md":"https://skillfed.io/packages/tfidf-matcher.md","pypi":"https://pypi.org/project/tfidf-matcher/"},"maintenance":{"status":"abandoned"},"meta":{"latest_release":"2023-04-06","license_spdx":null,"license_treatment":"permissive","name":"tfidf-matcher","python_support":"supports_current","summary":"A small package that enables super-fast TF-IDF based string matching."},"popularity":{"monthly_downloads":84264,"position":14011,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.3.0"}