{"categories":[{"label":"Information Analysis","url":"https://skillfed.io/packages/category/scientific-engineering-information-analysis"}],"enrichment":{"capability":"Provides streaming algorithms (sketches) for approximate answers to big-data queries like cardinality estimation, quantiles, and frequent items with proven error bounds and orders-of-magnitude speed gains.","skillfed_tags":["streaming-algorithms","approximate-computing","cardinality-estimation"],"use_cases":["Estimate cardinality (distinct count) in a data stream without storing all unique values.","Compute approximate quantiles and percentiles from unbounded data streams with controlled error.","Identify frequent items or heavy hitters in a stream with false-positive/negative guarantees.","Perform approximate set operations (union, intersection) on large datasets with memory efficiency.","Implement real-time analytics dashboards where exact results are unnecessary but speed is critical."],"what_it_does":"DataSketches is a Python binding to the Apache DataSketches library, a collection of streaming algorithms designed to answer expensive big-data queries approximately but with mathematically proven error bounds. Instead of computing exact results (which may require prohibitive compute time and resources), sketches trade precision for speed\u2014often delivering results orders of magnitude faster. The library includes algorithms for cardinality estimation (HLL, CPC), quantile approximation (KLL, REQ, Quantiles), frequent-item detection, set operations (Theta, Tuple), sampling (VarOpt, EBPPS), and specialized tools like count-min sketches and kernel density estimation.\n\nThe package wraps C++ implementations via nanobind, providing a Python API that closely mirrors the C++ interface. It depends on numpy and is distributed as precompiled wheels for modern Python versions and common platforms. The library is portable across languages when endianness matches, making it suitable for systems that need to serialize sketches across different environments. Use it when exact answers are infeasible but approximate results with error guarantees are acceptable\u2014typical scenarios include real-time analytics, interactive queries on massive datasets, and distributed systems where communication overhead dominates.","worth_installing":"Yes, if you need approximate answers to expensive big-data queries with proven error bounds and can tolerate a 531-day release gap. The library is stable, well-maintained under Apache governance, has no known vulnerabilities, and offers a mature API. Install friction is moderate due to C++ compilation, but prebuilt wheels reduce friction for standard platforms. Not suitable if you require exact results or need active feature development."},"id":"datasketches","links":{"html":"https://skillfed.io/packages/datasketches","md":"https://skillfed.io/packages/datasketches.md","pypi":"https://pypi.org/project/datasketches/"},"maintenance":{"status":"aging"},"meta":{"latest_release":"2025-03-01","license_spdx":null,"license_treatment":"permissive","name":"datasketches","python_support":"unspecified","summary":"The Apache DataSketches Library for Python"},"popularity":{"monthly_downloads":1205968,"position":4222,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"5.2.0"}