madoka

What it is and what it does

Madoka is a Python wrapper around a C++ Count-Min sketch library that trades exact counting for memory efficiency. It provides multiple classes (Sketch, CroquisFloat, CroquisDouble, CroquisUint8/16/32/64) to store different numeric types, each using a probabilistic data structure that approximates frequencies with bounded error. Unlike a standard Python dict or Counter, a sketch uses fixed memory regardless of the number of unique keys, making it suitable for streaming scenarios where you cannot store all keys in memory.

The package supports typical counter operations—increment, add, set, get—plus sketch-specific operations like merge, shrink, inner product, and median. It can serialize to disk, load from file, and extract top-K most common keys if initialized with a k parameter. The tradeoff is that you cannot enumerate all keys (only top-K), and values are approximate rather than exact.

Use it for:

• Approximate word frequency counting in large text streams where memory is constrained.
• Real-time traffic or event counting where exact counts are less important than memory efficiency.
• Merging frequency counts from multiple data sources with minimal memory overhead.
• Extracting top-K most frequent items from a stream without storing all unique items.
• Comparing similarity between two frequency distributions using inner product.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Madoka implements a Count-Min sketch data structure for memory-efficient counting of string-int pairs in streaming data, with lower memory overhead than Python dict or Counter.

Yes, if you need memory-efficient approximate counting for streaming data and can tolerate ~0.0911% error. The package is stable (Alpha status, 12+ years old), has no known vulnerabilities, and supports modern Python versions. Install friction is moderate due to compiled wheels. Not suitable if you require exact counts or need to enumerate all keys.

Install

madoka on PyPI

pip install madoka

uv add madoka

poetry add madoka

Installing madoka

Before you install

Medium install friction due to compiled C++ extension with wheels available for Python 3.8–3.14 across Linux, macOS, and Windows platforms. Last release 257 days ago; repository is active but aging.

License in practice

New BSD License (permissive) allows commercial and private use with minimal restrictions.

Quickstart

import madoka
sketch = madoka.Sketch()
sketch['key'] += 1
value = sketch['key']

Verify before relying

• Whether the 0.0911% counting error rate applies uniformly across all Croquis variants or only to Sketch class.
• Performance characteristics and memory savings relative to dict/Counter for typical workloads.
• Whether Free Threading support (listed in classifiers) is fully functional or experimental.

Package facts

Evidence: madoka-0.7.2.1-cp310-cp310-macosx_10_9_universal2.whl; madoka-0.7.2.1-cp310-cp310-macosx_10_9_x86_64.whl; madoka-0.7.2.1-cp310-cp310-macosx_11_0_arm64.whl; madoka-0.7.2.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl; madoka-0.7.2.1-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl; madoka-0.7.2.1-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl; madoka-0.7.2.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl; madoka-0.7.2.1-cp310-cp310-musllinux_1_2_aarch64.whl; madoka-0.7.2.1-cp310-cp310-musllinux_1_2_ppc64le.whl; madoka-0.7.2.1-cp310-cp310-musllinux_1_2_s390x.whl; madoka-0.7.2.1-cp310-cp310-musllinux_1_2_x86_64.whl; madoka-0.7.2.1-cp310-cp310-win32.whl; madoka-0.7.2.1-cp310-cp310-win_amd64.whl; madoka-0.7.2.1-cp310-cp310-win_arm64.whl; madoka-0.7.2.1-cp311-cp311-macosx_10_9_universal2.whl; madoka-0.7.2.1-cp311-cp311-macosx_10_9_x86_64.whl; madoka-0.7.2.1-cp311-cp311-macosx_11_0_arm64.whl; madoka-0.7.2.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl; madoka-0.7.2.1-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl; madoka-0.7.2.1-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl

Keywords: Count-Min Sketch, counter, word count

Tags

More Python Modules packages