{"categories":[{"label":"Text Processing","url":"https://skillfed.io/packages/category/text-processing/2"}],"enrichment":{"capability":"A Rust-based JSON tokenizer that accelerates JSON parsing for the json-stream library, available as a drop-in replacement or automatic dependency.","skillfed_tags":["json-parsing","performance-optimization","rust-extension"],"use_cases":["Use with json-stream 2.0 or later to get automatic Rust-accelerated tokenization without code changes.","Explicitly select RustTokenizer in older json-stream versions to override the default pure-Python implementation.","Process large or deeply nested JSON documents where tokenization overhead is significant.","Ensure consistent fast parsing across CPython environments without modifying application code."],"what_it_does":"json-stream-rs-tokenizer is a Rust-compiled tokenizer for the json-stream library, ported from json-stream's own pure-Python tokenizer. Since json-stream 2.0, it is automatically used as a dependency, so explicit installation is typically unnecessary unless you're using an older json-stream version or need to ensure the Rust implementation is loaded. The package trades medium install friction (compiled wheels for multiple platforms, or Rust build from source) for parsing speedup.\n\nThe tokenizer has no runtime dependencies and works transparently: json-stream will use it if available and fall back to pure Python if the Rust extension failed to build or load. Explicit use requires importing RustTokenizer and passing it to json-stream's load or visit functions. PyPy users see only modest speedup, and large-integer conversion may fall back to Python in PyPy builds or when using Python's limited C API.","worth_installing":"Yes, if you use json-stream 2.0 or later\u2014it is installed automatically and provides transparent speedup with no code changes. Yes-with-conditions if using an older json-stream version: install it explicitly only if profiling shows tokenization is a bottleneck. No if you are on PyPy and the speedup does not justify the build complexity, or if your JSON workload is small and tokenization is not a measured bottleneck."},"id":"json-stream-rs-tokenizer","links":{"html":"https://skillfed.io/packages/json-stream-rs-tokenizer","md":"https://skillfed.io/packages/json-stream-rs-tokenizer.md","pypi":"https://pypi.org/project/json-stream-rs-tokenizer/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-10","license_spdx":null,"license_treatment":"unclear","name":"json-stream-rs-tokenizer","python_support":"supports_current","summary":"A faster tokenizer for the json-stream Python library"},"popularity":{"monthly_downloads":1253425,"position":4158,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"0.5.3"}