---
id: razdel
version: "0.5.0"
license: MIT
license_treatment: permissive
maintenance: active
---

# razdel — Splits russian text into tokens, sentences, section. Rule-based

License: permissive · Maintenance: active · Downloads: 130.5K/mo

## What it is and what it does

razdel is a rule-based tokenizer and sentence splitter designed specifically for Russian text. It provides two main functions: tokenize() breaks text into words and punctuation with character offsets, and sentenize() splits text into sentences. The package returns Substring objects that preserve the original text span, making it easy to map results back to the source.

The library is optimized against four Russian corpora (SynTagRus, OpenCorpora, GICRYA, RNC) consisting mainly of news and fiction. It trades off perfect accuracy for practical performance: the description acknowledges that tokenization has no single correct answer and documents its segmentation choices explicitly. Benchmarks show it achieves competitive error rates on these domains while maintaining reasonable speed.

Use it for:

- Preprocess Russian news articles or literary texts before feeding into downstream NLP models
- Extract individual words and punctuation from Russian text while preserving their exact positions for annotation or highlighting
- Split Russian documents into sentences for batch processing or analysis at the sentence level
- Build Russian text search or indexing pipelines that require accurate token boundaries
- Evaluate or compare tokenization quality on Russian corpora using the included benchmarking tools

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Splits Russian text into sentences and tokens using rule-based heuristics, returning character-offset substrings for precise text reconstruction.

Yes, if your text is primarily Russian news or fiction. The package is lightweight, actively maintained, permissively licensed, and offers competitive accuracy on its target domains. Install with caution if your text is from social media, legal documents, or scientific articles—the description explicitly warns performance may degrade outside news/fiction.

## Install

pip install razdel
uv add razdel
poetry add razdel

## Installing razdel

Before you install: Low friction: pure Python wheel with no runtime dependencies. Actively maintained as of April 2026 with 286 repository stars.

License in practice: MIT license permits unrestricted use, modification, and distribution with minimal attribution requirements.

Quickstart:

from razdel import tokenize, sentenize

tokens = list(tokenize('Кружка-термос на 0.5л'))
sents = list(sentenize('Так в чем же дело? Не радуют.'))

Verify before relying:

- Whether performance on non-news/fiction domains (social media, legal, scientific) is documented or measurable
- Exact Python version support range beyond the '3.5+' claim in the description

## Package facts

- License: MIT (permissive)
- Python support: unspecified
- Install friction: low
- Maintenance: active
- Downloads: 130.5K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

russian text tokenization, russian sentence segmentation, cyrillic word tokenizer, russian nlp preprocessing, russian text splitting, morphological tokenization russian, sentence boundary detection russian, russian-nlp, rule-based, tokenization

[View on SkillFed](https://skillfed.io/packages/razdel) · [View on PyPI](https://pypi.org/project/razdel/)