mwxml

What it is and what it does

mwxml is a Python library for processing MediaWiki XML database dumps—the raw XML exports that contain Wikipedia and other wiki content. It addresses two core problems: the complexity of parsing large XML files and the performance overhead of naive approaches. The library provides a simple iterator interface that streams through dump files without loading them entirely into memory, making it practical to process dumps that would otherwise exhaust available RAM.

The package also supports distributed processing, allowing you to work with multiple dump files in parallel. It depends on mwtypes, mwcli, para, and jsonschema to handle MediaWiki-specific data types, command-line utilities, parallelization, and schema validation. It's designed for developers and researchers who need to extract, analyze, or transform content from wiki database exports.

Use it for:

• Extract revision history and metadata from Wikipedia or other MediaWiki dumps for linguistic or historical analysis.
• Process multiple large XML dump files in parallel to build indexes or data warehouses from wiki content.
• Stream through a dump file to filter and extract specific pages or revisions without loading the entire file into memory.
• Analyze edit patterns, contributor activity, or content evolution by iterating over revisions in a structured way.
• Build data pipelines that consume wiki dumps as input for downstream NLP or machine learning workflows.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Efficiently parse and stream-process MediaWiki XML database dumps with memory-conscious iteration and optional distributed processing across multiple files.

Yes. The package is actively maintained, has no known vulnerabilities, installs with low friction, and solves a real problem for anyone working with MediaWiki dumps. The MIT license imposes no restrictions. Install it if you need to process wiki database exports; skip it if you're not working with MediaWiki XML data.

Install

mwxml on PyPI

pip install mwxml

uv add mwxml

poetry add mwxml

Installing mwxml

Before you install

Low install friction with a pure-Python wheel distribution. Active maintenance with a recent release (128 days ago) and ongoing repository activity.

License in practice

MIT license permits unrestricted use, modification, and distribution with minimal legal constraints.

Quickstart

pip install mwxml

import mwxml

dump = mwxml.Dump.from_file(open("dump.xml"))
for page in dump:
    for revision in page:
        print(revision.id)

Verify before relying

• Whether the distributed processing strategy (map function) requires additional setup or dependencies beyond the listed runtime packages.
• Performance characteristics and memory overhead when processing very large dump files.
• Python version compatibility details, as requires_python is unspecified despite Python 3 Only classifier.

Package facts

Evidence: mwxml-0.3.8-py2.py3-none-any.whl

Tags

More Scientific/Engineering packages