pyspark-stubs
A collection of the Apache Spark stub files
Decision gist · record as of 2026-08-14
Yes, if you are using Spark 2.4 or 3.0 and want type checking support—but be aware the package is abandoned and no longer receives updates. For Spark 3.1+, use the type hints built into Apache Spark itself instead. The stubs are permissively licensed and install easily, but verify compatibility with your current mypy and PySpark versions before relying on them in production.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires a type checker (mypy, pytype, or equivalent) to be installed separately to use the stubs; stubs alone do not enforce types at runtime.
- Installation is straightforward with low friction, but the package is abandoned as of August 2022 and no longer maintained.
- Development was merged into Apache Spark 3.1+; this package only receives fixes for Spark 2.4 and 3.0 until their deprecation.
License · maintenance · safety
permissive license (permissive) — Licensed under the permissive Apache License, which allows use in commercial and private projects with minimal restrictions.
last release 2022-01-05 (1682 days) · last repo commit 2022-08-17 · 118 stars · archived
0 known vulnerabilities (OSV.dev, 2026-08-14) · 117,589 downloads/mo, #12,156 on PyPI
Alternatives
Verify before relying
pip install pyspark-stubs
# Type checker (e.g., mypy) will then recognize PySpark types
import pyspark
from pyspark.sql import SparkSession- Whether stub coverage remains adequate for modern PySpark workflows given the package is unmaintained since 2022.
- Compatibility with PySpark versions released after January 2022 (the latest release date).
- Whether the stubs work correctly with current versions of mypy or other type checkers, as the description notes rare incompatibilities with preceding mypy releases.
What it is and what it does
PySpark Stubs is a collection of type stub files (.pyi) for Apache Spark's Python API, generated by mypy's stubgen tool and manually refined with accurate type hints. It allows developers to use static type checkers like mypy and IDEs like PyCharm to catch type errors in PySpark code before runtime, and enables better autocompletion in editors.
The package was originally maintained as a standalone project but has been merged into the main Apache Spark repository for versions 3.1 and later. This PyPI package continues to serve Spark 2.4 and 3.0 users, though it is no longer actively developed. Installation overlays the stub files into your PySpark installation directory so type checkers can find them automatically.
Use it for
- Catch type errors in PySpark code during development using mypy or similar type checkers before submitting to production.
- Improve IDE autocompletion and inline documentation in PyCharm, VS Code, or other editors that support type hints.
- Enable static analysis in CI/CD pipelines to enforce type safety on Spark data processing jobs.
- Support legacy Spark 2.4 or 3.0 projects that need type hints but cannot upgrade to newer Spark versions with built-in stubs.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are using Spark 2.4 or 3.0 and want type checking support—but be aware the package is abandoned and no longer receives updates.
For Spark 3.1+, use the type hints built into Apache Spark itself instead. The stubs are permissively licensed and install easily, but verify compatibility with your current mypy and PySpark versions before relying on them in production.
Install
pyspark-stubs on PyPI
Before you install
Installation is straightforward with low friction, but the package is abandoned as of August 2022 and no longer maintained. Development was merged into Apache Spark 3.1+; this package only receives fixes for Spark 2.4 and 3.0 until their deprecation.
Requires a type checker (mypy, pytype, or equivalent) to be installed separately to use the stubs; stubs alone do not enforce types at runtime.
License in practice
Licensed under the permissive Apache License, which allows use in commercial and private projects with minimal restrictions.
Quickstart
pip install pyspark-stubs
# Type checker (e.g., mypy) will then recognize PySpark types
import pyspark
from pyspark.sql import SparkSessionVerify before relying
- Whether stub coverage remains adequate for modern PySpark workflows given the package is unmaintained since 2022.
- Compatibility with PySpark versions released after January 2022 (the latest release date).
- Whether the stubs work correctly with current versions of mypy or other type checkers, as the description notes rare incompatibilities with preceding mypy releases.
Package facts
| License | permissive license permissive |
| Python support | Not specified |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 1 packagepyspark |
| Maintenance | Abandoned 1,682 days since the last release |
| Last repo commit | repository archived |
| First released | |
| Downloads | 117,589 / month, #12,156 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 4 - BetaIntended Audience :: DevelopersLicense :: OSI Approved :: Apache Software LicenseProgramming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: Implementation :: CPythonProgramming Language :: Python :: Implementation :: PyPyTyping :: Typed |
Evidence: pyspark_stubs-3.0.0.post3-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “pyspark type hints”
- pyspark-stubsProvides type stubs for PySpark to enable static type checking and…
- tinselGenerates PySpark DataFrame schemas from Python NamedTuple and…
- sparkdanticConverts Pydantic models to PySpark schemas (StructType or JSON…
Give your agent the search over MCP, or paste the wish link into any chat.
More Quality Assurance packages
Coverage.py measures which lines of Python code are executed during test runs, reporting coverage percentages and identifying untested code paths.
Install it if you want to measure test completeness or enforce coverage thresholds in your project.
Ruff is a Python linter and code formatter written in Rust that combines linting, formatting, and code fixing into a single tool, replacing Flake8, Black, isort, and related utilities.
Pexpect spawns and controls interactive console applications by sending input and matching output patterns, automating tasks that would otherwise require manual interaction.
Black reformats Python source code to a consistent style by parsing entire files and rewriting them according to an opinionated, deterministic set of rules, eliminating manual formatting decisions.
pytest-xdist distributes pytest tests across multiple CPU cores or machines to speed up test execution, with the simplest usage being `pytest -n auto` to spawn workers equal to available CPUs.
Install it if your test suite takes long enough that parallelization would save meaningful time.
Validates AWS CloudFormation templates in YAML or JSON format against resource provider schemas and best practices, checking property values and configuration correctness.
Install it if you work with CloudFormation templates.
See also types-dataclasses · docutils-stubs · findspark · ob-metaflow-stubs · google-auth-stubs · types-futures · confluent-kafka-stubs · types-SQLAlchemy · types-backports · types-PyJWT