snowpipe-streaming

What it is and what it does

Snowpipe Streaming is a Python SDK for real-time data ingestion built with a Rust core and Python bindings using stable ABI (abi3) for forward compatibility across Python 3.9–3.13. It manages channel lifecycle, row buffering, and communication with streaming infrastructure, handling retries and backpressure automatically. The single runtime dependency is msgspec for fast serialization.

You instantiate a client with credentials and pipe metadata, open a channel, append rows as dictionaries, and close when done. It targets production streaming workloads where latency and throughput matter—typical use is continuous or near-continuous data flow, not one-off bulk loads.

Use it for:

• Real-time event streaming from application logs or sensors for live analytics
• Continuous ETL pipelines that ingest data from message queues or APIs
• Live dashboards and monitoring systems requiring sub-second data freshness
• IoT data ingestion where devices or edge systems stream measurements
• Change Data Capture workflows that stream database changes in near-real-time

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Streams data in real-time into Snowflake tables via a Python SDK with a Rust-based core for high performance.

Yes, if you need to stream data at scale. The package is actively maintained, has no known vulnerabilities, uses a permissive license, and offers prebuilt wheels for common platforms. Medium install friction is acceptable for a compiled Rust extension. Install only if you have streaming requirements; it is not a general-purpose data tool.

Install

snowpipe-streaming on PyPI

pip install snowpipe-streaming

uv add snowpipe-streaming

poetry add snowpipe-streaming

Installing snowpipe-streaming

Before you install

Medium install friction due to compiled Rust bindings; prebuilt wheels available for Linux (x86_64, aarch64), macOS (arm64), and Windows (x86_64) on Python 3.9+. Active maintenance with a release 22 days ago.

License in practice

Licensed under MIT or Apache-2.0 (permissive); either license allows commercial use, modification, and distribution with minimal restrictions.

Quickstart

pip install snowpipe-streaming

from snowpipe_streaming import StreamingIngestClient

client = StreamingIngestClient(
    client_name="my_client",
    db_name="my_database",
    schema_name="my_schema",
    pipe_name="my_pipe",
    properties={"account": "your_account", "user": "your_user", "private_key": "your_private_key"}
)
channel, status = client.open_channel("my_channel")
channel.append_row({"id": 1, "name": "John Doe"})
channel.close()
client.close()

Requires Python 3.9 or higher; Snowflake account credentials must be configured; glibc 2.26+ on Linux, macOS 11.0+ on macOS, Windows 10+ on Windows.

Verify before relying

• Throughput and latency benchmarks compared to alternative ingest methods
• Retry behavior and failure recovery specifics beyond 'built-in retry logic'
• Memory footprint under sustained high-volume streaming
• Backpressure handling thresholds and overflow behavior

Package facts

Evidence: snowpipe_streaming-1.7.0-cp39-abi3-macosx_11_0_arm64.whl; snowpipe_streaming-1.7.0-cp39-abi3-manylinux_2_24_x86_64.whl; snowpipe_streaming-1.7.0-cp39-abi3-manylinux_2_26_aarch64.whl; snowpipe_streaming-1.7.0-cp39-abi3-win_amd64.whl

Keywords: snowflake, streaming, ingest, data

Tags

More Database packages