{"categories":[{"label":"Software Development","url":"https://skillfed.io/packages/category/software-development/14"},{"label":"Libraries","url":"https://skillfed.io/packages/category/software-development-libraries/8"},{"label":"Python Modules","url":"https://skillfed.io/packages/category/software-development-libraries-python-modules/17"}],"enrichment":{"capability":"Encodes PyArrow RecordBatches into PostgreSQL's native binary format for efficient bulk loading via COPY.","skillfed_tags":["etl","bulk-load","data-pipeline"],"use_cases":["Load Parquet files from S3 or local disk into PostgreSQL without staging or intermediate formats.","Bulk import Arrow datasets with custom field encoding (e.g., string-to-JSONB) to match Postgres table schemas.","Stream large datasets through psycopg's COPY interface while avoiding row-by-row inserts or temporary CSV files.","Automate data pipelines that extract Arrow data and load it into Postgres for analytics or reporting."],"what_it_does":"pgpq is a Python library that translates PyArrow RecordBatches into PostgreSQL's native binary wire format, enabling direct bulk loading of Arrow data into Postgres via the COPY command. It handles schema translation, field encoding, and binary serialization so you can stream Arrow datasets (from Parquet, CSV, or other sources) directly into Postgres without intermediate conversions or row-by-row inserts.\n\nThe package sits between PyArrow and psycopg, providing an ArrowToPostgresBinaryEncoder that takes an Arrow schema, optionally customizes field encoders (e.g., mapping string columns to JSONB), and produces the byte stream PostgreSQL expects. You define a temporary or permanent table matching the encoder's schema, then pipe the encoded batches through psycopg's COPY interface. It's designed for data warehouse and ETL workflows where bulk loading speed matters.","worth_installing":"Yes, if you regularly load Arrow data into PostgreSQL and want to avoid row-by-row inserts or intermediate file formats. The package is actively maintained, has no known vulnerabilities, and solves a specific ETL bottleneck. Medium install friction (compiled wheels) is typical for performance-critical libraries. Not necessary if you use PostgreSQL's native tools or rarely bulk-load Arrow data."},"id":"pgpq","links":{"html":"https://skillfed.io/packages/pgpq","md":"https://skillfed.io/packages/pgpq.md","pypi":"https://pypi.org/project/pgpq/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-02-28","license_spdx":null,"license_treatment":"permissive","name":"pgpq","python_support":"supports_current","summary":"Arrow -> PostgreSQL encoder"},"popularity":{"monthly_downloads":173253,"position":10314,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.11.1"}