---
id: bigquery-schema-generator
version: "1.6.1"
license: Apache 2.0
license_treatment: permissive
maintenance: dormant
---

# bigquery-schema-generator — BigQuery schema generator from JSON or CSV data

License: permissive · Maintenance: dormant · Downloads: 787.9K/mo

## What it is and what it does

BigQuery Schema Generator is a command-line tool and Python library that infers BigQuery table schemas by scanning complete datasets. Unlike BigQuery's built-in auto-detection (which examines only the first 500 records), this tool processes all input records to produce a more accurate schema, especially useful when data fields are added organically over time and may not appear in early records.

The tool accepts newline-delimited JSON or CSV format on standard input and outputs a BigQuery-compatible schema in JSON format. It can be used as a standalone command-line utility or imported as a library for programmatic schema generation. It supports various options for handling nulls, quoted values, type inference modes, and schema merging with existing schemas.

Use it for:

- Generate accurate BigQuery schemas for datasets where fields appear sporadically across records, ensuring no data is lost during table creation.
- Pre-validate and generate schemas for CSV or JSON data before bulk loading into BigQuery using the bq load tool.
- Merge new schema information from incremental data exports with existing table schemas to keep them up-to-date.
- Debug and inspect inferred field types and structure from raw data files before committing to a table design.
- Automate schema generation in data pipelines where data format or structure may vary across sources.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Generates BigQuery table schemas from complete JSON or CSV datasets by analyzing all records, rather than relying on BigQuery's auto-detection which examines only the first 500 records.

Yes, with conditions. The tool fills a genuine gap in BigQuery's schema detection workflow. However, dormant maintenance (944 days since last release) and high install friction are concerns. Install if you regularly work with BigQuery data ingestion and need reliable schema inference across complete datasets; avoid if you need active maintenance or have strict dependency constraints.

## Install

pip install bigquery-schema-generator
uv add bigquery-schema-generator
poetry add bigquery-schema-generator

## Installing bigquery-schema-generator

Before you install: High install friction: the package is distributed as a source tarball with no runtime dependencies listed, suggesting potential build or environment setup requirements. Maintenance is dormant—last release was 944 days ago, though the repository remains active with a recent commit.

License in practice: Licensed under Apache 2.0 (permissive), allowing free use, modification, and distribution with minimal restrictions; suitable for both open-source and proprietary projects.

Quickstart:

pip3 install bigquery-schema-generator
generate-schema < file.data.json > file.schema.json
# Or for CSV:
generate-schema --input_format csv < file.data.csv > file.schema.json

Requires Python 3.6 or higher; the generate-schema command may be installed in different locations depending on your Python environment (e.g., /usr/bin, /usr/local/bin, or ~/.local/bin).

Verify before relying:

- Whether the high install friction is due to compiled dependencies, system libraries, or simply source-only distribution.
- Current compatibility with modern BigQuery API versions and schema features.
- Performance characteristics when processing very large datasets.

## Package facts

- License: Apache 2.0 (permissive)
- Python support: supports_current
- Install friction: high
- Maintenance: dormant
- Downloads: 787.9K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

bigquery schema generation, json to bigquery schema, csv to bigquery schema, schema inference from data, bigquery table schema generator, newline delimited json schema, data schema detection, bigquery, schema-inference, data-pipeline

[View on SkillFed](https://skillfed.io/packages/bigquery-schema-generator) · [View on PyPI](https://pypi.org/project/bigquery-schema-generator/)