{"categories":[{"label":"Database","url":"https://skillfed.io/packages/category/database/2"}],"enrichment":{"capability":"Generates BigQuery table schemas from complete JSON or CSV datasets by analyzing all records, rather than relying on BigQuery's auto-detection which examines only the first 500 records.","skillfed_tags":["bigquery","schema-inference","data-pipeline"],"use_cases":["Generate accurate BigQuery schemas for datasets where fields appear sporadically across records, ensuring no data is lost during table creation.","Pre-validate and generate schemas for CSV or JSON data before bulk loading into BigQuery using the bq load tool.","Merge new schema information from incremental data exports with existing table schemas to keep them up-to-date.","Debug and inspect inferred field types and structure from raw data files before committing to a table design.","Automate schema generation in data pipelines where data format or structure may vary across sources."],"what_it_does":"BigQuery Schema Generator is a command-line tool and Python library that infers BigQuery table schemas by scanning complete datasets. Unlike BigQuery's built-in auto-detection (which examines only the first 500 records), this tool processes all input records to produce a more accurate schema, especially useful when data fields are added organically over time and may not appear in early records.\n\nThe tool accepts newline-delimited JSON or CSV format on standard input and outputs a BigQuery-compatible schema in JSON format. It can be used as a standalone command-line utility or imported as a library for programmatic schema generation. It supports various options for handling nulls, quoted values, type inference modes, and schema merging with existing schemas.","worth_installing":"Yes, with conditions. The tool fills a genuine gap in BigQuery's schema detection workflow. However, dormant maintenance (944 days since last release) and high install friction are concerns. Install if you regularly work with BigQuery data ingestion and need reliable schema inference across complete datasets; avoid if you need active maintenance or have strict dependency constraints."},"id":"bigquery-schema-generator","links":{"html":"https://skillfed.io/packages/bigquery-schema-generator","md":"https://skillfed.io/packages/bigquery-schema-generator.md","pypi":"https://pypi.org/project/bigquery-schema-generator/"},"maintenance":{"status":"dormant"},"meta":{"latest_release":"2024-01-13","license_spdx":null,"license_treatment":"permissive","name":"bigquery-schema-generator","python_support":"supports_current","summary":"BigQuery schema generator from JSON or CSV data"},"popularity":{"monthly_downloads":787914,"position":5059,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"1.6.1"}