data-designer

What it is and what it does

Data Designer is a framework for building synthetic datasets that go beyond simple LLM prompting. It lets you define columns using statistical samplers (category, numeric distributions), LLM generators, or seed data, then orchestrates generation with dependency-aware field relationships. The package includes validators (Python, SQL, custom, and remote LLM-as-judge) to assess output quality and a preview mode to test configurations before full-scale runs.

The library runs on an async, cell-level engine that overlaps independent column generation and adapts concurrency per provider and model. It integrates with multiple LLM providers (NVIDIA Build, OpenAI, OpenRouter) and includes OpenTelemetry instrumentation for monitoring. Configuration is built programmatically via a builder API and can also be managed through CLI commands. Telemetry is enabled by default but can be disabled via environment variable.

Use it for:

• Generate diverse product review datasets with realistic correlations between category, rating, and review text for training classification models.
• Create synthetic customer records with demographic attributes, purchase history, and behavioral patterns for privacy-preserving testing and development.
• Build test datasets for data pipelines and analytics by sampling from statistical distributions and validating output against SQL or Python rules.
• Augment small seed datasets by generating synthetic variations while maintaining statistical properties and field dependencies.
• Evaluate LLM quality on domain-specific tasks using LLM-as-judge validators to score generated outputs before production use.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Generates high-quality synthetic datasets from scratch or seed data, with control over field relationships, statistical distributions, and built-in validation and quality scoring.

Yes. Data Designer is actively maintained, has low install friction, carries a permissive Apache-2.0 license, and addresses a real need for controlled synthetic data generation beyond simple prompting. It's suitable for developers building datasets for ML training, testing, or privacy-preserving development. The async engine and multi-provider support are practical for production workflows. No known security vulnerabilities. Start with a preview to test your schema before committing to full generation.

Install

data-designer on PyPI

pip install data-designer

uv add data-designer

poetry add data-designer

Installing data-designer

Before you install

Low friction installation as a pure-Python wheel. Active maintenance with a release 3 days old. Depends on 15 runtime packages including pandas, pydantic, and OpenTelemetry; most are common data and ML infrastructure libraries.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions; you must include a copy of the license and state significant changes.

Quickstart

pip install data-designer

import data_designer.config as dd
from data_designer.interface import DataDesigner

data_designer = DataDesigner()
config_builder = dd.DataDesignerConfigBuilder()
config_builder.add_column(
    dd.SamplerColumnConfig(
        name="product_category",
        sampler_type=dd.SamplerType.CATEGORY,
        params=dd.CategorySamplerParams(
            values=["Electronics", "Clothing", "Home & Kitchen", "Books"],
        ),
    )
)
preview = data_designer.preview(config_builder=config_builder)

Requires Python 3.10 or later. Requires at least one API key set (NVIDIA_API_KEY, OPENAI_API_KEY, or OPENROUTER_API_KEY) to generate LLM-based columns; statistical samplers work without external APIs.

Verify before relying

• Actual performance improvement from the async engine on typical workloads and how to measure it
• Whether the agent skill works reliably with coding agents other than Claude Code
• Telemetry data collection scope and frequency beyond model names and token counts

Package facts

Evidence: data_designer-0.9.1-py3-none-any.whl

Tags

More Software Development packages

Further reading