data-designer

What it is and what it does

Data Designer is a framework for building synthetic datasets that go beyond simple LLM prompting. It lets you define columns using statistical samplers (category, numeric distributions), LLM generators, or seed data, then orchestrates generation with dependency-aware field relationships. The package includes validators (Python, SQL, custom, and remote LLM-as-judge) to assess output quality and a preview mode to test configurations before full-scale runs.

The library runs on an async, cell-level engine that overlaps independent column generation and adapts concurrency per provider and model. It integrates with multiple LLM providers (NVIDIA Build, OpenAI, OpenRouter) and includes OpenTelemetry instrumentation for monitoring. Configuration is built programmatically via a builder API and can also be managed through CLI commands. Telemetry is enabled by default but can be disabled via environment variable.

Use it for

• Generate diverse product review datasets with realistic correlations between category, rating, and review text for training classification models.
• Create synthetic customer records with demographic attributes, purchase history, and behavioral patterns for privacy-preserving testing and development.
• Build test datasets for data pipelines and analytics by sampling from statistical distributions and validating output against SQL or Python rules.
• Augment small seed datasets by generating synthetic variations while maintaining statistical properties and field dependencies.
• Evaluate LLM quality on domain-specific tasks using LLM-as-judge validators to score generated outputs before production use.

Worth the install?

AI-flagged interpretation of the facts on this page. Verify before relying on it.

Install

data-designer on PyPI

Before you install

Low friction installation as a pure-Python wheel. Active maintenance with a release 3 days old. Depends on 15 runtime packages including pandas, pydantic, and OpenTelemetry; most are common data and ML infrastructure libraries.

Requires Python 3.10 or later. Requires at least one API key set (NVIDIA_API_KEY, OPENAI_API_KEY, or OPENROUTER_API_KEY) to generate LLM-based columns; statistical samplers work without external APIs.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions; you must include a copy of the license and state significant changes.

Quickstart

pip install data-designer

import data_designer.config as dd
from data_designer.interface import DataDesigner

data_designer = DataDesigner()
config_builder = dd.DataDesignerConfigBuilder()
config_builder.add_column(
    dd.SamplerColumnConfig(
        name="product_category",
        sampler_type=dd.SamplerType.CATEGORY,
        params=dd.CategorySamplerParams(
            values=["Electronics", "Clothing", "Home & Kitchen", "Books"],
        ),
    )
)
preview = data_designer.preview(config_builder=config_builder)

Package facts

Evidence: data_designer-0.9.1-py3-none-any.whl

Tags

More Software Development packages

See also data-designer-config · data-designer-engine · oil-reservoir-synthesizer · copulas · sdv · sdmetrics · deepecho · nemo-gym · ucimlrepo · nemo-evaluator

Further reading