{"enrichment":{"faq":[{"a":"airflow-dag-patterns teaches you to construct production-ready DAGs using proven design patterns. The skill covers structuring tasks with proper dependencies, implementing idempotent workflows, configuring retry logic and timeouts, and organizing code for maintainability. You'll learn to leverage the TaskFlow API, define clear task boundaries, and follow Airflow conventions for reliable orchestration.","q":"How to build Apache Airflow DAGs with best practices?"},{"a":"airflow-dag-patterns emphasizes production readiness through idempotent task design, robust error handling, and deployment strategies. Key practices include setting appropriate retry policies, using sensors effectively, structuring dependencies to prevent cascading failures, testing DAGs locally before production, and monitoring failed runs. The skill guides you through configuration patterns that ensure your workflows remain reliable at scale.","q":"What are airflow dag best practices for production deployment?"},{"a":"airflow-dag-patterns walks you through designing and orchestrating complete data pipelines. You'll learn to define task sequences, connect operators for data extraction and transformation, use sensors to wait for external conditions, and structure XCom communication between tasks. The skill covers real ETL patterns, batch job scheduling, and how to compose operators and sensors into cohesive workflows.","q":"How do you create a data pipeline with Airflow?"},{"a":"airflow-dag-patterns teaches custom operator and sensor implementation alongside built-in options. You'll learn when to use standard operators versus building custom ones, how sensors reschedule or poke for readiness, and patterns for task communication. The skill covers sensor modes, operator inheritance, and practical examples of extending Airflow's capabilities for your specific pipeline needs.","q":"What airflow operators and sensors should I implement?"},{"a":"airflow-dag-patterns provides testing strategies for both local development and production environments. You'll learn to validate DAG structure, test individual tasks, simulate scheduling behavior, and debug failed runs. The skill covers tools for local testing, logging practices, and techniques to identify issues before deployment, ensuring your workflows behave correctly across environments.","q":"How do you test and debug Airflow DAGs locally?"},{"a":"airflow-dag-patterns covers dependency setup patterns that prevent race conditions and ensure correct execution order. You'll learn idempotent task design principles\u2014making tasks safe to retry without side effects\u2014and configuring catchup settings, retry logic, and timeouts. The skill teaches how to structure workflows so failures don't corrupt data and retries succeed reliably.","q":"How should task dependencies and idempotent workflows be configured?"}],"shadow_tags":["workflow-orchestration","data-pipeline-design","task-scheduling","dag-patterns","airflow-operators","batch-processing","production-deployment","error-handling-retry"],"summary_rewrite":"Master production-grade Airflow DAG design with patterns for operators, sensors, and testing. Learn idempotent task design, dependency structures, and deployment strategies to orchestrate reliable data pipelines."},"files":[{"bytes":3135,"path":"plugins/data-engineering/skills/airflow-dag-patterns/SKILL.md","sha256":"99cf58e6d32b977e02a9c7ae2ed0ae4894e24ece95260134ac22925497c2a732","url":"https://skillfed.io/files/wshobson/agents/airflow-dag-patterns/bf76b000/SKILL.md"}],"id":"wshobson/agents/airflow-dag-patterns","links":{"html":"https://skillfed.io/wshobson/agents/airflow-dag-patterns","md":"https://skillfed.io/wshobson/agents/airflow-dag-patterns.md","repo":"https://github.com/wshobson/agents"},"meta":{"agents_supported":[],"first_seen":"2026-07-28","forks":4097,"language":"Python","last_updated":"2026-07-22","license":"MIT","name":"airflow-dag-patterns","publisher":"wshobson","stars":38308},"relations":{"similar":[{"id":"manutej/luxor-claude-marketplace/apache-airflow-orchestration"},{"id":"Aradotso/data-skills/apache-airflow-orchestration"},{"id":"personamanagmentlayer/pcl/airflow-expert"},{"id":"secondsky/claude-skills/ml-pipeline-automation"},{"id":"datadrivenconstruction/DDC_Skills_for_AI_Agents_in_Construction/airflow-dag"},{"id":"ancoleman/ai-design-components/transforming-data"},{"id":"pluginagentmarketplace/custom-plugin-data-engineer/etl-tools"},{"id":"Aradotso/data-skills/data-engineering-medallion-pipeline"},{"id":"Aradotso/data-skills/realtime-cinema-data-engineering-pipeline"},{"id":"Aradotso/data-skills/google-cloud-data-engineering-hub"}]},"slug":{"owner":"wshobson","repo":"agents","skill":"airflow-dag-patterns"},"version":"bf76b000"}