ingest
ingest is a category skill for transforming unstructured files into JSONL chunks on your machine. It handles PDF, DOCX, and PPTX formats through the document-processing skill, which uses Docling to produce search-ready output without requiring cloud infrastructure or AWS.
ingest converts unstructured documents like PDFs into search-ready JSONL chunks locally using Docling.
AI-generated summary based on this skill's SKILL.md
Install
opensearch-project/opensearch-agent-skills/ingest · repository language: Python
git clone https://github.com/opensearch-project/opensearch-agent-skills
cp -r opensearch-agent-skills/skills/opensearch-skills/ingest ~/.claude/skills/ingestnpx skillfed install opensearch-project/opensearch-agent-skills/ingestFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How does ingest convert PDF to JSONL chunks?
ingest transforms PDF, DOCX, and PPTX files into JSONL chunks using Docling, a document-processing engine that runs locally on your machine. The skill parses unstructured content, structures it into discrete chunks, and outputs them in JSONL format ready for indexing without cloud dependency.
Can ingest process unstructured documents for search?
Yes. ingest is designed to process unstructured documents for search by converting them into search-ready JSONL chunks. It evaluates chunk quality and structure to optimize search performance, then prepares the output for OpenSearch indexing or other downstream systems.
What file formats does ingest support?
ingest supports PDF, DOCX, and PPTX formats. The skill handles all three through its document-processing pipeline, converting each into consistently structured JSONL chunks suitable for ingestion into search systems.
Does ingest require cloud infrastructure?
No. ingest operates entirely locally on your machine, eliminating cloud dependency. All document processing, chunking, and quality assessment happen without AWS or external services, giving you full control over your data pipeline.
How does ingest assess chunk quality before indexing?
ingest evaluates chunk quality and structure locally before ingestion, examining factors like coherence, size, and metadata completeness. This assessment ensures optimal search performance when chunks are indexed into OpenSearch or similar systems.
What license does ingest use?
ingest is licensed under Apache-2.0, allowing free use, modification, and distribution under the terms of the Apache License 2.0.
SKILL.md
rendered from the published skill — quoted content, verbatim
Ingest
Category skill for local document processing — turning unstructured files into search-ready JSONL chunks.
Skills
| Skill | Description |
|---|---|
| document-processing | Process PDF/DOCX/PPTX into JSONL chunks via Docling (local, no AWS needed) |
Not Covered
This skill covers local document processing only (PDF/DOCX → JSONL chunks).
It does NOT cover:
- Cloud-scale ingestion via OSIS pipelines → see managed-ingestion-service
- Structured bulk-indexing (_bulk API)
- OpenSearch _ingest processor pipelines (grok, date, set, script)
- Log/metric shipping (Fluent Bit, Data Prepper,
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 3 files
skills/opensearch-skills/ingest/SKILL.md
skills/opensearch-skills/ingest/document-processing/SKILL.md
skills/opensearch-skills/ingest/document-processing/document_processing_guide.md