document-processing
Document Processing transforms unstructured files into indexed-ready JSONL chunks using the open-source Docling library, running entirely on your machine. Output includes text, headings, source file references, and page numbers for direct ingestion into OpenSearch.
Document Processing converts unstructured documents into search-ready JSONL chunks for OpenSearch indexing.
AI-generated summary based on this skill's SKILL.md
Install
opensearch-project/opensearch-agent-skills/document-processing · repository language: Python
git clone https://github.com/opensearch-project/opensearch-agent-skills
cp -r opensearch-agent-skills/skills/opensearch-skills/ingest/document-processing ~/.claude/skills/document-processingnpx skillfed install opensearch-project/opensearch-agent-skills/document-processingFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
What file formats does Document Processing support?
Document Processing handles PDF, DOCX, PPTX, and XLSX files. The skill processes these formats locally on your machine using the open-source Docling library, converting unstructured content into search-ready chunks without relying on cloud services.
How do I chunk documents for search indexing with this tool?
Document Processing automatically converts your files into indexed-ready JSONL chunks. Each chunk includes extracted text, headings, source file references, and page numbers, making them ready for direct ingestion into OpenSearch or other search engines.
Can Document Processing process unstructured documents locally?
Yes. Document Processing runs entirely on your machine without cloud services. It transforms unstructured files into search-ready chunks locally, giving you full control over your document data while preparing it for indexing.
What output format does Document Processing generate?
Document Processing outputs JSONL format optimized for OpenSearch ingestion. Each record contains extracted text, headings, source file references, and page numbers, allowing you to index documents with full context and traceability.
How can I evaluate document chunk quality before indexing?
Document Processing includes inspection capabilities to review chunk quality before indexing. You can examine the generated JSONL output, verify text extraction accuracy, and validate that headings and metadata are properly captured for your search engine.
Does Document Processing handle batch processing of multiple files?
Document Processing is designed to extract and split large documents into manageable indexed segments. It can process multiple PDFs, DOCX, PPTX, and XLSX files, converting them into search-ready chunks that maintain document structure and reference information.
SKILL.md
rendered from the published skill — quoted content, verbatim
Document Processing
Process unstructured documents into search-ready JSONL chunks using Docling (open-source, runs locally). No AWS credentials or cloud services needed.
When to Use
- User has unstructured documents (PDF, DOCX, PPTX, XLSX)
- User wants to prepare documents for OpenSearch indexing
- User wants to inspect or evaluate chunk quality
Output
JSONL files at .opensearch/chunks/<index>/<filename>.jsonl. Each line:
{"text": "...", "headings": ["Section Title"], "source_file": "doc.pdf", "chunk_id": 0, "page_number": 1}
The JSONL output can be ingested into any OpenSearch target: - **Local
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 2 files
skills/opensearch-skills/ingest/document-processing/SKILL.md
skills/opensearch-skills/ingest/document-processing/document_processing_guide.md