extract-webpage-data
Extract Webpage Data uses AI to pull structured information from web pages, transforming unorganized HTML into clean, actionable data. Choose from multiple extraction methods—natural language prompts, predefined schemas, or general scraping—depending on your needs. Handles product details, contact information, team rosters, and more.
Extract Webpage Data turns messy web pages into clean, structured information using AI-powered scraping.
AI-generated summary based on this skill's SKILL.md
Install
gooseworks-ai/goose-skills/extract-webpage-data · repository language: Python
git clone https://github.com/gooseworks-ai/goose-skills
cp -r goose-skills/skills/research-tools/capabilities/extract-webpage-data ~/.claude/skills/extract-webpage-dataFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
What can extract-webpage-data do with web pages?
Extract Webpage Data uses AI to pull structured information from web pages, transforming unorganized HTML into clean, actionable data. You can extract product details, contact information, team rosters, pricing, and other specific information. The skill handles both single-page extraction and multi-page crawling at scale, converting messy content into organized JSON format.
How do I scrape webpage content with this tool?
Extract Webpage Data offers multiple extraction methods tailored to your needs. You can use natural language prompts to describe what you want, apply predefined schemas for consistent structure, or perform general scraping. Simply provide the URL and specify what information you need—whether it's product listings, contact details, or table data—and the AI handles the parsing and structuring.
Can extract-webpage-data convert webpage to JSON format?
Yes. Extract Webpage Data transforms unstructured HTML content into clean, organized JSON automatically. This makes the extracted data immediately usable in your applications, databases, or workflows without manual reformatting or additional processing steps.
What types of data can I extract from websites?
Extract Webpage Data handles diverse data types including product information and prices, contact details and team member profiles, blog post titles and metadata, table data, and other structured or semi-structured content. The AI-powered approach adapts to different page layouts and content types, making it flexible across various websites and industries.
Does extract-webpage-data support large-scale scraping?
Yes. Extract Webpage Data can crawl multiple pages and extract data at scale, enabling you to gather information across entire websites or large datasets efficiently. The skill is designed to handle automated web scraping operations while maintaining data quality and organization.
What license does extract-webpage-data use?
Extract Webpage Data is released under the MIT license, allowing you to use, modify, and distribute it freely with minimal restrictions.
SKILL.md
rendered from the published skill — quoted content, verbatim
Extract Webpage Data
Setup
Read your credentials from ~/.gooseworks/credentials.json:
export GOOSEWORKS_API_KEY=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json'))['api_key'])")
export GOOSEWORKS_API_BASE=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json')).get('api_base','https://api.gooseworks.ai'))")
If ~/.gooseworks/credentials.json does not exist, tell the user to run: npx gooseworks login
All endpoints use Bearer auth: -H "Authorization: Bearer $GOOSEWORKS_API_KEY"
Extract structured data from any web page using AI. Turn messy HTML into clean, organized data.
When to Use
- User wants to extract specific data from a website
- User asks to scrape information from a page
- User needs structured data from unstructured content
- User wants to pull product info, contact details, etc.
- Converting web content to usable data
How It Works
Uses Olostep, Scrapegraph, or Riveter APIs for AI-powered data extraction.
Usage
Simple Scrape with Olostep
```bash curl
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 2 files
skills/research-tools/capabilities/extract-webpage-data/SKILL.md
skills/research-tools/capabilities/extract-webpage-data/skill.meta.json