ai-web-scraping-scrapegraph
This skill harnesses AI to pull structured information from web pages by simply describing what you want. Use natural language prompts to extract product details, search results, or entire site content, with support for JavaScript-heavy sites, pagination, and custom output schemas.
ai-web-scraping-scrapegraph extracts structured data from websites by describing what you need in natural language.
AI-generated summary based on this skill's SKILL.md
Install
gooseworks-ai/goose-skills/ai-web-scraping-scrapegraph · repository language: Python
git clone https://github.com/gooseworks-ai/goose-skills
cp -r goose-skills/skills/research-tools/capabilities/ai-web-scraping-scrapegraph ~/.claude/skills/ai-web-scraping-scrapegraphFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
What is ai-web-scraping-scrapegraph and how does it work?
ai-web-scraping-scrapegraph is an AI-powered web scraping tool that extracts structured data from websites using natural language prompts. Instead of writing complex selectors, you describe what information you need—like product prices or article titles—and the skill intelligently identifies and pulls that data from web pages, including JavaScript-heavy sites.
Can I extract data from websites using prompts with this skill?
Yes. ai-web-scraping-scrapegraph lets you extract data from websites using natural language prompts. Simply describe the information you want—such as product names, prices, or contact details—and the AI automatically locates and structures that data for you without requiring CSS selectors or manual configuration.
Does ai-web-scraping-scrapegraph support converting webpages to markdown?
Yes, ai-web-scraping-scrapegraph can scrape and convert web pages to markdown format for processing. This makes it easy to feed webpage content into downstream tools, documentation systems, or AI models while preserving structure and readability.
How does ai-web-scraping-scrapegraph handle multi-page crawling?
ai-web-scraping-scrapegraph crawls multiple pages and maps website structure automatically. It can follow pagination, navigate site hierarchies, and extract data across linked pages without manual intervention, making it ideal for collecting large datasets from complex websites.
Can I use ai-web-scraping-scrapegraph for price monitoring?
Yes. ai-web-scraping-scrapegraph supports monitoring prices and tracking data across e-commerce sites. You can set up automated extraction of product prices, availability, and other details from multiple pages, enabling competitive pricing analysis and market monitoring.
What license does ai-web-scraping-scrapegraph use?
ai-web-scraping-scrapegraph is released under the MIT license, which permits free use, modification, and distribution for both commercial and personal projects with minimal restrictions.
SKILL.md
rendered from the published skill — quoted content, verbatim
ScrapeGraph AI - Intelligent Web Scraping
Setup
Read your credentials from ~/.gooseworks/credentials.json:
export GOOSEWORKS_API_KEY=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json'))['api_key'])")
export GOOSEWORKS_API_BASE=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json')).get('api_base','https://api.gooseworks.ai'))")
If ~/.gooseworks/credentials.json does not exist, tell the user to run: npx gooseworks login
All endpoints use Bearer auth: -H "Authorization: Bearer $GOOSEWORKS_API_KEY"
Extract web content using AI with natural language prompts.
Capabilities
- Start SmartScraper: Extract content from a webpage using AI by providing a natural language prompt and a URL
- Start SearchScraper: Start a new AI-powered web search request
- Scrape: Extract raw HTML content from web pages with JavaScript rendering support
- Start SmartCrawler: Start a new web crawl request with AI extraction or markdown conversion
- **Start
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 2 files
skills/research-tools/capabilities/ai-web-scraping-scrapegraph/SKILL.md
skills/research-tools/capabilities/ai-web-scraping-scrapegraph/skill.meta.json