ollama
Ollama lets you deploy and serve open-weight language models on your machine with automatic GPU detection. It exposes an OpenAI-compatible API endpoint, integrates with PenguinHarness for model registration, and supports flexible context window configuration for agent workloads.
Ollama deploys and runs open-weight language models locally with automatic GPU detection and an OpenAI-compatible API.
AI-generated summary based on this skill's SKILL.md
Install
Prism-Shadow/penguin-harness/ollama · repository language: TypeScript
git clone https://github.com/Prism-Shadow/penguin-harness
cp -r penguin-harness/packages/skills/skills/ollama ~/.claude/skills/ollamanpx skillfed install Prism-Shadow/penguin-harness/ollamaFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I run local LLM models with Ollama?
Ollama lets you deploy and serve open-weight language models on your machine with automatic GPU detection. Install Ollama, then use `ollama pull <model-name>` to download a model (e.g., `ollama pull qwen`), and `ollama run <model-name>` to start it. Ollama automatically detects and uses your GPU for acceleration, making inference fast without cloud dependencies.
Does Ollama provide an OpenAI-compatible API endpoint?
Yes. Ollama exposes an OpenAI-compatible API endpoint that lets you integrate local models into applications expecting standard OpenAI interfaces. Once a model is running, you can query it via the local endpoint, making it easy to swap cloud APIs for self-hosted inference without rewriting client code.
How do I configure context length for Ollama models?
Ollama supports flexible context window configuration for agent sessions. When pulling or running a model, you can adjust parameters to control context length based on your workload requirements. This lets you balance between model capability and memory usage for different agent scenarios.
Can I register Ollama models with PenguinHarness?
Yes. Ollama integrates with PenguinHarness for model registration, allowing you to manage local models within the PenguinHarness agent framework. Register your running Ollama endpoint so agents can discover and use your locally-served models.
What's the difference between Ollama and vLLM for local serving?
Ollama and vLLM are both local model serving options. Ollama emphasizes ease of setup with automatic GPU detection and a simple pull-and-run workflow, while vLLM focuses on high-throughput inference optimization. Choose Ollama for quick deployment; consider vLLM if you need advanced batching or performance tuning.
How do I set up Ollama GPU acceleration?
Ollama automatically detects and uses your GPU for acceleration during setup. Install Ollama on Windows, Mac, or Linux, and it will configure GPU support without manual intervention. Verify GPU usage by checking logs when you run a model—Ollama will report which device (GPU or CPU) is being used.
SKILL.md
rendered from the published skill — quoted content, verbatim
Ollama Serving
Ollama runs open-weight models locally with automatic GPU detection and an OpenAI-compatible API on http://localhost:11434.
Before you start
If the user's message only invokes this skill (e.g. "use ollama skill") without a concrete request, ask the user what they want. Do not run any command until the goal is clear.
Ask the user which model to run; if they have no preference, recommend the small default Qwen/Qwen3.5-0.8B (ollama pull qwen3.5:0.8b). The model must fit the machine's RAM/VRAM.
Ollama runs everywhere — macOS, Linux and Windows, on CPUs as well as NVIDIA/AMD GPUs — so engine choice follows the user's preference: Ollama is the simple default, while vLLM targets high-throughput GPU serving. Check the current state first:
```bash ollama --version # is Ollama installed? ollama ps # is the service already
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 2 files
packages/skills/skills/ollama/SKILL.md
packages/skills/skills/ollama/icon.svg