ollama
Ollama lets you deploy and serve open-weight language models on your machine with automatic GPU detection. It exposes an OpenAI-compatible API endpoint, integrates with PenguinHarness for model registration, and supports flexible context window configuration for agent workloads.
Ollama deploys and runs open-weight language models locally with automatic GPU detection and an OpenAI-compatible API.
AI-generated summary based on this skill's SKILL.md
Decision gist · record as of 2026-07-27
Ollama deploys and runs open-weight language models locally with automatic GPU detection and an OpenAI-compatible API. Ollama lets you deploy and serve open-weight language models on your machine with automatic GPU detection. It exposes an OpenAI-compatible API endpoint, integrates with PenguinHarness for model registration, and supports flexible context window configuration for agent workloads.
Use it when
- Yes.
- Ollama supports flexible context window configuration for agent sessions.
Verify before relying
Read SKILL.md below before installing (2 files). Open directory: indexed for reading, not audited.
Install
Prism-Shadow/penguin-harness/ollama · repository language: TypeScript
Open directory. Skills are indexed for reading, not audited. Review a skill's body before installing it.
Frequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I run local LLM models with Ollama?
Ollama lets you deploy and serve open-weight language models on your machine with automatic GPU detection. Install Ollama, then use `ollama pull <model-name>` to download a model (e.g., `ollama pull qwen`), and `ollama run <model-name>` to start it. Ollama automatically detects and uses your GPU for acceleration, making inference fast without cloud dependencies.
Does Ollama provide an OpenAI-compatible API endpoint?
Yes. Ollama exposes an OpenAI-compatible API endpoint that lets you integrate local models into applications expecting standard OpenAI interfaces. Once a model is running, you can query it via the local endpoint, making it easy to swap cloud APIs for self-hosted inference without rewriting client code.
How do I configure context length for Ollama models?
Ollama supports flexible context window configuration for agent sessions. When pulling or running a model, you can adjust parameters to control context length based on your workload requirements. This lets you balance between model capability and memory usage for different agent scenarios.
Can I register Ollama models with PenguinHarness?
Yes. Ollama integrates with PenguinHarness for model registration, allowing you to manage local models within the PenguinHarness agent framework. Register your running Ollama endpoint so agents can discover and use your locally-served models.
What's the difference between Ollama and vLLM for local serving?
Ollama and vLLM are both local model serving options. Ollama emphasizes ease of setup with automatic GPU detection and a simple pull-and-run workflow, while vLLM focuses on high-throughput inference optimization. Choose Ollama for quick deployment; consider vLLM if you need advanced batching or performance tuning.
How do I set up Ollama GPU acceleration?
Ollama automatically detects and uses your GPU for acceleration during setup. Install Ollama on Windows, Mac, or Linux, and it will configure GPU support without manual intervention. Verify GPU usage by checking logs when you run a model—Ollama will report which device (GPU or CPU) is being used.
SKILL.md
Rendered from the published skill. Quoted content, verbatim.
Ollama Serving
Ollama runs open-weight models locally with automatic GPU detection and an OpenAI-compatible API on http://localhost:11434.
Before you start
If the user's message only invokes this skill (e.g. "use ollama skill") without a concrete request, ask the user what they want. Do not run any command until the goal is clear.
Ask the user which model to run; if they have no preference, recommend the small default Qwen/Qwen3.5-0.8B (ollama pull qwen3.5:0.8b). The model must fit the machine's RAM/VRAM.
Ollama runs everywhere — macOS, Linux and Windows, on CPUs as well as NVIDIA/AMD GPUs — so engine choice follows the user's preference: Ollama is the simple default, while vLLM targets high-throughput GPU serving. Check the current state first:
```bash ollama --version # is Ollama installed? ollama ps # is the service already
(truncated - see the full file via the links below)
File tree — 2 files
packages/skills/skills/ollama/SKILL.md
packages/skills/skills/ollama/icon.svg
Let your AI agent find skills like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 56,283 agent skills by what they can do, searchable in plain language.
wish › “Deploy and run open-weight language models locally with GPU support”
Give your agent the search over MCP, or paste the wish link into any chat. No install? Search from any chat →
Related skills
vLLM runs open-weight models on local GPUs with high-throughput inference behind an OpenAI-compatible endpoint, ready for chat and agent workloads. Configure context length and quantization to fit your hardware, then register the endpoint with PenguinHarness to make it available to your agents.
LlamaFactory streamlines LLM fine-tuning through YAML-driven configs, supporting LoRA, QLoRA, and full-parameter training methods like SFT and DPO. Register datasets, train your model, merge adapters, and export for standalone deployment or API serving. The skill guides you through GPU requirements, base model selection, and inference setup.
vllm-server guides you through deploying and configuring vLLM—a high-performance open-source LLM serving engine—for production workloads. Set up continuous batching, multi-GPU tensor parallelism, model quantization, and OpenAI-compatible API endpoints to serve models like Llama and Mistral at scale. Includes Docker deployment, performance tuning, monitoring with Prometheus metrics, and troubleshooting for common VRAM and throughput issues.
Launch agents with flexible LLM provider selection across Claude, OpenAI, CrewAI, Lyzr, GitHub Models, and other adapters. Supports local directories, remote git repositories, and one-shot prompts for immediate execution. Auto-detection identifies the best adapter from your repo's configuration.
This skill guides you through selecting an LLM serving engine by analyzing hardware topology, workload profile, and operational constraints rather than benchmark headlines. It maps seven engines—vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX—to their strengths across GPU clusters, edge devices, and single-user scenarios, then walks you through a five-step decision workflow to eliminate incompatible options and benchmark your top candidates.
Cloudflare Deploy guides you through deploying applications across Cloudflare's platform—from serverless functions and full-stack sites to databases and AI services. Navigate decision trees to find the right product, then follow detailed setup steps with authentication and troubleshooting support.