ml-llm-wiki
Search a self-contained markdown wiki covering transformer architectures, attention cost and efficiency, and long-context scaling. The skill provides read-only access to pre-compiled articles; use the companion llm-wiki skill to add or update content.
ml-llm-wiki lets you query a machine learning knowledge base on transformers and attention mechanisms.
AI-generated summary based on this skill's SKILL.md
Install
sammcj/agentic-coding/examples · repository language: HTML
git clone https://github.com/sammcj/agentic-coding
cp -r agentic-coding/Skills/llm-wiki/examples ~/.claude/skills/examplesFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How does attention work in transformers?
ml-llm-wiki provides compiled knowledge on transformer attention mechanisms. The skill offers read-only access to pre-compiled articles explaining how attention computes query-key-value interactions to weight information flow. Search the wiki for detailed breakdowns of the attention head computation, multi-head parallelization, and how attention enables transformers to model long-range dependencies in sequences.
What is the quadratic cost of attention and is it inevitable?
ml-llm-wiki covers attention cost analysis and efficiency trade-offs. The wiki explains why standard attention exhibits quadratic scaling with sequence length, but also documents that this quadratic limit is not inevitable—alternative implementations and sparse attention patterns can reduce complexity. Search for articles on attention complexity and optimization to explore practical approaches to lowering memory and compute requirements.
What long context scaling techniques does ml-llm-wiki document?
ml-llm-wiki contains articles on practical approaches to long-context scaling in language models. The wiki covers efficient attention implementations, sparse attention variants, and architectural modifications that enable transformers to handle longer sequences without proportional cost increases. Browse the knowledge base for citations and detailed comparisons of different scaling strategies.
Why is attention memory expensive and how can it be optimized?
ml-llm-wiki explains transformer memory requirements and attention efficiency trade-offs. The skill's articles detail how standard attention stores full query-key-value matrices, leading to high memory overhead. The wiki documents optimization techniques including sparse patterns, low-rank approximations, and kernel-based methods that reduce memory footprint while maintaining model quality.
How do I query the ML wiki for transformer architecture knowledge?
ml-llm-wiki provides read-only access to a self-contained markdown wiki on transformer architectures and attention. Search the knowledge base using keywords like 'attention mechanism,' 'scaling,' 'efficiency,' or 'long-context' to find relevant pre-compiled articles. To add or update content, use the companion llm-wiki skill for editing the underlying knowledge base.
What resources does ml-llm-wiki provide on attention quadratic scaling?
ml-llm-wiki's knowledge base includes articles analyzing attention complexity and the quadratic scaling myth. The wiki documents why standard attention has O(n²) complexity but also explores evidence and techniques showing this limit can be overcome. Search for 'attention complexity' or 'efficient attention implementations' to find citations and detailed technical discussions.
SKILL.md
rendered from the published skill — quoted content, verbatim
Machine Learning Wiki
A self-contained markdown knowledge base on transformer architectures, attention cost and efficiency, and long-context scaling. This skill is for querying it: the knowledge is already compiled into articles under wiki/, so read those rather than re-deriving from scratch.
Keep this current: as the wiki grows, update the name and description above so they describe what it actually covers and trigger on the right questions.
(Sample note: this example wiki lives in examples/ within the llm-wiki repo. To load it as a skill, place the directory in your skills path named ml-llm-wiki, so the directory matches the name above.)
Maintenance and deeper analysis - ingesting sources, superseding stale knowledge, linting, auditing, critiquing reasoning - is not done
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 14 files
Skills/llm-wiki/examples/CLAUDE.md
Skills/llm-wiki/examples/README.md
Skills/llm-wiki/examples/SKILL.md
Skills/llm-wiki/examples/local/prep/2026-06-04-attention-talk-prep.md
Skills/llm-wiki/examples/raw/machine-learning/2017-06-12-attention-is-all-you-need.md
Skills/llm-wiki/examples/raw/machine-learning/2022-05-27-flashattention.md
Skills/llm-wiki/examples/wiki/README.md
Skills/llm-wiki/examples/wiki/gaps.md
Skills/llm-wiki/examples/wiki/index.md
Skills/llm-wiki/examples/wiki/log.md
Skills/llm-wiki/examples/wiki/machine-learning/attention-cost.md
Skills/llm-wiki/examples/wiki/machine-learning/attention-efficiency.md
Skills/llm-wiki/examples/wiki/machine-learning/transformer-architectures.md
Skills/llm-wiki/examples/wiki/machine-learning/why-transformers-scale.md