{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/6"}],"enrichment":{"capability":"Implements local windowed attention for transformer models, allowing efficient attention computation over fixed-size windows rather than full sequences.","skillfed_tags":["transformer-attention","sparse-attention","language-modeling"],"use_cases":["Building efficient language models where full-sequence attention is computationally prohibitive for long contexts.","Implementing Reformer-style architectures with shared query-key spaces and local attention windows.","Replacing standard attention in transformer layers to reduce memory and compute while maintaining reasonable model capacity.","Prototyping sparse attention patterns in transformer research without writing custom CUDA kernels.","Training models on long sequences (e.g., 8192 tokens) where quadratic attention complexity becomes a bottleneck."],"what_it_does":"Local-attention is a PyTorch module that implements local windowed attention\u2014a restricted form of transformer attention where each query attends only to keys and values within a fixed window, rather than the entire sequence. This is useful for language modeling because it allows transformer models to use computationally cheaper local attention in lower layers while reserving global attention for higher layers to integrate information across the full context.\n\nThe package provides configurable windowed attention with support for causal (auto-regressive) masking, lookback/lookahead windows, shared query-key spaces (Reformer-style), and automatic padding. It depends on torch for tensor operations and einops for tensor reshaping, plus hyper-connections as a utility dependency. The implementation is described as battle-tested across multiple repositories.","worth_installing":"Yes, if you are building or experimenting with transformer models and need efficient local attention. The package has low install friction, permissive licensing, and no known vulnerabilities. However, the 394-day gap since the last release and aging maintenance status suggest it is stable but not actively developed\u2014suitable for production use in established architectures but not for cutting-edge research requiring frequent updates."},"id":"local-attention","links":{"html":"https://skillfed.io/packages/local-attention","md":"https://skillfed.io/packages/local-attention.md","pypi":"https://pypi.org/project/local-attention/"},"maintenance":{"status":"aging"},"meta":{"latest_release":"2025-07-16","license_spdx":null,"license_treatment":"permissive","name":"local-attention","python_support":"unspecified","summary":"Local attention, window with lookback, for language modeling"},"popularity":{"monthly_downloads":311627,"position":7728,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"1.11.2"}