pytorch-pretrained-bert
PyTorch version of Google AI BERT model with script to load Google pre-trained models
Decision gist · record as of 2026-08-14
Yes, if you need to work with BERT, GPT, or Transformer-XL in PyTorch and are comfortable with a package that has not received updates since April 2019. The implementation is well-tested, has low install friction, and carries permissive licensing. However, consider whether a more actively maintained alternative (such as HuggingFace Transformers, which this package predates) better suits your project's long-term maintenance needs.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires PyTorch 0.4.1 or 1.0.0+; torch must be installed separately.
- Optional: ftfy and SpaCy for original OpenAI GPT tokenization (ftfy pinned to 4.4.3 for Python 2).
- Low install friction with pure Python wheels for both Python 2 and 3.
License · maintenance · safety
Apache (permissive) — Licensed under Apache (permissive), allowing commercial and private use with minimal restrictions. You may use, modify, and distribute the code freely provided you include a copy of the license and notice of changes.
last release 2019-04-25 (2668 days) · last repo commit 2026-08-14 · 164,105 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 78,578 downloads/mo, #14,427 on PyPI
Alternatives
Verify before relying
pip install pytorch-pretrained-bert
from pytorch_pretrained_bert import BertModel, BertTokenizer
model = BertModel.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
encoded = tokenizer.encode('Hello world')
output = model(encoded)- Whether pre-trained model weights are still downloadable from the original sources (Google, OpenAI) given the package's age.
- Compatibility with modern PyTorch versions beyond 1.0.0.
- Performance parity claims (e.g., ~91 F1 on SQuAD) on current datasets and hardware.
What it is and what it does
This package provides op-for-op PyTorch reimplementations of four major pre-trained transformer models: BERT, OpenAI GPT, Transformer-XL, and GPT-2. It includes pre-trained weights converted from the original TensorFlow checkpoints and a suite of task-specific model heads for common NLP workflows like sequence classification, token classification, question answering, and masked language modeling.
The package is built on torch, numpy, boto3, requests, tqdm, and regex. It is designed for developers who want to fine-tune or adapt these models for downstream NLP tasks without reimplementing the transformer architecture. The repository includes examples, notebooks, and a command-line interface for checkpoint conversion, though the package itself has not seen a new release since April 2019.
Use it for
- Fine-tune BERT for text classification tasks like sentiment analysis or intent detection.
- Use pre-trained GPT or GPT-2 for language generation and completion tasks.
- Adapt Transformer-XL for long-sequence modeling and perplexity evaluation on language corpora.
- Perform question answering with BertForQuestionAnswering on SQuAD-like datasets.
- Extract contextual embeddings from BERT for downstream machine learning pipelines.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you need to work with BERT, GPT, or Transformer-XL in PyTorch and are comfortable with a package that has not received updates since April 2019.
The implementation is well-tested, has low install friction, and carries permissive licensing. However, consider whether a more actively maintained alternative (such as HuggingFace Transformers, which this package predates) better suits your project's long-term maintenance needs.
Install
pytorch-pretrained-bert on PyPI
Before you install
Low install friction with pure Python wheels for both Python 2 and 3. Active maintenance with recent commits; however, the package has not received a release since April 2019 despite ongoing repository activity, suggesting it may be in maintenance mode rather than active development.
Requires PyTorch 0.4.1 or 1.0.0+; torch must be installed separately. Optional: ftfy and SpaCy for original OpenAI GPT tokenization (ftfy pinned to 4.4.3 for Python 2).
License in practice
Licensed under Apache (permissive), allowing commercial and private use with minimal restrictions. You may use, modify, and distribute the code freely provided you include a copy of the license and notice of changes.
Quickstart
pip install pytorch-pretrained-bert
from pytorch_pretrained_bert import BertModel, BertTokenizer
model = BertModel.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
encoded = tokenizer.encode('Hello world')
output = model(encoded)Verify before relying
- Whether pre-trained model weights are still downloadable from the original sources (Google, OpenAI) given the package's age.
- Compatibility with modern PyTorch versions beyond 1.0.0.
- Performance parity claims (e.g., ~91 F1 on SQuAD) on current datasets and hardware.
Package facts
| License | Apache permissive |
| Python support | Not specified |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 6 packagestorchnumpyboto3requeststqdmregex |
| Maintenance | Actively maintained 2,668 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 78,578 / month, #14,427 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Intended Audience :: Science/ResearchLicense :: OSI Approved :: Apache Software LicenseProgramming Language :: Python :: 3Topic :: Scientific/Engineering :: Artificial Intelligence |
Evidence: pytorch_pretrained_bert-0.6.2-py2-none-any.whl; pytorch_pretrained_bert-0.6.2-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “BERT PyTorch implementation”
- pytorch-pretrained-bertProvides PyTorch implementations of BERT, GPT, Transformer-XL, and…
- curated-transformersCurated Transformers provides PyTorch implementations of…
- keras-hubKerasHub provides Keras 3 implementations of pretrained model…
Give your agent the search over MCP, or paste the wish link into any chat.
More Artificial Intelligence packages
LiteLLM provides a unified Python interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others) using OpenAI-compatible API format, available as both a Python SDK and a self-hosted AI Gateway proxy server.
Install it if you need to work with multiple LLM providers or want to centralize LLM routing in your organization.
Client library and CLI tool for downloading, uploading, and managing models, datasets, and repositories on the Hugging Face Hub platform.
Install it if you work with Hugging Face Hub models or datasets.
LangChain provides a framework for building agents and LLM-powered applications by composing language models, tools, and memory through a unified API that abstracts over multiple model providers.
hf-xet provides chunk-based deduplication and efficient file transfer for the Hugging Face Hub, enabling faster uploads and downloads of large files with local disk caching.
Tokenizers converts raw text into token sequences for NLP models, with support for training custom vocabularies and using pre-built tokenizers (BPE, WordPiece) optimized for speed via Rust.
Transformers provides a unified framework for loading, fine-tuning, and running state-of-the-art pretrained models across text, vision, audio, video, and multimodal tasks using PyTorch, JAX, or TensorFlow.
Install it if you need to run or train any transformer-based model for NLP, vision, audio, or multimodal tasks.
See also fair-esm · spacy-transformers · torchtext · curated-transformers · spacy-curated-transformers · loralib · bertopic · bert-score · sentence-transformers · transformer-lens