--- id: pytorch-pretrained-bert version: "0.6.2" license: Apache license_treatment: permissive maintenance: active --- # pytorch-pretrained-bert — PyTorch version of Google AI BERT model with script to load Google pre-trained models License: permissive · Maintenance: active · Downloads: 78.6K/mo ## What it is and what it does This package provides op-for-op PyTorch reimplementations of four major pre-trained transformer models: BERT, OpenAI GPT, Transformer-XL, and GPT-2. It includes pre-trained weights converted from the original TensorFlow checkpoints and a suite of task-specific model heads for common NLP workflows like sequence classification, token classification, question answering, and masked language modeling. The package is built on torch, numpy, boto3, requests, tqdm, and regex. It is designed for developers who want to fine-tune or adapt these models for downstream NLP tasks without reimplementing the transformer architecture. The repository includes examples, notebooks, and a command-line interface for checkpoint conversion, though the package itself has not seen a new release since April 2019. Use it for: - Fine-tune BERT for text classification tasks like sentiment analysis or intent detection. - Use pre-trained GPT or GPT-2 for language generation and completion tasks. - Adapt Transformer-XL for long-sequence modeling and perplexity evaluation on language corpora. - Perform question answering with BertForQuestionAnswering on SQuAD-like datasets. - Extract contextual embeddings from BERT for downstream machine learning pipelines. ## Worth the install? AI-flagged interpretation of the facts above — verify before relying. Provides PyTorch implementations of BERT, GPT, Transformer-XL, and GPT-2 models with pre-trained weights and fine-tuning examples for natural language processing tasks. Yes, if you need to work with BERT, GPT, or Transformer-XL in PyTorch and are comfortable with a package that has not received updates since April 2019. The implementation is well-tested, has low install friction, and carries permissive licensing. However, consider whether a more actively maintained alternative (such as HuggingFace Transformers, which this package predates) better suits your project's long-term maintenance needs. ## Install pip install pytorch-pretrained-bert uv add pytorch-pretrained-bert poetry add pytorch-pretrained-bert ## Installing pytorch-pretrained-bert Before you install: Low install friction with pure Python wheels for both Python 2 and 3. Active maintenance with recent commits; however, the package has not received a release since April 2019 despite ongoing repository activity, suggesting it may be in maintenance mode rather than active development. License in practice: Licensed under Apache (permissive), allowing commercial and private use with minimal restrictions. You may use, modify, and distribute the code freely provided you include a copy of the license and notice of changes. Quickstart: pip install pytorch-pretrained-bert from pytorch_pretrained_bert import BertModel, BertTokenizer model = BertModel.from_pretrained('bert-base-uncased') tokenizer = BertTokenizer.from_pretrained('bert-base-uncased') encoded = tokenizer.encode('Hello world') output = model(encoded) Requires PyTorch 0.4.1 or 1.0.0+; torch must be installed separately. Optional: ftfy and SpaCy for original OpenAI GPT tokenization (ftfy pinned to 4.4.3 for Python 2). Verify before relying: - Whether pre-trained model weights are still downloadable from the original sources (Google, OpenAI) given the package's age. - Compatibility with modern PyTorch versions beyond 1.0.0. - Performance parity claims (e.g., ~91 F1 on SQuAD) on current datasets and hardware. ## Package facts - License: Apache (permissive) - Python support: unspecified - Install friction: low - Maintenance: active - Downloads: 78.6K/month (top 15,000 on PyPI) - Known vulnerabilities: none known ## Tags BERT PyTorch implementation, pre-trained transformer models, GPT language model PyTorch, fine-tune BERT models, transformer NLP models, pre-trained language models, BERT question answering, transformer sequence classification, transformer-models, language-models, nlp [View on SkillFed](https://skillfed.io/packages/pytorch-pretrained-bert) · [View on PyPI](https://pypi.org/project/pytorch-pretrained-bert/)