{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/11"}],"enrichment":{"capability":"Provides PyTorch implementations of BERT, GPT, Transformer-XL, and GPT-2 models with pre-trained weights and fine-tuning examples for natural language processing tasks.","skillfed_tags":["transformer-models","language-models","nlp"],"use_cases":["Fine-tune BERT for text classification tasks like sentiment analysis or intent detection.","Use pre-trained GPT or GPT-2 for language generation and completion tasks.","Adapt Transformer-XL for long-sequence modeling and perplexity evaluation on language corpora.","Perform question answering with BertForQuestionAnswering on SQuAD-like datasets.","Extract contextual embeddings from BERT for downstream machine learning pipelines."],"what_it_does":"This package provides op-for-op PyTorch reimplementations of four major pre-trained transformer models: BERT, OpenAI GPT, Transformer-XL, and GPT-2. It includes pre-trained weights converted from the original TensorFlow checkpoints and a suite of task-specific model heads for common NLP workflows like sequence classification, token classification, question answering, and masked language modeling.\n\nThe package is built on torch, numpy, boto3, requests, tqdm, and regex. It is designed for developers who want to fine-tune or adapt these models for downstream NLP tasks without reimplementing the transformer architecture. The repository includes examples, notebooks, and a command-line interface for checkpoint conversion, though the package itself has not seen a new release since April 2019.","worth_installing":"Yes, if you need to work with BERT, GPT, or Transformer-XL in PyTorch and are comfortable with a package that has not received updates since April 2019. The implementation is well-tested, has low install friction, and carries permissive licensing. However, consider whether a more actively maintained alternative (such as HuggingFace Transformers, which this package predates) better suits your project's long-term maintenance needs."},"id":"pytorch-pretrained-bert","links":{"html":"https://skillfed.io/packages/pytorch-pretrained-bert","md":"https://skillfed.io/packages/pytorch-pretrained-bert.md","pypi":"https://pypi.org/project/pytorch-pretrained-bert/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2019-04-25","license_spdx":null,"license_treatment":"permissive","name":"pytorch-pretrained-bert","python_support":"unspecified","summary":"PyTorch version of Google AI BERT model with script to load Google pre-trained models"},"popularity":{"monthly_downloads":78578,"position":14427,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.6.2"}