PyText Documentation

PyText is a deep-learning based NLP modeling framework built on PyTorch. PyText addresses the often-conflicting requirements of enabling rapid experimentation and of serving models at scale. It achieves this by providing simple and extensible interfaces and abstractions for model components, and by using PyTorch’s capabilities of exporting models for inference via the optimized Caffe2 execution engine. We use PyText at Facebook to iterate quickly on new modeling ideas and then seamlessly ship them at scale.

Core PyText Features:

• Production ready models for various NLP/NLU tasks:
  • Text classifiers
    • Yoon Kim (2014): Convolutional Neural Networks for Sentence Classification
    • Lin et al. (2017): A Structured Self-attentive Sentence Embedding
  • Sequence taggers
    • Lample et al. (2016): Neural Architectures for Named Entity Recognition
  • Joint intent-slot model
    • Zhang et al. (2016): A Joint Model of Intent Determination and Slot Filling for Spoken Language Understanding
  • Contextual intent-slot models
• Extensible components that allow easy creation of new models and tasks
• Ensemble training support
• Distributed-training support (using the new C10d backend in PyTorch 1.0)
• Reference implementation and a pre-trained model for the paper: Gupta et al. (2018): Semantic Parsing for Task Oriented Dialog using Hierarchical Representations

How To Use

Please follow the tutorial series in Getting Started to get a sense of how to train a basic model and deploy to production.

After that, you can explore more options of builtin models and training methods in Training More Advanced Models

If you want to use PyText as a library and build your own models, please check the tutorial in Extending PyText

Note

All the demo configs and test data for the tutorials can be found in source code. You can either install PyText from source or download the files manually from GitHub.

Getting Started

• Installation
• Train your first model
• Execute your first model
• Visualize Model Training with TensorBoard
• Use PyText models in your app
• Serve Models in Production
• Config Files Explained
• Config Commands

Training More Advanced Models

• Train Intent-Slot model on ATIS Dataset
• Hierarchical intent and slot filling
• Multitask training with disjoint datasets
• Data Parallel Distributed Training
• XLM-RoBERTa

Extending PyText

• Architecture Overview
• Custom Data Format
• Custom Tensorizer
• Using External Dense Features
• Creating A New Model
• Hacking PyText

References

• pytext
  • config
  • data
  • exporters
  • loss
  • metric_reporters
  • models
  • optimizer
  • task
  • trainers
• pytext package
  • Subpackages
  • Submodules
  • pytext.builtin_task module
  • pytext.main module
  • pytext.workflow module
  • Module contents

Indices and tables

• Index
• Search Page