GPT2BPETokenizer.ConfigΒΆ
Component: GPT2BPETokenizer
-
class
GPT2BPETokenizer.
Config
[source] Bases:
ConfigBase
All Attributes (including base classes)
- bpe_encoder_path: str =
'manifold://pytext_training/tree/static/vocabs/bpe/gpt2/encoder.json'
- bpe_vocab_path: str =
'manifold://pytext_training/tree/static/vocabs/bpe/gpt2/vocab.bpe'
- lowercase: bool =
False
Default JSON
{
"bpe_encoder_path": "manifold://pytext_training/tree/static/vocabs/bpe/gpt2/encoder.json",
"bpe_vocab_path": "manifold://pytext_training/tree/static/vocabs/bpe/gpt2/vocab.bpe",
"lowercase": false
}