pytext.config package

Submodules

pytext.config.component module

class pytext.config.component.Component(config=None, *args, **kwargs)

Bases: object

classmethod from_config(config, *args, **kwargs)

class pytext.config.component.ComponentMeta

Bases: type

class pytext.config.component.ComponentType

Bases: enum.Enum

An enumeration.

BATCHER = 'batcher'

BATCH_SAMPLER = 'batch_sampler'

COLUMN = 'column'

DATA_HANDLER = 'data_handler'

DATA_SOURCE = 'data_source'

DATA_TYPE = 'data_type'

EXPORTER = 'exporter'

FEATURIZER = 'featurizer'

LOSS = 'loss'

MASKING_FUNCTION = 'masking_function'

METRIC_REPORTER = 'metric_reporter'

MODEL = 'model'

MODEL2 = 'model2'

MODULE = 'module'

OPTIMIZER = 'optimizer'

PREDICTOR = 'predictor'

PRIVACY_ENGINE = 'privacy_engine'

SCHEDULER = 'scheduler'

SPARSIFIER = 'sparsifier'

TASK = 'task'

TENSORIZER = 'tensorizer'

TOKENIZER = 'tokenizer'

TRAINER = 'trainer'

class pytext.config.component.Registry

Bases: object

classmethod add(component_type: pytext.config.component.ComponentType, cls_to_add: Type[CT_co], config_cls: Type[CT_co])

classmethod configs(component_type: pytext.config.component.ComponentType) → Tuple[Type[CT_co], ...]

classmethod get(component_type: pytext.config.component.ComponentType, config_cls: Type[CT_co]) → Type[CT_co]

classmethod subconfigs(config_cls: Type[CT_co]) → Tuple[Type[CT_co], ...]

classmethod values(component_type: pytext.config.component.ComponentType) → Tuple[Type[CT_co], ...]

exception pytext.config.component.RegistryError

Bases: Exception

pytext.config.component.create_component(component_type: pytext.config.component.ComponentType, config: Any, *args, **kwargs)

pytext.config.component.create_data_handler(data_handler_config, *args, **kwargs)

pytext.config.component.create_exporter(exporter_config, *args, **kwargs)

pytext.config.component.create_featurizer(featurizer_config, *args, **kwargs)

pytext.config.component.create_loss(loss_config, *args, **kwargs)

pytext.config.component.create_metric_reporter(module_config, *args, **kwargs)

pytext.config.component.create_model(model_config, *args, **kwargs)

pytext.config.component.create_optimizer(optimizer_config, model: torch.nn.modules.module.Module, *args, **kwargs)

pytext.config.component.create_predictor(predictor_config, *args, **kwargs)

pytext.config.component.create_privacy_engine(privacy_engine_config, *args, **kwargs)

pytext.config.component.create_scheduler(scheduler_config, optimizer, *args, **kwargs)

pytext.config.component.create_sparsifier(sparsifier_config, *args, **kwargs)

pytext.config.component.create_trainer(trainer_config, model: torch.nn.modules.module.Module, *args, **kwargs)

pytext.config.component.get_component_name(obj)

Return the human-readable name of the class of obj. Document the type of a config field and can be used as a Union value in a json config.

pytext.config.component.register_tasks(task_cls: Union[Type[CT_co], List[Type[CT_co]]])

Task classes are already added to registry during declaration, pass them as parameters here just to make sure they’re imported

pytext.config.config_adapter module

pytext.config.config_adapter.create_parameter(config, path_str, value)

pytext.config.config_adapter.delete_parameter(config, path_str)

pytext.config.config_adapter.deprecate(json_config, t)

pytext.config.config_adapter.doc_model_deprecated(json_config)

Rename DocModel to DocModel_Deprecated.

pytext.config.config_adapter.downgrade_one_version(json_config)

pytext.config.config_adapter.ensemble_task_deprecated(json_config)

Rename tasks with new API consistently

pytext.config.config_adapter.find_dicts_containing_key(json_config, key)

pytext.config.config_adapter.find_parameter(config, path_str)

pytext.config.config_adapter.fix_fl_local_optimizer_and_trainer(json_config)

a) Change FL local optimizer from optimizer:{SGD:{lr=0.1, momentum=0.2}} to optimizer:{lr=0.1, momentum=0.2} b) Replace trainer:{FLSyncTrainer:{foo}} by trainer:{fl_trainer:{foo, type:SyncTrainer}} Same for FLAsyncTrainer

pytext.config.config_adapter.flatten_deprecated_ensemble_config(json_config)

pytext.config.config_adapter.get_json_config_iterator(json_config, lookup_key)

pytext.config.config_adapter.get_name_from_options(export_config)

Reverse engineer which model is which based on recognized export configurations. If the export configurations don’t adhere to the set of recognized backends, then set target name to unknown

pytext.config.config_adapter.is_type_specifier(json_dict)

If a config object is a class, it might have a level which is a type specifier, with one key corresponding to the name of whichever type it is. These types should not be explicitly named in the path.

pytext.config.config_adapter.lm_model_deprecated(json_config)

Rename LM model to _Deprecated (LMTask is already deprecated in v5)

pytext.config.config_adapter.migrate_to_new_data_handler(task, columns)

pytext.config.config_adapter.move_epoch_size(json_config)

pytext.config.config_adapter.new_tasks_rename(json_config)

Rename tasks with new API consistently

pytext.config.config_adapter.old_tasks_deprecated(json_config)

Rename tasks with data_handler config to _Deprecated

pytext.config.config_adapter.register_adapter(from_version)

pytext.config.config_adapter.register_down_grade_adapter(from_version)

pytext.config.config_adapter.remove_docclassificationtask_deprecated(json_config)

pytext.config.config_adapter.remove_lmtask_deprecated(json_config)

pytext.config.config_adapter.rename(json_config, old_name, new_name)

pytext.config.config_adapter.rename_bitransformer_inputs(json_config)

In “BiTransformer” model, rename input “characters” -> “bytes” and update subfields.

pytext.config.config_adapter.rename_fl_task(json_config)

pytext.config.config_adapter.rename_parameter(config, old_path, new_path, transform=<function <lambda>>)

A powerful tool for writing config adapters, this allows you to specify a JSON-style path for an old and new config parameter. For instance

rename_parameter(config, “task.data.epoch_size”, “task.trainer.batches_per_epoch”)

will look through the config for task.data.epoch_size, including moving through explicitly specified types. If it’s specified, it will delete the value and set it in task.trainer.num_batches_per_epoch instead, creating trainer as an empty dictionary if necessary.

pytext.config.config_adapter.rename_tensorizer_vocab_params(json_config)

pytext.config.config_adapter.update_to_version(json_config, expected_version=26)

pytext.config.config_adapter.upgrade_export_config(json_config)

Upgrade model export related config fields to the new “export” section.

pytext.config.config_adapter.upgrade_if_xlm(json_config)

Make XLMModel Union changes for encoder and tokens config. Since they are now unions, insert the old class into the config if no class name is mentioned.

pytext.config.config_adapter.upgrade_one_version(json_config)

pytext.config.config_adapter.upgrade_padding(json_config)

Upgrade config option padding_control to seq_padding_control.

pytext.config.config_adapter.upgrade_to_latest(json_config)

pytext.config.config_adapter.v0_to_v1(json_config)

pytext.config.config_adapter.v12_to_v13(json_config)

remove_output_encoded_layers(json_config)

pytext.config.config_adapter.v1_to_v2(json_config)

pytext.config.config_adapter.v22_to_v23(json_config)

Upgrade by adding read_chunk_size option

pytext.config.config_adapter.v23_to_v22(json_config)

Upgrade by removing read_chunk_size option

pytext.config.config_adapter.v23_to_v24(json_config)

No-op since export_list is optional

pytext.config.config_adapter.v24_to_v23(json_config)

Downgrade by removing export_list option

pytext.config.config_adapter.v24_to_v25(json_config)

Upgrade by adding max_input_text_length option and default to None

pytext.config.config_adapter.v25_to_v24(json_config)

Downgrade by removing max_input_text_length option for SentencePieceTokenizer

pytext.config.config_adapter.v25_to_v26(json_config)

pytext.config.config_adapter.v26_to_v25(json_config)

Downgrade by removing target option from all exports in export_list

pytext.config.config_adapter.v2_to_v3(json_config)

Optimizer and Scheduler configs used to be part of the task config, they now live in the trainer’s config.

pytext.config.config_adapter.v3_to_v4(json_config)

Key for provding the path for contextual token embedding has changed from pretrained_model_embedding to contextual_token_embedding. This affects the `features section of the config.

pytext.config.config_adapter.v6_to_v7(json_config)

Make LabelTensorizer expansible. If the labels field should be an instance of LabelTensorizer, convert it to`{LabelTensorizer: labels}`.

pytext.config.contextual_intent_slot module

class pytext.config.contextual_intent_slot.ExtraField

Bases: object

DOC_WEIGHT = 'doc_weight'

RAW_WORD_LABEL = 'raw_word_label'

TOKEN_RANGE = 'token_range'

UTTERANCE = 'utterance'

WORD_WEIGHT = 'word_weight'

class pytext.config.contextual_intent_slot.ModelInput

Bases: object

CHAR = 'char_feat'

CONTEXTUAL_TOKEN_EMBEDDING = 'contextual_token_embedding'

DENSE = 'dense_feat'

DICT = 'dict_feat'

SEQ = 'seq_word_feat'

TEXT = 'word_feat'

class pytext.config.contextual_intent_slot.ModelInputConfig(**kwargs)

Bases: pytext.config.module_config.Module.Config

char_feat = None

contextual_token_embedding = None

dense_feat = None

dict_feat = None

seq_word_feat = <pytext.config.field_config.WordFeatConfig object>

word_feat = <pytext.config.field_config.WordFeatConfig object>

pytext.config.doc_classification module

class pytext.config.doc_classification.ExtraField

Bases: object

RAW_TEXT = 'text'

class pytext.config.doc_classification.ModelInput

Bases: object

CHAR_FEAT = 'char_feat'

CONTEXTUAL_TOKEN_EMBEDDING = 'contextual_token_embedding'

DENSE_FEAT = 'dense_feat'

DICT_FEAT = 'dict_feat'

SEQ_LENS = 'seq_lens'

WORD_FEAT = 'word_feat'

class pytext.config.doc_classification.ModelInputConfig(**kwargs)

Bases: pytext.config.module_config.Module.Config

char_feat = None

contextual_token_embedding = None

dense_feat = None

dict_feat = None

word_feat = <pytext.config.field_config.WordFeatConfig object>

pytext.config.field_config module

pytext.config.field_config.CharFeatConfig

alias of pytext.config.field_config.CharFeatConfig

pytext.config.field_config.ContextualTokenEmbeddingConfig

alias of pytext.config.field_config.ContextualTokenEmbeddingConfig

pytext.config.field_config.DictFeatConfig

alias of pytext.config.field_config.DictFeatConfig

class pytext.config.field_config.DocLabelConfig(**kwargs)

Bases: pytext.config.pytext_config.ConfigBase

export_output_names = ['doc_scores']

label_weights = {}

target_prob = False

class pytext.config.field_config.EmbedInitStrategy

Bases: enum.Enum

An enumeration.

RANDOM = 'random'

ZERO = 'zero'

class pytext.config.field_config.FeatureConfig(**kwargs)

Bases: pytext.config.module_config.Module.Config

char_feat = None

contextual_token_embedding = None

dense_feat = None

dict_feat = None

seq_word_feat = None

word_feat = <pytext.config.field_config.WordFeatConfig object>

class pytext.config.field_config.FloatVectorConfig(**kwargs)

Bases: pytext.config.pytext_config.ConfigBase

dim = 0

dim_error_check = False

export_input_names = ['float_vec_vals']

pytext.config.field_config.MLPFeatConfig

alias of pytext.config.field_config.MLPFeatConfig

class pytext.config.field_config.Target

Bases: object

DOC_LABEL = 'doc_label'

TARGET_LABEL_FIELD = 'target_label'

TARGET_LOGITS_FIELD = 'target_logit'

TARGET_PROB_FIELD = 'target_prob'

pytext.config.field_config.WordFeatConfig

alias of pytext.config.field_config.WordFeatConfig

class pytext.config.field_config.WordLabelConfig(**kwargs)

Bases: pytext.config.pytext_config.ConfigBase

export_output_names = ['word_scores']

use_bio_labels = False

pytext.config.module_config module

class pytext.config.module_config.Activation

Bases: enum.Enum

An enumeration.

GELU = 'gelu'

GLU = 'glu'

LEAKYRELU = 'leakyrelu'

RELU = 'relu'

TANH = 'tanh'

class pytext.config.module_config.CNNParams(**kwargs)

Bases: pytext.config.pytext_config.ConfigBase

causal = False

dilated = False

kernel_num = 100

kernel_sizes = [3, 4]

weight_norm = False

class pytext.config.module_config.ExporterType

Bases: enum.Enum

An enumeration.

INIT_PREDICT = 'init_predict'

PREDICTOR = 'predictor'

pytext.config.module_config.ModuleConfig

alias of pytext.config.module_config.ModuleConfig

class pytext.config.module_config.PerplexityType

Bases: enum.Enum

An enumeration.

EOS = 'eos'

MAX = 'max'

MEAN = 'mean'

MEDIAN = 'median'

MIN = 'min'

class pytext.config.module_config.PoolingType

Bases: enum.Enum

An enumeration.

LOGSUMEXP = 'logsumexp'

MAX = 'max'

MEAN = 'mean'

NONE = 'none'

class pytext.config.module_config.SlotAttentionType

Bases: enum.Enum

An enumeration.

CONCAT = 'concat'

DOT = 'dot'

MULTIPLY = 'multiply'

NO_ATTENTION = 'no_attention'

pytext.config.pair_classification module

class pytext.config.pair_classification.ExtraField

Bases: object

UTTERANCE_PAIR = 'utterance'

class pytext.config.pair_classification.ModelInput

Bases: object

TEXT1 = 'text1'

TEXT2 = 'text2'

class pytext.config.pair_classification.ModelInputConfig(**kwargs)

Bases: pytext.config.module_config.Module.Config

text1 = <pytext.config.field_config.WordFeatConfig object>

text2 = <pytext.config.field_config.WordFeatConfig object>

pytext.config.pytext_config module

class pytext.config.pytext_config.ConfigBase(**kwargs)

Bases: object

items()

class pytext.config.pytext_config.ConfigBaseMeta

Bases: type

annotations_and_defaults()

class pytext.config.pytext_config.ExportConfig(**kwargs)

Bases: pytext.config.pytext_config.ConfigBase

accelerate = []

batch_padding_control = None

export_caffe2_path = None

export_lite_path = None

export_onnx_path = '/tmp/model.onnx'

export_torchscript_path = None

inference_interface = None

seq_padding_control = None

target = ''

torchscript_quantize = False

exception pytext.config.pytext_config.InvalidMethodInvocation(message)

Bases: Exception

class pytext.config.pytext_config.LogitsConfig(**kwargs)

Bases: pytext.config.pytext_config.TestConfig

batch_size = 16

dump_raw_input = False

fp16 = False

gpus = 1

ndigits_precision = 0

output_columns = None

use_gzip = False

class pytext.config.pytext_config.PlaceHolder

Bases: object

class pytext.config.pytext_config.PyTextConfig(**kwargs)

Bases: pytext.config.pytext_config.ConfigBase

accelerate

auto_resume_from_snapshot = False

batch_padding_control

debug_path = '/tmp/model.debug'

distributed_world_size = 1

export = <pytext.config.pytext_config.ExportConfig object>

export_caffe2_path

export_check(method_name)

export_list = []

export_onnx_path

export_torchscript_path

get_export_accelerate(index)

get_export_batch_padding_control(index)

get_export_caffe2_path(index)

get_export_inference_interface(index)

get_export_onnx_path(index)

get_export_seq_padding_control(index)

get_export_target(index)

get_export_torchscript_path(index)

get_export_torchscript_quantize(index)

get_first_config()

gpu_streams_for_distributed_training = 1

include_dirs = None

inference_interface

load_snapshot_path = ''

modules_save_dir = ''

random_seed = 0

Seed value to seed torch, python, and numpy random generators.

read_chunk_size = 1000000000

report_eval_results = False

report_test_results = True

save_all_checkpoints = False

save_module_checkpoints = False

save_snapshot_path = '/tmp/model.pt'

seq_padding_control

set_export_accelerate(acc, index)

set_export_batch_padding_control(batch_padding_control, index)

set_export_caffe2_path(p, index)

set_export_inference_interface(inference_interface, index)

set_export_onnx_path(p, index)

set_export_seq_padding_control(seq_padding_control, index)

set_export_target(tgt, index)

set_export_torchscript_path(p, index)

set_export_torchscript_quantize(quantize, index)

target

test_out_path = '/tmp/test_out.txt'

torchscript_quantize

use_config_from_snapshot = True

use_cuda_for_testing = True

use_cuda_if_available = True

use_deterministic_cudnn = False

Whether to allow CuDNN to behave deterministically.

use_fp16 = False

use_tensorboard = True

class pytext.config.pytext_config.TestConfig(**kwargs)

Bases: pytext.config.pytext_config.ConfigBase

field_names = None

Field names for the TSV. If this is not set, the first line of each file will be assumed to be a header containing the field names.

test_out_path = ''

test_path = 'test.tsv'

use_cuda_if_available = True

use_fp16 = False

use_tensorboard = True

pytext.config.query_document_pairwise_ranking module

class pytext.config.query_document_pairwise_ranking.ModelInput

Bases: object

NEG_RESPONSE = 'neg_response'

POS_RESPONSE = 'pos_response'

QUERY = 'query'

class pytext.config.query_document_pairwise_ranking.ModelInputConfig(**kwargs)

Bases: pytext.config.module_config.Module.Config

neg_response = <pytext.config.field_config.WordFeatConfig object>

pos_response = <pytext.config.field_config.WordFeatConfig object>

query = <pytext.config.field_config.WordFeatConfig object>

pytext.config.serialize module

exception pytext.config.serialize.ConfigParseError

Bases: Exception

exception pytext.config.serialize.EnumTypeError

Bases: pytext.config.serialize.ConfigParseError

exception pytext.config.serialize.IncorrectTypeError

Bases: Exception

exception pytext.config.serialize.MissingValueError

Bases: pytext.config.serialize.ConfigParseError

exception pytext.config.serialize.UnionTypeError

Bases: pytext.config.serialize.ConfigParseError

exception pytext.config.serialize.ValueSerializationError

Bases: Exception

pytext.config.serialize.build_subclass_dict(subclasses)

pytext.config.serialize.component_config_type_from_type_name(cls, type_name: str) → Type[CT_co]

pytext.config.serialize.config_from_json(cls, json_obj, ignore_fields=())

pytext.config.serialize.config_to_json(cls, config_obj)

pytext.config.serialize.parse_config(config_json)

Parse PyTextConfig object from parameter string or parameter file

pytext.config.serialize.pytext_config_from_json(json_obj, ignore_fields=(), auto_upgrade=True)

pytext.config.utils module

pytext.config.utils.cast_str(to_type, value)

pytext.config.utils.find_param(root, suffix, parent='')

Recursively look at all fields in config to find where suffix would fit. This is used to change configs so that they don’t use default values. Return the list of field paths matching.

pytext.config.utils.is_component_class(obj)

pytext.config.utils.replace_param(root, path_list, value)

pytext.config.utils.resolve_optional(type_v)

Deal with Optional implemented as Union[type, None]

Module contents