pytext.utils package

Submodules

pytext.utils.ascii_table module

pytext.utils.ascii_table.ascii_table(data, human_column_names=None, footer=None, indentation='', alignments=())

pytext.utils.ascii_table.ascii_table_from_dict(dict, key_name, value_name, indentation='')

pytext.utils.ascii_table.ordered_unique(sequence)

pytext.utils.config_utils module

class pytext.utils.config_utils.MockConfigLoader(config_base_path: str, reset_paths: {<class 'str'>: <class 'str'>} = None, replace_paths: {<class 'str'>: <class 'str'>} = None)

Bases: object

disable_cuda(config)

fix_paths(config)

make_config(config_filename, disable_tensorboard=True)

make_config_from_dict(config, disable_tensorboard)

pytext.utils.cuda module

pytext.utils.cuda.FloatTensor(*args)

pytext.utils.cuda.GetTensor(tensor)

pytext.utils.cuda.LongTensor(*args)

pytext.utils.cuda.Variable(data, *args, **kwargs)

pytext.utils.cuda.device()

pytext.utils.cuda.tensor(data, dtype)

pytext.utils.cuda.var_to_numpy(v)

pytext.utils.cuda.zerovar(*size)

pytext.utils.data module

class pytext.utils.data.ResultRow(name, metrics_dict)

Bases: object

class pytext.utils.data.ResultTable(metrics, class_names, labels, preds)

Bases: object

class pytext.utils.data.Slot(label: str, start: int, end: int)

Bases: object

B_LABEL_PREFIX = 'B-'

I_LABEL_PREFIX = 'I-'

NO_LABEL_SLOT = 'NoLabel'

b_label_name

i_label_name

token_label(use_bio_labels, token_start, token_end)

token_overlap(token_start, token_end)

pytext.utils.data.align_slot_labels(token_ranges: List[Tuple[int, int]], slots_field: str, use_bio_labels: bool = False)

pytext.utils.data.byte_length(text: str) → int

Return the string length in term of byte offset

pytext.utils.data.char_offset_to_byte_offset(text: str, char_offset: int) → int

Convert a char offset to byte offset

pytext.utils.data.get_substring_from_offsets(text: str, start: Optional[int], end: Optional[int], byte_offset: bool = True) → str

Access substring of a text using byte offset, if the switch is turned on. Otherwise return substring as the usual text[start:end]

pytext.utils.data.is_number(string)

pytext.utils.data.merge_token_labels_by_bio(token_ranges, labels)

pytext.utils.data.merge_token_labels_by_label(token_ranges, labels)

pytext.utils.data.merge_token_labels_to_slot(token_ranges, labels, use_bio_label=True)

pytext.utils.data.no_tokenize(s: Any) → Any

pytext.utils.data.parse_and_align_slot_labels_list(token_ranges: List[Tuple[int, int]], slots_field: str, use_bio_labels: bool = False)

pytext.utils.data.parse_json_array(json_text: str) → List[str]

pytext.utils.data.parse_slot_string(slots_field: str) → List[pytext.utils.data.Slot]

pytext.utils.data.parse_token(utterance: str, token_range: List[int]) → List[Tuple[str, Tuple[int, int]]]

pytext.utils.data.simple_tokenize(s: str) → List[str]

pytext.utils.data.strip_bio_prefix(label)

pytext.utils.data.unkify(token: str)

pytext.utils.distributed module

pytext.utils.distributed.dist_init(distributed_rank: int, world_size: int, init_method: str, device_id: int, backend: str = 'nccl', gpu_streams: int = 1)

1. After spawn process per GPU, we want all workers to call init_process_group around the same time or times out. 2. After dist_init, we want all workers to start calling all_reduce/barrier around the same time or NCCL timeouts.

pytext.utils.distributed.force_print(*args, **kwargs)

pytext.utils.distributed.get_shard_range(dataset_size: int, rank: int, world_size: int)

In case dataset_size is not evenly divided by world_size, we need to pad one extra example in each shard shard_len = dataset_size // world_size + 1

Case 1 rank < remainder: each shard start position is rank * shard_len

Case 2 rank >= remainder: without padding, each shard start position is rank * (shard_len - 1) + remainder = rank * shard_len - (rank - remainder) But to make sure all shard have same size, we need to pad one extra example when rank >= remainder, so start_position = start_position - 1

For example, dataset_size = 21, world_size = 8 rank 0 to 4: [0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10, 11], [12, 13, 14] rank 5 to 7: [14, 15, 16], [16, 17, 18], [18, 19, 20]

pytext.utils.distributed.suppress_output()

pytext.utils.documentation module

pytext.utils.documentation.find_config_class(class_name)

Return the set of PyText classes matching that name. Handles fully-qualified class_name including module.

pytext.utils.documentation.get_class_members_recursive(obj)

Find all the field names for a given class and their default value.

pytext.utils.documentation.get_config_fields(obj)

Return a dict of config help for this object, where: - key: config name - value: (default, type, options)

• default: default value for this key if not specified
• type: type for this config value, as a string
• options: possible values for this config, only if type = Union

If the type is “Union”, the options give the lists of class names that are possible, and the default is one of those class names.

pytext.utils.documentation.get_subclasses(klass, stop_classes=(<class 'pytext.models.module.Module'>, <class 'pytext.config.component.Component'>, <class 'torch.nn.modules.module.Module'>))

pytext.utils.documentation.pretty_print_config_class(obj)

Pretty-print the fields of one object.

pytext.utils.documentation.replace_components(root, component, base_class)

Recursively look at all fields in config to find where component would fit. This is used to change configs so that they don’t use default values. Return the chain of field names, from child to parent.

pytext.utils.embeddings module

class pytext.utils.embeddings.PretrainedEmbedding(embeddings_path: str = None, lowercase_tokens: bool = True, skip_header: bool = True, delimiter: str = ' ')

Bases: object

Utility class for loading/caching/initializing word embeddings

cache_pretrained_embeddings(cache_path: str) → None

Cache the processed embedding vectors and vocab to a file for faster loading

filter_criteria(token: str) → bool

initialize_embeddings_weights(str_to_idx: Dict[str, int], unk: str, embed_dim: int, init_strategy: pytext.config.field_config.EmbedInitStrategy) → torch.Tensor

Initialize embeddings weights of shape (len(str_to_idx), embed_dim) from the pretrained embeddings vectors. Words that are not in the pretrained embeddings list will be initialized according to init_strategy. :param str_to_idx: a dict that maps words to indices that the model expects :param unk: unknown token :param embed_dim: the embeddings dimension :param init_strategy: method of initializing new tokens :returns: a float tensor of dimension (vocab_size, embed_dim)

load_cached_embeddings(cache_path: str) → None

Load cached embeddings from file

load_pretrained_embeddings(raw_embeddings_path: str, append: bool = False, dialect: str = None, lowercase_tokens: bool = True, skip_header: bool = True, delimiter: str = ' ') → None

Loading raw embeddings vectors from file in the format: num_words dim word_i v0, v1, v2, …., v_dim word_2 v0, v1, v2, …., v_dim …. Optionally appends _dialect to every token in the vocabulary (for XLU embeddings).

normalize_token(token: str) → str

Apply normalizations to the input token for the embedding space

pytext.utils.embeddings.append_dialect(word: str, dialect: str) → str

pytext.utils.file_io module

pytext.utils.file_io.chunk_file(file_path, chunks, work_dir)

Splits a large file by line into number of chunks and writes them into work_dir

pytext.utils.file_io.register_http_url_handler()

support reading file from url starting with “http://”, “https://”, “ftp://”

pytext.utils.label module

pytext.utils.label.get_label_weights(vocab_dict: Dict[str, int], label_weights: Dict[str, float])

pytext.utils.lazy module

class pytext.utils.lazy.Infer(resolve_fn)

Bases: object

A value which can be inferred from a forward pass. Infer objects should be passed as arguments or keyword arguments to Lazy objects; see Lazy documentation for more details.

classmethod dimension(dim)

A helper for creating Infer arguments looking at specific dimensions.

class pytext.utils.lazy.Lazy(module_class, *args, **kwargs)

Bases: torch.nn.modules.module.Module

A module which is able to infer some of its parameters from the inputs to its first forward pass. Lazy wraps any other nn.Module, and arguments can be passed that will be used to construct that wrapped Module after the first forward pass. If any of these arguments are Infer objects, those arguments will be replaced by calling the callback of the Infer object on the forward pass input.

For instance, >>> Lazy(nn.Linear, Infer(lambda input: input.size(-1)), 4) Lazy()

takes its in_features dimension from the last dimension of the input to its forward pass. This can be simplified to

>>> Lazy(nn.Linear, Infer.dimension(-1), 4)

or a partial can be created, for instance

>>> LazyLinear = Lazy.partial(nn.Linear, Infer.dimension(-1))
>>> LazyLinear(4)
Lazy()

Finally, these Lazy objects explicitly forbid treating themselves normally; they must instead be replaced by calling init_lazy_modules on your model before training. For instance,

>>> ll = lazy.Linear(4)
>>> seq = nn.Sequential(ll)
>>> seq
Sequential(
    0: Lazy(),
)
>>> init_lazy_modules(seq, torch.rand(1, 2)
Sequential(
    0: Linear(in_features=2, out_features=4, bias=True)
)

forward(*args, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

classmethod partial(module_class, *args, **kwargs)

resolve()

Must make a call to forward before calling this function; returns the full nn.Module object constructed using inferred arguments/dimensions.

exception pytext.utils.lazy.UninitializedLazyModuleError

Bases: Exception

A lazy module was used improperly.

pytext.utils.lazy.init_lazy_modules(module: torch.nn.modules.module.Module, dummy_input: Tuple[torch.Tensor, ...]) → torch.nn.modules.module.Module

Finalize an nn.Module which has Lazy components. This will both mutate internal modules which have Lazy elements, and return a new non-lazy nn.Module (in case the top-level module itself is Lazy).

Parameters:

• module – An nn.Module which may be lazy or contain Lazy subcomponents
• dummy_input – module is called with this input to ensure that Lazy subcomponents have been able to infer any parameters they need

Returns:

The full nn.Module object constructed using inferred arguments/dimensions.

class pytext.utils.lazy.lazy_property(fget)

Bases: object

More or less copy-pasta: http://stackoverflow.com/a/6849299 Meant to be used for lazy evaluation of an object attribute. property should represent non-mutable data, as it replaces itself.

pytext.utils.lazy.replace_lazy_modules(module)

pytext.utils.loss module

class pytext.utils.loss.LagrangeMultiplier

Bases: torch.autograd.function.Function

static backward(ctx, grad_output)

Defines a formula for differentiating the operation.

This function is to be overridden by all subclasses.

It must accept a context ctx as the first argument, followed by as many outputs did forward() return, and it should return as many tensors, as there were inputs to forward(). Each argument is the gradient w.r.t the given output, and each returned value should be the gradient w.r.t. the corresponding input.

The context can be used to retrieve tensors saved during the forward pass. It also has an attribute ctx.needs_input_grad as a tuple of booleans representing whether each input needs gradient. E.g., backward() will have ctx.needs_input_grad[0] = True if the first input to forward() needs gradient computated w.r.t. the output.

static forward(ctx, input)

Performs the operation.

This function is to be overridden by all subclasses.

It must accept a context ctx as the first argument, followed by any number of arguments (tensors or other types).

The context can be used to store tensors that can be then retrieved during the backward pass.

pytext.utils.loss.build_class_priors(labels, class_priors=None, weights=None, positive_pseudocount=1.0, negative_pseudocount=1.0)

build class priors, if necessary. For each class, the class priors are estimated as (P + sum_i w_i y_i) / (P + N + sum_i w_i), where y_i is the ith label, w_i is the ith weight, P is a pseudo-count of positive labels, and N is a pseudo-count of negative labels.

Parameters:

• labels – A Tensor with shape [batch_size, num_classes]. Entries should be in [0, 1].
• class_priors – None, or a floating point Tensor of shape [C] containing the prior probability of each class (i.e. the fraction of the training data consisting of positive examples). If None, the class priors are computed from targets with a moving average.
• weights – Tensor of shape broadcastable to labels, [N, 1] or [N, C], where N = batch_size, C = num_classes`
• positive_pseudocount – Number of positive labels used to initialize the class priors.
• negative_pseudocount – Number of negative labels used to initialize the class priors.

Returns:

A Tensor of shape [num_classes] consisting of the

weighted class priors, after updating with moving average ops if created.

Return type:

class_priors

pytext.utils.loss.false_postives_upper_bound(labels, logits, weights)

false_positives_upper_bound defined in paper: “Scalable Learning of Non-Decomposable Objectives”

Parameters:

• labels – A Tensor of shape broadcastable to logits.
• logits – A Tensor of shape [N, C] or [N, C, K]. If the third dimension is present, the lower bound is computed on each slice [:, :, k] independently.
• weights – Per-example loss coefficients, with shape broadcast-compatible with that of labels. i.e. [N, 1] or [N, C]

Returns:

A Tensor of shape [C] or [C, K].

pytext.utils.loss.lagrange_multiplier(x)

pytext.utils.loss.range_to_anchors_and_delta(precision_range, num_anchors)

Calculates anchor points from precision range.

Parameters:

• precision_range – an interval (a, b), where 0.0 <= a <= b <= 1.0
• num_anchors – int, number of equally spaced anchor points.

Returns:

A Tensor of [num_anchors] equally spaced values

in the interval precision_range.

delta: The spacing between the values in precision_values.

Return type:

precision_values

Raises:

ValueError – If precision_range is invalid.

pytext.utils.loss.true_positives_lower_bound(labels, logits, weights)

true_positives_lower_bound defined in paper: “Scalable Learning of Non-Decomposable Objectives”

Parameters:

• labels – A Tensor of shape broadcastable to logits.
• logits – A Tensor of shape [N, C] or [N, C, K]. If the third dimension is present, the lower bound is computed on each slice [:, :, k] independently.
• weights – Per-example loss coefficients, with shape [N, 1] or [N, C]

Returns:

A Tensor of shape [C] or [C, K].

pytext.utils.loss.weighted_hinge_loss(labels, logits, positive_weights=1.0, negative_weights=1.0)

Parameters:

• labels – one-hot representation Tensor of shape broadcastable to logits
• logits – A Tensor of shape [N, C] or [N, C, K]
• positive_weights – Scalar or Tensor
• negative_weights – same shape as positive_weights

Returns:

3D Tensor of shape [N, C, K], where K is length of positive weights or 2D Tensor of shape [N, C]

pytext.utils.meter module

class pytext.utils.meter.Meter

Bases: object

avg

reset()

update(val=1)

class pytext.utils.meter.TimeMeter

Bases: pytext.utils.meter.Meter

Computes the average occurrence of some event per second

avg

elapsed_time

reset()

update(val=1)

pytext.utils.mobile_onnx module

pytext.utils.mobile_onnx.add_feats_numericalize_ops(init_net, predict_net, vocab_map, input_names)

pytext.utils.mobile_onnx.create_context(init_net)

pytext.utils.mobile_onnx.create_vocab_index(vocab_list, net, net_workspace, index_name)

pytext.utils.mobile_onnx.create_vocab_indices_map(init_net, vocab_map)

pytext.utils.mobile_onnx.get_numericalize_net(init_net, predict_net, vocab_map, input_names)

pytext.utils.mobile_onnx.pytorch_to_caffe2(model, export_input, external_input_names, output_names, export_path, export_onnx_path=None)

pytext.utils.model module

pytext.utils.model.get_mismatched_param(models: Iterable[torch.nn.modules.module.Module], rel_epsilon: Optional[float] = None, abs_epsilon: Optional[float] = None) → str

Return the name of the first mismatched parameter. Return an empty string if all the parameters of the modules are identical.

pytext.utils.model.to_onehot(feat: pytext.utils.cuda.Variable, size: int) → pytext.utils.cuda.Variable

Transform features into one-hot vectors

pytext.utils.onnx module

pytext.utils.onnx.add_feats_numericalize_ops(c2_prepared, vocab_map, input_names)

pytext.utils.onnx.convert_caffe2_blob_name(blob_name)

pytext.utils.onnx.create_vocab_index(vocab_list, net, net_workspace, index_name)

pytext.utils.onnx.create_vocab_indices_map(c2_prepared, init_net, vocab_map)

pytext.utils.onnx.export_nets_to_predictor_file(c2_prepared, input_names, output_names, predictor_path, extra_params=None)

pytext.utils.onnx.get_numericalize_net(c2_prepared, vocab_map, input_names)

pytext.utils.onnx.pytorch_to_caffe2(model, export_input, external_input_names, output_names, export_path, export_onnx_path=None)

pytext.utils.onnx.validate_onnx_export(model)

pytext.utils.path module

pytext.utils.path.get_absolute_path(file_path: str) → str

pytext.utils.path.get_pytext_home()

pytext.utils.path.is_absolute_path(file_path: str) → bool

pytext.utils.precision module

pytext.utils.precision.delay_unscale()

pytext.utils.precision.maybe_float(tensor)

pytext.utils.precision.maybe_half(tensor)

pytext.utils.precision.pad_length(n)

pytext.utils.precision.set_fp16(fp16_enabled: bool)

pytext.utils.tensor module

pytext.utils.test module

pytext.utils.test.import_tests_module(packages_to_scan=None)

pytext.utils.timing module

class pytext.utils.timing.HierarchicalTimer

Bases: object

pop()

push(label, caller_id)

snapshot()

time(label)

class pytext.utils.timing.Snapshot

Bases: object

report(report_pep=False)

class pytext.utils.timing.SnapshotList

Bases: list

lists are not weakref-able by default.

class pytext.utils.timing.Timings(sum: float = 0.0, count: int = 0, max: float = -inf, times: List[T] = None)

Bases: object

add(time)

average

p50

p90

p99

pytext.utils.timing.format_time(seconds)

pytext.utils.timing.report_snapshot(fn)

pytext.utils.torch module

pytext.utils.usage module

pytext.utils.usage.log_accelerator_feature_usage(feature)

pytext.utils.usage.log_class_usage(klass)

pytext.utils.usage.log_feature_usage(feature)

pytext.utils.usage.log_flow_usage(flow_name)

Module contents

pytext.utils.cls_vars(cls)

pytext.utils.recursive_map(seq, func)

This is similar to the build-in map function but works for nested lists. Useful for transforming tensors serialized with .tolist()

pytext.utils.round_seq(seq, ndigits)

Rounds a nested sequence of floats to ndigits precision. Useful for rounding tensors serialized with .tolist()

pytext.utils.set_random_seeds(seed, use_deterministic_cudnn)