pytext.models.seq_models package¶
Submodules¶
pytext.models.seq_models.attention module¶
-
class
pytext.models.seq_models.attention.
DecoupledMultiheadAttention
(embed_dim: int, context_dim: int, num_heads: int, dropout: float, unseen_mask=False, src_length_mask=True)[source]¶ Bases:
torch.nn.modules.module.Module
Multiheaded Scaled Dot Product Attention. This function has the same exact signature as the one used in pytorch_translate with the added benefit of supporting torchscript
-
forward
(decoder_state: torch.Tensor, source_hids: torch.Tensor, src_len_mask: Optional[torch.Tensor], squeeze: bool = True) → Tuple[torch.Tensor, torch.Tensor][source]¶ Computes MultiheadAttention with respect to either a vector or a tensor
- Inputs:
- decoder_state: (bsz x decoder_hidden_state_dim) or
- (bsz x T x decoder_hidden_state_dim)
source_hids: srclen x bsz x context_dim src_lengths: bsz x 1, actual sequence lengths squeeze: Whether or not to squeeze on the time dimension.
Even if decoder_state.dim() is 2 dimensional an explicit time step dimension will be unsqueezed.- Outputs:
- [batch_size, max_src_len] if decoder_state.dim() == 2 & squeeze
- or
- [batch_size, 1, max_src_len] if decoder_state.dim() == 2 & !squeeze
- or
- [batch_size, T, max_src_len] if decoder_state.dim() == 3 & !squeeze
- or
- [batch_size, T, max_src_len] if decoder_state.dim() == 3 & squeeze & T != 1
- or
[batch_size, max_src_len] if decoder_state.dim() == 3 & squeeze & T == 1
-
-
class
pytext.models.seq_models.attention.
DotAttention
(decoder_hidden_state_dim, context_dim, force_projection=False, src_length_masking=True)[source]¶ Bases:
torch.nn.modules.module.Module
-
forward
(decoder_state, source_hids, src_lengths)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.attention.
MultiheadAttention
(embed_dim, num_heads, dropout, kdim=None, vdim=None, bias=True)[source]¶ Bases:
pytext.models.seq_models.base.PyTextIncrementalDecoderComponent
Refer Attention is All You Need for more details.
This is a simplified implementation of multihead attention optimized for exporting using torchscript. Usage of nn.Linear() instead of F.Linear() helps to quantize the linear layers.
Query represents the output from last decoder step. Key and Values are obtained from encoder. Attention weights are obtained from the dot product of query and key. Attention weights multiplied by the value gives output.
-
forward
(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, key_padding_mask: Optional[torch.Tensor], need_weights: bool, incremental_state: Optional[Dict[str, torch.Tensor]] = None) → Tuple[torch.Tensor, Optional[torch.Tensor]][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.seq_models.base module¶
-
class
pytext.models.seq_models.base.
PlaceholderAttentionIdentity
[source]¶ Bases:
torch.nn.modules.module.Module
-
forward
(query, key, value, need_weights: bool = None, key_padding_mask: Optional[torch.Tensor] = None, incremental_state: Optional[Dict[str, torch.Tensor]] = None) → Tuple[torch.Tensor, Optional[torch.Tensor]][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.base.
PlaceholderIdentity
[source]¶ Bases:
torch.nn.modules.module.Module
-
forward
(x, incremental_state: Optional[Dict[str, torch.Tensor]] = None)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.base.
PyTextIncrementalDecoderComponent
[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
-
get_incremental_state
(incremental_state: Dict[str, torch.Tensor], key: str) → Optional[torch.Tensor][source]¶ Helper for getting incremental state for an nn.Module.
-
pytext.models.seq_models.contextual_intent_slot module¶
-
class
pytext.models.seq_models.contextual_intent_slot.
ContextualIntentSlotModel
(default_doc_loss_weight, default_word_loss_weight, *args, **kwargs)[source]¶ Bases:
pytext.models.joint_model.IntentSlotModel
Joint Model for Intent classification and slot tagging with inputs of contextual information (sequence of utterances) and dictionary feature of the last utterance.
Training data should include: doc_label (string): intent classification label of either the sequence of utterances or just the last sentence word_label (string): slot tagging label of the last utterance in the format of start_idx:end_idx:slot_label, multiple slots are separated by a comma text (list of string): sequence of utterances for training dict_feat (dict): a dict of features that contains the feature of each word in the last utterance
Following is an example of raw columns from training data:
doc_label reply-where word_label 10:20:restaurant_name text [“dinner at 6?”, “wanna try Tomi Sushi?”] dict_feat - {“tokenFeatList”: [{“tokenIdx”: 2, “features”: {“poi:eatery”: 0.66}},
- {“tokenIdx”: 3, “features”: {“poi:eatery”: 0.66}}]}
pytext.models.seq_models.conv_decoder module¶
-
class
pytext.models.seq_models.conv_decoder.
ConvDecoderConfig
(**kwargs)[source]¶ Bases:
pytext.config.pytext_config.ConfigBase
-
combine_pos_embed
= 'concat'¶
-
decoder_embed_dim
= 128¶
-
decoder_input_dim
= 128¶
-
decoder_learned_pos
= False¶
-
decoder_normalize_before
= False¶
-
decoder_output_dim
= 128¶
-
dropout
= 0.1¶
-
max_target_positions
= 128¶
-
no_token_positional_embeddings
= False¶
-
positional_embedding_type
= 'learned'¶
-
-
class
pytext.models.seq_models.conv_decoder.
LightConvDecoder
(target_dict, embed_tokens, layers, decoder_config)[source]¶ Bases:
pytext.models.seq_models.conv_decoder.LightConvDecoderBase
-
forward
(prev_output_tokens: torch.Tensor, encoder_out: Dict[str, torch.Tensor], incremental_state: Optional[Dict[str, torch.Tensor]] = None, timestep: Optional[int] = None) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.conv_decoder.
LightConvDecoderBase
(target_dict, embed_tokens, layers, decoder_config)[source]¶ Bases:
pytext.models.seq_models.base.PyTextIncrementalDecoderComponent
-
forward_unprojected
(prev_output_tokens: torch.Tensor, encoder_out: Dict[str, torch.Tensor], incremental_state: Optional[Dict[str, torch.Tensor]] = None, timestep: Optional[int] = None) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶
-
-
class
pytext.models.seq_models.conv_decoder.
LightConvDecoderLayer
(attention_dropout, decoder_attention_heads, self_attention_heads, decoder_conv_dim, decoder_conv_type, attention_type, self_attention_type, decoder_embed_dim, decoder_ffn_embed_dim, decoder_glu, decoder_normalize_before, dropout, input_dropout, relu_dropout, need_attention, convolution_type, conv=None, self_attention=None, attention=None)[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
-
extra_repr
()[source]¶ Set the extra representation of the module
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
-
forward
(x, encoder_out: torch.Tensor, encoder_padding_mask: Optional[torch.Tensor], decoder_padding_mask: Optional[torch.Tensor], incremental_state: Optional[Dict[str, torch.Tensor]])[source]¶ Parameters: - x (Tensor) – input to the layer of shape (seq_len, batch, embed_dim)
- encoder_padding_mask (ByteTensor) – binary ByteTensor of shape
(batch, src_len) where padding elements are indicated by
1
.
Returns: encoded output of shape (batch, src_len, embed_dim)
-
-
class
pytext.models.seq_models.conv_decoder.
LightConvDecoupledDecoder
(target_dict, embed_tokens, layers, decoder_config, ontology_generation_only, decoupled_attention_heads, model_output_logprob)[source]¶ Bases:
pytext.models.seq_models.conv_decoder.LightConvDecoderBase
-
forward
(prev_output_tokens: torch.Tensor, encoder_out: Dict[str, torch.Tensor], incremental_state: Optional[Dict[str, torch.Tensor]] = None, timestep: Optional[int] = None) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.seq_models.conv_encoder module¶
-
class
pytext.models.seq_models.conv_encoder.
ConvEncoderConfig
(**kwargs)[source]¶ Bases:
pytext.config.pytext_config.ConfigBase
-
combine_pos_embed
= 'concat'¶
-
dropout
= 0.1¶
-
embedding_dim
= 128¶
-
encoder_embed_dim
= 128¶
-
encoder_learned_pos
= False¶
-
encoder_normalize_before
= False¶
-
max_source_positions
= 1024¶
-
max_target_positions
= 100¶
-
no_token_positional_embeddings
= False¶
-
positional_embedding_type
= 'learned'¶
-
-
class
pytext.models.seq_models.conv_encoder.
LightConvEncoder
(dictionary, embed_tokens, layers, encoder_config)[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
,pytext.models.seq_models.nar_modules.NAREncoderUtility
-
extra_repr
()[source]¶ Set the extra representation of the module
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
-
forward
(src_tokens: torch.Tensor, src_embeddings: torch.Tensor, src_lengths: torch.Tensor) → Dict[str, torch.Tensor][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.conv_encoder.
LightConvEncoderLayer
(dropout, encoder_conv_dim, encoder_conv_type, self_attention_type, encoder_embed_dim, encoder_ffn_embed_dim, self_attention_heads, encoder_glu, encoder_normalize_before, input_dropout, relu_dropout, convolution_type, conv=None, self_attention=None)[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
-
extra_repr
()[source]¶ Set the extra representation of the module
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
-
forward
(x, encoder_padding_mask: Optional[torch.Tensor] = None)[source]¶ Parameters: - x (Tensor) – input to the layer of shape (seq_len, batch, embed_dim)
- encoder_padding_mask (ByteTensor) – binary ByteTensor of shape
(batch, src_len) where padding elements are indicated by
1
.
Returns: encoded output of shape (batch, src_len, embed_dim)
-
pytext.models.seq_models.conv_model module¶
-
class
pytext.models.seq_models.conv_model.
CNNModel
(encoder, decoder, source_embedding)[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
-
forward
(src_tokens: torch.Tensor, additional_features: List[List[torch.Tensor]], src_lengths, prev_output_tokens, src_subword_begin_indices: Optional[torch.Tensor] = None) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.seq_models.light_conv module¶
-
class
pytext.models.seq_models.light_conv.
LightweightConv
(input_size, kernel_size, convolution_type: str, num_heads, weight_softmax, bias)[source]¶ Bases:
pytext.models.seq_models.base.PyTextIncrementalDecoderComponent
-
extra_repr
()[source]¶ Set the extra representation of the module
To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.
-
forward
(x, incremental_state: Optional[Dict[str, torch.Tensor]] = None)[source]¶ Assuming the input, x, of the shape T x B x C and producing an output in the shape T x B x C :param x: Input of shape T x B x C, i.e. (timesteps, batch_size, input_size) :param incremental_state: A dict to keep the state
-
pytext.models.seq_models.mask_generator module¶
-
class
pytext.models.seq_models.mask_generator.
BeamRankingAlgorithm
[source]¶ Bases:
enum.Enum
An enumeration.
-
AVERAGE_TOKEN_LPROB
= 'AVERAGE_TOKEN_LPROB'¶
-
LENGTH_CONDITIONED_AVERAGE_TOKEN_LPROB
= 'LENGTH_CONDITIONED_AVERAGE_TOKEN_LPROB'¶
-
LENGTH_CONDITIONED_AVERAGE_TOKEN_LPROB_MULTIPLIED
= 'LENGTH_CONDITIONED_AVERAGE_TOKEN_LPROB_MULTIPLIED'¶
-
LENGTH_CONDITIONED_RANK
= 'LENGTH_CONDITIONED_RANK'¶
-
LENGTH_CONDITIONED_RANK_MUL
= 'LENGTH_CONDITIONED_RANK_MUL'¶
-
LEN_ONLY
= 'LEN_ONLY'¶
-
TOKEN_LPROB
= 'TOKEN_LPROB'¶
-
-
class
pytext.models.seq_models.mask_generator.
EmbedQuantizeType
[source]¶ Bases:
enum.Enum
An enumeration.
-
BIT_4
= '4bit'¶
-
BIT_8
= '8bit'¶
-
NONE
= 'None'¶
-
-
class
pytext.models.seq_models.mask_generator.
MaskedSequenceGenerator
(config, model, length_prediction_model, trg_vocab, beam_size, use_gold_length, beam_ranking_algorithm, quantize, embed_quantize)[source]¶ Bases:
pytext.models.module.Module
-
forward
(src_tokens: torch.Tensor, dict_feats: Optional[Tuple[torch.Tensor, torch.Tensor, torch.Tensor]], contextual_embed: Optional[torch.Tensor], char_feats: Optional[torch.Tensor], src_lengths: torch.Tensor, src_subword_begin_indices: Optional[torch.Tensor] = None, target_lengths: Optional[torch.Tensor] = None, beam_size: Optional[int] = None, src_index_tokens: Optional[torch.Tensor] = None) → Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
classmethod
from_config
(config, model, length_prediction, trg_vocab, quantize=False, embed_quantize=False)[source]¶
-
generate_hypo
(tensors: Dict[str, torch.Tensor]) → Tuple[Tuple[torch.Tensor, torch.Tensor], torch.Tensor][source]¶ Generates hypotheses using beam search, also returning their scores
- Inputs:
- tensors: dictionary containing needed tensors for generation
- Outputs:
- (hypos, lens): tuple of Tensors
- hypos: Tensor of shape (batch_size, beam_size, MAX) containing the generated tokens. MAX refers to the longest sequence in batch.
- lens: Tensor of shape (batch_size, beam_size) containing generated sequence lengths
- _hypo_scores: Tensor of shape (batch_size, beam_size) containing the scores for each generated sequence
-
get_encoder_out
(src_tokens: torch.Tensor, dict_feats: Optional[Tuple[torch.Tensor, torch.Tensor, torch.Tensor]], contextual_embed: Optional[torch.Tensor], char_feats: Optional[torch.Tensor], src_subword_begin_indices: Optional[torch.Tensor], src_lengths: torch.Tensor, src_index_tokens: Optional[torch.Tensor] = None) → Dict[str, torch.Tensor][source]¶
-
-
pytext.models.seq_models.mask_generator.
avg_token_lprob
(token_lprob: torch.Tensor, length_lprob: torch.Tensor, target_lengths: torch.Tensor) → torch.Tensor[source]¶
-
pytext.models.seq_models.mask_generator.
get_beam_ranking_function
(ranking_algorithm: pytext.models.seq_models.mask_generator.BeamRankingAlgorithm)[source]¶
-
pytext.models.seq_models.mask_generator.
length
(token_lprob: torch.Tensor, length_lprob: torch.Tensor, target_lengths: torch.Tensor) → torch.Tensor[source]¶
-
pytext.models.seq_models.mask_generator.
length_conditioned_avg_lprob_rank
(token_lprob: torch.Tensor, length_lprob: torch.Tensor, target_lengths: torch.Tensor) → torch.Tensor[source]¶
-
pytext.models.seq_models.mask_generator.
length_conditioned_avg_lprob_rank_mul
(token_lprob: torch.Tensor, length_lprob: torch.Tensor, target_lengths: torch.Tensor) → torch.Tensor[source]¶
-
pytext.models.seq_models.mask_generator.
length_conditioned_rank
(token_lprob: torch.Tensor, length_lprob: torch.Tensor, target_lengths: torch.Tensor) → torch.Tensor[source]¶
-
pytext.models.seq_models.mask_generator.
length_conditioned_rank_mul
(token_lprob: torch.Tensor, length_lprob: torch.Tensor, target_lengths: torch.Tensor) → torch.Tensor[source]¶
pytext.models.seq_models.nar_length module¶
-
class
pytext.models.seq_models.nar_length.
ConvLengthPredictionModule
(embed_dim: int, conv_dim: int, max_target_positions: int, length_dropout: float, glu: bool, activation, pooling_type, conv_layers)[source]¶ Bases:
pytext.models.module.Module
-
forward
(x: torch.Tensor, encoder_padding_mask: Optional[torch.Tensor] = None)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.nar_length.
MaskedLengthPredictionModule
(embed_dim: int, length_hidden_dim: int, max_target_positions: int, length_dropout: float)[source]¶ Bases:
pytext.models.module.Module
-
forward
(x: torch.Tensor, encoder_padding_mask: Optional[torch.Tensor] = None) → Tuple[torch.Tensor, torch.Tensor][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.seq_models.nar_modules module¶
-
class
pytext.models.seq_models.nar_modules.
NAREncoderUtility
[source]¶ Bases:
object
-
prepare_for_nar_inference
(length_beam_size: int, encoder_out: Dict[str, torch.Tensor]) → Dict[str, torch.Tensor][source]¶ During masked NAR inference, multiple lengths are predicted for each item in the batch. Hence tiling has to be done in such a way that all new rows related to each item should be placed together. This is the assumption that we are following in the rest of the nar generation code. Eg: [row1, row2, row3] should be tiled as [row1, row1, row1, row2, row2, row2, row3, row3, row3] NOT [row1, row2, row3, row1, row2, row3, row1, row2, row3]
-
pytext.models.seq_models.nar_output_layer module¶
-
class
pytext.models.seq_models.nar_output_layer.
NARSeq2SeqOutputLayer
(target_names: Optional[List[str]] = None, loss_fn: Optional[pytext.loss.loss.Loss] = None, *args, **kwargs)[source]¶ Bases:
pytext.models.output_layers.output_layer_base.OutputLayerBase
Non-autoregressive seq2seq output layer.
-
classmethod
from_config
(config: pytext.models.seq_models.nar_output_layer.NARSeq2SeqOutputLayer.Config, vocab: pytext.data.utils.Vocabulary)[source]¶
-
get_loss
(model_outputs: Tuple[torch.Tensor, Dict[str, torch.Tensor]], targets: Tuple[Tuple[torch.Tensor, torch.Tensor], torch.Tensor], context: Dict[str, Any] = None, reduce=True) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶ label_logits: B x T x V_1 label_targets: B x T length_logits: B x V_2 length_targets: B
-
classmethod
pytext.models.seq_models.positional module¶
-
class
pytext.models.seq_models.positional.
LearnedPositionalEmbedding
(num_embeddings: int, embedding_dim: int, padding_idx: int)[source]¶ Bases:
torch.nn.modules.sparse.Embedding
This module learns positional embeddings up to a fixed maximum size. Padding ids are ignored by either offsetting based on padding_idx or by setting padding_idx to None and ensuring that the appropriate position ids are passed to the forward function.
-
class
pytext.models.seq_models.positional.
PostionalEmbedCombine
[source]¶ Bases:
enum.Enum
An enumeration.
-
CONCAT
= 'concat'¶
-
SUM
= 'sum'¶
-
-
class
pytext.models.seq_models.positional.
PostionalEmbedType
[source]¶ Bases:
enum.Enum
An enumeration.
-
HYBRID
= 'hybrid'¶
-
LEARNED
= 'learned'¶
-
SINUSOIDAL
= 'sinusoidal'¶
-
-
class
pytext.models.seq_models.positional.
SinusoidalPositionalEmbedding
(embedding_dim, padding_idx, init_size=124, learned_embed=False)[source]¶ Bases:
torch.nn.modules.module.Module
This module produces sinusoidal positional embeddings of any length.
Padding symbols are ignored.
-
pytext.models.seq_models.positional.
build_positional_embedding
(positional_embedding_type: pytext.models.seq_models.positional.PostionalEmbedType, combine_pos_embed: pytext.models.seq_models.positional.PostionalEmbedCombine, max_target_positions: int, input_embed_dim: int, embed_dim: int, padding_idx: int, no_token_positional_embeddings: bool)[source]¶
pytext.models.seq_models.projection_layers module¶
-
class
pytext.models.seq_models.projection_layers.
DecoderWithLinearOutputProjection
(src_dict, dst_dict, out_embed_dim=512, *args, **kwargs)[source]¶ Bases:
torch.nn.modules.module.Module
Simple linear projection from the hidden vector to vocab.
-
forward
(encoder_out: Dict[str, torch.Tensor], decoder_out: Tuple[torch.Tensor, Dict[str, torch.Tensor]], incremental_state: Optional[Dict[str, torch.Tensor]] = None)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.projection_layers.
DecoupledDecoderHead
(src_dict, dst_dict, out_embed_dim=512, encoder_hidden_dim=None, pointer_attention_heads=1, fixed_generation_vocab=None, attention_dropout=0.2, model_output_logprob=True)[source]¶ Bases:
torch.nn.modules.module.Module
-
fixed_generation_vocab_expanded
= typing_extensions.Final[torch.Tensor]¶
-
forward
(encoder_out: Dict[str, torch.Tensor], decoder_out: Tuple[torch.Tensor, Dict[str, torch.Tensor]], incremental_state: Optional[Dict[str, torch.Tensor]] = None) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶ B: Batch T_src: Length of source sequence T_trg: Length of target seuqence C: hidden dimension V_ont: Size of ontology vocabulary V_trg: Size of full target vocabulary
-
pytext.models.seq_models.rnn_decoder module¶
-
class
pytext.models.seq_models.rnn_decoder.
DecoderWithLinearOutputProjection
(out_vocab_size, out_embed_dim=512)[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
Common super class for decoder networks with output projection layers.
-
forward
(input_tokens, encoder_out: Dict[str, torch.Tensor], incremental_state: Optional[Dict[str, torch.Tensor]] = None, timestep: int = 0) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.rnn_decoder.
RNNDecoder
(out_vocab_size, embed_tokens, encoder_hidden_dim, embed_dim, hidden_dim, out_embed_dim, cell_type, num_layers, dropout_in, dropout_out, attention_type, attention_heads, first_layer_attention, averaging_encoder)[source]¶ Bases:
pytext.models.seq_models.rnn_decoder.RNNDecoderBase
,pytext.models.seq_models.rnn_decoder.DecoderWithLinearOutputProjection
-
class
pytext.models.seq_models.rnn_decoder.
RNNDecoderBase
(embed_tokens, encoder_hidden_dim, embed_dim, hidden_dim, out_embed_dim, cell_type, num_layers, dropout_in, dropout_out, attention_type, attention_heads, first_layer_attention, averaging_encoder)[source]¶ Bases:
pytext.models.seq_models.base.PyTextIncrementalDecoderComponent
RNN decoder with multihead attention. Attention is calculated using encoder output and output of decoder’s first RNN layerself. Attention is applied after first RNN layer and concatenated to input of subsequent layers.
-
forward_unprojected
(input_tokens, encoder_out: Dict[str, torch.Tensor], incremental_state: Optional[Dict[str, torch.Tensor]] = None) → Tuple[torch.Tensor, Dict[str, torch.Tensor]][source]¶
-
pytext.models.seq_models.rnn_encoder module¶
-
class
pytext.models.seq_models.rnn_encoder.
BiLSTM
(num_layers, bidirectional, embed_dim, hidden_dim, dropout)[source]¶ Bases:
torch.nn.modules.module.Module
Wrapper for nn.LSTM
Differences include: * weight initialization * the bidirectional option makes the first layer bidirectional only (and in that case the hidden dim is divided by 2)
-
forward
(embeddings: torch.Tensor, lengths: torch.Tensor, enforce_sorted: bool = True)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
pytext.models.seq_models.rnn_encoder.
LSTMSequenceEncoder
(embed_dim, hidden_dim, num_layers, dropout_in, dropout_out, bidirectional)[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
RNN encoder using nn.LSTM for cuDNN support / ONNX exportability.
-
forward
(src_tokens: torch.Tensor, embeddings: torch.Tensor, src_lengths) → Dict[str, torch.Tensor][source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.seq_models.rnn_encoder_decoder module¶
-
class
pytext.models.seq_models.rnn_encoder_decoder.
RNNModel
(encoder, decoder, source_embeddings)[source]¶ Bases:
pytext.models.seq_models.base.PyTextSeq2SeqModule
-
forward
(src_tokens: torch.Tensor, additional_features: List[List[torch.Tensor]], src_lengths, prev_output_tokens, incremental_state: Optional[Dict[str, torch.Tensor]] = None)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.seq_models.seq2seq_model module¶
-
class
pytext.models.seq_models.seq2seq_model.
Seq2SeqModel
(model: pytext.models.seq_models.rnn_encoder_decoder.RNNModel, output_layer: pytext.models.seq_models.seq2seq_output_layer.Seq2SeqOutputLayer, src_vocab: pytext.data.utils.Vocabulary, trg_vocab: pytext.data.utils.Vocabulary, dictfeat_vocab: pytext.data.utils.Vocabulary, generator_config=None)[source]¶ Bases:
pytext.models.model.Model
Sequence to sequence model using an encoder-decoder architecture.
-
arrange_model_inputs
(tensor_dict) → Tuple[torch.Tensor, Optional[Tuple[torch.Tensor, torch.Tensor, torch.Tensor]], torch.Tensor, torch.Tensor][source]¶
-
forward
(src_tokens: torch.Tensor, dict_feats: Optional[Tuple[torch.Tensor, torch.Tensor, torch.Tensor]], contextual_token_embedding: Optional[torch.Tensor], src_lengths: torch.Tensor, trg_tokens: torch.Tensor)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
pytext.models.seq_models.seq2seq_output_layer module¶
-
class
pytext.models.seq_models.seq2seq_output_layer.
Seq2SeqOutputLayer
(target_names: Optional[List[str]] = None, loss_fn: Optional[pytext.loss.loss.Loss] = None, *args, **kwargs)[source]¶ Bases:
pytext.models.output_layers.output_layer_base.OutputLayerBase
-
classmethod
from_config
(config: pytext.models.seq_models.seq2seq_output_layer.Seq2SeqOutputLayer.Config, vocab: pytext.data.utils.Vocabulary)[source]¶
-
get_loss
(model_outputs: Tuple[torch.Tensor, Dict[str, torch.Tensor]], targets: Tuple[torch.Tensor, torch.Tensor], context: Dict[str, Any] = None, reduce=True) → torch.Tensor[source]¶ Compute and return the loss given logits and targets.
Parameters: - logit (torch.Tensor) – Logits returned
Model
. - target (torch.Tensor) – True label/target to compute loss against.
- context (Optional[Dict[str, Any]]) – Context is a dictionary of items
that’s passed as additional metadata by the
DataHandler
. Defaults to None. - reduce (bool) – Whether to reduce loss over the batch. Defaults to True.
Returns: Model loss.
Return type: torch.Tensor
- logit (torch.Tensor) – Logits returned
-
classmethod
pytext.models.seq_models.seqnn module¶
-
class
pytext.models.seq_models.seqnn.
SeqNNModel
(embedding: pytext.models.embeddings.embedding_base.EmbeddingBase, representation: pytext.models.representations.representation_base.RepresentationBase, decoder: pytext.models.decoders.decoder_base.DecoderBase, output_layer: pytext.models.output_layers.output_layer_base.OutputLayerBase)[source]¶ Bases:
pytext.models.doc_model.DocModel
Classification model with sequence of utterances as input. It uses a docnn model (CNN or LSTM) to generate vector representation for each sequence, and then use an LSTM or BLSTM to capture the dynamics and produce labels for each sequence.
-
class
pytext.models.seq_models.seqnn.
SeqNNModel_Deprecated
(embedding: pytext.models.embeddings.embedding_base.EmbeddingBase, representation: pytext.models.representations.representation_base.RepresentationBase, decoder: pytext.models.decoders.decoder_base.DecoderBase, output_layer: pytext.models.output_layers.output_layer_base.OutputLayerBase)[source]¶ Bases:
pytext.models.model.Model
Classification model with sequence of utterances as input. It uses a docnn model (CNN or LSTM) to generate vector representation for each sequence, and then use an LSTM or BLSTM to capture the dynamics and produce labels for each sequence.
DEPRECATED: Use SeqNNModel
pytext.models.seq_models.utils module¶
-
pytext.models.seq_models.utils.
make_positions
(input, padding_idx: int)[source]¶ Replace non-padding symbols with their position numbers.
Position numbers begin at padding_idx+1. Padding symbols are ignored.
-
pytext.models.seq_models.utils.
prepare_full_key
(instance_id: str, key: str, secondary_key: Optional[str] = None)[source]¶