TransformerRepresentation.Config

Component: TransformerRepresentation

class TransformerRepresentation.Config

Bases: ConfigBase

All Attributes (including base classes)

num_layers: int = 3

num_attention_heads: int = 4

ffnn_embed_dim: int = 32

dropout: float = 0.0

Default JSON

{
    "num_layers": 3,
    "num_attention_heads": 4,
    "ffnn_embed_dim": 32,
    "dropout": 0.0
}