MemoryEfficientFP16OptimizerFairseq.Config

Component: MemoryEfficientFP16OptimizerFairseq

class MemoryEfficientFP16OptimizerFairseq.Config

Bases: FP16Optimizer.Config

All Attributes (including base classes)

init_loss_scale: int = 128

scale_window: Optional[int] = None

scale_tolerance: float = 0.0

threshold_loss_scale: Optional[float] = None

min_loss_scale: float = 0.0001

Default JSON

{
    "init_loss_scale": 128,
    "scale_window": null,
    "scale_tolerance": 0.0,
    "threshold_loss_scale": null,
    "min_loss_scale": 0.0001
}