model.shattered_gradients

Attributes

weights

Classes

ShatteredGradientsNN

Implements a neural network model using a linear layer followed by an activation function.

Module Contents

class model.shattered_gradients.ShatteredGradientsNN(weights: torch.Tensor, act_fun: str = 'Relu')

Bases: torch.nn.Sequential

Implements a neural network model using a linear layer followed by an activation function.

This model is designed to exhibit shattered gradients. To generate a model that exhibits shattered gradients, use shattered_grad.py, located in datagenerator.

Inherits from:

torch.nn.Sequential: Parent class for implementing neural networks with modules defined in a sequential manner.

Initializes a ShatteredGradientsNN object.

Parameters:

  • weights (torch.Tensor) – The weights to be applied to the linear layer.

  • act_fun (str) – The activation function to be used. Valid options are “Relu”, “Gelu”, or “Sigmoid”. Defaults to ‘Relu’.

_default_activation_function(act_fun: str) → torch.nn.Module

Returns the default activation function based on the provided string.

The _default_activation_function method maps the provided activation function name to the corresponding PyTorch activation function module. Supported activation functions include “Relu” (ReLU), “Gelu” (GELU), and “Sigmoid” (Sigmoid). If the provided name is not in the supported list, a KeyError is raised.

Parameters:

act_fun (str) – The activation function name or class.

Returns:

The corresponding PyTorch activation function module.

Return type:

torch.nn.Module

Raises:

KeyError – If the given activation function is not supported.

model.shattered_gradients.weights