weight_initialization(model)

Initializes the weights of a given linear model using a normal distribution. This function is typically used to initialize the weights of models in a neural network before training begins.

Parameters:
  • model (Module) –

    The model whose weights will be initialized. The initialization is performed in-place and only affects linear layers.

Process
  • If the model is an instance of nn.Linear, its weights are initialized using a normal distribution with mean 0 and standard deviation 0.02.
Side Effects
  • Modifies the model's weights in-place, affecting only the linear layers (nn.Linear instances).
Example

model = nn.Linear(10, 2) weight_initialization(model)

Model weights are now initialized.

Source code in utils.py
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
def weight_initialization(model):
    """
    Initializes the weights of a given linear model using a normal distribution.
    This function is typically used to initialize the weights of models in a neural network before training begins.

    Args:
        model (nn.Module): The model whose weights will be initialized. The initialization is performed in-place and only affects linear layers.

    Process:
        - If the model is an instance of nn.Linear, its weights are initialized using a normal distribution with mean 0 and standard deviation 0.02.

    Side Effects:
        - Modifies the model's weights in-place, affecting only the linear layers (nn.Linear instances).

    Example:
        >>> model = nn.Linear(10, 2)
        >>> weight_initialization(model)
        # Model weights are now initialized.
    """
    if isinstance(model, nn.Linear):
        init.normal_(model.weight.data, mean=0, std=0.02)