Neuron Correlation Hidden Layer

Modules

prosper_nn.utils.neuron_correlation_hidden_layers.corrcoef(m: Tensor, rowvar: bool = True) → Tensor[source]

Return Pearson product-moment correlation coefficients.

Implementation of the numpy function of the same name.

Parameters:

  • m (A 1-D or 2-D torch.Tensor containing multiple variables and observations.) – Each row of m represents a variable, and each column a single observation of all those variables.

  • rowvar (If rowvar is True, then each row represents a) – variable, with observations in the columns. Otherwise, the relationship is transposed: each column represents a variable, while the rows contain observations.

Returns:

The correlation matrix of the variables.

Return type:

torch.Tensor

prosper_nn.utils.neuron_correlation_hidden_layers.hl_size_analysis(output_layer: Tensor, min_absolute_corr: float = 0.5, print_values: bool = True, xlabel: str = 'Neuron Index', ylabel: str = 'Neuron Index', title: str = 'Correlation of neurons in layer') → List[Tensor][source]

Analyses the correlation of neurons in a layer to see if more neurons are needed. If the strongest correlation is small, it can be helpful to increase the number of neurons. Plots correlation matrix of layer neurons, gives correlation matrix and strongest correlation with corresponding neurons.

Parameters:

  • output_layer (torch.Tensor) – The output of the layer you want to investigate.

  • min_absolute_corr (float) – See plot_correlation. The default is 0.5.

  • print_values (boolean) – See find_max_correlation.

  • xlabel (str) – Set the label for the x-axis.

  • ylabel (str) – Set the label for the y-axis.

  • title (str) – Set a title for the axes.

Returns:

Contains the correlation matrix as a 2D PyTorch tensor, the value of the strongest correlation and the indices of the corresponding neurons.

Return type:

List[torch.Tensor]

prosper_nn.utils.neuron_correlation_hidden_layers.hl_size_analysis_Sequential(loaded_model: Sequential, model_input: Tensor, index_layer: int | None = None, name_layer: str | None = None, min_absolute_corr: float = 0.5, print_values: bool = True, xlabel: str = 'Neuron Index', ylabel: str = 'Neuron Index', title: str = 'Correlation of neurons in layer') → List[Tensor][source]

Analyses the correlation of neurons in a layer to see if more neurons are needed. If the strongest correlation is small, it can be helpful to increase the number of neurons. Plots correlation matrix of layer neurons, gives correlation matrix and strongest correlation with corresponding neurons. In contrast to hl_size_analysis, the layer output is automatically computed, using the loaded model, the model input and the name OR the index of the layer(module) in the model.

Be careful, always check the print-out to see if you are actually analyzing the model you want to analyze. If the layers in your model are not initialized in the order they are used in the sequential, you might analyze the wrong model.

Parameters:

  • loaded_model (nn.Sequential) – The loaded model which you want to analyze.

  • model_input (torch.Tensor) – The input for the model.

  • index_layer (integer) – The index of the layer you want to analyze in list(loaded_model.modules()). Bear in mind that the first entry in this list is the pre-trained model itself and that submodules cannot be analyzed individually but only the top level module they are part of. EITHER index_layer OR name_layer has to be forwarded.

  • name_layer (string) – The name of the layer you want to analyze. For this to suffice, all modules at the top level have to be named. Bear in mind that submodules cannot be analyzed individually but only the top level module they are part of. EITHER index_layer OR name_layer has to be forwarded.

  • min_absolute_corr (float) – See plot_correlation.

  • print_values (boolean) – See find_max_correlation.

  • xlabel (str) – Set the label for the x-axis.

  • ylabel (str) – Set the label for the y-axis.

  • title (str) – Set a title for the axes.

Returns:

Contains the correlation matrix as a 2D PyTorch tensor, the value of the strongest correlation and the indices of the corresponding neurons.

Return type:

List[torch.Tensor]

Example

X = torch.rand([10, 100])
net = prosper_nn.models.feedforward.FFNN(100, 5, 1)

# with trained model
net.eval()
corr_matrix2, max_corr2, ind_neurons2 = hl_size_analysis_Sequential(
    net, name_layer="hidden", model_input=X)