Deep Feed Forward Neural Network

Module

class prosper_nn.models.deep_feed_forward.deep_feed_forward.DeepFeedForward(input_dim: int, hidden_dim: int, output_dim: int, deepness: int, activation: ~typing.Type[~torch.autograd.function.Function] = <built-in method tanh of type object>, dropout_rate: float = 0.0)[source]

Bases: Module

Create a Deep Feed Forward Neural Network

Parameters:

  • input_dim (int) – The input dimension of the model.

  • hidden_dim (int) – The size of the hidden layer. All hidden layers have the same dimension.

  • output_dim (int) – The output dimension of the returned Deep-Feed-Forward Neural Network. It is equal for all levels of deepness.

  • deepness (int) – The number of hidden layers in the Neural Network. It corresponds with the amount of Feed Forward Neural Network paths to an output layer. A deepness equals one leads to a normal Feed-Forward Neural Network.

  • activation (nn.functional, optional) – The activation function that is applied on the output of the hidden layers. The same function is used on all hidden layers. No function is applied if no function is given.

  • dropout_rate (float, optional) – The rate to which input nodes of the model are set so zero during training of the model. The value of the dropout_rate should be in the range of [0, 1). If the value is zero or no value is given, no dropout is applied.

Return type:

None

forward(x: Tensor) → tensor[source]
Parameters:

x (torch.Tensor) – The input tensor given to the Deep-Feed-Forward Neural Network. It should have shape=(batchsize, input_dim).

Returns:

A output of a level in the model is stored in the tensor with the index that is equal to the level. Therefore the first dimension corresponds to the level of deepness in the model. The complete output shape is: shape=(deepness, batchsize, output_dim).

Return type:

torch.Tensor

Example

import torch
from prosper_nn.models.deep_feed_forward.deep_feed_forward import DeepFeedForward

# Set model and data parameter
input_dim = 10
hidden_dim = 15
output_dim = 1
n_batches = 100
batchsize = 5
deepness = 3

# Initialise Deep Feedforward Neural Network
deepff = DeepFeedForward(input_dim=input_dim,
                         hidden_dim=hidden_dim,
                         output_dim=output_dim,
                         deepness=deepness)

X = torch.randn([n_batches, batchsize, input_dim])
Y = torch.randn([n_batches, batchsize, output_dim])

# Train Model
optimizer = torch.optim.Adam(deepff.parameters())
loss_function = torch.nn.MSELoss()

for epoch in range(10):
    for x, y in zip(X, Y):
        output = deepff(x)

        deepff.zero_grad()
        loss = sum([loss_function(output[i], y) for i in range(deepness)]) / deepness
        loss.backward()
        optimizer.step()