Question

Consider a fully connected neural network model taking 4 input data attributes with one hidden layer of 16 neurons and 1 neuron in the output layer. How many parameters are needed to specify the model? Explain in depth

Answer 1

A neural network can be thought of as a network of “neurons” which are organised in layers. The predictors (or inputs) form the bottom layer, and the forecasts (or outputs) form the top layer. There may also be intermediate layers containing “hidden neurons”.

In the given definition of model,

input layes - 4

hidden layers - 1 with 16 neurons

output layer - 1

Main parameters needed:

The leftmost layer, known as the input layer, consists of a set of neurons {xi|x1,x2,x3,x4} representing the input features.

Each neuron in the hidden layer transforms the values from the previous layer with a weighted linear summation w1x1+w2x2+...+w4x4, followed by a non-linear activation function g(⋅):R→R - like the hyperbolic tan function.

The output layer receives the values from the last hidden layer and transforms them into output value, y.

List of weight matrices, where weight matrix at index i represents the weights between layer i and layer i+1.

Llist of bias vectors, where the vector at index i represents the bias values added to layer i+1.

Initially here we need parameters:

- input X = [x1,x2,x3,x4]

- a weight matrix of 4X16 , W as:

.....................................

.............................. ]

and bias matrix b of 1X16 as:

[b1,b2,.........................b16]