Neural Networks

Neural Network is a machine learning algorithm inspired from biological neurons. Neurons (also known as neurones, nerve cells and nerve fibers) are electrically excitable cells in the nervous system that function to process and transmit information,similarly neural network enable machine to perform activity like computer vision, language processing, decision making etc.

A newly created neural network model can be compared with human brain. More we train more that individual will learn. For example we want a child to classify between apple image and orange image so first we feed the brain with some images of apples and oranges and gradually that child will be trained enough to distinguish between the images.

A neural network consist of an input layer, output layer and hidden layers. Input layer takes the input, hidden layer process the input and output layers gives the output.

Preceptron

Forward propagation is the multiplication of the input with the randomly intialized weights of lower value and addition of bias (initialized with zero), adding the non linearity using the activation funtions (sigmoid, tanh, Relu). Then loss is calulated using loss function (crossentropy). Aim is to reduce the loss using gradient descent.

Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent, weights and bias are subtracted from the change of loss with respect to weight into learning rate (alphs).

If learning rate alpha is too small it will converge slowly.If alpha is too large loss of parameter may not decrease on every iteration and may not converge.