Feed Forward and Back Propagation Neural Network ( Method/Example )

Example for classification using Neural Network

FEEDFORWARD STEPS
 I. INPUT LAYER OUTPUTS
II. HIDDEN LAYER INPUT
III. HIDDEN LAYER NEURON OUTPUT
IV. INPUT  TO OUTPUT LAYER
V. OUTPUT LAYER NEURON OUTPUT
VI. CALCULATE ERROR FOR OUTPUT LAYER
VII. FIND THE ERROR FOR HIDDEN LAYER NEURON


BACKPROPAGATION STEPS
I. UPDATE THE WEIGHTS CONNECTING HIDDEN LAYER TO OUTPUT LAYER
II. UPDATE THE WEIGHTS CONNECTING INPUT LAYER TO HIDDEN LAYER

FEEDFORWARD

I. INPUT LAYER OUTPUTS

Input of each neuron will be compared with threshold of each neuron. Since possible inputs are 0 and 1 we need not to compare it to threshold value we can directly take them as output. Hence no need of threshold and comparision.
Output of N1:    1
Output of N2:    1
Output of N3:    0
Output of N4:    0

II. HIDDEN LAYER INPUT

Input to N5: =(1*0.3) + (1*0.4) +(0*0.5) +(0*0.5)  = 0.7   

III. HIDDEN LAYER NEURON OUTPUT

N5 and N6 function as below:

Ex:{NOTE: a=1.2 ; activation function: sigmoid i.e
y=1/(1+e^(-x));  output=0.76852478349; }


IV. INPUT  TO OUTPUT LAYER
Input to N8=(0.768*0.23)+(0.322*0.5)+(0.678*0.85)
                    =0.91394

V. OUTPUT LAYER NEURON OUTPUT

N5 and N6 function as below: Ex:


{NOTE: a=1.41394 ; activation function: sigmoid i.e
y=1/(1+e^(-x));  output=0.80376594363; }


VI. CALCULATE ERROR FOR OUTPUT LAYER

OUTPUT_ERROR = (TARGET – ACTUAL) * sigmoid_derivative (ACTUAL)

N8 NEURON ERROR = (1-0.8037)*sig_der(0.8037)
                      =  0.03101685654
{Note: sigmoid_derivative(val)=(val*(1-val))}

N8 NEURON ERROR =0.03101
N9 NEURON ERROR =0.76


VII. FIND THE ERROR FOR HIDDEN LAYER NEURON

n8_error=0.03101   n9_error=0.76
        weight_five_to_eight=0.23;
        weight_five_to_nine=0.88;

n5_error=[ ( n8_error * weight_five_to_eight ) +
            ( n9_error * weight_five_to_nine ) ] * derivative_of_actn_func(N5_output)



BACKPROPAGATION
 I. UPDATE THE WEIGHTS CONNECTING HIDDEN LAYER TO OUTPUT LAYER
This is the output from output layer

we have
n5_output=0.768;
n6_output=0.322;
n7_output=0.678;
weight_five_to_eight=0.23;
weight_five_to_nine=0.88;

n8_error=0.03101   n9_error=0.76

weight updates:
weight_five_to_eight+=(learning_rate*n8_error* n5_output)
weight_five_to_nine+=(learning_rate*n9_error* n5_output)

II. UPDATE THE WEIGHTS CONNECTING INPUT LAYER TO HIDDEN LAYER
weight_one_to_five=0.3
weight_one_to_six=0.7
weight_one_to_seven=0.25

weight_one_to_five+=(learn_rate*n5_error*output_f_n1)+
                    (learn_rate*n6_error*output_f_n1)+
                     (learn_rate*n7_error*output_f_n1)