History of NN

(20th-September-2020)

History of Neural network is following time line.

Difficulty in learning

• Over learning: The learning error is small but the test error does not decrease. - When initializing the connection weights randomly and starting.

• Empirical knowledge = "Information does not propagate to the lower layers" - "Gradients spread"

To represent the learning data, only the upper layer suffices