Historical Trends in Deep Learning

(17th-April-2021)

• It is easiest to understand deep learning with some historical context. Rather than providing a detailed history of deep learning, we identify a few key trends:

• Deep learning has had a long and rich history, but has gone by many names reflecting different philosophical viewpoints, and has waxed and waned in popularity.

• Deep learning has become more useful as the amount of available training data has increased.

• Deep learning models have grown in size over time as computer infrastructure (both hardware and software) for deep learning has improved.

• Deep learning has solved increasingly complicated applications with increasing accuracy over time.

The high-level organization