It’s shocking how a few of the primary topics in machine studying are nonetheless unknown by researchers and regardless of being basic and customary to make use of, appear to be mysterious. It’s a enjoyable factor about machine studying that we construct issues that work after which determine why they work in any respect!
Right here, I goal to research the unknown territory in some machine studying ideas in an effort to present whereas these concepts can appear primary, in actuality, they’re constructed by layers upon layers of abstraction. This helps us to observe questioning the depth of our information.
In this text, we discover a number of key phenomena in deep studying that problem our conventional understanding of neural networks.
- We begin with Batch Normalization and its underlying mechanisms that stay not totally understood.
- We study the counterintuitive statement that overparameterized fashions usually generalize higher, contradicting the classical machine studying theories.
- We discover the implicit regularization results of gradient descent, which appear to naturally bias neural networks in direction of less complicated, extra…
