Information Theory and Foundations of the Learning Behaviour of Deep Neural Networks

We believe that an information theoretic view of deep learning can give us an excellent framework to describe and understand the dynamics of learning. We study deep neural networks with statistical mechanics and information theory to uncover the emergence of structure during the learning process.