Condensed Matter Seminar: What Makes Data Suitable for Deep Learning?

Deep learning is delivering unprecedented performance when applied to various data modalities, yet there are data distributions over which it utterly fails. The question of what makes a data distribution suitable for deep learning is a fundamental open problem in the field.  In this talk I will present a recent theory aiming to address the problem via tools from quantum physics.  The theory establishes that certain neural networks are capable of accurate prediction over a data distribution if and only if the data distribution admits low quantum entanglement under certain partitions of features.  This brings forth practical methods for adaptation of data to neural networks, and vice versa.  Experiments with widespread models over various datasets will demonstrate the findings.  An underlying theme of the talk will be the potential of physics to advance our understanding of the relation between deep learning and real-world data.

Works covered in the talk were in collaboration with my graduate students Noam Razin, Yotam Alexander, Nimrod De La Vega and Tom Verbin.