Strange Geometry in High Dimensions and its Implication for Machine Learning

Tom Fletcher
Department of Electrical and Computer Engineering, Department of Computer Science
University of Virginia

Abstract Modern data lives in high dimensions, e.g., the number of pixels in an image, the number of words in a document, etc. In this talk, I will present some of the geometric oddities of random samples in high-dimensional Euclidean space. Our intuition about distances, angles, and volumes, which we acquire from 2D and 3D reasoning, doesn’t serve us well in higher dimensions. This has important implications for machine learning. One of the most famous is the existence of adversarial examples, which are data that can be slightly perturbed to change a correct classification into an incorrect one. I will outline a couple of existing conjectures for how high-dimensional geometry leads to adversarial examples, but also argue that these explanations are not fully satisfactory. Finally, I will present some recent work on how to detect vulnerability to adversarial attacks using nonlinear manifold geometry.