Ayan Chakrabarti: Learning Visual Inference and Measurement

Computer vision systems must solve ill-posed estimation problems by relying on machine learning tools to leverage the statistical structure present in natural scenes.

March 28, 2017
4:30 pm - 5:30 pm
Location
Kemeny Hall 007
Sponsored by
Computer Science Department
Audience
Public
More information
Sandra Hall

Computer vision systems must solve ill-posed estimation problems by relying on machine learning tools to leverage the statistical structure present in natural scenes. In the first part of my talk, I will describe a new framework for structured inference. This framework is designed to allow reliable inference of continuous-valued maps of scene properties---such as depth, shape, reflectance, motion, etc.---from image data, while enforcing different notions of spatial structure at different scales across regions of the visual field. In this context, I will describe a method that uses this framework to estimate scene depth from a single color image, by training a neural network to produce dense probabilistic estimates of different elements of local geometric structure, and harmonizing these estimates to produce consistent depth maps.

Beyond building better computational methods for inference, designing optimized optical sensors and measurement strategies can also contribute to the success and accuracy of vision systems. In the second part of the talk, I will describe an approach for learning, automatically from data, aspects of a camera's optical design. Using color imaging as an example task, I will introduce a framework where the camera's measurement process is encoded as a neural-network "layer", whose learnable weights parameterize the possible measurement choices for the sensor. This measurement layer is then trained end-to-end with a network that carries out inference on the corresponding measurements, with the objective of maximizing the quality of the final output. I'll show that this approach is able to automatically discover a measurement strategy that, when used with the jointly learned inference network, significantly outperforms traditional sensor designs.

Bio: Ayan Chakrabarti is currently a Research Assistant Professor at the Toyota Technological Institute at Chicago. He completed his PhD. in Engineering Sciences from Harvard University in 2011, advised by Prof. Todd Zickler, and was a post-doctoral fellow at Harvard from 2011-14. Dr. Chakrabarti works on applying tools from machine learning to problems in computer vision and computational photography---dealing with the design of accurate and efficient algorithms for visual inference, and of new kinds of high-capability sensors and cameras. His research seeks solutions to these problems by considering both the statistical structure of natural images and scenes, and the physics of image formation.

Location
Kemeny Hall 007
Sponsored by
Computer Science Department
Audience
Public
More information
Sandra Hall