250 Hutchison Rd, Rochester, NY 14620


Understanding Human Material Perception Using Deep Generative Models 


Unsupervised learning with Generative Adversarial Networks (GAN) provides a powerful approach in automatic image generation and a plausible framework for understanding human vision.  This talk will discuss how a Style-Based Generative Network (StyleGAN) can identify latent representation of material appearance. In everyday life, we encounter materials with complex appearances, like textiles and foodstuffs. They pose extraordinary challenges for theories of vision. Different from object recognition, there is little semantically labeled image data of materials, partially due to the large variations of material appearance under different scene contexts such as lighting, 3D shapes, viewing points, and a lack of precise and sufficient verbal descriptions. To mediate this, we train an StyleGAN on large unlabeled photographs of real-world materials and learn a latent space that can produce perceptually realistic images. The scale-specific latent space also allows us to manipulate images along certain scene features (changing material properties without varying 3D shapes) and predict material attributes of novel images. I will discuss how these model predictions compare with human translucency perception. Furthermore, we generate images of several distinct materials with unique visual qualities via transfer learning. This allows us to create novel material appearances by linearly interpolating between the learned models. Using this expanded and morphable space, I will compare the representation derived from human visual similarity of materials, human semantic descriptions, and the image-based representation derived from the GAN’s learned latent code. Together, this reveals the relationship between high-level interpretation and mid-level image features in material perception.  



Bei Xiao is an associate professor of computer science and a faculty member of the Center of Behavioral Neuroscience at American University. Her research lies in the intersection between human vision and computer vision.  Prior to joining AU, she was a postdoctoral associate in Brain and Cognitive Science and CSAIL at MIT. She received her PhD in Neuroscience from the University of Pennsylvania. Her research has been funded by NIH, NSF, and Google Research.  More about her research can be found at: https://sites.google.com/site/beixiao/

Event Details

  • Abhishek Tyagi

1 person is interested in this event



Meeting ID: 992 1227 7995

User Activity

No recent activity