Article Source

• Title: Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Authors: Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Figure 1: The generator of 3D Generative Adversarial Network (3D-GAN)

Figure 2: Shapes synthesized by 3D-GAN

Abstract

We study the problem of 3D object generation. We propose a novel framework, namely 3D Generative Adversarial Network (3D-GAN), which generates 3D objects from a probabilistic space by leveraging recent advances in volumetric convolutional networks and generative adversarial nets. The benefits of our model are three-fold: first, the use of an adversarial criterion, instead of traditional heuristic criteria, enables the generator to capture object structure implicitly and to synthesize high-quality 3D objects; second, the generator establishes a mapping from a low-dimensional probabilistic space to the space of 3D objects, so that we can sample objects without a reference image or CAD models, and explore the 3D object manifold; third, the adversarial discriminator provides a powerful 3D shape descriptor which, learned without supervision, has wide applications in 3D object recognition. Experiments demonstrate that our method generates high-quality 3D objects, and our unsupervisedly learned features achieve impressive performance on 3D object recognition, comparable with those of supervised learning methods.

Spotlight Video

If you cannot access YouTube, please download our video here.

Also see “AI Makes 3D Models From Photos” from Two Minute Papers on YouTube.

Publication

• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling, Jiajun Wu*, Chengkai Zhang*, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum

NIPS 2016 Paper Slides Poster arXiv BibTeX (* indicates equal contributions)

Downloads

• Pretrained models and code for shape synthesis

Related Publication

• MarrNet: 3D Shape Reconstruction via 2.5D Sketches
Jiajun Wu*, Yifan Wang*, Tianfan Xue, Xingyuan Sun, William T. Freeman, and Joshua B. Tenenbaum

NIPS 2017 Paper Project Page BibTeX (* indicates equal contributions)