Ersin Yumer

I am a Staff Research Scientist at Uber ATG, starting a new research team in the San Francisco office (Yes, we are hiring! see below for more info). Previously, I was at Argo AI where I led the Perception Machine Learning team. Before that, I spent three years at Adobe Research. I have a PhD from Carnegie Mellon University, during which I visited Google Research for several summers as well.

My interests lie at the intersection of machine learning, and 3D computer vision. I develop end-to-end learning systems and holistic machine learning applications that bring signals of the visual world together: images, depth scans, videos, 3D shapes and points clouds. More specifically, I am actively working on deep structured inference from the limited view of the world obtained by a learning agent, under realistic assumptions of limited data and experience. To address this challenge, my work focuses on: (1) incorporating our knowledge about the real world into deep neural networks as custom physically-based differentiable layers, (2) exploiting synthetic data and augmentation to bootstrap lack of training data.

Research Opportunities at Uber ATG

Full-time: If you are looking to join Uber ATG as a full-time researcher, please reach out. A strong publication record in conferences such as NeurIPS, ICML, CVPR, ICCV, ECCV, ICRA, RSS, IROS is required. You can also apply here: Uber ATG Open Roles.

Internship: If you are a student interested in doing a research internship at Uber ATG and/or collaborating on a project with me, please reach out. It will be very helpful if you include a resume, and a summary of your current interests. You can also apply here: Uber ATG Open Roles.

Publications

2018

Generating Multimodal Human Dynamics with a Transformation based Representation.
Xinchen Yan, Akash Rastogi, Ruben Villegas, Eli Shechtman, K. Sunkavalli, S. Hadap, Ersin Yumer, Honglak Lee.
European Conference on Computer Vision (ECCV 2018).

BibTeX arXiv Project Page

Real-Time Hair Rendering using Sequential Adversarial Networks.
Lingyu Wei, Liwen Hu, Vladimir Kim, Ersin Yumer, Hao Li.
European Conference on Computer Vision (ECCV 2018).

BibTeX PDF Project Page [soon]

Real-Time Blind Video Temporal Consistency.
Wei-Sheng Lai, Jia-Bin Huang, Oliver Wang, Eli Shechtman, Ersin Yumer, Ming-Hsuan Yang.
European Conference on Computer Vision (ECCV 2018).

BibTeX arXiv Project Page

BodyNet: Volumetric Inference of 3D Human Body Shapes.
Gul Varol, Duygu Ceylan, Bryan Russell, Jimei Yang, Ersin Yumer, Ivan Laptev, Cordelia Schmid.
European Conference on Computer Vision (ECCV 2018).

BibTeX arXiv Project Page

SeeThrough: Finding Chairs in Heavily Occluded Indoor Scene Images.
Moos Hueting, Pradyumna Reddy, Ersin Yumer, Vladimir G. Kim, Nathan Carr, Niloy J. Mitra.
International Conference on 3D Vision (3DV 2018).

BibTeX arXiv Project Page

Learning Design Semantics for Mobile Apps.
Thomas Liu, Mark Craft, Jason Situ, Ersin Yumer, Radomir Mech, Ranjitha Kumar.
ACM Symposium on User Interface Software and Technology (UIST 2018).

BibTeX PDF

ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing.
Chen-Hsuan Lin, Ersin Yumer, Oliver Wang, Eli Shechtman, Simon Lucey.
Computer Vision and Pattern Recognition (CVPR 2018).

BibTeX arXiv Code (Github)

PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image.
Chen Liu, Jimei Yang, Duygu Ceylan, Ersin Yumer, Yasutaka Furukawa.
Computer Vision and Pattern Recognition (CVPR 2018). (SPOTLIGHT)

BibTeX arXiv Project Page (Code & Data)

Extracting Regular FOV Shots from 360 Event Footage.
Anh Truong, Sarah Chen, Ersin Yumer, Wilmot Li, David Salesin.
Human-Computer Interaction (CHI 2018).

BibTeX ACM Open Pub.

Learning Local Shape Descriptors with View-based Convolutional Neural Networks.
Haibin Huang, Evangelos Kalegorakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G. Kim, Ersin Yumer.
ACM Transactions on Graphics (To be presented at SIGGRAPH 2018).

BibTeX arXiv Project Page (Code & Data)

2017

Self-supervised Learning of Motion Capture.
Hsiao-Yu Tung , Wei Tung, Ersin Yumer, Katerina Fragkiadaki.
Neural Information Processing Systems (NeurIPS 2017). (SPOTLIGHT)

BibTeX arXiv Project Page (Code & Data)

Learning to Predict Indoor Illumination from a Single Image.
Marc-Andre Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Jean-Francois Lalonde.
ACM Transactions on Graphics (SIGGRAPH Asia 2017).

BibTeX arXiv

3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks.
Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, Derek Hoiem.
International Conference on Computer Vision (ICCV 2017).

BibTeX arXiv Code (Github) Data (.zip)

Material Editing using a Physically Based Rendering Network.
Guilin Liu, Duygu Ceylan, Ersin Yumer, Jimei Yang, Jyh-Ming Lien.
International Conference on Computer Vision (ICCV 2017).

BibTeX arXiv Project Page (Code & Data)

Learning Hierarchical Shape Segmentation and Labeling from Online Repositories.
Li Yi, Leonidas Guibas, Aaron Hertzmann, Vladimir G. Kim, Hao Su, Ersin Yumer.
ACM Transactions on Graphics (SIGGRAPH 2017).

BibTeX arXiv Project Page (Code & Data)

GRASS: Generative Recursive Autoencoders for Shape Structures.
Jun Li, Kevin Xu, Siddhartha Chaudhuri, Ersin Yumer, Richard Zhang, Leonidas Guibas.
ACM Transactions on Graphics (SIGGRAPH 2017).

BibTeX arXiv Project Page (Code & Data)

Convolutional Neural Networks on Surfaces via Seamless Toric Covers.
Haggai Maron, Meirav Galun, Noam Aigerman, Miri Trope, Nadav Dym, Ersin Yumer, Vladimir G. Kim, Yaron Lipman.
ACM Transactions on Graphics (SIGGRAPH 2017).

BibTeX PDF Code

Skippy: Single View 3D Curve Interactive Modeling.
Vojtech Krs, Ersin Yumer, Nathan Carr, Bedrich Benes, Radomir Mech.
ACM Transactions on Graphics (SIGGRAPH 2017).

BibTeX PDF Project Page

Neural Face Editing with Intrinsic Image Disentangling.
Zhixin Shu, Ersin Yumer, Sunil Hadap, Kalyan Sunkavalli, Eli Shechtman, Dimitris Samaras.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017). (ORAL)

BibTeX arXiv Project Page (Code & Data)

Physically-Based Rendering for Indoor Scene Understanding Using CNNs.
Yinda Zhang*, Shuran Song*, Ersin Yumer, Manolis Savva, J-Y Lee, Hailin Jin, Thomas Funkhouser.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017).
(*equal contribution)

BibTeX arXiv Project Page (Code & Data)

Transformation-Grounded Image Generation Network for Novel 3D View Synthesis.
Eunbyung Park, Jimei Yang, Ersin Yumer, Duygu Ceylan, Alex Berg.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017).

BibTeX arXiv Project Page (Code & Data)

Learning to Extract Semantic Structure from Documents Using Multimodal FCNNs.
Xiao Yang, Ersin Yumer, Paul Asente, Mike Kraley, Daniel Kifer, C. Lee Giles.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017). (SPOTLIGHT)

BibTeX arXiv Project Page (Data)

2016

Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision.
Xinchen Yan, Jimei Yang, Ersin Yumer, Yijie Guo, Honglak Lee.
Neural Information Processing Systems (NeurIPS 2016).

BibTeX arXiv Project Page (Code & Data)

Learning Semantic Deformation Flows with 3D Convolutional Networks.
Ersin Yumer, Niloy Mitra.
European Conference on Computer Vision (ECCV 2016).

DOI BibTeX Pdf Supplementary Pdf Project Page (Data)

Spectral Style Transfer for Human Motion between Independent Actions.
Ersin Yumer, Niloy Mitra.
ACM Transactions on Graphics (SIGGRAPH 2016).

ACM Open Access BibTeX Pdf Supplementary Pdf Video (Youtube)

Shape Synthesis from Sketches via Procedural Models and Convolutional Networks.
Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, Radomir Mech.
IEEE Transactions on Visualization and Computer Graphics, 2016.

DOI BibTeX Pdf Supplementary Pdf Code (.zip) Project Page

2015 and Earlier

Procedural Modeling Using Autoencoder Networks.
Ersin Yumer, Paul Asente, Radomir Mech, Levent Burak Kara.
ACM Symposium on User Interface Software and Technology (UIST 2015).

DOI BibTeX Pdf Video (Youtube)

Semantic Shape Editing Using Deformation Handles.
Ersin Yumer, Siddhartha Chaudhuri, Jessica K. Hodgins, Levent Burak Kara.
ACM Transactions on Graphics (SIGGRAPH 2015).

DOI BibTeX Pdf Supplementary Pdf Video (Youtube)

Co-Constrained Handles for Deformation in Shape Collections.
Ersin Yumer, Levent Burak Kara.
ACM Transactions on Graphics (SIGGRAPH Asia 2014).

DOI BibTeX Pdf Supplementary Pdf Video (Youtube)

Co-Segmentation of Textured 3D Shapes with Sparse Annotations.
Ersin Yumer, Won Chun, Ameesh Makadia.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014).

BibTeX Pdf Supplementary Pdf

Learning 3D Part Detection from Sparsely Labeled Data.
Ameesh Makadia, Ersin Yumer.
International Conference on 3D Vision (3DV 2014).

DOI BibTeX Pdf Supplementary Pdf

Co-Abstraction of Shape Collections.
Ersin Yumer, Levent Burak Kara.
ACM Transactions on Graphics (SIGGRAPH Asia 2012).

DOI BibTeX Pdf Supplementary Pdf Video (Youtube)

Surface Creation on Unstructured Point Sets Using Neural Networks.
Ersin Yumer, Levent Burak Kara.
Computer-Aided Design, 2012.

DOI BibTeX Pdf

Sketch-Based Image Editing Using Clothoid Curves.
Gunay Orbay, Ersin Yumer, Levent Burak Kara.
Journal of Visual Languages and Computing, 2012.

Conceptual Design and Modification of Freeform Surfaces Using Dual Shape Representations in Augmented Reality Environments.
Mark Fuge, Ersin Yumer, Gunay Orbay, Levent Burak Kara.
Computer-Aided Design, 2012.