Bardienus Duisterhof

Robotics + 3D perception PhD at CMU. I build systems that help robots see and reason in 3D.

About

I am a final-year Ph.D. student at Carnegie Mellon University (CMU)'s Robotics Institute, advised by Jeffrey Ichnowski, and I frequently collaborate with Deva Ramanan and Bowen Wen. I am currently a research intern at World Labs on the pre-training team, under Justin Johnson. Previously, I was a research intern in the DUSt3R Group at NAVER Labs Europe, advised by Jérome Revaud and Vincent Leroy. My interests lie at the intersection of perception and robot manipulation of challenging objects, such as transparent and deformable items.

During my first year at CMU I worked with Sebastian Scherer on geometric camera calibration. Prior to CMU I completed a Bachelor's and Master's degree in Aerospace Engineering at Delft University of Technology, advised by Guido de Croon, studying efficient bio-inspired algorithms for fully autonomous nano drones. In 2019 I was a visiting student at Vijay Janapa Reddi's Edge Computing lab at Harvard University. I am a recipient of the 2023 CMLH Fellowship in Digital Health Innovation and the 2024 CMLH Fellowship in Generative AI in Healthcare.

I'm on the job market for a research scientist or similar role.

Selected Research
Modality Forcing for Scalable Spatial Generation

Bardienus P. Duisterhof, Deva Ramanan, Jeffrey Ichnowski, Justin Johnson, Keunhong Park

arXiv preprint, 2026

Modality Forcing turns a pretrained text-to-image model into a joint image-depth generator with a simple post-training recipe: one DiT, separate noise levels per modality, and per-modality decoders that allow training on sparse real-world depth. Depth accuracy scales with T2I pre-training (300M → 3B), and our strongest model is competitive with state-of-the-art monocular depth estimators, reducing AbsRel by 57% over prior joint image-depth generative models.

3PoinTr: 3D Point Tracks for Learning Manipulation from Unconstrained Human Videos

Adam Hung, Bardienus P. Duisterhof, Jeffrey Ichnowski

arXiv preprint, 2026

3PoinTr learns manipulation from unconstrained human videos: videos where the human demonstrator can act freely rather than mimicking target robot kinematics. 3PoinTr first predicts dense 3D point tracks — how the scene should move to complete the task — and then conditions a closed-loop multitask policy on these tracks. 3PoinTr outperforms strong behavior cloning and learning-from-video baselines across simulated and real-world evaluations.

Wiggle and Go! System Identification for Zero-Shot Dynamic Rope Manipulation

Arthur Jakobsson, Abhinav Mahajan, Karthik Pullalarevu, Krishna Suresh, Yunchao Yao, Yuemin Mao, Bardienus P. Duisterhof, Shahram Syed, Jeffrey Ichnowski

Preprint, 2025

Wiggle and Go! enables zero-shot dynamic rope manipulation through system identification: the robot first wiggles a rope to identify its physical parameters, then uses learned simulation priors to plan an accurate, goal-conditioned throw — without large real-world datasets or iterative retries. The two-stage framework completes dynamic rope tasks accurately on the first attempt across varied ropes and payloads.

RaySt3R: Predicting Novel Depth Maps for Zero-Shot Object Completion

Bardienus P. Duisterhof, Jan Oberst, Bowen Wen, Stan Birchfield, Deva Ramanan, Jeffrey Ichnowski

NeurIPS 2025

Imagine if robots could fill in the blanks in cluttered scenes. Enter RaySt3R ✨: a single masked RGB-D image in, complete 3D out. It infers depth, object masks, and confidence for novel views, then merges the predictions into a single point cloud.

MASt3R-SfM teaser
MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion

Bardienus P. Duisterhof*, Lojze Zust*, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, Jérome Revaud

International Conference on 3D Vision (3DV) 2025, Oral, Best Student Paper Award

MASt3R for SfM with 1000+ unordered images. We contribute a memory-efficient algorithm that leverages the MASt3R encoder for image retrieval without any overhead. MASt3R-SfM has overall linear complexity in the number of images, and handles any set of ordered or unordered images.

DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation

Bardienus P. Duisterhof, Zhao Mandi, Yunchao Yao, Jia-Wei Liu, Jenny Seidenschwarz, Mike Zheng Shou, Deva Ramanan, Shuran Song, Stan Birchfield, Bowen Wen, Jeffrey Ichnowski

Proc. Algorithmic Foundations of Robotics (WAFR) 2024

Deformable objects are common in household, industrial, and healthcare settings; tracking them would unlock applications across robotics, gen-AI, and AR. DeformGS performs dense 3D tracking and dynamic novel-view synthesis on real-world deformable cloths.

DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstruction

Jenny Seidenschwarz, Qunjie Zhou, Bardienus P. Duisterhof, Deva Ramanan, Laura Leal-Taixé

International Conference on 3D Vision (3DV) 2025

Online 3D tracking can unlock many new applications in robotics, AR, and VR. Most prior work targets offline tracking on full sequences. DynOMo simultaneously performs 3D tracking, 3D reconstruction, novel-view synthesis, and pose estimation.

Residual-NeRF: Learning Residual NeRFs for Transparent Object Manipulation

Bardienus P. Duisterhof, Yuemin Mao, Si Heng Teng, Jeffrey Ichnowski

IEEE International Conference on Robotics and Automation (ICRA) 2024
🌟 Spotlight 🌟 presentation at the ICCV23 — TRICKY Workshop

Residual-NeRF improves depth perception and training speed for transparent objects. By first learning a background NeRF of the workspace without the transparent objects to be manipulated, we recover better depth quality and faster convergence.

All Publications

The complete bibliography. The selected works above are repeated here in compact form.

News
Show earlier news
  • Aug 2024 DeformGS has been accepted at WAFR 2024!
  • July 2024 I started my internship at NAVER Labs Europe! Excited to work with Jérome Revaud, Vincent Leroy and the rest of the DUSt3R team.
  • Jan 2024 2 papers accepted at ICRA 2024! See you in Japan 🇯🇵.
  • Nov 2023 Check out our recent work on MD-Splatting, a method for dense tracking and novel view synthesis of cloth 🧣.
  • July 2023 Our paper on NeRFs for transparent objects has been accepted for a 🌟 spotlight 🌟 presentation at the ICCV23 — TRICKY Workshop.
  • April 2023 Thanks to the CMLH Fellowship in Digital Health Innovation for generously supporting my research!