--- title: "Giving Robots the Power of Perception: Interacting with the Physical World – Employee Interview: Computer Vision Lead" description: "Meet Jeronimo, Lead Computer Vision Engineer who studied under Prof. Takeo Kanade at CMU, debugged a critical trade show demo straight off an overseas flight at Narita, and built Mujin's foundational 3D vision stack from scratch." type: blog publishDate: "2021-06-22" tags: ["Interview", "ComputerVision", "Engineering", "MachineIntelligence"] --- Welcome to another edition of our "Mujinians Voice" series! Today, we sit down with **Jeronimo**, who leads our Computer Vision engineering team. He worked on the same robotics project with Mujin CTO Rosen Diankov at Carnegie Mellon University (CMU) and later joined Mujin as one of our very first core engineers. From academic research at CMU to debugging mission-critical robotics demos on the frontline, Jeronimo reflects on our founding years, the philosophy of machine perception, and what it takes to build vision systems that never fail.
Jeronimo, Lead Computer Vision Engineer
Computer Vision Team Lead: Jeronimo
**From CMU to Tokyo** Mentored by Prof. Takeo Kanade
Studied under the "father of computer vision" and collaborated with CTO Rosen on generic 3D pose estimation for shiny automotive components. **The Narita 6 AM Rescue** DMS 2014 Crisis Averted
Flew across the Pacific after an emergency call 2 days before the expo, reaching the trade floor to get the demo live 2 hours before opening. **Perception Loop & Autonomy** Transforming Machines into Robots
Without perception, hardware is just a blind machine waiting for human buttons. Real-time 3D modeling enables genuine machine intelligence. --- ## 1. Early Life in Portugal & Carnegie Mellon University *-- Could you tell us about your background and how you entered robotics?* > 💬 **Jeronimo:** > I was born in Portugal. While growing up, I developed a deep fascination with physics, computers, and mathematics. For my undergraduate studies, I majored in Electrical and Computer Engineering, covering power grids, computer science, electronics, telecommunications, and robotics. > > When pursuing my master's degree, I chose to specialize in robotics. While exploring diverse topics such as autonomous navigation, distributed formation control, and system identification, my primary research focus crystallized around **computer vision for shape recognition, object detection, and 3D classification**. > > I had the privilege of presenting our work at premier international conferences including IEEE CVPR (Conference on Computer Vision and Pattern Recognition). That breakthrough opened doors to doctoral research at the Robotics Institute (RI) at Carnegie Mellon University. --- ## 2. Why Computer Vision? The Line Between "Machines" and "Robots" *-- Why did you become specifically drawn to computer vision?* > 💬 **Jeronimo:** > I was always captivated by robots and relentlessly searched for ways to make them truly autonomous. In human daily life, vision plays the defining role in nearly every physical action. I realized that applying that same perception loop to robotic systems would be the cornerstone of real-world autonomy. :::note[The Fundamental Line Separating Robots from Machines] The definitive difference between a true "robot" and a simple "machine" is the presence of an active **Perception Loop** commanding physical actions. If a machine cannot perceive its surroundings, a human operator must push buttons or feed pre-programmed trajectories. In that case, it is merely automated factory machinery with severely constrained operational flexibility. ::: > 💬 **Jeronimo:** > By empowering robots with computer vision, they gain the ability to perceive, dynamically model, and interact with the physical world in real time. > > A smart robot can map an unfamiliar space, localize its own 6-DoF pose, plan collision-free paths, dynamically react to shifting payloads, manipulate delicate objects, and orchestrate other automation systems. **Giving robots the power of vision transforms them from blind mechanisms into intelligent collaborators.** --- ## 3. Meeting CTO Rosen: "Coding Faster Than I Could Move My Fingers" *-- How did you first meet Mujin CTO Rosen Diankov?*
Rosen setting up research computing hardware at Mujin
Rosen setting up the first computing hardware after joining Mujin
Dinner with Prof. Takeo Kanade and Rosen at CMU
Dinner with Prof. Takeo Kanade (right) and Rosen (center) during the CMU Robotics Institute years
*-- What was it like collaborating with Rosen at CMU?* > 💬 **Jeronimo:** > Toward the end of my master's studies, I met Prof. Takeo Kanade—widely recognized across the academic world as the "father of computer vision"—who was a prominent advisor at CMU's Robotics Institute and later became Mujin's Chief Technology Advisor. Speaking with Prof. Kanade convinced me to enter CMU for my doctorate, where I met Rosen. > > From day one, I was awestruck by Rosen's **unmatched intensity, tireless work ethic, and ability to conquer extraordinarily complex challenges while making them look effortless**. During technical presentations, if he wanted to prove a point, he would program complex robotic simulations live on stage—typing code faster than I could move my fingers! It was utterly mesmerizing. > > Soon after, Rosen and I teamed up on a joint bin-picking project between CMU and a major Japanese automotive manufacturer. At that time, estimating 6-DoF poses of textureless, highly reflective metal parts from 2D and 3D sensors, followed by real-time motion planning to pick them from random jumbles, was groundbreaking work. We flew to Japan together to demonstrate the operational system. I still remember Rosen saying to me, *"Wow, awesome hard work!"* > > Within a year, Rosen completed his doctorate at CMU and moved to Tokyo as a post-doctoral researcher at the University of Tokyo, co-founding Mujin in 2011 alongside CEO Issei Takino. Meanwhile, I continued advancing the bin-picking project at CMU, while completing internships at Qualcomm and Google developing mobile computer vision for Augmented Reality (AR). --- ## 4. Touching Down at Narita at 6 AM: The Legendary DMS 2014 Rescue *-- How did you transition from CMU to joining Mujin full-time?* > 💬 **Jeronimo:** > As Mujin's industrial deployments gathered momentum, the need for an in-house, production-grade computer vision stack tightly coupled with Rosen's autonomous motion planning became mission-critical. When Prof. Kanade visited the Tokyo office, Rosen and Issei asked him if he knew of an exceptional computer vision engineer, and Prof. Kanade recommended me. While still completing my PhD coursework, I began writing computer vision algorithms for Mujin remotely from the US.
Robotics exhibition demonstration at DMS 2014
The metal part bin-picking demonstration running live at DMS 2014
*-- And then came the famous trade show crisis...* > 💬 **Jeronimo:** > Exactly (*laughs*). Just two days before "DMS 2014"—a massive manufacturing exhibition in Tokyo—Mujin was preparing a live industrial robot demonstration using the vision algorithms I had written. Suddenly, my phone rang with an urgent call from Rosen: > > *"Hey, we just integrated the hardware and software on the actual industrial cell, but the system isn't running and the transport truck is waiting outside right now! Can you jump on a plane to Tokyo and fix it on the show floor?"* > > Immediately after hanging up, I booked the earliest flight across the Pacific and boarded the next morning. Navigating across international date lines, **I landed at Narita Airport at 6:00 AM on the very morning the exhibition opened**. With my suitcase in hand, I dashed straight to Makuhari Messe, arriving at 8:00 AM. > > Kneeling next to the robot cell with my laptop, I diagnosed and patched the software pipeline frantically. **At exactly 10:00 AM—the very minute the exhibition doors swung open to thousands of visitors—the robot began running flawlessly!** > > After the exhibition, Rosen and the founding team repeatedly insisted: *"Join, join, join!"* Experiencing your own code command a multi-ton industrial robot in physical reality provided a thrilling rush that academic paper-writing could never match. Seeing Mujin's bold ambition to solve real-world industrial labor shortages, I officially joined full-time two months later. --- ## 5. Crashing at CEO Issei's House: Deep Human Bonds in Early Mujin *-- What was life like during Mujin's founding chapter?*
Founding team members living and building together
Early days: International engineers living and working side-by-side in Tokyo
> 💬 **Jeronimo:** > In those early days, almost all of our engineers were foreigners from overseas. Building a revolutionary robotics product from ground zero was daunting enough, but adapting to daily life in a completely new country with zero Japanese was equally challenging. > > That was where CEO Issei's incredible humanity shone. > > When I first arrived in Japan, Issei welcomed me into his personal home. I shared his living room, his couch, his bathroom, and his kitchen. Issei cooked hearty meals for us engineers almost every night, telling us: *"Eat everything in the fridge, don't worry about anything except building great robots."* And believe me, I ate an enormous amount (*laughs*). > > **Mujin's true strength lies not just in world-class software, but in the unbreakable trust among our people.** Because Issei took such genuine care of us as human beings, our team formed an unbreakable bond that allowed us to pour 100% of our energy into engineering. --- ## 6. Closing the Gap Between Ideal & Reality: Robust Vision at Scale *-- As the first computer vision engineer, what did you build?* > 💬 **Jeronimo:** > There was no existing code repository for vision when I joined; everything had to be architected from ground zero. We didn't have standardized off-the-shelf products yet, so we rapidly engineered bespoke, high-performance systems for demanding client demonstrations. > > Through that crucible, we architected reusable, modular 3D point cloud libraries. We showcased early capabilities at international robotics conferences like ICRA and trade shows like iREX, winning our first industrial deployments in automotive bin-picking. Soon after, we expanded into logistics piece-picking—a domain where no automated commercial solution had ever succeeded in production.
Jeronimo leading architectural design discussions
Whiteboard discussions and algorithmic review sessions at Tokyo headquarters
*-- What are you and your team focused on today?* > 💬 **Jeronimo:** > Today, Mujin operates multiple mature robotics product lines spanning manufacturing and global logistics. Our dual mission is continuously pushing the scientific envelope with cutting-edge vision algorithms while making our existing production systems rock-solid. > > In real warehouses and factories, cameras encounter ambient sunlight flare, crushed boxes, warped pallets, and reflective shrink wrap—chaos that clean academic benchmarks never prepare you for. Bridging that punishing gap between theoretical models and real-world 99.99% uptime is the essence of true vision engineering. > > As team lead, I make sure every engineer understands the "Big Picture." We don't optimize algorithms in an isolated vacuum; we see how our perception code keeps an entire automated supply chain flowing and frees human workers from dangerous, grueling labor. --- ## 7. Who Belongs on the Team: Passion and a Never-Surrender Spirit *-- What kind of engineers thrive in Mujin's Computer Vision team?*
Jeronimo in the Tokyo robotics lab
Jeronimo testing perception pipelines with production manipulators at Mujin's Tokyo Robotics Lab
> 💬 **Jeronimo:** > We look for exceptionally sharp, analytical thinkers with an unyielding determination to make intelligent robots work in physical reality. > > The challenges we face daily don't have answers in textbooks or academic papers. Some problems demand deep mathematical and optical rigor; others require daring, creative algorithmic leaps. > > But above all, the decisive quality is an authentic **passion for robotics and automation paired with an uncompromising Never-Surrender spirit**. When a bug appears on the physical floor at midnight, having the resilience to diagnose the hardware, analyze point clouds, and conquer the problem is what defines a true Mujinian. *-- Any closing thoughts on Mujin's journey ahead?* > 💬 **Jeronimo:** > Human progress has always been propelled by automation. Steam engines and modern transport compressed weeks of walking into hours; computers and internet search replaced endless hours of manual library cataloging. Automation frees human beings from repetitive toil so they can focus on creative, fulfilling pursuits. > > Yet across modern supply chains, millions of people are still trapped doing strenuous, monotonous physical labor. Our mission is to have intelligent robots shoulder those burdens. > > The mountain we are climbing is steep, but **I have never been more thrilled about the automated future Mujin is building!** --- :::tip[Shape the Future with Our Computer Vision Team] Mujin's Computer Vision team is continuously recruiting talented roboticists and vision engineers to advance 3D point cloud algorithms, 6-DoF pose estimation, physics simulation, and autonomous sensor calibration. - Explore technical stacks & team culture: [Computer Vision Team Overview](/teams/mujin-os/computer-vision) - Technical deep dive: [The Essence of Vision Engineering: Beyond Deep Learning](/blog/vision-engineering-beyond-deep-learning) - Current open roles: [Mujin Careers Portal](https://jobs.lever.co/mujininc?department=1%20-%20Software%20Development&team=Computer%20Vision%20Team) :::