Sudeep Fadadu Email:

Webpage:
LinkedIn:
fa [email protected]
sudeepf.github.io
linkedin.com/in/fadadu

Summary
I’m a Staff Engineer with 7+ years of industry experience building AI products and doing applied ML research in
field of Computer Vision(CV) and Robotics. I’m currently at Aurora Innovations, where I have led and worked on
various CV projects to build Self Driving Technology. Previously, I worked in Applied Research at Vicarious AI on
real-time geometry-based CV and Robotics. I did my graduate studies from Carnegie Mellon where I focused on
my research on multi-modal ML at the intersection of CV and Natural Language Processing (NLP). I am looking
to work on cutting edge ML research in the domain of CV, NLP and Robotics for real-world applications.

Experience
Aurora Innovations - Previously Uber Advanced Technologies Group San Francisco, CA
Staff Software Engineer | Perception Team Jan ’20 - current
• Leading ground up design and development of fail-safe autonomy in perception - media coverage
– Built a team of 7+ to develop the Fault Management System for Perception.
– Designed a process to effectively and thoroughly identify failure modes and their effects.
– Lead a team to build accurate fault detectors and designed a scalable framework to effectively communi-
cate cause and effects to downstream systems for graceful handling.
– Worked on determining fault handling strategies for graceful risk mitigation to acceptable levels.
– Lead research towards making deep-learning perception models aware and robust against faults.
• Developed Machine Learning methods to improve detection capabilities in Perception
– Led the development to improve detection and tracking of VRUs (such as Motorcycles) by geometry aware
multi-modal sensor fusion.
– Developed loss selection strategies to handle class-imbalance in dataset for rare objects.
– Improved a image-centric detector model to run on multi-view, keeping latency and detection performance
unchanged by migrating computation to GPU and effectively leveraging Tensor RT.

Vicarious AI - Now Intrinsic, an Alphabet Company San Francisco, CA

Research Engineer | Computer Vision Team Mar ’18 - Jan ’20
• Efficient target tracking algorithm for resource constrained robotic-arm
– Built visual tracking system using known geometric constraints and other sensory data for arm-mounted
camera and moving target in closed loop precision insertion.
– Developed a stand-alone system inspired from structure-from-motion to generate 3D model of any object
from sequence of images for grasping position estimation.

Uurmi Systems - A MathWorks company Hyderabad, India

Project Lead May ’15 - Jun ’16
• Self-driving technology for low-cost sensor platform
– Led a team of 10+ to develop stereo-camera based perception, localization and mapping algorithms,low-
level control systems and drive-by-wire systems.
– Successfully secured funding from leading Indian automobile manufacturer - (media coverage).

Other industry experiences

• Intern at Uber Advanced Technologies Group | Deployed a real-time vision centric object localization system
by lifting 2D detection to 3D with stero and map geometry features.

• Capstone project member at Amazon Lab126 | Devised a novel learning method to estimate 3D human pose
in varying background by train-time gradient fusion from 2D labels. (Video)
Publications and Contributions
Multi-View Fusion of Sensor Data for Perception and Prediction (published at WACV-22) Apr ’20 - Jul ’20
Proposed an efficient end-to-end method for fusing multiple sensor modalities (LiDAR, Cameras) and views
(Bird’s Eye View, RangeView) for improved detection and trajectory prediction performance. - (arxiv)
Robust Long-Range Perception Against Sensor Misalignment (submitted at WACV-24) Apr ’23 - Jul ’24
Proposed an efficient multitask learning method for predicting and self-correcting extrinsic miscalibration
between different sensor modalities. - (arxiv)
Generative Adversarial Attacks on Vision Language Models Aug ’17 - Jan ’18
Developed a novel method to successfully fool SOTA VQA models with more than 70% attack success rate
(Paper | Successive work published at NeurIPS workshop 2018)
Improving Activity Recognition from Videos with Linguistic Prior Jan ’17 - May ’17
Enabled deep VQA model to use learnt representation of concepts from text (Improving benchmark from
14.3% to 19.21% - Report)
Other Projects
• Lightweight Fast Dense Inertial Visual SLAM for Stereo (Demo Video)
• HDR Image Reconstruction from LDR Images (Report | Result)
• Human Detection and Tracking in 3D space (Video)

Education
Carnegie Mellon University Pittsburgh, PA
Masters of Science in Computer Vision Aug ’16 - Dec ’17
Robotics Institute, School of Computer Science
Courses: Visual Learning and Recognition, Geometry-based Methods in Vision, Machine Learning, Advanced Multimodal
Machine Learning, Deep Reinforcement Learning
Teaching Assistant: Mobile Robots, Sensing and Sensors

International Institute of Information Technology (IIIT-H) Hyderabad, India

Bachelors in Electronics and Communication Engineering Aug ’11 - May ’15
Honors: Robotics
Courses: Mobile Robotics, Artificial Neural Network, Computer Vision, Digital Image Processing, Design of Mechanism,
Advances in Robotics and Control, Probability and Random Processes, Algorithms, Data Structures, Functional Analysis

Skills
Programming and Scripting: Proficient: C/C++, Python | Familiar: CUDA
Libraries and tools: Proficient: PyTorch, TensorFlow, OpenCV, PCL, ROS, MATLAB, Git