Statement of Purpose

Sameer Dharur – Applicant to the MS in Computer Science, Columbia University, Fall 2019.

During my spirited presentation at the finals of Qualcomm India’s annual innovation contest in 2018,
it took me merely a few minutes to recognize the ample scope for improvement in the suboptimal
performance metrics of my new Artificial Intelligence (AI)-based background removal feature for
video calls. The state-of-the-art MobileNet Single Shot Detector-based deep neural network (DNN) of
my training ran no more than 16 inferences per second on the GPU of the newest Snapdragon 845
chipset, consuming as many as 1640 milliwatts of power for each inference. In many ways, this
anecdote succinctly encapsulates my desired direction of work as well as the current limitations of
our field. I subsequently ranked among the top 5 contestants in a pool of over 300 from around the
country for my maiden end-to-end project in on-device AI. And yet, it became evident to me that our
ability to create solutions for the optimal inferencing of DNNs fully on-the-edge, by addressing
sensitive constraints of performance and power, will be the decisive springboard that helps AI soar to
its long-desired ubiquity.

I credit the genesis of my trajectory in this field to intuitive choices exercised during my
undergraduate study at BITS Pilani, India’s top-ranked private engineering university. I spent my final
year almost exclusively conducting research - a luxury made possible by my department’s highly
competent research ecosystem and its flexible academic curriculum - working first under the tutelage
of Prof. G. Geethakumari on a project to develop a cloud forensics tool sponsored by the
Government of India. Subsequently, I had the career-altering experience of doing my undergraduate
thesis on a machine learning solution for energy conservation in an IoT-powered smart building with
Prof. Chittaranjan Hota, in collaboration with the Vrije Universiteit in Amsterdam as part of the
Indo-Dutch Joint Research Programme for ICT, 2016. My initial reservations about the prospect of
plunging into a full year of open-ended problem solving were dispelled by reaching consistent
milestones that served as timely reminders of the firm practicality of concepts gleaned from my
courses. During this impressionable phase, I was blessed to have been mentored by professors who
were Senior Members of IEEE with an outstanding track record of research. Working in this
stimulating milieu propelled me to publish work from my thesis in two research papers presented at
IEEE’s international conferences held in India. Despite graduating in the first division with consistently
improving numbers, my wealth of lessons garnered from countless hours spent in the department’s
research labs and mentoring students through a teaching assistantship are regrettably unrepresented
in my grade sheet. In totality, I remain grateful for a varied experience that eventually left me with a
clear modus operandi in a field that felt my own.

For the better part of the last two years since the conclusion of my undergraduate degree, I have
spent my time at Qualcomm learning the tools of the trade that contribute to the universalization of
AI on smart devices. This endeavor was bolstered in March 2018 when I joined the company’s first
Indian team working on machine learning – an exclusive group of eight employees handpicked from
around the country. We were tasked to work on the Snapdragon Neural Processing Engine (SNPE): a
software framework that facilitates DNN inferences on the various processing cores of a smartphone
chip - the CPU, GPU and DSP - while meeting key performance indicators of inference speeds,
accuracy, memory and power consumption. I have also been entrusted with the responsibility of
carrying out similar work on Google's Android Neural Networks API within my team, allowing me to
extract valuable knowledge in the field from yet another popular vantage point. As a member of the
founding team for this division in India, I have had a panoramic perspective on getting things up and
running from scratch in various modules of the team's activities – from implementing operations of
DNN layers on the chip for a range of use-cases, to training models of various deep architectures for
profiling their performance to the creation of test cases that identify and overcome bottlenecks. I
consider this stint a particularly enriching one, for it has helped me dive headlong into the minutiae
of running DNNs on chipsets and presented me with holistic insights to aid their performance at
various levels of the software stack.

Perhaps the most gratifying outcomes through this period resulted from my attempts to plumb the
field’s myriad challenges by interacting with senior members of Qualcomm’s engineering teams
across divisions. Our collective deliberations helped me piece together a coherent view of the
requirements of an AI practitioner in this space. Some of these efforts yielded immensely satisfying
results, including the presentation of two posters at the company’s annual Power Summit 2018 in
San Diego - one outlining a DNN-based personalized algorithm for estimating battery life on a
smartphone chipset, and the other prescribing an ML-driven power adaptive solution for
coverage-enhanced devices to reduce the number of uplink repetitions, a notoriously
battery-intensive operation among 5G LTE devices. Diversifying into solving problems on sequence
models, I reached out to the audio systems team to build a deep learning-based active noise
cancellation feature during voice calls that made it to the top 15 from over 300 submissions in the
2018 edition of the company’s innovation contest. In this project, I was responsible for a gated
recurrent unit (GRU)-based optimization in a speech processing pipeline. My work here has also led
to the unique privilege of filing two patents for my ideas – one for a 5G narrow-band IoT and
machine learning-based project to accurately predict groundwater in an agricultural setting, and the
other on leveraging motion vectors to minimize the execution of DNN inferences to achieve crucial
gains in performance during object detection and semantic segmentation on smartphone chipsets.

It is my humble opinion that the greatest validation of research comes from the adoption and impact
of its solutions – a yardstick on which Columbia’s CS department, a venerable pole star in the field,
has consistently shone over the years. I see strong resonance in the work of numerous researchers in
your department. Prof. Shih-Fu Chang and the Digital Video Multimedia Lab’s dizzying array of
projects in object detection and semantic segmentation – most notably Detection and Recognition of
Overlay Text in Video, CU-VIREO374, Large Scale Dynamic Event Detection, among others – dovetails
seamlessly with my own efforts at Qualcomm over the past couple of years. Likewise, Shree K Nayar
and the Columbia Vision Laboratory’s projects in visual recognition – Gaze Locking and The RAD in
particular – align closely with my demonstrated work and future areas of interest in this field.
Moreover, Prof. Carl Vondrick’s recent research in attempting to enhance fundamental areas of
Computer Vision as exemplified by his work in tracking emerges by colorizing videos, visualizing
object detection features and scaling up video annotations – are all enormously exciting to me as an
ardent student of this field. Prof. Suman Jana’s efforts at setting in place frameworks for the
automated testing of DNNs through his hugely interesting tools DeepTest and DeepXplore
significantly overlap with and extend my similar work in functional testing of DNNs on Qualcomm’s
SNPE. Additionally, Prof. Julia Hirschberg and the Spoken Language Processing Group’s projects are
natural extensions to my work in on-device speech processing due to which I could make meaningful
contributions to anomaly detection and identifying deceptive speech.
Fortunate as I am to currently belong to a team whose work aligns wholesomely with my interests
and is regularly manifested in millions of devices around the world, I remain a bullish advocate of the
primacy of academia in pushing the envelope of a richly complex field. Much like a well-trained
reinforcement learning agent, I find the need to counterbalance exploitation with exploration - an
idea at the cornerstone of academia, where I view my future self as an avid graduate researcher
building on my lessons from undergraduate research and the challenges of practical adoption from
the industry, combined with the experience of producing papers and patents, to help drive progress
across a range of fields in AI. In so doing, I wish to remain conscious of the viability of my solutions to
thrive on resource-crunched embedded devices – to the extent that breeds no more misgivings at
future presentations!