Microsoft
Software Engineering Intern, Azure OpenAI Inference Team
Built AI infrastructure for Azure OpenAI that improves prompt-cache efficiency and LLM inference performance at scale.
Zhuohao Ni
I am an M.S.E. student in Robotics at the University of Pennsylvania. Previously, I studied Computer Engineering at the University of British Columbia.
I am broadly interested in AI systems for multimodal and embodied intelligence. My work spans foundation-model infrastructure, multimodal learning, and robot learning.
Experience
Industry
ShotAI
Co-Founder / Software Engineer, AI Systems
A multimodal creative agent that understands video, plans edits, and turns intent into a finished cut.
LEAP A.I.
Founding Engineer
An AI advising agent that turns scattered institutional knowledge into grounded, personalized guidance.
Walnut
Co-Founder / Tech Lead
An AI copilot for the classroom—from live lecture to personalized study.
Research
WashU NLP Group
Research Assistant, Multimodal Learning
Language-first instruction tuning that helps multimodal models learn more with less data.
UBC TEA Lab
Research Assistant / Undergraduate Thesis
Making medical AI more interpretable—from localized visual concepts to data-driven brain communities.
Westlake AI4S Lab
Research Assistant, Scientific Machine Learning
Learning reusable dynamics from complex physical systems governed by time-dependent PDEs.
Publications
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Introduces a text-heavy visual instruction-tuning strategy that transfers instruction-following ability and domain knowledge across modalities, matching vision-heavy mixtures across 12 benchmarks with as little as half the training tokens.
FiLMed: Fine-Grained Visual Tokens Align with Localized Semantics
Introduces a fine-grained concept-bottleneck framework that aligns learnable attribute tokens with localized visual evidence, enabling spatially grounded and clinically interpretable skin-lesion diagnosis.
Learnable Community-Aware Transformer for Brain Connectome Analysis with Token Clustering
Brings learnable community discovery to connectomic analysis by coupling transformer representations with token clustering, replacing fixed brain partitions with data-driven functional communities.
* Equal contribution
Projects
AI Systems
Prompt Cache Advisor
Built an AI infrastructure system for Azure OpenAI that improves inference efficiency by diagnosing prompt-cache performance at scale and translating system signals into actionable optimization strategies.
Multimodal Video-Editing Agent
Built ClickCut, a multimodal planner-executor agent that converts creative intent into structured, executable video edits. Orchestrated multiple foundation models through tool calling and replayable workflows, making complex editing tasks reliable, inspectable, and easy to refine.
AI Learning Platform
Co-founded Walnut, an AI learning platform that helps international students follow lectures through real-time transcription, translation, and context-aware study assistance. Led the technical direction across multimodal note-taking, retrieval over course materials, and the product infrastructure that turns live classroom content into searchable, personalized study workflows.
Robotics
Autonomous Drone Racing
Developed a PPO-based autonomous racing policy with gate-aware observations, reward design, and domain randomization for robust high-speed flight. Bridged simulation and physical deployment on Crazyflie with ROS 2, placing 1st out of 39 teams.
Automated Robotic Sorting Arm
Built an end-to-end edge robotics system on NVIDIA Jetson Nano, combining YOLO perception, TensorRT inference, and autonomous manipulation for real-time component sorting. Designed modular perception, control, and monitoring layers to support reliable deployment and rapid debugging.
Learning in Robotics
Built a suite of robotics learning systems that connect estimation, mapping, reinforcement learning, neural rendering, and model-predictive control across complete perception-to-action pipelines. Implemented and evaluated these methods in simulation, with an emphasis on how learned policies and classical robotics components interact.
Education
University of PennsylvaniaM.S.E. in Robotics
University of British ColumbiaB.A.Sc. in Computer Engineering, Graduated with Distinction