Research

From what a model sees to what a robot does.

My research asks how vision-language models represent space and physics, and how that knowledge, and the robot's own embodiment, reach the actions a policy takes. Below: the question I am working on now, the two robotics papers it grows from, and a medical-robotics system where the same perception-to-action loop meets patients.

CV

01 / CURRENT WORK

Spatial knowledge in VLA policies

04 Evaluation loop03 Actions and interventions02 Representations01 Inputs

CMU · Prof. Guannan Qu · 2026–

Do VLM representations carry spatial knowledge into actions?

Step 2 of 4 · Representations

Work in progress: this is the experimental design, not a set of findings.

RelatedGVLA · embodiment-conditioned actions ↗GCA-Bench · execution-level evaluation ↗

02 / ECCV 2026

Gripper-aware Vision-Language-Action Models

GVLA and MiGA overview across multiple grippers, robots, and manipulation settings

Accepted · Second Author

GVLA studies how manipulation policies can adapt to different gripper embodiments instead of treating all end-effectors as interchangeable.

MY CONTRIBUTIONCore contributor (2nd author): co-designed the gripper-conditioning module from inception, implemented and ablated multiple architectural variants, ran experiments, and co-wrote the paper.

VLARobotic ManipulationGripper EmbodimentMulti-Gripper Datasetpi0.5

03 / IROS 2026

GCA-Bench: Benchmarking Complex Grasping from Detection to Execution

GCA-Bench overview from language instructions and scene understanding to grasp execution

Accepted · Contributing Author

GCA-Bench evaluates robotic grasping as a complete execution pipeline, from semantic and scene reasoning to trajectory-level task success.

MY CONTRIBUTIONContributed real-robot data collection, dataset organization, and manuscript preparation for a benchmark spanning detection, semantic reasoning, and trajectory-level execution.

BenchmarkComplex GraspingVLA EvaluationRobotic ArmSim-to-Real

04 / MEDICAL ROBOTICS

Autonomous magnetic capsule endoscopy

Jifu Medical makes magnetically controlled capsule endoscopy. The patient swallows a capsule camera, and an external magnet on a robotic arm steers it through the stomach and intestine. A full examination needs no anaesthesia, which matters for elderly patients who cannot be sedated, and it spares patients the discomfort of a conventional scope.

The arm was driven by hand by a doctor. I joined to help build the first VLA-style autonomous version: perceive the patient and the capsule's view, judge whether the whole stomach has been covered, and move the magnet to steer the capsule on its own.

Jifu Medical · 2025 · first autonomous version

From a hand-driven magnet arm to a VLA-style loop

Step 1 of 4 · Patient posture

Early-stage engineering system; not a clinical result.

Explore the system flow →

The groundwork I built: before a policy can drive the magnet arm, it needs demonstrations. I set up the imitation-learning data pipeline on SO-101 dual arms: teleoperation, synchronized multi-view recording, and joint-state / action logging, with 50+ episodes recorded and validated.

Jifu Medical · 2025

SO-101 dual-arm imitation-learning pipeline

Step 1 of 4 · Teleoperation

System flow →
Select shoulder_panSelect shoulder_liftSelect elbow_flexSelect wrist_flexSelect wrist_rollSelect gripperSelect camera

Inspect the arm · SO-101

elbow_flex

Bends the forearm; together with shoulder_lift it places the wrist in space.

Joint state · what each frame logs

Joint names follow LeRobot's SO-101 convention; ranges and link lengths are illustrative.

05 / EARLIER WORK

Multimodal learning

Before robotics, I worked on multimodal representation learning for medical imaging: how to fuse, complete and align modalities when some are missing or noisy.