PG-SAM: Prior-Guided SAM with Medical LLMs for Multi-organ Segmentation
Yiheng Zhong*, Zihong Luo*, et al.
Research
My research asks how vision-language models represent space and physics, and how that knowledge, and the robot's own embodiment, reach the actions a policy takes. Below: the question I am working on now, the two robotics papers it grows from, and a medical-robotics system where the same perception-to-action loop meets patients.
CV01 / CURRENT WORK
CMU · Prof. Guannan Qu · 2026–
Step 2 of 4 · Representations
Work in progress: this is the experimental design, not a set of findings.
RelatedGVLA · embodiment-conditioned actions ↗GCA-Bench · execution-level evaluation ↗
02 / ECCV 2026

Accepted · Second Author
GVLA studies how manipulation policies can adapt to different gripper embodiments instead of treating all end-effectors as interchangeable.
MY CONTRIBUTIONCore contributor (2nd author): co-designed the gripper-conditioning module from inception, implemented and ablated multiple architectural variants, ran experiments, and co-wrote the paper.
03 / IROS 2026

Accepted · Contributing Author
GCA-Bench evaluates robotic grasping as a complete execution pipeline, from semantic and scene reasoning to trajectory-level task success.
MY CONTRIBUTIONContributed real-robot data collection, dataset organization, and manuscript preparation for a benchmark spanning detection, semantic reasoning, and trajectory-level execution.
04 / MEDICAL ROBOTICS
Jifu Medical makes magnetically controlled capsule endoscopy. The patient swallows a capsule camera, and an external magnet on a robotic arm steers it through the stomach and intestine. A full examination needs no anaesthesia, which matters for elderly patients who cannot be sedated, and it spares patients the discomfort of a conventional scope.
The arm was driven by hand by a doctor. I joined to help build the first VLA-style autonomous version: perceive the patient and the capsule's view, judge whether the whole stomach has been covered, and move the magnet to steer the capsule on its own.
Jifu Medical · 2025 · first autonomous version
Step 1 of 4 · Patient posture
Early-stage engineering system; not a clinical result.
The groundwork I built: before a policy can drive the magnet arm, it needs demonstrations. I set up the imitation-learning data pipeline on SO-101 dual arms: teleoperation, synchronized multi-view recording, and joint-state / action logging, with 50+ episodes recorded and validated.
Jifu Medical · 2025
Step 1 of 4 · Teleoperation
Inspect the arm · SO-101
Bends the forearm; together with shoulder_lift it places the wrist in space.
Joint state · what each frame logs
Joint names follow LeRobot's SO-101 convention; ranges and link lengths are illustrative.
05 / EARLIER WORK
Before robotics, I worked on multimodal representation learning for medical imaging: how to fuse, complete and align modalities when some are missing or noisy.
Yiheng Zhong*, Zihong Luo*, et al.
Chengzhi Liu*, Zile Huang*, Zihong Luo, et al.
Zihong Luo*, Chengzhi Liu*, et al.