Research

How planners fail, and how robots should make themselves clear.

Two lines of work: how language-model planners fail, measured against machine-checked ground truth, and how robots should move and speak so people can calibrate their trust. Every figure is backed by committed data a stranger can re-run. Academic CV (PDF)

SubmittedIEEE RA-L, Aug 2026

Certified bounds on achievable legibility under a path cost budget

A benchmark measures what a planner achieved. This certifies what nothing could achieve: an upper bound on legibility over every admissible trajectory, not only the ones somebody tried. All 32 world and budget pairs hold with zero violations, and the narrowest gap anywhere is 0.0064.

Where it stops. The observer model is exactly reproducible and has never been validated against people, so these are bounds on a stated objective, not on what a human would infer.

Certified intervals, by world
door_pair
0.0269
narrow_gap
0.0456
fan_outer
0.0282
wall_choice
0.0128
fan_middle
0.0567
legibility
0.40.60.81.0
gap

Each bar runs from a trajectory that exists to a ceiling nothing can exceed. Five of the eight worlds, each at the budget where its interval is widest.

Working paperSole author, preprint on ZenodoLive demo

plan-failure-bench: a machine-checkable benchmark of how language model planners fail

Every instruction either admits a valid plan or plants exactly one trap, and every ground-truth label carries a proof re-verified in CI. A deterministic checker scores each response, with no human or LLM judge. Across 18 runs on four models the failures are distinct and stable: Gemini 3.6 Flash survives full semantic obfuscation, while smaller models split between silent compliance and surface-anchored refusal.

Where it stops. Two symbolic PDDL environments and four models. Detection is never reported without its paired false positive count.

60

instructions, each with a machine-checked proof of ground truth

18

complete model runs, every record committed

589

tests and label proofs, re-run on every commit

0

human or LLM judgements in the scoring loop

Under review · Robotics (MDPI)MSc research, University of Hertfordshire, supervised by Dr Patrick Holthaus

Enhancing human-robot companionship: verbal and non-verbal communication cues in humanoid robots

A full sim-to-real pipeline on the JD humanoid, from an Isaac Sim digital twin through ROS2 hardware integration, built so a gestural and a spoken condition could be delivered identically to 20 participants. Speech conveyed intent significantly more clearly than gesture, while engagement and warmth held comparable across both. The study was run during the MSc (2023-2024); the manuscript was written up for journal submission in 2025.

Where it stops. One deliberately mundane scenario, the robot asking to be recharged, with 20 participants. Every gestural failure was a failure of joint attention, which is why the design target is hybrid.

Intent identified correctly
The JD humanoid robot used in the study
Speech
95%
Gesture
80%

Clarity effect d = 0.58, p = 0.017. Within-subjects, counterbalanced, RoSAS-validated, N = 20.

PilotIn active development

legible-motion-bench: do language models know when their motion is legible?

Measures legibility against path cost and constraint satisfaction, computed exactly, with no human rater and no model judge. Asked to plan in the same worlds, three language models called 116 of their 120 trajectories legible, including all 25 that were not physically possible.

Where it stops. Three models at one temperature is a pilot, not a finding. Next: whether the metric tracks what people actually perceive.

Claimed vs possible, of 40 decodes
Qwen 2.5 7B
40
26
Llama 3.3 70B
40
29
Gemini 3.6 Flash
36
40
called legible by the modelphysically possible
Publications and preprints

How to cite.

  1. Kazmi, S. M. (2026). Certified bounds on achievable legibility under a path cost budget. Preprint, Zenodo. doi:10.5281/zenodo.21834955. Submitted to IEEE Robotics and Automation Letters.
  2. Kazmi, S. M. (2026). plan-failure-bench: a machine-checkable benchmark of how language model planners fail. Working paper, Zenodo. doi:10.5281/zenodo.21756817.
  3. Kazmi, S. M. (2026). What clarity costs: measuring legible robot motion against path cost and constraint satisfaction. Pilot, preprint (PDF), no DOI yet.
  4. Kazmi, S. M. (2025). Enhancing human-robot companionship: verbal and non-verbal communication cues in humanoid robots. Under review at Robotics (MDPI); write-up (PDF). MSc research, University of Hertfordshire, supervised by Dr Patrick Holthaus.
  5. Kazmi, S. M. (2026). The frozen blowup profile of the Okamoto, Sakajo and Wunsch family: a control variate for its spectral exponent. Note, PDF.

Each entry's data and code are in the linked repository, and every figure on this site is checked against them by tools/drift.py.

Next

Looking for a PhD student?

hello@munawarkazmi.com