Vision-language navigation agents are typically evaluated on their ability to follow route-like instructions toward a fixed goal. Real navigation instructions, however, often depend on the observed state of the environment: if a condition holds, then follow one path, otherwise take another. Such instructions require an agent to evaluate scene evidence, select the correct logical branch, and execute the corresponding navigation behavior. Existing evaluations provide limited control over conditional branch execution, making it difficult to determine whether agents fail because of perception, grounding, navigation, or logical decision-making.
We introduce CondVLN, a scene-graph-grounded benchmark for diagnosing conditional branching in vision-language navigation. CondVLN programmatically generates instructions whose branch conditions are grounded in verifiable 3D scene-graph predicates, with controlled variation in branch depth, dependency chain length, spatial composition, evidence observability, and instruction horizon. The current instantiation contains over 2,000 generated conditional instructions across AI2-THOR, Matterport3D, Gibson, and ReplicaCAD, and evaluates agents using both standard VLN metrics and branch-specific diagnostics: Branch Selection Accuracy and Conditional Success Rate.
Evaluating five recent VLN agents, including VLN-Zero, NaVid, NaVILA, and Open-Nav, shows that conditional branching exposes failures that are not captured by standard success rate or path length alone: agents can navigate plausibly while committing to a branch inconsistent with the observed scene condition. Finally, we present a lightweight neurosymbolic branch-selection layer that separates condition grounding from navigation execution, improving Conditional Success Rate and Branch Selection Accuracy by 12% and 15% each under controlled information access. CondVLN provides a reusable testbed for measuring whether embodied agents can not only follow instructions, but follow the right instruction under the right condition.
Overview of CondVLN. CondVLN features a programmatic benchmark generation pipeline for evaluating neurosymbolic reasoning in vision-language navigation, where agents must combine semantic-spatial grounding in embodied environments with symbolic reasoning over explicit logical and conditional structures. The pipeline consists of two parts: dataset generation and evaluation.
Simple Conditional
Nested Conditional
Deep Nested Conditional
Chain Conditional
Long Chain Conditional
Nested Chain Conditional
Simple Conditional
Nested Conditional
Deep Nested Conditional
Chain Conditional
Long Chain Conditional
Nested Chain Conditional
Simple Conditional
Nested Conditional
Deep Nested Conditional
Chain Conditional
Long Chain Conditional
Nested Chain Conditional
