Human-Centered VLA Survey & outlook

Towards Human-Centered Safety in Vision-Language-Action Robot Control

A lifecycle view of how safety understanding shapes robot action: from learning and runtime safeguards to validation.

1National University of Singapore 2IAIC, A*STAR 3University of Southern California 4University of Alberta 5University of California, Los Angeles 6Nankai University 7Nanyang Technological University

*Equal contributions†Corresponding author

Overview

A robot can recognize a hazard and still act unsafely. Human-centered safety depends on whether that understanding remains consequential for physical behavior.

Vision-language-action models connect perception and language to control in environments shared with people. The same action can have different consequences depending on human state, object properties, task context, social expectations, or interaction history.

This survey examines that relationship across learning, execution, and evaluation, with human well-being, environmental integrity, and social compliance as complementary safety concerns.

Human well-beingEnvironmental integrity
01 / SEMANTICS

Specify what matters.

Identify the people, objects, context, and consequences that make an action admissible or unsafe.

02 / FORMULATION

Make safety actionable.

Compare learning objectives, control constraints, temporal logic, executable predicates, and uncertainty estimates.

03 / MITIGATION

Choose where to intervene.

Connect alignment and data coverage with action filters, plan verification, clarification, and recovery.

04 / DIRECTIONS

Find the missing evidence.

Examine whether safety distinctions change actions and whether interventions repair the pathway to failure.

Lifecycle taxonomy

The survey organizes safety across design, deployment, and validation. Use these three stages to navigate the comparison tables below.

Taxonomy of design-time, deployment-time, and validation-time methods for human-centered VLA safety.Open figure
HUMAN-CENTERED VLA SAFETY

Safety must hold across the full lifecycleThe taxonomy connects learning, runtime intervention, and validation across human, environmental, and social constraints.

  • Lifecycle Safety must hold from learning through physical execution and evaluation.
  • Learning Design-time evidence shapes what the system can recognize and prefer.
  • Verification Deployment safeguards and validation test whether those preferences survive physical execution.

Research landscape, milestones & findings

Where the manuscript corpus concentrates, how the research has developed, and what the evidence reveals.

The manuscript's analysis of paper volume, robot embodiments, lifecycle stages, and safety scenarios.Open figure
SAFETY EVALUATION LANDSCAPE

Human-involved evaluation remains limitedThe corpus maps publication volume, robot forms, lifecycle stages, and the safety semantics tested in practice.

  • Scope Real-world safety semantics vary widely across tasks and robot forms.
  • Dimensions Evaluation spans human well-being, environmental integrity, and social compliance.
  • Evidence gap Human-involved evaluations account for about 43% of safety-evaluated papers and include simulated interactions.
Timeline of research milestones in semantic action safety, safety data, safeguards, and validation.Open figure
VLA SAFETY MILESTONES

The field is moving toward explicit and testable safetyProgress shifts from functional data and fixed filters toward consequence reasoning, contextual recovery, and generative stress tests.

  • Data The field is moving from functional data toward safety-critical data.
  • Intervention Action costs and fixed filters are giving way to consequence reasoning and contextual recovery.
  • Validation General benchmarks are expanding toward generative stress testing.

Manuscript comparison tables

Five tables from the Overleaf manuscript. Open a table to inspect its methods, annotations, and incident coverage; search across all 97 entries.

5 manuscript tables · 97 entries

I

Design time

How is safe behavior learned?

01Learning-based alignmentTraining signals · Constraints · Safety dimensions24 entries

Scroll horizontally to inspect all columns

Design-time learning-based alignment
MethodEmbodimentAlignmentFsafe Safety signalConstraintsDimensionsSinc Incident set coverage
RsnCtrlHumEnvSoc
Direct Control Alignment
ViNT ↗NavigationImitation-drivenDemonstrations for collision aversion✕✕✕✓✕Obstacle collision; off-route navigation
NoMaD ↗NavigationImitation-drivenDemonstrations for collision aversion✕✕✕✓✕Obstacle collision; failed goal-reaching
NavDP ↗NavigationImitation-drivenDemonstrations for trajectory safety✕✕✕✓✕Obstacle collision; unsafe trajectory rollout
Multi-Hug RL ↗NavigationControl-constrainedPenalty for collision and intervention aversion✕✓✓✓✓Human collision; obstacle collision; intervention triggers
S2E ↗NavigationControl-constrainedPenalty for pedestrian-aware collision aversion✕✓✓✓✓Pedestrian collision; obstacle collision; social intrusion
SafeVLA ↗Mob. Manip.Control-constrainedPenalty for unsafe-action aversion✕✓✕✓✓Obstacle collision; unsafe manipulation / behaviors
Reasoning-Augmented Control Alignment
TOKEN ↗DrivingReasoning-mediatedObject cues for hazard grounding✓✕✓✓✕Missed traffic object / scenario / collision risk
AutoVLA ↗DrivingReasoning-mediatedReasoning traces for rule compliance✓✕✓✓✕Missed traffic-rule grounding; unsafe maneuver; collision risk
Poutine ↗DrivingReasoning-mediatedSafety reasoning for hazard anticipation✓✕✓✓✕Missed traffic hazard / collision-aware reasoning
Pinocchio ↗DrivingReasoning-mediatedConsistency critic from reasoning to action✓✓✓✓✓Human / environment collision; socially unsafe driving
StyleVLA ↗DrivingReasoning-mediatedStyle reasoning for risk-calibrated driving✓✕✓✓✓Risky driving style; rule violation; socially unsafe maneuver
RationalVLA ↗Manip.Reasoning-mediatedFeasibility reasoning for action rejection✓✕✓✓✕Infeasible action; human harm; object damage
Gemini Robotics ↗Manip.Reasoning-mediatedSemantic reasoning for unsafe-action rejection✓✕✓✓✓Human harm; object damage; socially inappropriate action
Think2Drive ↗DrivingWorld modelReasoning traces & score for future-risk avoidance✕✓✓✓✕Predicted collision; unsafe future state; traffic-rule violation
IRL-VLA ↗DrivingWorld modelReward for trajectory-risk aversion✕✓✓✓✕Missed collision awareness; unsafe trajectory preference
ELF-VLA ↗DrivingWorld modelFailure data and correction for recovery behavior✓✓✓✓✕Driving failure; unrecovered hazard; unsafe correction
TakeVLA ↗DrivingWorld modelTakeover data & boundary for human intervention✓✓✓✓✕Delayed takeover; unsafe boundary; collision risk
Curious-VLA ↗DrivingWorld modelConstraint for safe exploration✓✓✓✓✕Unsafe exploration; collision awareness; traffic violation
CF-VLA ↗DrivingWorld modelCounterfactuals and score for causal-risk avoidance✓✓✓✓✕Missed causal action understanding / collision risk
Alpamayo-R1 ↗DrivingHybridCausal reasoning & score for control-risk avoidance✓✓✓✓✓Human / environment collision; socially unsafe driving
LCDrive ↗DrivingHybridLatent reasoning & score for control-risk avoidance✓✓✓✓✓Human / environment collision; socially unsafe driving
Agentic Action Alignment
HarmBench ↗GeneralistReasoning-mediatedUnsafe-instruction labels for refusal behavior✓✕✓✓✓Harmful instruction compliance; object misuse
Robot Constitutions ↗GeneralistReasoning-mediatedPrompted reasoning for rule compliance✓✕✓✓✓Rule violation; harmful action; socially inappropriate behavior
Safety Thinking ↗GeneralistReasoning-mediatedPhysical reasoning for feasibility preservation✓✕✓✓✕Physical infeasibility; human harm; environment damage
TABLE 01 / LEARNING-BASED ALIGNMENT

How safety enters the training objectiveCompare demonstrations, penalties, rewards, constraints, and reasoning traces by the incidents and safety dimensions they cover.

II

Deployment time

How are actions checked and interventions triggered?

02Runtime safeguardsRuntime critics · Action granularity · Incident coverage26 entries

Scroll horizontally to inspect all columns

Deployment-time runtime safeguards
MethodGranularityParadigmCsafe Runtime criticConstraintsDimensionsSinc Incident set coverage
RsnCtrlHumEnvSoc
Control-Level Safeguards: (a) Control-theoretic filtering
PACS ↗Step-wiseReachabilityReachability safety set over robot-object states✕✓✓✓✕Human/object collision
SALT ↗Step-wiseReachabilityReach-avoid value network over robot-goal states✕✓✕✓✕Wall collision; failed pickup
LatentSafe ↗Step-wiseReachabilityLearned safety value over latent states✕✓✕✓✓Candy spilling; block toppling
UNISafe ↗Step-wiseReachabilityLearned safety value, OOD over latent states✕✓✕✓✓Jenga collapse; block toppling
AnySafe ↗Step-wiseReachabilityLearned adaptive safety value over latent states✕✓✕✕✓Preference over forbidden region/layout; sweeping task
AEGIS ↗Step-wiseCBFCBF margin over VLM-grounded obstacles✕✓✕✓✕Tabletop obstacle collision
Safe-Night ↗Step-wiseCBFCBF margin over obstacle, thermal temperatures✕✓✕✓✓Hot object; buried target; mirror illusion
CORE ↗Step-wiseCBFCBF over VLM-inferred contextual safe sets✕✓✓✓✓Hazard/restricted zone; unsafe terrain; person proximity
Control-Level Safeguards: (b) Predictive filtering
SEAL ↗Trajectory-wiseValue-based predictionVLM alignment score over candidate actions in sim✓✓✕✕✓Wrong object; wrong placement; reasoning-action gap
NavDP ↗Trajectory-wiseValue-based predictionLearned critic score over candidate trajectories✕✓✕✓✕Wall/obstacle collision
SocialNav ↗Trajectory-wiseValue-based predictionVLM etiquette score over candidate paths✓✓✓✓✓Human/obstacle collision; social intrusion; ladder proximity
FOREWARN ↗Trajectory-wiseNeural world predictionVLM alignment score over imagined futures✓✓✕✓✓Chip crushing; object slipping; sanitation; failed pickup
ReOI ↗Trajectory-wiseNeural world predictionVLM alignment score over reimagined futures✓✓✕✓✕Obstacle collision; rollout hallucination
Drive-WM ↗Trajectory-wiseNeural world predictionLearned visual reward score over driving futures✓✓✓✓✓Traffic collision; lane violation; road hazard
Task-Level Safeguards
SafetyChip ↗Task-wiseRuntime verificationLTL admissibility over action traces✓✕✕✕✓Forbidden order; unmet obligation; invalid sequence
SELP ↗Task-wiseRuntime verificationAutomaton admissibility over action traces✓✕✕✕✓Drone ordering; block ordering; illegal continuation
Safety-Sup. ↗Task-wiseRuntime verificationLTL admissibility + obstacle margin over executions✓✓✕✓✓Task violation; motion collision; virtual obstacle
SafePlan ↗Task-wiseRuntime verificationInvariant satisfaction over task plans✓✕✓✓✓Child harm; toxic object; unsafe request
RoboSafe ↗Task-wiseRuntime verificationExecutable predicate truth over action traces✓✓✕✓✓Fire hazard; flood hazard; appliance misuse
SAFEL ↗Task-wiseRuntime verificationTransition-risk score over symbolic plans✓✕✓✓✓Foil microwave; missing unplugging; wrong order
IS Bench ↗Task-wiseRuntime verificationTriggered safety-goal satisfaction over traces✓✕✓✓✓Food contamination; stove left on; water-electricity
Bluesky ↗Task-wiseRuntime verificationAccess-control admissibility over actions✓✕✕✕✓Private diary; personal newspaper; movable dishes
SAFE-AUDIT ↗Task-wiseSemantic verificationSemantic audit score over reasoning traces✓✕✓✓✓Human harm; environment harm; robot-self harm
AutoRT ↗Task-wiseSemantic verificationConstitution compliance over proposed tasks✓✕✓✓✓Sharp object; electrical device; infeasible skill
Safety-as-Policy ↗Task-wiseSemantic verificationConsequence-risk score over TAMP code✓✓✓✓✕Power strip; candle / flour; chemicals / blades
Gemini Robotics 2 ↗Task-wiseSemantic verificationSemantic reasoning over constraints and confidence✓✓✓✓✓Unsafe task, human proximity, low-confidence action
TABLE 02 / RUNTIME SAFEGUARDS

How unsafe execution is blocked or redirectedCompare reasoning-level and control-level guardrails by their safety criterion, intervention point, and incident coverage.

03OOD triggers and responsesNovelty · Anomaly · Uncertainty · Intervention7 entries

Scroll horizontally to inspect all columns

OOD safety: runtime triggers and responses
MethodYearEmbodimentTriggerResponse
Semantic Anomaly ↗2023AgenticAnomalyFlag
KnowNo ↗2023AgenticUncertaintyEscalate
AESOP ↗2024NavigationAnomalyReplan
FORTRESS ↗2025NavigationNoveltyReplan
UNISafe ↗2025ManipulationUncertaintyReplan
UPS ↗2026AgenticUncertaintyEscalate
Gemini Robotics 2 ↗2026AgenticUncertaintyReplan/Escalate
TABLE 03 / OOD RESPONSE

What triggers intervention when conditions changeNovelty, anomaly, and uncertainty signals lead to detection, stopping, replanning, fallback behavior, or human input.

III

Validation time

How are safety claims and failure modes evaluated?

04Safety benchmarksEvaluation settings · Action outputs · Safety dimensions21 entries

Scroll horizontally to inspect all columns

Safety benchmarks and evaluation
BenchmarkEmbodimentSettingAction outputClosed-loopSafety dimensionsLong-tailedSinc Incident set coverage
TextCtrlHumanEnvSocial
Reasoning-focused evaluation
WaymoQA ↗DrivingReal✓✕✕✓✓✓✓Occluded road users; detours;
oncoming traffic; signal risks
WOMD-Reasoning ↗DrivingReal✓✓✕✓✓✓✓Interactions; overtaking;
yielding; right-of-way
ASIMOV-1 ↗AgenticReal✓✕✕✓✓✓✕Allergens; burns; sharp objects
placement; norm violations
ASIMOV-2.0 ↗AgenticReal✓✕✕✓✓✓✕Burns; children; lifting injury
Electrical/biohazards; alerts
ASIMOV-Agentic ↗AgenticReal✓✕✕✓✓✓✓Human proximity; constraints;
unsafe/ambiguous requests
SciFi-Benchmark ↗AgenticReal✓✕✕✕✕✓✕Fictional robot/AI dilemmas;
ethical/social-norm violations
IS-Bench ↗AgenticSim✓✕✓✕✓✕✕Appliance and chemical risks;
contamination; fire hazards
AGENTSAFE ↗AgenticSim✓✕✓✕✓✕✕Hazardous/jailbreak requests;
liquids near electronics; spills
SafePlan-Bench ↗AgenticSim✓✕✓✓✓✕✕Falls; burns; appliance misuse;
chemical/electrical hazards
Control or joint reasoning-control evaluation
NAVSIM ↗DrivingSim✕✓✕✓✓✕✕Road-user/obstacle collisions;
unsafe spacing; road departure
Bench2Drive ↗DrivingSim✓✓✓✓✓✓✕Pedestrian/vehicle interactions;
red lights; emergency yielding
WOD-E2E ↗DrivingReal✓✓✕✓✓✓✓Pedestrian/cyclist interactions;
unsafe paths; traffic obstacles
Fail2Drive ↗DrivingSim✕✓✓✓✓✓✓Pedestrian/obstacle collisions;
stop/yield failures
DynaNav ↗NavigationSim✓✓✓✓✓✕✕Moving human collisions;
obstacle collisions
SidewalkBench ↗NavigationSim✕✓✓✓✓✓✕Pedestrian/obstacle collisions;
boundary/yield/gesture failures
SafetyChores ↗Mobile Manip.Sim✕✓✓✕✓✓✕Fragile objects; falls; collisions;
precarious placement
VLA-Arena ↗ManipulationSim✕✓✓✕✓✕✕Obstacles; sharp objects; spills;
appliance and fire hazards
ANNIE-Bench ↗ManipulationSim✕✓✓✓✓✕✕Human proximity; sharp objects;
boundary/workspace collisions
HazardArena ↗ManipulationSim✕✓✓✓✓✕✕Burns; electrical/toxic hazards;
children; sharp objects
OopsieVerse ↗ManipulationSim✕✓✓✕✓✕✕Mechanical and thermal damage;
fluid damage
LIBERO-Safety ↗ManipulationSim✓✓✓✓✓✕✕Human-hand/tabletop collisions;
fragile object toppling
TABLE 04 / SAFETY BENCHMARKS

Where safety evidence is actually testedCompare robot forms, action outputs, closed-loop interaction, long-tail coverage, human involvement, and incident coverage.

05Adversarial and backdoor attacksAttack level · Physical evaluation · Defense coverage19 entries

Scroll horizontally to inspect all columns

Adversarial and backdoor attacks
PaperEmbodimentPhys.Task fail.SafetyDefense
HumEnvSoc
Control-Level Adversarial Attacks
AdvDO ↗Driving✕✓✕✓✕✓
EVA-VLA ↗Manipulation✓✓✕✓✕✓
AdvGrasp ↗Manipulation✓✓✕✓✕✕
FreezeVLA ↗Manipulation✕✓✕✕✕✕
RedVLA ↗Manipulation✓✕✕✓✕✓
ObeyPatch ↗Manipulation✓✓✕✓✕✕
Control-Level Backdoor Attacks
VLNBackdoor ↗Navigation✓✓✕✓✕✕
GoBA ↗Manipulation✓✓✕✓✕✕
AttackVLA ↗Manipulation✓✓✕✓✕✓
BadVLA ↗Manipulation✓✓✕✓✕✓
DropVLA ↗Manipulation✓✓✕✓✕✕
Task-Level Jailbreaks
BadNAVer ↗Navigation✓✕✓✓✓✕
BADROBOT ↗Manipulation✓✕✓✓✓✓
ROBOPAIR ↗Generalist✓✓✓✓✓✕
DecisionAttack ↗Generalist✕✓✓✓✓✕
PathManip ↗Navigation✕✓✕✕✕✓
Task-Level Backdoor Attacks
BadVLMDriver ↗Driving✓✓✓✕✕✓
BALD ↗Generalist✓✓✓✓✕✓
TrojanRobot ↗Manipulation✓✓✕✓✕✓
TABLE 05 / ADVERSARIAL TESTING

How attacks reach planning and controlCompare adversarial and backdoor attacks by physical evaluation, task failure, safety impact, and reported defenses.

VLA architectures

VLA broadly refers to systems that connect vision and language to action. The three panels compare whether safety-relevant information appears inside one policy, through an intermediate layer, or through an external agent interface. These architectures also encompass world-action models (WAMs), depending on where action-conditioned future prediction is integrated into the system.

01 / Monolithic VLAs

Semantic information remains mostly implicit inside an end-to-end policy that maps observations and language to actions.

02 / Hierarchical VLAs

Reasoning, plans, or predicted futures appear before low-level control.

03 / Agent-based VLAs

Plans, skills, programs, or tool calls are exposed and executed through external controllers.

Architectures of monolithic, hierarchical, and agent-based VLA systems and their action pipelines.Open figure
REPRESENTATIVE VLA PARADIGMS

Architecture determines where safety can interveneMonolithic, hierarchical, and agent-based systems expose different interfaces between semantic information and physical control.

Research outlook

Four linked requirements connect safety semantics to action. The synthesis figure summarizes the requirements and the corresponding open challenges.

Four requirements for semantic action safety: boundary coverage, semantic-to-action fidelity, contextual intervention and recovery, and incident-path validation.Open figure
SEMANTIC ACTION SAFETY SYNTHESIS

Four linked requirements connect semantics to actionBoundary coverage, fidelity, intervention, and incident-path validation form a cycle for improving the next system iteration.

  • Evidence Safety evaluation should follow a realizable incident pathway.
  • Attribution The pathway links the precursor, required response, and consequence.
  • Feedback The resulting repair should improve learning, safeguards, and future validation.
01CAPTURESafety boundary coverage

Training evidence must expose the contextual distinctions that change whether an action is admissible. More data can repeat familiar conditions while leaving these boundaries unresolved.

Research direction Use failures, near misses, takeovers, and counterfactual scenarios to discover missing safety distinctions and guide targeted data acquisition.

02PRESERVESemantic-to-action fidelity

Recognizing a hazard or producing a plausible explanation does not establish that the represented safety information influenced physical action.

Research direction Test whether changing a safety-relevant condition causes the expected action change, while irrelevant contextual changes preserve behavior.

03ENFORCEContextual intervention & recovery

A correct risk judgment must lead to a suitable response while recovery is still possible. Ambiguity, capability mismatch, and imminent physical danger may require different interventions.

Research direction Connect risk estimates to intervention timing, recoverability, clarification, replanning, and human escalation.

04EVIDENCEIncident-path validation

Aggregate success, refusal, and collision metrics can reveal a failure without identifying where safety-relevant information stopped influencing behavior.

Research direction Evaluate complete pathways from precursor and context to expected response, executed action, and consequence; check whether a correction repairs the same pathway.

Reference collection

Search the accompanying bibliography by lifecycle stage, topic, year, and venue. Scroll within the collection to browse all 182 references.

Survey reference collection
YearVenuePaper and topics
2026AAAI10 Open Challenges Steering the Future of Vision-Language-Action Models
BackgroundContext, Standards, and Surveys
2026IEEE/ASME Transactions on MechatronicsA Dual-System Vision-Language-Action Model for Rational Manipulation
Design timeReasoning-Mediated Safety
2026IEEE Transactions on Neural Networks and Learning SystemsA Survey on Vision-Language-Action Models for Embodied AI
BackgroundContext, Standards, and Surveys
2026arXivACME: A Multi-Cultural, Multi-Embodiment Social-Navigation Dataset
Validation timeControl-Centric Evaluation
2026arXivAction Hallucination in Generative Vision-Language-Action Models
BackgroundSafety Perspectives and Position Papers
2026CVPRAGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
Design timeDeployment timeValidation timeAgentic Action Alignment · Semantic Verification · Reasoning-Centric Evaluation
2026ICRAAnySafe: Adapting Latent Safety Filters at Runtime via Safety Constraint Parameterization in the Latent Space
Deployment timeReachability-Based Filters
2026IROSAsk, Reason, Assist: Robot Collaboration via Natural Language and Temporal Logic
Deployment timeFormal Runtime Verification
2026Science RoboticsBeyond Alignment: Why Robotic Foundation Models Need Context-Aware Safety
BackgroundSafety Perspectives and Position Papers
2026ICMLCaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2026arXivContextual Safety Reasoning and Grounding for Open-World Robots
Deployment timeControl-Barrier Function Filters
2026ICLRCosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2026CVPRCounterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning
Design timeWorld-Model-Based Safety
2026CVPR FindingsDevil is in Narrow Policy: Unleashing Exploration in Driving VLA Models
Design timeWorld-Model-Based Safety
2026arXivDo Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning
Design timeReasoning-Mediated Safety
2026ICRADo What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification
Deployment timeValue Estimation
2026IROSDropVLA: An Action-Level Backdoor Attack on Vision–Language–Action Models
Validation timeControl-Level Backdoors
2026arXivFail2Drive: Benchmarking Closed-Loop Driving Generalization
Validation timeControl-Centric Evaluation
2026IROSFailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models
Design timeSafety-Critical Data Generation
2026ICRAFrom Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies
Deployment timeReachability-Based Filters
2026IEEE Robotics and Automation LettersFrom Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection
Deployment timeValue Estimation
2026ICLRFrom Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning
Design timeControl-Constrained Safety
2026Google DeepMind technical reportGemini Robotics 2: Safety Evaluations
Deployment timeValidation timeSemantic Verification · Uncertainty-Aware Control Prediction · Reasoning-Centric Evaluation
2026arXivGuided Action Flow: Value-Guided Sampling for Frozen Vision-Language-Action Policies
Deployment timeGuided Action Generation
2026RSSGuided Streaming Stochastic Interpolant Policy
Deployment timeGuided Action Generation
2026arXivHazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
Validation timeControl-Centric Evaluation
2026CoRLHow Well Do Latent World Models Understand Partially Observable Safety Constraints?
Deployment timeReachability-Based Filters
2026NVIDIA technical blogInside NVIDIA Halos for Robotics: A Full-Stack Functional Safety System for Physical AI
BackgroundContext, Standards, and Surveys
2026AAAIIS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks
Deployment timeValidation timeFormal Runtime Verification · Reasoning-Centric Evaluation
2026CVPRLatent Chain-of-Thought World Modeling for End-to-End Driving
Design timeHybrid Safety
2026arXivLearning from Mistakes: Post-Training for Driving VLA with Takeover Data
Design timeWorld-Model-Based Safety
2026ECCVLIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models
Validation timeControl-Centric Evaluation
2026RSSmimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2026CoRLMolmoAct2: Action Reasoning Models for Real-World Deployment
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2026ICRANavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance
Design timeDeployment timeImitation-Driven Safety · Safety-Critical Data Generation · Value Estimation
2026arXivNeuro-Symbolic Safety Guidance for Vision-Language-Action Models via Constrained Flow Matching
Deployment timeGuided Action Generation
2026RSSOopsieVerse: A Safety Benchmark with Damage-Aware Simulation for Robot Manipulation
Validation timeControl-Centric Evaluation
2026ICLROpenfly: A comprehensive platform for aerial vision-language navigation
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2026arXivOpenLongTail: Generative Scaling of Long-Tail Driving Data
Design timeSafety-Critical Data Generation
2026AAAIPhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems
Validation timeTask-Level Jailbreaks and Backdoors
2026ICML Position PaperPosition: Good Embodied Reward Models Need Bad Behavior Data
BackgroundSafety Perspectives and Position Papers
2026ICRAPreventing Robotic Jailbreaking via Multimodal Domain Adaptation
Validation timeMitigation Evaluation
2026arXivRed-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policies
Validation timePerturbation Testing
2026arXivRedVLA: Physical Red Teaming for Vision-Language-Action Models
Validation timeControl-Level Adversarial Attacks
2026ICRAReimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control
Deployment timeNeural World Modeling
2026ICLR ESR WorkshopRoboSafe: Safeguarding Embodied Agents via Executable Safety Logic
Deployment timeFormal Runtime Verification
2026IROSSafe-Night VLA: Seeing the Unseen via Thermal-Perceptive Vision-Language-Action Models for Safety-Critical Manipulation
Deployment timeControl-Barrier Function Filters

Handles perceptual blind spots—such as hot objects and reflections—using safety constraints and a runtime safety filter.

2026arXivSafeClaw-R: Towards Safe and Secure Multi-Agent Personal Assistants
BackgroundContext, Standards, and Surveys
2026IEEE Robotics and Automation LettersSafety Guardrails for LLM-Enabled Robots
Deployment timeFormal Runtime Verification
2026arXivSafety in Embodied AI: A Survey of Risks, Attacks, and Defenses
BackgroundContext, Standards, and Surveys
2026PreprintSafety of Vision-Language-Action Models: A Survey from Lifecycle Perspectives
BackgroundContext, Standards, and Surveys
2026CVPRScaling-Aware Data Selection for End-to-End Autonomous Driving Systems
Design timeSafety-Critical Scenario Sampling
2026RSSSelf-Supervised Bootstrapping of Action-Predictive Embodied Reasoning
Design timeReasoning-Mediated Safety
2026arXivSidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks
Validation timeControl-Centric Evaluation
2026arXivSpanVLA: Learning from Negative-Recovery Samples with Fast Action Bridging for Vision-Language-Action Model
Design timeReasoning-Mediated Safety
2026CoRLStressDream: Steering Video World Models for Robust Policy Evaluation and Improvement
Validation timeSafety-Critical Scenario Generation
2026arXivStyleVLA: Driving Style-Aware Vision Language Action Model for Autonomous Driving
Design timeReasoning-Mediated Safety
2026ICMLTIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments
Design timeValidation timeReasoning-Mediated Safety · Control-Centric Evaluation
2026ACM Computing SurveysTowards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks
BackgroundContext, Standards, and Surveys
2026arXivUnleashing VLA Potentials in Autonomous Driving via Explicit Learning from Failures
Design timeWorld-Model-Based Safety
2026IROSUnsupervised Discovery of Failure Taxonomies from Deployment Logs
Design timeSafety-Critical Scenario Sampling
2026arXivVision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms
BackgroundContext, Standards, and Surveys
2026ICMLVLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models
Validation timeControl-Centric Evaluation
2026IROSVLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer
Deployment timeControl-Barrier Function Filters
2026EMNLP FindingsWaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Driving
Validation timeReasoning-Centric Evaluation
2026CVPRWhen Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models
Validation timeControl-Level Adversarial Attacks
2026RSSWhen to Act, Ask, or Learn: Uncertainty-Aware Policy Steering
Deployment timeUncertainty-Aware Control Prediction
2026CVPRWOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-Tail Scenarios
Validation timeControl-Centric Evaluation
2026CoRLWorld Action Models Are Zero-Shot Policies
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2026arXivπ0.7: A Steerable Generalist Robotic Foundation Model with Emergent Capabilities
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2025ICCV3D Gaussian Splatting Driven Multi-View Robust Physical Adversarial Camouflage Generation
Validation timeControl-Level Adversarial Attacks
2025arXivA Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents
Validation timeReasoning-Centric Evaluation
2025IEEE Robotics and Automation SocietyA Pathway Study for Future Humanoid Standards
BackgroundSafety Perspectives and Position Papers
2025arXivAdversarial Attacks on Robotic Vision-Language-Action Models
Validation timeControl-Level Adversarial Attacks
2025IJCAIAdvGrasp: Adversarial Attacks on Robotic Grasping from a Physical Perspective
Validation timeControl-Level Adversarial Attacks
2025arXivAlpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
Design timeHybrid Safety
2025arXivANNIE: Be Careful of Your Robots
Validation timeControl-Centric Evaluation
2025arXivAttackVLA: Benchmarking Adversarial and Backdoor Attacks on Vision-Language-Action Models
Validation timeControl-Level Backdoors
2025arXivAttention-Guided Patch-Wise Sparse Adversarial Attacks on Vision-Language-Action Models
Validation timeControl-Level Adversarial Attacks
2025NeurIPSAutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
Design timeReasoning-Mediated Safety
2025arXivBadNAVer: Exploring Jailbreak Attacks on Vision-and-Language Navigation
Validation timeTask-Level Jailbreaks and Backdoors
2025ICLRBadRobot: Jailbreaking Embodied LLM Agents in the Physical World
Design timeValidation timeAgentic Action Alignment · Task-Level Jailbreaks and Backdoors
2025arXivBadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization
Validation timeControl-Level Backdoors
2025arXivCan AI Perceive Physical Danger and Intervene?
Design timeValidation timeAgentic Action Alignment · Reasoning-Centric Evaluation
2025ICLRCan We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems
Validation timeTask-Level Jailbreaks and Backdoors
2025arXivCosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
Design timeSafety-Critical Data Generation
2025Waymo blogDemonstrably Safe AI for Autonomous Driving
BackgroundContext, Standards, and Surveys
2025AAAIDiffScene: Diffusion-Based Safety-Critical Scenario Generation for Autonomous Vehicles
Validation timeSafety-Critical Scenario Generation
2025ICMLDINO-WM: World Models on Pre-Trained Visual Features Enable Zero-Shot Planning
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2025IEEE Robotics and Automation LettersDon't Let Your Robot Be Harmful: Responsible Robotic Manipulation via Safety-as-Policy
Deployment timeSemantic Verification
2025arXivEmbodied AI: Emerging Risks and Opportunities for Policy Action
BackgroundSafety Perspectives and Position Papers
2025TMLREMMA: End-to-End Multimodal Model for Autonomous Driving
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2025arXivEvaluating Gemini Robotics Policies in a Veo World Simulator
Design timeValidation timeSafety-Critical Data Generation · Safety-Critical Scenario Generation
2025ICCVExploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics
Validation timeControl-Level Adversarial Attacks
2025arXivFreezeVLA: Action-Freezing Attacks against Vision-Language-Action Models
Validation timeControl-Level Adversarial Attacks
2025arXivFrom Foresight to Forethought: VLM-in-the-Loop Policy Steering via Latent Alignment
Deployment timeNeural World Modeling
2025arXivGAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving
Design timeSafety-Critical Data Generation
2025arXivGemini Robotics: Bringing AI into the Physical World
Design timeReasoning-Mediated Safety
2025RSSGeneralizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis
Deployment timeReachability-Based Filters
2025arXivGenerating Robot Constitutions & Benchmarks for Semantic Safety
Design timeValidation timeAgentic Action Alignment · Reasoning-Centric Evaluation
2025arXivGoal-Oriented Backdoor Attack against Vision-Language-Action Models via Physical Objects
Validation timeControl-Level Backdoors
2025arXivGR00T N1: An Open Foundation Model for Generalist Humanoid Robots
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2025arXivIRL-VLA: Training an Vision-Language-Action Policy via Reward World Model
Design timeWorld-Model-Based Safety
2025ICRAJailbreaking LLM-Controlled Robots
Design timeValidation timeAgentic Action Alignment · Task-Level Jailbreaks and Backdoors
2025arXivModel-Agnostic Adversarial Attack and Defense for Vision-Language-Action Models
Validation timeControl-Level Adversarial Attacks
2025NVIDIA safety reportNVIDIA Autonomous Vehicles Safety Report
BackgroundContext, Standards, and Surveys
2025arXivPoutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving
Design timeReasoning-Mediated Safety
2025arXivPredictive Red Teaming: Breaking Policies Without Breaking Robots
Validation timePerturbation Testing
2025CoRLReal-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning
Deployment timeUncertainty-Aware Control Prediction
2025ICCVRethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparities
Validation timePerturbation Testing
2025IROSRobots That Suggest Safe Alternatives
Deployment timeReachability-Based Filters
2025arXivRoboView-Bias: Benchmarking Visual Bias in Embodied Agents for Robotic Manipulation
BackgroundContext, Standards, and Surveys
2025NeurIPSRvLLM: LLM Runtime Verification with Domain Knowledge
Deployment timeFormal Runtime Verification

General LLM runtime verification, evaluated on legal and mathematical tasks.

2025arXivSafePlan: Leveraging Formal Logic and Chain-of-Thought Reasoning for Enhanced Safety in LLM-based Robotic Task Planning
Deployment timeValidation timeFormal Runtime Verification · Reasoning-Centric Evaluation
2025Foundations and Trends in Privacy and SecuritySafety at Scale: A Comprehensive Survey of Large Model and Agent Safety
BackgroundContext, Standards, and Surveys
2025NeurIPS SpotlightSafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning
Design timeValidation timeControl-Constrained Safety · Control-Centric Evaluation
2025arXivSciFi-Benchmark: Leveraging Science Fiction to Improve Robot Behavior
Validation timeReasoning-Centric Evaluation
2025ICRASelf-Improving Autonomous Underwater Manipulation
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2025ICRASELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models
Deployment timeFormal Runtime Verification
2025KDDSSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation
Design timeSafety-Critical Scenario Sampling
2025EMNLPSubtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making
Deployment timeFormal Runtime Verification
2025CoRLUncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
Deployment timeReachability-Based Filters · Uncertainty-Aware Control Prediction
2025ICRAUpdating Robot Safety Representations Online from Natural Language Feedback
Deployment timeTask-Level Guardrail
2025arXivVidar: Embodied Video Diffusion Model for Generalist Manipulation
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2025arXivWhen Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models
Validation timeControl-Level Adversarial Attacks
2025ICMLWOMD-Reasoning: A Large-Scale Dataset for Interaction Reasoning in Driving
Validation timeReasoning-Centric Evaluation
2025RSSπ₀: A Vision-Language-Action Flow Model for General Robot Control
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2024EngineeringA Survey on an Emerging Safety Challenge for Autonomous Vehicles: Safety of the Intended Functionality
BackgroundContext, Standards, and Surveys
2024arXivAutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
Deployment timeSemantic Verification
2024CoRLAvoid Everything: Model-Free Collision Avoidance with Expert-Guided Fine-Tuning
Design timeSafety-Critical Data Generation
2024NeurIPS Datasets and BenchmarksBench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-to-End Autonomous Driving
Validation timeControl-Centric Evaluation
2024SACMATBlueSky: How to Raise a Robot - A Case for Neuro-Symbolic AI in Constrained Task Planning for Humanoid Assistive Robots
Deployment timeFormal Runtime Verification
2024CVPRDriving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving
Deployment timeNeural World Modeling
2024IROSEnsuring Safety in LLM-Driven Robotics: A Cross-Layer Sequence Supervision Mechanism
Deployment timeFormal Runtime Verification
2024NeurIPSEveryday Object Meets Vision-and-Language Navigation Agent via Backdoor
Validation timeControl-Level Backdoors
2024arXivExploring the Robustness of Decision-Level through Adversarial Attacks on LLM-Based Embodied Models
Validation timeTask-Level Jailbreaks and Backdoors
2024CoRLFREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversariality
Validation timeSafety-Critical Scenario Generation
2024arXivGemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2024ICMLHarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Design timeAgentic Action Alignment
2024arXivHuman-AI Safety: A Descendant of Generative AI and Control Systems Safety
BackgroundSafety Perspectives and Position Papers
2024IEEE Robotics and Automation LettersLTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-Based Planning
Deployment timeGuided Action Generation
2024IROSMalicious Path Manipulations via Exploitation of Representation Vulnerabilities of Vision-Language Navigation Systems
Validation timeTask-Level Jailbreaks and Backdoors
2024NeurIPS Datasets and BenchmarksNAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
Validation timeControl-Centric Evaluation
2024ICRANoMaD: Goal Masked Diffusion Policies for Navigation and Exploration
Design timeImitation-Driven Safety
2024RSSOcto: An Open-Source Generalist Robot Policy
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2024CoRLOpenVLA: An Open-Source Vision-Language-Action Model
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2024arXivPhysical Backdoor Attack Can Jeopardize Driving with Vision-Large-Language Models
Validation timeTask-Level Jailbreaks and Backdoors
2024ICRAPlug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents
Deployment timeFormal Runtime Verification
2024RSSReal-Time Anomaly Detection and Reactive Planning with Large Language Models
Deployment timeSemantic Anomaly Detection
2024RSSRoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2024arXivRobot Collapse: Supply Chain Backdoor Attacks Against VLM-based Robotic Manipulation
Validation timeTask-Level Jailbreaks and Backdoors
2024Communications PsychologyStudying and Improving Reasoning in Humans and Machines
BackgroundContext, Standards, and Surveys
2024ECCVThink2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)
Design timeWorld-Model-Based Safety
2024CoRLTokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
Design timeReasoning-Mediated Safety
2024arXivTowards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
BackgroundSafety Perspectives and Position Papers
2023IEEE Transactions on Intelligent Transportation SystemsA Survey on Safety-Critical Driving Scenario Generation—A Methodological Perspective
BackgroundContext, Standards, and Surveys
2023ICRACode as Policies: Language Model Programs for Embodied Control
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2023arXivGAIA-1: A Generative World Model for Autonomous Driving
Design timeSafety-Critical Data Generation
2023TPAMIHuman-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation
Design timeControl-Constrained Safety
2023CoRLRobots That Ask for Help: Uncertainty Alignment for Large Language Model Planners
Deployment timeSemantic Anomaly Detection
2023Autonomous RobotsSemantic Anomaly Detection with Large Language Models
Deployment timeSemantic Anomaly Detection
2023Center for AI SafetyStatement on AI Extinction Risk
BackgroundContext, Standards, and Surveys
2023ACM Computing SurveysTaxonomy of Machine Learning Safety: A Survey and Primer
BackgroundContext, Standards, and Surveys
2023CoRLViNT: A Foundation Model for Visual Navigation
Design timeImitation-Driven Safety
2023CoRLVoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2022IEEE Transactions on Emerging Topics in Computational IntelligenceA Survey of Embodied AI: From Simulators to Research Tasks
BackgroundContext, Standards, and Surveys
2022ECCVAdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Validation timeControl-Level Adversarial Attacks
2022CoRLBEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2022CoRLDo as I Can, Not as I Say: Grounding Language in Robotic Affordances
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2022Robotics and Autonomous SystemsPerceived safety in physical human–robot interaction—A survey
BackgroundSafety Perspectives and Position Papers
2021ICCVLarge Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset
BackgroundGeneral VLA Systems, Robotics, and Benchmarks
2020Soft ComputingHuman-Robot Collisions Detection for Safe Human-Robot Interaction Using One Multi-Input-Output Neural Network
BackgroundContext, Standards, and Surveys
2020Safety ScienceSafety Bounds in Human Robot Interaction: A Survey
BackgroundContext, Standards, and Surveys
2019IEEEEthically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems
BackgroundContext, Standards, and Surveys
2019VikingHuman Compatible: Artificial Intelligence and the Problem of Control
BackgroundContext, Standards, and Surveys
2016ISO/TS 15066ISO/TS 15066:2016 Robots and Robotic Devices – Collaborative Robots
BackgroundContext, Standards, and Surveys
2014ISO/IEC Guide 51ISO/IEC Guide 51:2014 Safety Aspects – Guidelines for Their Inclusion in Standards
BackgroundContext, Standards, and Surveys
2004Spectra (ebook)I, Robot
BackgroundContext, Standards, and Surveys
1985Journal of Dynamic Systems, Measurement, and ControlImpedance Control: An Approach to Manipulation: Part I—Theory
BackgroundContext, Standards, and Surveys
1985ICRAReal-Time Obstacle Avoidance for Manipulators and Mobile Robots
BackgroundContext, Standards, and Surveys
1950In *I, Robot*Runaround
BackgroundContext, Standards, and Surveys
n.d.Tesla websiteSafety at Tesla
BackgroundContext, Standards, and Surveys

Citation & contribution

Use the repository citation below.

REPOSITORY CITATION / BIBTEX
@misc{chung2026humancenteredsafety,
  title = {Towards Human-Centered Safety in Vision-Language-Action Robot Control: A Survey and Outlook},
  author = {Nhat Chung and Toan Nguyen and Tianrun Hu and Yun Xing and Puming Jiang and Tuan-Anh Vu and Bohao Qu and Yedi Zhang and Yue Cao and Jie Zhang and Qing Guo and Yue Wang and Ming-Ming Cheng and Jin Song Dong and Cheston Tan and Harold Soh and David Hsu and Daniel Seita and Ivor Tsang},
  year = {2026},
  howpublished = {Survey repository},
  url = {https://github.com/Human-Centered-VLA/Awesome-Human-Centered-VLA-Safety}
}

Survey figure

Figures open fitted to the window. Use “View 100%” to inspect embedded text, or Open original image.