Physical work
Accomplishing work through a body
A robot is a powered physical system that senses and acts to perform assigned tasks. It need not operate independently: teleoperation uses continuing human input, while autonomous operation delegates some perception, decisions, and execution to the machine. The useful distinction is how responsibility is divided under particular task and environmental conditions, not whether a product is labeled intelligent. NIST's autonomy terminology makes those conditions and human roles explicit.
Embodiment means that behavior depends on the machine's body, measurements, and possible movements. A sensor measures a physical quantity. An actuator converts supplied energy into motion or force, as an electric motor does. Manipulation acts on objects through moving appendages; mobility moves the robot through its surroundings. Software cannot choose a useful action without respecting those physical capabilities. The sensor and actuator foundations connect these terms to their hardware meanings.
Consider placing a part in a fixed holder. The engineering problem includes reaching the holder, carrying the load, and meeting a placement tolerance. Moving through changing surroundings adds a different obligation: yesterday's clear route may be obstructed now. Human access introduces further constraints on permissible movement. Useful automation therefore starts with a physical task and its conditions, not a model selected in isolation.
Closing the physical loop
The physical loop begins with measurements. State estimation infers current conditions from them; planning proposes future behavior; a controller turns desired behavior into actuator commands. A policy is a rule or learned process for selecting actions. These are responsibilities, not necessarily separate programs: one model may perform several, while several controllers may share execution.
Closed-loop control uses measured consequences to change subsequent commands. An open-loop sequence proceeds without those corrections. If a wheel turns less than requested, the next decision must use the measured movement rather than assume the command succeeded. This resembles observation-based agent decisions, but the physical world continues evolving while computation runs. Runtime correction need not change any learned parameters.
Feedback also has levels. Joint measurements can establish that a motion target was tracked within a tolerance. They do not establish that an object remained grasped or was placed correctly. A complete system needs both movement feedback and observations of task effects, with protective supervision governing what execution may do.
Continuing foundations
Robotics developed several architectural traditions because repeatable motion, goal-directed planning, and timely reaction are different problems. The following developments explain why modern systems still combine them.
| Development | Contribution |
|---|---|
| Programmed Article Transfer — filed 1954; granted 1961 | George Devol's patent combined recorded position commands with sensed-position comparison and coordinated gripping. Reprogrammable physical work did not require learning from data. |
| Unimate — 1961 industrial application | GM records an installation moving hot die-cast parts into cooling pools at its Ternstedt plant in Trenton. This was a concrete handling task, distinct from the patent's design disclosure. |
| Shakey — 1966–1972 | SRI's mobile-robot project integrated perception, an environment model, planning, and execution. Its action routines included correction, and PLANEX monitored execution; symbolic planning was not simply an unobserved command list. |
| Layered control — March 1986 | Rodney Brooks's layered-control paper organized asynchronous, task-achieving behaviors. Higher layers could influence lower ones while useful sensor-driven behavior continued without waiting for a complete central model. |
These approaches coexist. A learned task policy can select a goal, a planner can find a route, and a feedback controller can track it while a separate monitor restricts motion. New learning methods change how some decisions are obtained; they do not remove the need to measure, react, and verify effects.
Bodies and measurements
Expressing positions in robot coordinates
A displacement vector gives direction and distance through components along axes. A coordinate frame defines an origin and oriented axes; pose combines position and orientation relative to a frame.
A quarter-turn example re-expresses coordinates of the same stationary point, separating rotation of its components from translation between frame origins. MIT's spatial-algebra notes develop the notation.
One point in two frames
ExampleThe point does not move.
Coordinate conversion
Axis arrows show coordinate directions; their lengths are illustrative.
- 1. A x
- 2. A y
- 3. B x
- 4. B y
- 5. B components
- 6. P
Read coordinates and regions as data
X: -0.5–2.5 m; Y: -0.5–2.5 m, increasing up. Equal scale on both axes.
(0, 0); (0.8, 0)
(0, 0); (0, 0.8)
(1, 0); (1, 0.8)
(1, 0); (0.2, 0)
(1, 0); (1, 2); (0, 2)
(0, 2)
A: (0, -0.2)
B: (1, -0.2)
P: A=(0,2), B=(2,1): (0.05, 2.22)
+2 along B x: (1.15, 1.3)
+1 along B y: (0.5, 1.8)
A pixel is not yet such a point: several depths can project onto the same pixel. Image-to-scene geometry explains that ambiguity. Camera intrinsics describe projection within the image; extrinsics relate physical frames. Hand-eye calibration estimates the camera-to-gripper relationship from robot movements. Given a measured camera-frame 3D point, that relationship and the robot's configuration connect the observation to base coordinates.
A moving camera adds time to the contract. The image timestamp should identify acquisition, and its frame relationship must describe that same time. Using a newer transform answers a different geometric question and can misplace an otherwise correct measurement.
From joints to tool pose
Robot links connect through turning (revolute) or sliding (prismatic) joints. A degree of freedom is an independent coordinate; linkage constraints can couple joints. A configuration specifies body arrangement, and configuration space contains these arrangements. Workspace describes physical space or selected tool coordinates. The end effector is the task-facing tool.
Forward kinematics computes tool pose from configuration; inverse kinematics seeks configurations for a desired pose. Targets may have several solutions or none. The kinematics notes explain why equal tip position need not mean equal posture.
The Jacobian maps joint velocities to instantaneous tool velocity. At a singularity, that mapping loses one or more achievable tool-motion directions. Near such a configuration, even a small requested velocity in those directions can require large joint velocities. Other directions can remain available. Modern Robotics explains this directional limitation.
Same tip position, different motion directions
Two one-metre links. q1 is the base joint; q2 turns the second link relative to the first. The tip is (cos q1 + cos(q1+q2), sin q1 + sin(q1+q2)). This model considers translational tip velocity, not a complete industrial-arm pose.
Link geometry
Available instantaneous velocity
Tip (1.000, 1.000) m · final-link angle 90° · velocity semiaxes 1.618, 0.618 m/s. Both instantaneous translation directions remain available.
Velocity = J × joint rate, with J = [-1.000, -1.000; 1.000, 0.000] m/rad. The pictured set uses √(q̇1² + q̇2²) ≤ 1 rad/s, a joint-rate norm bound—not independent per-joint limits. Geometry and velocity axes use equal scales within each plot.
What sensors measure
Proprioception measures the robot itself; exteroception measures its surroundings. Neither supplies complete state. Useful readings depend on operating range, resolution, and update frequency. Occlusion hides surfaces; noise varies measurements; bias is a persistent offset; sampling and delivery delay make readings old. Robot Sensors explains these measurement categories.
| Measurement | Useful information | What remains unresolved |
|---|---|---|
| Camera image | Visible appearance in image coordinates | A pixel alone supplies neither metric depth nor hidden surfaces. |
| Depth or laser range | Distance along measured directions | Unobserved space is not established as free. |
| Joint encoder | Mechanical rotation or displacement | Joint position alone does not establish object retention. |
| Inertial measurement unit, or IMU | Angular velocity and acceleration-related measurements | Orientation, gravity, and bias affect their interpretation. |
| Force and tactile sensing | Mechanical interaction and local contact changes | Local contact evidence does not describe the entire scene. |
Units and missing-value conventions belong to the interface. ROS 2's JointState uses radians or metres for position, corresponding rates for velocity, and newton-metres or newtons for effort; all entries describe one measurement time. Its Imu message includes covariance fields describing uncertainty. An all-zero covariance field means uncertainty is unknown, not zero; a first entry of minus one marks an unavailable estimate that consumers should disregard.
Raw accelerometer readings are not directly gravity-free translational acceleration. Orientation determines their conversion to a global frame and gravity removal. In an isolated constant-bias model, acceleration bias creates velocity error and position error after time . This explains why integration alone can drift; it is not a prediction for every fused estimator. Woodman's inertial-navigation report develops the error sources.
Complementary sensing matters most when the physical claim changes. Visual recognition may locate an object; tactile sensing can detect motion at its contact with the gripper. GelSight, for example, observes deformation and relative movement at a contacted surface to detect slip. Recognizing the object and establishing a stable grasp are different observations.
Estimating the world
Predicting and correcting state
A robot's state contains the physical quantities needed for its decisions. A belief represents uncertainty over possible states. Estimation predicts how that belief changes under motion, then corrects it using an observation likelihood: how compatible each possible state is with the measurement. A wrong motion or measurement model can produce a confidently wrong estimate.
In his 1960 filtering paper, R. E. Kalman developed a linear estimation method that propagated both state estimates and their uncertainty. Tracking how uncertainty changed supplied time-varying weights for the filter's corrections. This connected prediction and measurement correction in a recursive computation, accommodating changing statistical conditions within one formulation.
The standard Bayesian Kalman filter assumes linear state transitions and measurements, with Gaussian initial uncertainty and noise. A Gaussian distribution is bell-shaped: its mean locates its center, while its covariance describes the spread and how components vary together. Prediction propagates the mean and covariance and adds process uncertainty. Correction weights the measurement residual—the measured observation minus its prediction—using the Kalman gain, computed from the modeled uncertainties. The result is exact under these assumptions, not for arbitrary robot uncertainty. Bayesian Filtering and Smoothing develops the method.
Uncertainty determines the correction
Scalar position: predicted mean 0 m and variance P = 1 m²; observed position z = 2 m. Identity measurement model z = x + noise, with independent Gaussian measurement noise of variance R.
| Curve / style | Mean (m) | Variance (m²) |
|---|---|---|
| Prediction · solid cyan | 0.000 | 1.000 |
| Measurement likelihood · dashed gold | 2.000 | 1.000 |
| Posterior · dotted violet | 1.000 | 0.500 |
Gain K = P/(P+R) = 0.500; corrected mean = 0 + K(2−0) = 1.000 m; corrected variance = (1−K)P = 0.500 m².
Nonlinear systems require other methods or approximations, such as local linearization. One Gaussian cannot preserve two separated plausible poses; weighted particles can represent alternatives, though finite sampling can miss them. Combining estimates also requires accounting for shared information: treating correlated evidence as independent can overstate what was learned.
Observability asks which differences in state or calibration the measurements can distinguish. It is unrelated to whether software emits plentiful logs. Repeating an uninformative motion may leave a parameter unresolved. Visual-inertial observability analysis identifies such degeneracies for its particular model. Kalibr makes the practical requirement concrete: camera–IMU calibration needs informative rotational and translational motion, consistent timestamps, and appropriate sensor-noise models. More samples do not substitute for those conditions.
Localization and maps
Localization estimates robot pose relative to a map. Mapping estimates the surroundings; simultaneous localization and mapping, or SLAM, estimates both. Odometry accumulates motion estimates, so errors can accumulate too. Recognizable places provide additional constraints, but associating an observation with the wrong place can mislead the estimate.
Elfes's 1989 occupancy-grid article represented space as cells with occupancy probabilities, preserving uncertainty rather than immediately declaring definite objects. In its ideal range example, cells before a measured surface are free, the surface cell is occupied, and cells beyond remain unknown. Unknown does not mean empty. An occupancy estimate also differs from an object label or a judgment that the robot can traverse the area. The original article explains the sensor-model updates.
Loop closure adds a constraint when current sensing matches a previously mapped place. In Cartographer, published in 2016, Wolfgang Hess and colleagues built local submaps, then jointly optimized scan and submap poses using revisits to reduce accumulated inconsistency. Repeated-looking structures could create false matches, so robust optimization reduced outlier influence. The reported five-centimetre grid resolution was a map discretization, not a universal localization-error bound.
When several poses remain plausible, active localization chooses an action that obtains a distinguishing observation. Movement can therefore improve knowledge as well as change location. Even a well-localized robot still needs current obstacle observations: a stored map describes earlier information, not a guarantee that the route remains clear.
Planning and control
Grounding goals in skills
A goal describes a desired outcome; a skill is an executable behavior intended to produce an effect. Its contract needs preconditions, completion observations, and failure conditions. A grasp proposal is useful only if the object is accessible and the robot can execute the movement. Task and motion planning couples discrete choices—such as which object to move—with geometric feasibility. General agent planning supplies the broader decomposition idea; robotics adds these physical obligations.
SayCan, released by Google and Everyday Robots in April 2022, combined a language model's score for a skill's usefulness with an estimate of its success in the current state. That state-dependent opportunity is an affordance. The selected skill executed through an existing robot behavior, rather than making the language model a motor controller. The project explanation also identifies a limitation: current-step value estimates need not provide enough feedback about failures or environmental changes.
Sometimes the next useful skill obtains information. Opening a drawer can reveal its contents; moving an occluder can reveal space behind it. Garrett and colleagues' ICRA 2020 belief-space replanning system coupled such observation actions with motion choices, updating the plan after execution. This is why a complete symbolic plan cannot always be settled before movement begins: actions change both physical conditions and what the robot knows.
Paths and feasible motion
A path specifies geometric configurations; a trajectory assigns timing. Velocity is the rate of position change, and acceleration is the rate of velocity change. Force changes translational motion; torque changes rotational motion, with inertia describing resistance to acceleration. Gravity, contact, and coupled links affect required actuation. A reachable motion can therefore exceed actuator limits at a chosen timing.
Collision checking must consider configurations and their connections, not only endpoints. Clear start and goal positions can be separated by an obstructed movement. Coarse intermediate checking can miss thin obstacles. A path planner's geometry must represent the relevant body and carried objects, not just the tool tip.
Clear endpoints, obstructed movement
ExampleA center path can miss an obstacle while the body's swept region intersects it.
Fixed orientation, horizontal translation
The square's center moves from (1,1) to (5,1). The obstacle lies above the centerline but inside the body's swept extent.
- 1. Swept body region
- 2. Obstacle
- 3. Start body
- 4. End body
- 5. Center path
Read coordinates and regions as data
X: 0–6 m; Y: 0–3 m, increasing up. Equal scale on both axes.
(0.5, 0.5); (5.5, 0.5); (5.5, 1.5); (0.5, 1.5)
(2.7, 1.2); (3.3, 1.2); (3.3, 2.2); (2.7, 2.2)
(0.5, 0.5); (1.5, 0.5); (1.5, 1.5); (0.5, 1.5)
(4.5, 0.5); (5.5, 0.5); (5.5, 1.5); (4.5, 1.5)
(1, 1); (5, 1)
Clear start: (1, 0.25)
Clear end: (5, 0.25)
Obstacle overlaps swept body: (3, 2.45)
Rapidly exploring random trees, introduced by Steven LaValle in 1998, grow by sampling a configuration, finding a nearby tree node, and extending toward the sample. LaValle's account explains their exploration of complicated spaces without exhaustive enumeration. Probabilistic completeness concerns increasing success probability with sufficient sampling; it does not guarantee a solution within a finite planning budget.
Trajectory optimization instead chooses timed states and controls to reduce a cost subject to dynamics and other constraints. Obstacle avoidance can make that optimization nonconvex. Search can find a useful route; optimization can refine timed motion. Neither removes the need to track the result on the actual robot.
Tracking references with feedback
A reference is the desired signal; tracking error is reference minus measurement. Feedforward anticipates required actuation, while feedback corrects observed error. Proportional–integral–derivative control, or PID, combines present error, its accumulated history, and its rate of change.
Saturation means the hardware cannot apply the requested output. Continued integration can then cause windup and delayed recovery; anti-windup limits or corrects that accumulation. Gains and tolerated delay belong to the complete physical loop, not universal tuning constants. Feedback Systems develops these implementation constraints.
Model predictive control, or MPC, repeatedly optimizes a constrained future sequence, applies its first input, measures again, and solves a shifted problem. Its model need not be learned. Predictive action selection explains the planning pattern, while world-model construction covers learned dynamics. Nominal optimization does not automatically establish stability, feasibility under model error, or completion before the next deadline.
Execution interfaces should expose the relevant result. The ROS Joint Trajectory Controller interpolates timed waypoints; its action interface provides feedback and tolerances. A tolerance violation aborts the goal and requests a position hold. Its topic interface lacks equivalent result notification. A requested hold still depends on the hardware's ability to hold its load.
Controlling contact
Touching a surface changes the problem: motion creates reaction forces. Compliance is yielding under force; impedance describes the dynamic relationship between motion and interaction force. Neville Hogan's March 1985 impedance-control papers made that relationship a design objective, rather than prescribing position or force alone. The three-part study connected theory, implementation, and task-appropriate choices.
Stiffness relates displacement to force; damping opposes relative velocity. In a static spring example, : stiffness newtons per metre and displacement metres give newton—not a safe setting. Surface uncertainty can turn position error into force. In the contact's frame, normal means perpendicular to its surface and tangential means parallel. Both force components use that frame. Impedance shapes the response; hybrid control can regulate tangential motion separately from normal force.
Connect the contact frame to sticking forces
Reaction on a contacting block
Force components · μs = 0.5
Example assumptions: μs = 0.5; component demands (Ft, Fn) = (1, 4) N and (3, 4) N. The left arrow is a physical reaction direction; the right points represent force components. Boundaries are Ft = ±0.5Fn with Fn ≥ 0; tensile normal force is excluded.
A contact mode identifies which contacts and constraints currently apply. A foot in flight, heel contact, full-foot contact, and toe contact has different movement constraints. Hybrid models combine continuous motion within a mode with discrete events and state changes at transitions, such as an impact changing velocity. Planning through contact explains why a free-space controller cannot simply assume its previous dynamics remain valid.
Contact feedback can change the next action. In the GelSight grasp experiment, detected slip prompted stopping, lowering and releasing the object, then retrying with a higher contact threshold. That is a reported recovery sequence, not a universal retry policy. Compliance and slip detection improve interaction knowledge; neither alone establishes physical safety.
Timing and authority
Cadence is not freshness
Sensing, policy inference, and actuator updates can run at different rates: a multirate system. Cadence describes intervals between updates. Latency measures delay between named events; jitter is variation in timing; observation age measures elapsed time since acquisition. Real-time correctness includes meeting a deadline, not merely having low average latency. Scheduling, allocation, page faults, and blocking can all disturb that contract. ROS 2's real-time introduction distinguishes these concerns.
Communication belongs in the budget. In Rishabh Garg's control-systems talk, ten assumed 100-bit messages consume about one millisecond on a one-megabit-per-second Controller Area Network, or CAN, bus. Serialized with a two-millisecond policy, the simplified iteration takes three milliseconds. Pipelining can improve update cadence, but it creates handoff and synchronization obligations.
There are two distinct ways this can fail. A policy that misses a scheduled transmit slot can leave a command queued until the next one, producing a burst. Separately, a delayed receive thread can make an on-time policy reuse old measurements. Correct outgoing intervals therefore do not prove fresh inputs. Synchronization must align each measurement with the intended computation and command, not merely keep threads busy.
Track acquisition, receive handoff, policy start and completion, and transmission separately. Their differences expose observation age, computation time, and queued communication; none measures the actuator’s physical response.
Regular commands can still use old measurements
Illustrative schedule: samples acquired every 2 ms; receive takes 0.5 ms; policy calls start at 0.5, 2.5 and 4.5 ms and normally take 2 ms. Calls may overlap. Each uses the newest acquired sample whose handoff has completed.
Transmit openings: 2.5, 4.5, 6.5, 8.5 ms. At an opening, drain already-completed commands in completion order, ties by P0–P2. Each transmission takes 0.5 ms without overlap; a missed opening queues the command. Ready exactly at an opening qualifies. This transport example does not model command rejection or actuator response.
Circle: acquisition. Cyan span: receive. Dashed gold: delayed handoff; gold square: available to policy. Violet: compute. Gold blocks: transmit.
| Call / sample | Acquired | Policy starts | Finishes | Transmit | Queue wait | Age at transmit end |
|---|---|---|---|---|---|---|
| P0 / S0 | 0 | 0.5 | 2.5 | 2.5–3 | 0 | 3 |
| P1 / S1 | 2 | 2.5 | 4.5 | 4.5–5 | 0 | 3 |
| P2 / S2 | 4 | 4.5 | 6.5 | 6.5–7 | 0 | 3 |
Current output completion times: 3, 5, 7 ms. P1 consumes S1, aged 0.5 ms when its policy starts.
Static stale-input comparison: nominal P1 uses S1, aged 0.5 ms at start. With a 1 ms S1 handoff delay and no overrun, P1 reuses S0, aged 2.5 ms. Both schedules finish transmissions at 3, 5, 7 ms: outgoing cadence alone misses the stale input.
External bus capture can reveal delivery irregularity: a late message followed by an on-time one produces a long interval followed by a short interval. It cannot establish acquisition time or actuator response by itself. Remote inference adds another delay and availability dependency; local execution still needs assessment under sustained load. Energy and heat limits explain why short device benchmarks need not describe sustained performance.
Accepting current commands
A computed command needs a validity contract before execution: the state it used, its coordinate meaning, when it may apply, who owns actuation, and what happens when required updates disappear. Delivering every message is not necessarily desirable when late messages describe an obsolete situation.
| Policy | Meaning | Not established |
|---|---|---|
| History and depth | How many samples may be retained | That retained data are current |
| Lifespan | When samples expire | That every surviving sample is useful |
| Deadline | Expected maximum interval between messages | End-to-end actuator response |
| Liveliness | Whether a publisher continues asserting life | Correct measurements or physical progress |
Deadline and liveliness callbacks report conditions; the application chooses the response. Geometry has a separate validity check: tf2 retrieves a transform that converts source-frame coordinates into target-frame coordinates at a requested time. The latest available transform is not necessarily current or appropriate for an older measurement. Waiting for the required transform can fail; substituting a newer one changes the measurement's geometric interpretation.
A watchdog monitors whether required updates arrive in time and triggers a configured response when they do not. Nav2's Collision Monitor provides a concrete example: it sits in the velocity-command path and can request stopping when configured sensor sources time out. This changes the commanded motion rather than merely publishing a warning. The application must still verify command delivery and the robot's response.
Authority is separate. ros2_control's ResourceManager allows one command-interface claimant and shared state readers, but cannot exclude unrelated hardware access. Hardware read/write ERROR stops controllers using the affected command and state interfaces. Controller update ERROR deactivates the controller or chain and attempts configured fallback activation, subject to interface availability. Configuring a fallback does not establish that it can execute.
Learning behavior
Demonstrations and corrections
Behavior cloning fits a policy to demonstrated observation–action pairs. An episode is one task attempt, whose recordings connect what the operator could observe with the commands supplied. Training changes learned parameters; it is not simply placing examples in a language-model prompt.
Learning physical control predates today's foundation models. Dean Pomerleau's ALVINN, associated with the 1988 conference and 1989 proceedings volume, learned steering from simulated road images using camera and laser-range inputs. The paper reported bounded campus-path following; learning directly from human driving was proposed future work. Its contribution was fitting a perception-to-steering mapping rather than hand-designing the entire mapping.
In ALOHA, the human moves a leader arm; the robot's follower arm tracks its commands. Their joint-position difference influences applied force through the lower-level controller. ALOHA therefore records leader positions as action targets, while follower positions and camera views supply observations. Using follower positions as targets would replace requested motion with measured response and change that supervision. Collection runs at 50 Hz; learned absolute joint-position targets are tracked by lower-level controllers.
A policy error changes the next observation, potentially moving execution outside demonstrated states. DAgger, introduced by Ross, Gordon, and Bagnell in 2011, addresses this by collecting learner-visited states, requesting expert actions there, aggregating the examples, and retraining. The algorithm supplies corrective supervision, not protective hardware execution; expert availability and physical intervention remain separate concerns.
Collection consequently needs an organized record of task, operator, episode, and annotations. Physical Intelligence's data-engine account describes leader-arm teleoperation followed by episode tracking and annotation-based filtering. Those operations make experience selectable for training; they do not by themselves establish that the selected experience covers recovery states.
Predicting action sequences
An action chunk jointly predicts several actions. Diffusion Policy (2023) learns observation-conditioned chunks by iteratively refining noise into actions. The paper applies generative denoising to control; averaging incompatible successful motions need not work.
The prediction horizon counts proposed actions; the execution horizon counts those applied before observing and predicting again. Shorter execution allows earlier correction. Unlike MPC, this samples behavior without explicit dynamics-constrained optimization.
Action Chunking with Transformers, or ACT, was introduced by Zhao, Kumar, Levine, and Finn at the 2023 Robotics: Science and Systems conference for bimanual manipulation—using two arms together. ACT uses camera observations and joint positions to predict an action sequence with a transformer. During training, an encoder compresses demonstrated sequences into a latent style variable, a numerical representation of variation in the demonstrations. At deployment, that encoder is removed. The policy receives a fixed default value—the mean of the model's prior distribution over that variable—rather than an encoded demonstration or a randomly selected style. The authors' architecture explanation distinguishes those stages.
ACT's temporal ensembling queries repeatedly and averages overlapping predictions for the same execution time. It does not average adjacent actions along a trajectory. The weighting trades responsiveness against smoothing, with additional inference work. Averaging these predictions still supplies no guarantee that incompatible motions become feasible.
Replace an unexecuted tail or combine one execution time
Replacement: new observation at step 2
| Prediction time | Execute 0 | Execute 1 | Execute 2 | Execute 3 | Execute 4 | Execute 5 |
|---|---|---|---|---|---|---|
| At 0 | A0 executed | A1 executed | A2 discarded | A3 discarded | — | — |
| At 2 | — | — | B2 replacement prediction | B3 replacement prediction | B4 prediction | B5 prediction |
A0/A1 already executed. B2/B3 supersede A2/A3; B4/B5 remain predictions. Discarding the old tail does not undo past actions.
Temporal ensembling: combine one column
| Prediction time | Execute 0 | Execute 1 | Execute 2 | Execute 3 | Execute 4 |
|---|---|---|---|---|---|
| At 0 | P0→0 | P0→1 | P0→2 ★ | — | — |
| At 1 | — | P1→1 | P1→2 ★ | P1→3 | — |
| At 2 | — | — | P2→2 ★ | P2→3 | P2→4 |
★ P0→2, P1→2 and P2→2 contribute to the command for execution time 2. These are predictions from different rows for the same time, not adjacent actions. Weight values are not specified here.
Learning through interaction
Reinforcement learning improves action selection using rewards associated with outcomes. Return accumulates rewards over an attempt, so useful feedback may arrive well after the action that mattered. Credit assignment develops the optimization; environment responsibilities separate observations, actions, transitions, and rewards. In robotics, obtaining those transitions exposes a physical machine and its surroundings to the chosen actions.
| Experience source | Training information | Important limitation |
|---|---|---|
| Demonstrations | Expert action for an observation | Missing corrections at learner-induced states |
| Fixed interaction dataset | Recorded actions, successor states, and rewards | Alternatives outside recorded coverage must be estimated without new exploration |
| New physical interaction | Consequences of actions chosen during collection | Additional supervision, reset work, and exposure |
Offline learning uses a fixed dataset; online collection obtains new experience. Offline reinforcement learning can seek better behavior than imitation, but evaluating unrecorded alternatives introduces distribution shift. More copies of the same restricted behavior do not supply missing action consequences.
SERL illustrates the surrounding physical work. Demonstrations can seed learning, image-based success classifiers can supply rewards, and separate forward and backward policies can perform tasks and resets. A policy can exploit a classifier's errors; a reset policy still physically returns objects. Sample efficiency therefore concerns how much useful improvement is obtained per interaction, while the collection system must also account for restoration and supervision.
Interaction can improve a policy or a learned dynamics model that predicts consequences. These are different targets: one selects actions; the other forecasts what actions do. World Models explains the latter. A better predictor can support planning, but its forecasts remain distinct from newly observed physical outcomes.
Language and cross-robot transfer
A vision-language model processes visual information and language. A vision-language-action model, or VLA, uses these inputs, often with robot state, to produce robot-action representations. Adapting a language-capable backbone therefore requires an action interface and execution timing appropriate to the body, not just changing the requested answer format.
RT-2, introduced in 2023, represented discretized robot actions as tokens and trained with robot trajectories and vision-language tasks. Its paper distinguishes semantic transfer from new motor skills: web knowledge helped apply learned skills, but did not supply motions absent from robot training. Large-model inference cost also constrained high-frequency control.
Open X-Embodiment brought together heterogeneous robot trajectories to study shared training. Its dataset and RT-X study separates the full collection from the training mixture and evaluated systems. Mapping observations and actions into a shared format supports learning; it does not make joint geometry, units, or mechanical limits interchangeable.
| Changed condition | Claim requiring assessment |
|---|---|
| Object or instruction | Applying existing capabilities to a new referent or request |
| Physical skill | Producing a previously unlearned movement capability |
| Environment | Completing work under new arrangements and conditions |
| Robot body | Transferring through different sensing, action semantics, and mechanics |
Physical Intelligence's π0.5, published in April 2025, combined high-level subtasks with continuous action chunks. Its reported setup normalized heterogeneous action dimensions while distinguishing joint-space and end-effector control. Mobile manipulators executed targets at 50 Hz through lower-level controllers; that is not necessarily full-model inference frequency. Tests included three unseen homes, with scores roughly measuring completed task steps rather than only binary completion. The practitioner account shows the motivation for this architecture, while the trial contract bounds the generalization claim.
Simulation and transfer
Where simulation differs
A simulation computes consequences under a model. In control terminology, the plant is the physical system being controlled; a simulator substitutes a modeled plant. Sim-to-real transfer applies behavior developed in simulation to hardware. Fidelity concerns the properties needed for that use, not how realistic a rendered scene looks. Simulator validation develops that distinction.
A common API preserves command syntax, not physical consequences. K-Scale's 2025 platform account described the same gRPC interface for simulated and physical robots, allowing a target change through an address. That simplifies application portability without establishing matched sensing, actuation, or contact.
A test double may not model those consequences at all. ros2_control GenericSystem normally mirrors commands into states. Perfect apparent tracking can therefore follow from construction. It can test integration and injected missing feedback, but does not establish friction, saturation, or load behavior.
Even a dynamics simulation can differ materially. Peng and colleagues showed different measured joint trajectories when executing identical targets in simulation and hardware. Their physical puck-pushing tests also used motion capture for object state. That observation access is part of the result. Privileged state means simulator information unavailable to the deployed policy; it must not silently replace the actual sensing problem. Observation boundaries and physical resets deserve explicit contracts.
Fitting the physical system
Calibration estimates how measurements relate to physical quantities and frames. System identification estimates dynamics from recorded inputs and motion. With known mechanical structure, one approach records joint positions and torques, estimates velocities and accelerations, and fits parameters to the resulting dynamics equations. Tedrake's identification notes explain why noisy derivatives, physically impossible parameter values, and indistinguishable parameter combinations complicate that fit.
Informative excitation means collecting movements that distinguish the parameter effects of interest. Repeating nearly identical movements can add samples without identifying those effects. A parameter estimate is not necessarily unique, even if it predicts some measurements well. Constraints and prior estimates can help fitting, but cannot manufacture information absent from the experiment.
Assess the fitted model on independent data with consistent preprocessing and relevant input variation. A residual is measured output minus predicted output. Temporal structure in residuals or correlation with past inputs can expose behavior left unexplained by an aggregate fit score. Interpretation depends on the model and collection conditions; feedback can itself create some correlations. Held-out validation and residual analysis describe complementary checks.
One-step prediction and free-running simulation are different tests. Refreshing the model with measurements can conceal errors that accumulate when predictions become subsequent inputs. Good local fit may coexist with poor simulated trajectories. Accumulated prediction error explains why that matters when a planner searches for favorable futures.
Randomization and adaptation
Domain randomization trains across deliberately varied simulated conditions. Adaptation instead uses target-domain information to adjust behavior. These approaches address different uncertainty: variation can reduce dependence on one assumed setting, while measurements can identify how the actual system differs.
Tobin and colleagues' 2017 visual-randomization study varied textures, lighting, cameras, objects, and distractors to train detectors for real images. Its Fetch grasp experiments combined perception with motion planning. This supports a visual-transfer mechanism, not a claim that randomized appearance solves contact dynamics.
Peng and colleagues' 2018 dynamics-randomization study varied masses, damping, friction, controller gains, observation noise, and action intervals. A recurrent policy used observation–action history to adapt without receiving true dynamics parameters. Its instrumented puck-pushing result does not establish arbitrary visual manipulation or coverage of every contact and hardware fault.
Choose variation from the intended operating conditions, declare the training ranges, and hold out transfer conditions for assessment. Wider ranges alone establish neither relevance nor improvement. Test important combinations too: an interaction occurs when added delay has a different effect under one contact condition than another. Independent one-factor checks can miss it.
For a diagnostic comparison, keep the task, controller, measurement window, and remaining conditions comparable. Repeat fresh trials and randomize order so warm-up or drift does not systematically align with one setting. Reset relevant state or record carryover. A controlled simulator result still describes that simulator; transfer requires assessment of the physical response.
Protection and recovery
Preventing harm during motion
A hazard is a potential source of harm: collision, crushing, dropped loads, or loss of support. Risk assessment also considers severity, exposure, likelihood, and avoidance. It must include programming, maintenance, faults, and human access, not only intended automatic motion. OSHA's robot-application guidance connects those conditions to safeguarding and checks of actual stopping performance.
A safeguard prevents or limits hazardous exposure, for example through guarding or presence sensing. Speed-and-separation monitoring can reduce speed and stop before contact; monitored standstill can retain power while checking that motion has ceased. Detection and stopping take time. A planned endpoint outside a hazardous area says nothing about exposure during travel or what happens to a supported load.
Application software and protective functions must not be conflated. Nav2's velocity limiting is not a safety-rated emergency stop. Publishing zero velocity does not implement protective wiring, monitored standstill, brakes, or restart interlocks. Their effectiveness requires assessment of the complete sensing-to-physical-response path.
An emergency stop is a complementary protective measure, not a replacement for hazard prevention. The UR16e manual requires assessment and resolution of the activation cause before reset. Removing power is not a universal response for every gravity-loaded mechanism; the resulting load and brake behavior matter.
Standards also have application boundaries. ISO 10218-2:2025, published in February 2025, covers industrial robot applications and cells across their lifecycle. Its public scope excludes household consumer robots, publicly accessible service applications, medical uses, and mobile-platform mobility hazards. An industrial arm inside a different application does not make the whole application covered. These distinctions explain the engineering obligations; they do not establish compliance for a particular installation.
Recovery and restart
Recovery restores conditions under which work can permissibly continue. It is not synonymous with retry. Waiting, gathering a new observation, relocalizing, retreating, and transferring control have different effects. Nav2's Behavior Server provides behaviors such as waiting, backing up, and spinning, with collision-checking inputs and limits. The supervisor must still decide whether a behavior fits the observed failure.
A failed grasp may justify a new observation; uncertain contact may make blind retraction hazardous. A fallback controller may lack the interfaces it needs. In ros2_control's error handling, fallback activation is conditional, and controller deactivation does not specify physical braking or gravity response. Recovery needs an available behavior, not just a configured name.
Human takeover transfers command responsibility. Kelly and colleagues' HG-DAgger, in its March 2019 revision, let the human decide when to intervene and retain exclusive control until handing back. Corrective trajectories then supplied training data. This relied on human recognition and response; it was not a demonstrated automatic protective handoff. A practical handoff should communicate state uncertainty, supported loads, active and queued commands, and remaining hazards.
Clearing a sensor, resetting a stop, and restarting motion are distinct events. In Universal Robots' documented safeguard arrangement, a person can pass through a light curtain and remain inside after the beam clears. Whole-body access therefore needs manual reset or another means preventing restart without establishing absence. Fresh state and application-specific restart checks must precede renewed motion; replaying an old plan is not a physical reset.
Whole-system validation
Choosing the test boundary
A test establishes behavior of the components that actually participated. Recorded replay assesses predictions on captured observations; interactive simulation changes later observations in response to the policy's actions. A changed policy can take a different path, so replay cannot supply all of its counterfactual consequences. The EMMA presentation explicitly separated its reported open-loop prediction results from additional simulation and road testing.
| Test boundary | What participates | What remains outside |
|---|---|---|
| Mock hardware | Controller software and substitute interfaces | Production electronics and unmodeled physical effects |
| Software-in-the-loop | Compiled controller against a simulated plant | Target processor and physical interfaces |
| Processor-in-the-loop | Controller on its target processor | Real-time interaction is not automatic; the host may wait |
| Hardware-in-the-loop | Controller physical I/O connected to a real-time plant simulation | Physical phenomena absent from that model |
| Bounded physical trials | Actual robot, objects, and selected surroundings | Unexercised tasks and operating conditions |
Hardware-in-the-loop, or HIL, can expose latency, jitter, and signal-exchange problems through real interfaces. Its scenarios still need representative traffic, load, and faults. A simulated plant cannot reveal omitted physical effects merely because the controller is real. The testing-boundary explanation makes this distinction explicit.
Mock tests remain useful for narrower obligations. A command-mirroring setup can exercise configuration and injected missing-feedback handling; host timing characterizes that host and workload, not production deadlines. Start from the claim being tested, then choose the smallest boundary that can challenge it. Evaluation-case design explains how to preserve those conditions.
Measuring useful performance
An operating envelope names the conditions under which a capability claim applies. A physical trial must specify the task, starting arrangement, tolerances, permitted assistance, reset procedure, allowed attempts, and assessment window. Hold out objects or environments according to the intended generalization claim. Repeating one convenient arrangement does not establish broader usefulness.
Keep intermediate and final outcomes separate. A partial-step score can improve while whole-task completion remains unchanged. CloudChef's cooking-robot account describes separate assessments of cooking-state estimates, motion speed, and final food quality through in-house blind tasting. Those are different properties, and the talk does not supply a complete sample-size or scoring protocol.
| Dimension | Record |
|---|---|
| Completion | Whole-task result and physical tolerance |
| Partial progress | Which steps were achieved, if this measure is used |
| Assistance | Interventions, recovery work, and operator involvement |
| Time | Named start/end boundaries and included reset time |
| Constraints | Observed violations and monitored physical properties |
| Coverage | Assessed conditions, missing observations, and untested cases |
Unavailable measurements are not zero events. Assessment coverage belongs beside performance, and statistical uncertainty limits conclusions from finite trials. Trials with no observed harmful events establish neither zero future risk nor the effectiveness of an untested protective function. Application-level stopping and safeguarding checks remain separate.
When behavior fails, reconstruct the consequential chain: acquisition time and frame, state estimate, policy input identity, proposed command, delivered command, and measured motion. Preserve the hardware, calibration, controller, and model configuration that ran; boundary recording and system identity explain the general recordkeeping. A regular bus trace with reused inputs points to a different investigation from irregular command delivery. Neither observation alone proves the complete cause.
Use a controlled follow-up to distinguish the explanations. A sensing problem calls for different work from a timing fault, mechanical limitation, controller mismatch, missing policy experience, or unsuitable task definition. The engineering objective is a useful physical outcome under stated conditions—not a larger model, a smoother animation, or a better isolated score.
Open questions
Broad physical transfer remains difficult because shared data formats do not make sensing, action semantics, or mechanics equivalent. Progress would distinguish transfer across objects, skills, environments, and bodies through controlled physical evaluations, rather than treating them as one generalization score.
Action-sequence coherence competes with timely correction. Longer commitments can postpone reactions to changed contact or stale observations; more frequent inference adds execution work. Progress would measure the complete acquisition-to-response path and evaluate horizon choices under matched physical disturbances.
Safe intervention must remain available before the robot reaches a state from which a human or fallback cannot recover. Human recognition, command transfer, and physical response all take time. Progress would establish those boundaries jointly instead of equating configured takeover with effective protection.
Contact-rich simulation must represent the errors that change decisions, not merely produce convincing images. Progress would link independently measured dynamics and contact variation to held-out physical tests, including combinations of timing and mechanical conditions that isolated tests miss.
