Add planner-oriented atomic skill benchmark

scripts/benchmark/__main__.py

@@ -51,7 +51,7 @@ def _run_rl_cli(_: argparse.Namespace) -> None:
 
 
 def _run_motion_generation_cli(args: argparse.Namespace) -> None:
-    """Run the free-space motion-generation benchmark."""
+    """Run the shared planner and Atomic Task benchmark."""
     from scripts.benchmark.motion_generation.run_benchmark import run_from_args
 
     run_from_args(args)
@@ -118,7 +118,7 @@ def main(argv: Sequence[str] | None = None) -> None:
 
     motion_generation_parser = subparsers.add_parser(
         "motion-generation",
-        help="Benchmark free-space motion generation with cuRobo as baseline.",
+        help="Benchmark planners on fixed motion and Atomic Task cases.",
     )
     add_parser_arguments(motion_generation_parser)
     motion_generation_parser.set_defaults(func=_run_motion_generation_cli)

scripts/benchmark/motion_generation/BENCHMARK_DESIGN.md

@@ -15,6 +15,28 @@ The default comparison should be NMG versus cuRobo. IK plus interpolation and
 TOPPRA should remain optional diagnostic baselines rather than define the main
 leaderboard.
 
+## Implemented vertical slice
+
+The first physics-backed slice is available as
+`suites/atomic_franka_pgi_curobo.yaml`. It runs Franka + PGI with cuRobo only,
+and covers `MoveEndEffector` plus antipodal-grasp `PickUp`. Both skills pin
+`MotionPolicy(strategy="motion_gen", planner="curobo")` and compile through the
+same `AtomicActionEngine`; scenario code never calls cuRobo directly.
+
+The shared runner now selects planners, scenarios, robots, Atomic Action case
+providers, and object kinds through registries. Cases freeze the full robot
+start state, explicit targets/grasp, object configuration, difficulty factors,
+and independent sequential-IK evidence before measured planner calls. Reports
+keep planning, kinematic motion validity, controller execution, and physical
+task success separate while retaining exactly three tables.
+
+Run the slice with:
+
+```bash
+python -m scripts.benchmark.motion_generation.run_benchmark \
+  --suite atomic_franka_pgi_curobo --device cuda
+```
+
 ## Motivation
 
 The existing NeuralPlanner benchmark provides useful latency, memory, rollout,

scripts/benchmark/motion_generation/README.md

@@ -1,36 +1,58 @@
-# Motion Generation Benchmark
+# Planner Motion Generation & Atomic Skill Benchmark
 
-Free-space motion-generation suite with cuRobo as the default primary baseline.
+Shared planner benchmark framework for fixed motion-generation cases and
+physics-backed Atomic Actions. All Atomic Actions call the selected planner
+through the adapter-owned `MotionGenerator`.
 
 Design background and roadmap: see [`BENCHMARK_DESIGN.md`](./BENCHMARK_DESIGN.md).
 
 ## Run
 
 ```bash
-embodichain benchmark motion-generation --suite smoke
-embodichain benchmark motion-generation --suite coverage
-embodichain benchmark motion-generation --extra-baselines ik_interpolate toppra
-embodichain benchmark motion-generation --path-shapes direct l_turn --start-state-bins nominal near_singularity
+python -m scripts.benchmark.motion_generation.run_benchmark --suite smoke
+python -m scripts.benchmark.motion_generation.run_benchmark --suite coverage
+python -m scripts.benchmark.motion_generation.run_benchmark \
+  --suite atomic_franka_pgi_curobo --device cuda
+python -m scripts.benchmark.motion_generation.run_benchmark \
+  --extra-baselines ik_interpolate toppra
 ```
 
-Artifacts land under `outputs/benchmarks/motion_generation/<timestamp>/`
+Artifacts land under `outputs/benchmarks/<suite-name>/<timestamp>/`
 (`resolved_suite.yaml`, `case_manifest.json`, `trials.jsonl`, `aggregates.json`,
 `report.md` with exactly three tables).
 
 ## Implemented
 
-- Extensible planner/scenario registries and track-based suite YAML
+- Extensible planner, scenario, robot, Atomic Action, and object registries
 - `free-space-common` track with fixed manifests and start-state bins
+- `atomic-task` track with frozen robot/object/task manifests and common physics replay
+- Initial Atomic Task slice: Franka + PGI, cuRobo, `MoveEndEffector`, and
+  antipodal-grasp `PickUp` on a declarative cube
 - Default matrix: cuRobo (`primary_baseline`); IK / TOPPRA optional diagnostics
 - NMG adapter stub (`candidate`, disabled until a checkpoint is ready)
 - Lifecycle timing: construct / prepare / cold / warm
-- Ordered waypoint matching and external `motion_valid` (separate from
-  `PlanResult.success`)
+- Distinct planning, motion-valid, execution, and physical task-success stages
+- Planning latency, execution wall time, end-to-end time, nominal trajectory
+  duration, simulated task-completion time, controller tracking RMSE, and
+  task-specific object lift
 - One Markdown report: Time & Memory, Success & Other Metrics, Leaderboard
 
-## Not implemented yet
+## Extend
+
+- Planner: register a `PlannerAdapter`, expose its `MotionGenerator`, and
+  declare the `atomic_action` capability.
+- Robot: register a `RobotProvider` and select it under `robot`; PickUp suites
+  also declare the gripper control part and open/grasp qpos under `gripper`.
+- Object: add another `objects` entry for built-in `cube`/`mesh`, or register a
+  new object-kind factory.
+- Atomic skill: implement and register an `AtomicSkillCaseProvider`; the runner,
+  artifact schema, aggregation, and report stay unchanged.
+
+## Current limits
 
 - Real NMG checkpoint adapter
-- `collision-deployment` and `atomic-task` tracks
-- Physics execution / task-success metrics
+- `collision-deployment` and obstacle-aware common-input tracks
+- Atomic Task execution is currently `B=1`; the supplied suite covers only
+  Franka + PGI and cuRobo
+- Remaining skills: `MoveHeldObject`, `Place`, `Press`, and action chains
 - Latency-budget Pareto sweeps, confidence intervals, subprocess isolation