A short clip can make an engineering result legible in seconds. Evaluating it takes a little more structure. The questions below form our editorial checklist. They are a framework for reporting, rather than a rating of any particular manufacturer.
1. Name the task precisely
Describe the start and finish. “Moves a tote from a fixed stand to a conveyor” tells a reader more than “works in a warehouse.” Include the object, environment, and success condition. The narrower description makes later comparisons more meaningful.
2. Identify the operating mode
Look for an explicit account of autonomy, teleoperation, supervision, and human intervention. Attribute that account. If the publisher says the run was autonomous, record it as the publisher’s disclosure unless you have separate evidence.
3. Look at the continuity
Edits can make a video readable while leaving the sequence of events unclear. Playback speed, cuts, and the duration of the visible run affect what a viewer can infer about task timing. Record what is shown and seek the full sequence when timing matters.
4. Ask about repetition
How many attempts were made? How often did the robot succeed? Did the trials vary the object or its position? A continuous successful run answers a different question from a test across many starting conditions. Both can contribute useful evidence when their scope is stated.
5. Follow the exceptions
A dropped item or blocked path can reveal how the system responds to the unexpected. Ask whether the robot detects the problem, retries safely, requests help, or needs a reset. Include any recovery time in operational measurements.
6. Find the next layer of evidence
A research paper may describe the experimental setup. A customer announcement may identify a working deployment. An independent test may provide a separate assessment. Keep those sources visible so a reader can trace each claim.
We will update this checklist as we report more stories. The aim is to give impressive work the context required to understand and compare it.