Summary
Twelve Labs has released Pegasus 1.6, a vision-language model designed to turn first-person video into structured training data for robotics and physical AI. The model segments actions, identifies objects and tools, describes sequences and attaches time-based metadata in formats defined by users. It supports action labeling, detailed captions, quality assessment, search and curation, plus privacy and compliance detection. The company points to the Ego-Exo4D dataset as evidence of the manual bottleneck: labeling 1,286 hours reportedly required more than 200,000 hours of work. Twelve Labs says an unnamed US robotics-data company is already using Pegasus to assess factory footage, reject low-quality material and add descriptions, processing hundreds of thousands of hours each day. The report does not provide independent accuracy or cost benchmarks.
Hana’s perspective
I think the important idea is that video understanding can become production infrastructure, not only a way to generate new images. Animation and VFX teams also spend large amounts of time segmenting footage, tagging actions, finding shots and checking assets. A system that returns reliable time-coded structure could make archives and reference libraries much more usable. But the robotics example is a company-reported deployment, and automated labels can carry subtle errors into training data at scale. Teams should measure precision on their own footage, retain human review for ambiguous interactions and treat privacy detection as a safeguard to validate rather than a guarantee.
Are you a writer with a great story? We'd love to hear from you: APPLY HERE