The blog, Which autonomous vehicle training data do your models still need?, follows that question through a demo for the AI and infrastructure teams of a global automaker. Using public autonomous driving datasets and one Diskover scan, Bill Burns, Director of Digital Archaeology at Diskover, showed how metadata answers it from the edge.

What the demo found.

• 2,067 recordings on one page. A plugin read each MCAP file during the scan and added its sensors, capture time, duration, and GPS position to the catalog.
• Weather the files never recorded. A workflow step looked up the historical weather at each capture place and hour, and attached it to 1,047 recordings.
• 179 rain candidates. If the models need more rain, the selection is 179 of 2,067 recordings, under 9% of the set.
• Gaps found early. No snow, no sub-zero driving, and about half the recordings with a suspect clock, all flagged before any training.

Move only what a model needs.

• Decide centrally, keep the bytes at the edge. Diskover indexes the storage at each site, and only metadata travels to the center on every scan.
• AI-assisted, human-approved. Proposed tags stay proposals until a person approves them, in the Diskover interface or through the AI Connector.
• Policy does the moving. A Diskover policy copies the tagged recordings to the training store, and every other recording stays where it is until its retention period ends.

Every file has a where and a when.

The same four steps work well beyond vehicles: read what each file already carries, add the context it lacks, let the people who own the question set the criteria, and move only the files that meet them. As the automaker’s AI data architect put it: “The more we grow, the more important these capabilities become.”