
260 PB
of unstructured media indexed across 6 data centers
100B+
files cataloged with searchable metadata
30+
data and operational users
1
metadata catalog feeding Snowflake and Cortex AI
At a glance.
The company holds 260 PB of unstructured media across production, post, broadcast, archives, and theatrical assets, spread over six data centers and more than five file systems. Feeding that estate into AI and BI pipelines ran through a general-purpose lakehouse platform that could not scale to unstructured data without heavy overhead, and preparing training datasets from raw media was a manual scripting job. Business users who wanted answers about the estate filed tickets and waited.
Diskover indexed every file across all six data centers into one global metadata catalog and enriched it with production context from the company’s Autodesk production tracking system. Data teams tag, down-select, and curate datasets in the catalog with saved searches instead of scripts. The curated, tagged metadata streams into Snowflake, where Cortex AI lets operational and business users ask questions about the entire media estate in plain language.
The lakehouse workflow for unstructured media was decommissioned, and its license and overhead spend recovered. AI and BI pipelines are fed from 260 PB of media without manual preparation, ticket-driven queries have moved to self-service, and the company runs fewer tools and pipelines on one metadata catalog across its content estate.
Customer profile
A global entertainment company valued at more than $180 billion, with studios, broadcast networks, streaming services, and theme parks serving audiences worldwide.
Its production, post-production, broadcast, and archive operations generate unstructured media at petabyte scale across six data centers and more than five file systems.
Data platform teams support 30+ operational and business users who need to find, understand, and reuse that media for analytics and AI.
Data
Unstructured media across production, post-production, broadcast, archives, and theatrical assets.
Storage indexed
260 PB across six data centers, on Dell PowerScale and NetApp file systems and AWS S3 object storage.
Users
Data platform teams curating datasets, with 30+ users across 6 data centers.
Locations
Studios, networks, and business units worldwide, served from six data centers.
Challenges.
Petabytes past the platform.
Unstructured media at 260 PB demanded overhead the existing lakehouse deployment could not absorb, so the AI and BI pipeline stalled at the source.
Business users blocked.
Operational and business teams could not query the media estate without filing a ticket or waiting for a script, so questions went unasked.
Curation by script.
Tagging, down-selecting, and preparing AI training datasets from raw media was a manual scripting burden carried by a small data platform team.
Six sites, no shared view.
Media sat across six data centers and more than five file systems with no single inventory to curate from.
Solutions.
One catalog, 260 PB indexed.
Diskover indexed every file across six data centers and more than five file systems into one global metadata catalog, with the files staying where they are.
Tag, down-select, curate.
Saved searches and tags let data teams find the relevant slice of the estate and mark it as a dataset, replacing the scripts and the lakehouse workflow that did this before.
Production context in the index.
A plugin pulls project, shot, and status data from the company’s Autodesk production tracking system into each file’s record, so curation works on business meaning, not just paths.
Snowflake in, Cortex AI out.
Curated, tagged metadata streams into Snowflake tables continuously, and file content is pulled on demand only for the datasets a pipeline needs. Operational and business users ask Cortex AI questions across the entire media estate in plain language.
“Snowflake is built for data you can query. Media is 260 petabytes of files that nobody could query, so the first job was to turn every file into a record: what it is, which production it belongs to, its format, its status. Once that catalog exists, curation is a search, not a script. The metadata that matters streams into Snowflake continuously, the file itself is pulled only when a pipeline needs it, and a business user asks Cortex AI a question instead of filing a ticket.”
Olivier Rivard, VP of Product, Diskover
260 PB of media, curated for AI.
1. Index
Every file, 260 PB, metadata only, such as name, path, size, owner, and format.
2. Enrich and curate
A production tracking plugin adds project, shot, and status. Data teams down-select saved datasets instead of writing scripts.
3. Stream
Curated, tagged metadata streams into Snowflake, and operational and business users ask Cortex AI questions in plain language.
A lakehouse is built for tables. Media is files, more than 100 billion files of them, and most of what an AI or BI team needs to know about a file is metadata: what it is, which production it belongs to, its format, age, owner, and status.
Outcomes.
Pipeline costs recovered.
The lakehouse workflow for unstructured media was decommissioned, and its license and overhead spend recovered.
Answers without tickets.
30+ business and operational users get plain-language answers, and data teams stopped fielding ticket-driven queries.
AI-ready data at scale.
Curated metadata reaches Snowflake and Cortex AI from 260 PB of media without manual preparation.
Fewer tools, one catalog.
One unified metadata catalog across the content estate replaced separate tools and pipelines.
Questions people ask.
Why is unstructured media hard to use for AI and analytics?
Analytics platforms are built for tables, and media is files: here, more than 100 billion of them across six data centers. Without a catalog, preparing a training dataset means scripting against raw files, and business users cannot ask questions without filing a ticket.
How do you get media into Snowflake without copying the files?
Turn every file into a metadata record first. Diskover streams curated, tagged metadata into Snowflake tables continuously, and file content is pulled on demand only for the datasets a pipeline needs.
What does Cortex AI add for business users?
Once the metadata is in Snowflake, operational and business users ask Cortex AI questions about the entire media estate in plain language, instead of filing a ticket and waiting for a script.
How do data teams curate datasets without scripts?
With saved searches and tags in the Diskover catalog. Teams find the relevant slice of the estate, enriched with project, shot, and status from the production tracking system, and mark it as a dataset. Explore curation and orchestration.
What happened to the lakehouse workflow?
The general-purpose lakehouse workflow for unstructured media was decommissioned, and its license and overhead spend recovered. One metadata catalog now feeds the AI and BI pipelines.
Related resources.
See it on your own data.
We’ll index a sample of your environment and show you
what’s recoverable against your current storage spend.
