The storage got bigger. The problems kept their shape.

In 1999, the fastest way to move a project between two edit bays was often to unplug a hard drive and carry it down the hall. 

Networks were too slow for studio deadlines, so people moved the data on foot. Now we use shared high-throughput storage that feeds every stage of a production pipeline, from film scans and renders to color correction and final delivery.

Mike Bott, Senior Systems Engineer at Diskover, has designed, installed, and supported visual effects (VFX) storage for thirty years. He worked at Integrix, DataDirect Networks, Isilon, Talon Data Systems, FusionStorm (now Computacenter), Qumulo, and Hammerspace before he joined Diskover in early 2025. 

Every generation he worked on made storage bigger and faster. When we asked which problems those generations solved for good, his answer was short.

Here is that shape, and what changes when you can finally see every file, on every storage system, in one place.

In 1999, the fastest network was a pair of sneakers.

Mike started in 1995 in channel sales and technical support, selling SPARC clones and RAID storage. He learned quickly that the job was “understanding problems and proposing solutions,” and that is still how he works today.

By 1999, he was at DataDirect Networks, designing and installing Fibre Channel storage area networks (SANs) for high performance computing (HPC) and media customers. Studios asked for two things.

The first was speed, because playback and film scanning “were unforgiving and demanded sustained throughput.” The second was sharing those frames across capture, scanning, editorial, VFX, and quality control. 

Until shared storage caught up, the network for that job was the hallway. The industry called it “sneakernet”: a drive under someone’s arm, walked from one edit bay to the next, or one geography to another. DataDirect Networks even sold drive chassis built for the trip, moved “from workstation to workstation, server to server, edit bay to edit bay.”

The answer was the shared SAN file system, and Mike remembers a detail that still matters: “They all used a similar architecture: a separate metadata service accessible over ethernet/TCP/IP and direct data access to block devices over Fibre Channel.” The description of the data traveled on its own path, apart from the data. 

Twenty-five years later, that same separation is how an index can describe an entire estate quickly while the files stay where they are. In 1999, the frames rode on a portable drive through the hallway. Today, the metadata does the walking.

Bigger, faster, and the same three problems.

The growth is easy to measure. Mike’s first hard drive held 80 MB. He later installed 5.25-inch full-height 9 GB Seagate drives in a SPARC chassis. Today a half-height 3.5-inch drive holds more than 30 TB, which he calls “4 orders of magnitude larger in a volume less than ¼ of the old stuff.” The storage grew by orders of magnitude. The number of people who have to know what is on it did not.

From 2008 to 2013, Mike led engineering at Talon Data Systems in Burbank. Its product, NetFlight, moved uncompressed film frames between facilities, from Los Angeles to London for some customers. It relied on the user datagram protocol (UDP) to transfer data, avoiding costly-at-high-latency TCP handshakes to fill long-haul 10-gigabit pipes.

The network got faster, but the storage became the bottleneck. In 2008, only a few high-end file systems could read and write fast enough at both ends to keep up.

The networks are bigger now, and so are the files. Diskover’s own CloudSoda uses the same idea, with UDP-blasting, large buffers, and many threads to move data as fast as the network allows. And when a data set is big enough, a studio still considers shipping drives. Across every generation, three problems kept their shape:

1. Knowing what you have: what sits on each system, who owns it, and whether it still belongs there.
2. Moving the data between places, teams, and tiers, fast enough for the deadline.
3. Serving different teams who look at the same storage and ask different questions.

Which of these three costs your team the most time this month?

What VFX storage and a gene sequencer have in common.

At FusionStorm, Mike designed storage for media companies and genomics customers at the same time. A render farm and a gene sequencer sound like different worlds. His answer: “They are all image sequences!” 

Early sequencers wrote tiny images for a week, then a compute farm read them and wrote back results that researchers could view. 

In VFX and animation, a render farm reads artists’ assets and writes image sequences, which are then read back for playback.

Both are “a mix of machine-created and human-created data,” Mike says. The difference is where the pressure lands. The slow sequencer needed little write throughput, but its compute farm needed a lot of aggregate read throughput. 

In VFX, more stages are demanding: rendering needs aggregate throughput, while playback and color correction need fast single streams.

VFX storage is sized by the most demanding stage of the pipeline, and in a studio that stage keeps changing, from rendering early on to color correction at the end. Studios moved to scale-out file systems because they add speed and capacity together, one node at a time, so the same storage keeps up with whichever stage is pushing hardest. That is also why knowing how much storage a studio has tells you very little about how it’s being used.

Meeting Isilon again, twenty years later.

From 2005 to 2008, Mike was a field engineer for Isilon, installing and supporting clustered scale-out network-attached storage (NAS) for broadcast and film production in Southern California, Phoenix, and Las Vegas. 

That platform is now Dell PowerScale, and Diskover indexes it. “It’s fun to see the same architecture again,” Mike says. “Twenty years ago I was building these clusters. Now I get to show teams everything that’s on them.” From the index side, the engineering work is about speed: keeping response times low and running scans in parallel across the cluster’s nodes, so even a very large cluster is indexed quickly.

The bigger change is what the index can show. Diskover collects dozens of metadata fields from PowerScale, including values that file-sharing protocols do not carry, such as protection overhead, SmartLink status, and SmartPool location. It shows them next to every file, alongside owner, age, and path.

That answers questions a file list cannot: how much capacity goes to protecting a project, which tier its files sit on, and which files are links to data stored elsewhere. It also gives tiering decisions a factual starting point: a team can see which projects have gone cold on the fastest tier and set SmartPools policies to move them. Mike’s summary is simple: those values are “easy to see in Diskover!” Diskover with Dell PowerScale puts them in the same catalog as every other system in the studio.

The first look at the index.

When a studio sees its whole estate in one searchable index for the first time, Mike hears two reactions. 

• The first is “What the heck is that?” about a data set or workflow nobody expected. 
• The second sounds more like: “Argh! Go talk to [REDACTED] again, they’re storing camera originals in their home directory.”

What has changed is who can find these answers. “Surfacing unexpected data sets, unknown workflows, and misplaced files has always been what Diskover does best,” Mike says. “What changed is who can ask. With a chatbot, a producer or creative gets the same answer an engineer would.” 

A producer can ask the catalog a question in plain language and get the file list or analysis back, with a recommended action that a person approves before anything happens. AI-assisted, human-approved.

Two recent analyses at major studios show how far metadata alone can go. 

At the first, an analysis across storage systems worldwide “uncovered evidence of an old ransomware attack, including the specific affected files, what archive they appeared in, and the time/date of the attack.” It worked from path, file type, extension, and timestamps alone. An attack without a clear record stays a risk long after it ends, because any restore from that archive can bring the damaged files back. With the exact files and the date in front of them, the security team had a defined scope to clean up. 

At the second, the analysis confirmed the team’s policy was keeping old data off expensive primary storage. Mike also found 100 tebibytes (TiB) of potentially unnecessary duplicates and 50 TiB of cold data still on premises. It also identified the studio’s applications and workflows from extensions, directory layout, dates, and production and vendor names in the paths. Every duplicate and every cold file is capacity the studio still powers, protects, and backs up. Those two numbers gave the team a concrete list to review before buying more storage.

If an analysis like that ran on your storage tonight, what would surprise you first?

Two teams, never the same request.

“They’re never the same request,” Mike says of storage teams and creative teams. “Both of these personas have very different lenses and value systems.”

WHAT the storage team asks	WHAT the creatives, technical directors, and producers ask
Capacity, file age, and heat by user, production, vendor, or client	Which shot or sequence costs the most
The cost profile of the whole storage environment, including data protection	Which project is growing fastest
What sits on expensive primary storage that does not belong there	Whether a frame range is complete, or has patches
Where the efficiency wins are	When vendor deliveries arrived, and how capacity grows across a shoot

Mike spent most of his career on the left side of that table. “It took me quite some time after arriving at Diskover to get a better handle on the needs of the creatives, the executives, the TDs,” he says. 

Today he sees the larger value on the right: “we provide more value to those that are close to the actual business data.” The same catalog answers both columns, which is also why the AI worth having in post-production works on the files around the creative work, never on the work itself.

The old system is now the new problem.

Mike studied cybernetics at UCLA, the science of feedback and control. A storage policy that tags data, notifies the owners, and then moves it is a feedback loop, and he treats every loop with care. 

“There is an axiom in systems design: ‘The old system is now the new problem,’“ he says. “Every systems engineer with some years of experience has dealt with an automation solution that is worse than its predecessor.”

The same caution applies to AI. He points to the vigilance decrement, first studied in radar operators during World War II: people who watch a system for long periods get worse at spotting small problems, especially when the system rarely needs them. “These systems can make mistakes, but we all tend to get comfortable over time with NOT looking at their output carefully. Attention to detail is still required.”

For a storage team, vigilance is a habit, not a setting. Before approving a cleanup the analysis recommends, someone opens a sample of the flagged files and checks their owners and paths. A report that comes back clean week after week still gets read, because a quiet report is exactly where the vigilance decrement sets in. The approval step protects the studio only while the person approving is still looking closely.

What will look old in five years.

Mike expects intelligent indexing to change how media asset management pipelines are built, with business metadata such as release dates and budgets in the same view as location, age, and cost. 

For the right side of the table, he sees AI chat as “a more effective UI for a nontechnical creative or executive.” 

For himself, he keeps a limit: “pry my deterministic CLI from my cold dead hands.” He means the greybeard part: “I actually have a grey beard now! I’m told it makes me look old distinguished.” 

The storage will keep getting bigger. The questions a studio asks about it will keep their shape: where is it, who owns it, and does it still belong here. The difference now is that you can ask them about the whole estate at once, in plain language, and get an answer back.