Data discovery

Data discovery is the work of building an accurate picture of what an organization holds, across every system. For unstructured data, it works from file attributes rather than contents, which is what lets it cover a petabyte data estate at the same speed as a small one.

Key takeaways

Discovery shows how much data each project holds, what goes unused, and what looks sensitive.
For unstructured data, it starts from file metadata indexed into one catalog.
Cross-vendor discovery answers questions that span the whole estate, not one storage system at a time.

Why does data discovery matter?

Because an organization cannot manage, protect, or pay sensibly for data it cannot see. Discovery is the first step for almost every data project: reducing storage cost, applying retention, finding sensitive files, planning a migration, or choosing data for AI. Skipping it means each of those projects starts by guessing.

What questions can data discovery answer?

Once file metadata from every storage system is indexed into one searchable catalog, teams can answer questions such as how much data each project holds, who owns it, what has not been opened in years, and which files look sensitive.

What is the difference between storage vendor reporting and cross-vendor discovery?

A storage vendor tool reports accurately on its own platform and is blind to everything else. Cross-vendor discovery reads PowerScale, ONTAP, S3, Azure Blob, and the rest into one index. Most decisions, such as what to tier or what to retire, are estate decisions, so a per-vendor view answers the wrong question.

What are the main data discovery methods?

There are three common methods:

Metadata scanning reads file attributes, such as path, size, owner, and dates, across storage systems, which is fast and works at scale.
Content scanning opens files to look for patterns, which is slower and more invasive.
Manual surveys ask teams what they hold, which is quick to start but goes out of date fast.

Most programs start with metadata scanning to see the whole estate, then apply deeper checks only where they are needed.

What is a disk usage analyzer, and when is it not enough?

A disk usage analyzer shows how space is used on one drive or file system, usually as a tree or chart of the largest folders. It works well for a single server. It reaches its limit when data spreads across many storage systems, sites, and clouds, when billions of files are involved, or when teams need to search, report, and act on files by attributes such as owner, project, or age across the whole estate. At that point, a metadata catalog across every system takes over.

How do you start data discovery?

Start small and widen:

Pick one question that matters now, such as what is filling the main file system.
Index the systems that hold the answer, reading metadata only.
Review the first results with the people who own the data.
Add the remaining systems once the first results are trusted, until one catalog covers the whole estate.