Data catalog

A data catalog is a catalog of structured assets: databases, tables, schemas, and the lineage between them. It describes what a database already knows about itself, which is why it does not cover files and why most organizations end up needing both kinds of catalog.

Key takeaways

A data catalog describes structured data, such as databases, tables, and schemas.
Files are described by a metadata catalog, which indexes unstructured data across storage systems.
A data fabric connects data across systems through shared metadata, and a metadata catalog is one of its parts.

Why does an organization need a data catalog?

So people can find and trust the structured data they already have. A data catalog shows which tables exist, what each column means, where the data came from, and who owns it. Analysts spend less time searching and asking around, the same report is not rebuilt twice, and governance teams can see where sensitive fields are stored.

How is a metadata catalog different from a data catalog?

A data catalog describes structured assets: databases, tables, schemas, and the lineage between them. A metadata catalog for unstructured data describes files no database is aware of. Most organizations need both, and the two do not overlap.

What is a data fabric, and how does it relate to a metadata catalog?

A data fabric is an architecture that connects data across many systems through shared metadata and common services, so applications can find and use data without knowing where it sits. For unstructured data, a metadata catalog is one of the parts a data fabric needs: it supplies the current description of every file, such as its path, owner, size, and age. The catalog describes the files, and the rest of the fabric decides how they are reached.

Can a data catalog include files?

Most data catalogs are built for databases and tables, and they see files only after the files are loaded into a database or a lakehouse table. The files that stay on file systems and object storage, which is most of them, remain outside their view. A metadata catalog fills that gap by indexing files where they live, and connecting the two catalogs lets one search reach both structured and unstructured data.

How does Diskover help?

Diskover builds an enriched metadata catalog for your files: one searchable index across every storage system, so unstructured data can be found and used alongside the structured data your teams already catalog.

Explore the enriched metadata catalog

Book a demo