DATASHEET | DATA MOBILITY

Discover every share. Set the policy. Move data at scale.


Pages of the Data Mobility Datasheet

50 Gbps

data mobility with a single DataDelivery agent

1.4B

files indexed in 2 hours by DataDiscovery

30-50%

of storage reclaimed by tiering and removing ROT data

3-6

months to payback after deployment

At a glance.

The challenge.

New shares and buckets appear faster than anyone can register them. Tiering lives in cron jobs and one-off scripts nobody owns, and file metadata never reaches the analytics stack, so data teams rebuild file inventories again and again.

The solution.

DataDiscovery indexes every NFS, SMB, and S3 source. A DataDelivery policy selects files by name, path, size, or age, runs a dry run first, then copies, moves, or syncs them on a schedule at up to 50 Gbps with a single agent.

Why it matters.

Tiering runs as policy instead of tickets. Every job is recorded in the activity log, so you know what ran and when, and DataDiscovery metadata feeds your lakehouse, so data teams work from the current state of the estate.

Core capabilities.

Read-only scans.

DataDiscovery workers mount volumes right before a scan, read-only by default. Credentials are encrypted and passed in memory, never written to disk.

High-speed transfers.

DataDelivery agents copy, move, and sync at up to 50 Gbps with a single agent. Several agents attached to the same storage systems share one job.

Policy creation.

DataDelivery policies select files by name, path, size, or age, then copy, move, or sync them on a schedule. Need a one-off move? Run the same policy on demand.

Activity log.

Every DataDelivery job is recorded in the activity log, so you know what ran and when. No more guessing which script ran last night.

Dry run first.

Dry-run any DataDelivery transfer to see its scope and cost before the job runs. Adjust the policy until the scope is right, then run it with confidence.

Open data out.

DataDiscovery reports export chosen metadata fields to CSV or Parquet on a schedule, for Snowflake, Databricks, Athena, and Spark. DataDelivery catalogs publish file metadata to Kafka topics.

How it works.

1. Discover

STORAGE OR IT TEAM

Enter each NFS server, SMB server, or S3 endpoint once, and DataDiscovery finds its exports, shares, and buckets.

Example: a new bucket is found without a ticket.

2. Index

AUTOMATION

Linux task workers scan read-only, with credentials kept in memory. Built-in plugins enrich every file, and one index holds the metadata for the whole estate.

Example: every share shows its size, owner, and age.

3. Decide

ANY AUTHORIZED TEAM

A DataDelivery policy selects files by name, path, size, or age and chooses copy, move, or sync, on a schedule or on demand, with a dry run first.

Example: files not modified in 90 days move to a warm bucket.

4. Move and share

BY POLICY

DataDelivery agents run each step at up to 50 Gbps, and DataDiscovery reports export metadata to Parquet or CSV for your lakehouse.

Result: tiering runs on policy, and data teams query the estate like a table.

Technical specifications.

Sources and storage

DataDiscovery: NFS, SMB, S3, and S3-compatible. DataDelivery: NFS, SMB, S3, Azure Blob, GCS, Wasabi, and PowerScale.

Policy actions

Copy, move, and sync, on a schedule or on demand, with a dry run first. Each move copies the file, verifies its checksum, then removes the source.

Sign-in

DataDiscovery: LDAP/AD, Okta, and Entra ID. DataDelivery: Okta, Entra ID, and SAML.

Exports and streaming

CSV or Parquet reports on a schedule, and Kafka topics through DataDelivery catalogs.

Outcomes teams can measure.

Savings that hold.

Tier down what nobody uses, defer the next capacity purchase, and let policy keep your storage lean and clean.

Decisions you can defend.

Dry-run any change, so cost and scope are known before the job runs.

Hours back to your team.

Rules and schedules replace ticket queues, spreadsheets, and one-off scripts.

Protected and provable.

Agent traffic is encrypted, files are checksum verified, and every job is recorded in the activity log.

Questions people ask.

How do DataDiscovery and DataDelivery work together?

DataDiscovery maps and indexes your storage. DataDelivery policies and agents move the files. Together they show what you have and move what matters.

How does tiering across buckets work?

Hot data lands in one bucket. One DataDelivery policy moves files not modified in 90 days to a second bucket, and another moves them to a third after a year. Each move copies the file, verifies its checksum, then removes the source.

Can file metadata reach Snowflake or Databricks?

Yes. Scheduled DataDiscovery reports export chosen fields, including plugin fields, to Parquet for Snowflake, Databricks, Athena, or Spark.

Are scans safe for production storage?

Yes. DataDiscovery workers mount volumes read-only by default, and credentials are encrypted and kept in memory, never written to disk.

How often does Diskover update the catalog?

Diskover indexes storage by volume, and you decide the indexing schedule for each one, based on its size and on how often its files change. Active project volumes are often re-indexed every 10 to 30 minutes, while backup and archive volumes are usually re-indexed once a week or once a month. Rescans are faster than the first index, about 500,000 to 600,000 files per second, because Diskover skips folders that have not changed.

Scroll to Top