Duplicates Finder Plugin

This plugin is included with the following Diskover editions:

Community Edition	Essential	Professional	Enterprise	AJA Diskover Media Edition	Life Science
	✔	✔	✔	✔	✔

Visit our Solutions page for more details about our different editions.

Overview

The Diskover Duplicates Finder Plugin, aka dupes finder, leverages the post-processing of indices to check for duplicates, across all file systems or subsets thereof. The plugin supports and uses xxhash, md5, sha1, and sha256 hash values to compare duplicates, allowing for extremely precise validation.

The plugin is designed for multiple use cases:

• To check for duplicate files across a single or all file systems (single or multiple indices) and index the file docs in index/indices that are dupes.
• To calculate the file checksums/hashes for all duplicate files or all files and index hashes to file docs in index.
• Reports can be customized to easily locate all duplicate files for clean-up efforts and/or to secure sensitive information.
• The results can be engaged in workflows for automated cleanup.

Calculating file hash checksums is an expensive CPU/disk operation. The dupes finder provides configuration options to control which files in the index get a hash calculated and marked as a dupe. The duplicates plugin harvests hash values that can be stored only for duplicates or for all files.

In addition, the dupes finder provides additional optimization mechanisms as described in the Diskover Configuration and Administration Guide.

Extra Fields Indexed by Diskover

Diskover can currently index four different hash values and adds an extra field for duplicates found. This extra metadata is viewable in the file attributes window and is also searchable, reportable, and actionable.

	MD5	SHA1	SHA256	XXHASH	DUPES
Field name	hash.md5	hash.sha1	hash.sha256	hash.xxhash	is_dupe

Duplicates View Via the Search Page

Duplicates View Via the File Attributes

No video available at the moment.