Data Deduplication Tools
Data Deduplication Tools Overview
What are Data Deduplication Tools?
Data deduplication tools are important for backup and restore operations where large quantities of data are backed up at regular intervals. Frequent backup always means copying and storing large data sets for recovery purposes. As much of this data is duplicate data, storing it all repeatedly would quickly lead to unmanageably large data storage requirements. It is essential to deduplicate these data streams to optimize data backup storage.Deduplication is achieved by means of a deduplication algorithm which is capable of examining an incoming data stream, and comparing data segments to data that has been stored previously. However, there are several things to consider when looking for a product as not all deduplication products work in the same way:
- Source versus target deduplication: Software running on a server which is the source of data is deduped before it is transmitted to the storage device. The advantages of this approach are that a smaller quantity of data is transmitted to the target storage solution and this method therefore uses less bandwidth for data transmission Source deduplication can increase processing time, which is often an important consideration in virtualized environments where there is a very large quantity of data duplication. The alternative is target deduplication where the data is all transmitted to a storage NAS device or tape library, and is deduped once it has been sent. This method reduces the storage capacity required for backup data, but does not reduce the amount of data sent across a LAN or WAN during backup.
- Inline deduplication versus post-processing deduplication: Inline processing means that the deduplication process happens in real time as the data is being transmitted to storage. In post-processing deduplication, the backup data is all written to a disk cache before it starts the deduplication process.
- Global deduplication: Global deduplication is an important consideration as most deduplication processes are designed to remove duplicated data from a single storage device. Global deduplication is removing redundant data across the entire data storage infrastructure. Global deduplication allows administrators to efficiently manage the entire backup data storage environment.
Data Deduplication Products
Barracuda Backup
Barracuda Backup is a data recovery and restore, and data deduplication, option from Barracuda Networks.
Data Domain
Data Domain from EMC is a data protection, backup, storage and deduplication solution.
Veritas NetBackup Appliance
Symantec NetBackup Appliance is a storage and deduplication solution.
Compellent
Since it's acquisition in 2011, Compellent is now a Dell product line of storage solutions.
NetApp FAS series
NetApp's FAS series systems offers a storage array system for enterprises.
IBM ProtecTier
ProtecTier from IBM is a data storage, backup, and data deduplication solution from IBM.
Nexsan Data Deduplication Appliance
Nexsan, from Imation, offers data deduplication in their storage solutions.
HP Data Deduplication Appliance
Hewlett-Packard's storage and recovery solutions offer data deduplication capability.
Fujitsu Data Deduplication Appliance
Fujitsu offers data deduplication solutions.
Spectra nTier Deduplication Appliance
Spectra Logic's nTier data storage product line offers data deduplication.
Quest DR Series
Quest DR Series is a backup and recovery solution, featuring data deduplication.
Falconstor VTL
Falconstor Virtual Tape Library (VTL) is the eponymous data center backup and recovery solution from the company headquartered in Melville, New York.
Falconstor FDS
Falconstor FDS is a data management / deduplication solution from Falconstor.
Hitachi Protection Platform Backup Appliance
Since the 2014 acquisition, technologies formerly offered by Sepaton (with DeltaStor data deduplication software) are now part of the Hitachi Protection Platform.
Nexsan DeDupe SG
Nexsan DeDupe SG is a disk based backup and data deduplication option, from Imation company Nexsan.
Microsoft DPM Appliance
In early 2015 Microsoft added data deduplication capability to their System Center 2012 R2 Data Protection Manager.
Clear Analytics
Clear Analytics is a business intelligence solution that enables non technical end users to perform analytics by leveraging existing knowledge of Excel coupled with a built in query builder. Some key features include: Dynamic Data Refresh, Data Share and In-Excel Collaboration.
StarDQ
StarDQ is a real time solution for cleansing, de-duping, and enriching enterprise data. By integrating StarDQ Solution, organizations can cleanse, match and unify data across multiple data sources and data domains. According to the vendor, the goal is to ensure that data is a...
DupeCatcher
DupeCatcher is a free de-duplication tool from the maker of Cloudingo.
Atempo
Atempo's integrated suite of solutions include data backup and disaster recovery offerings (Time Navigator and Live Navigator) and as well an an innovative archiving solution (Atempo Digital Archive) which features intuitive retrieval interfaces and supports a broad scope of platforms. Based on...
StorReduce
StorReduce is a software defined, scale-out deduplication service in the cloud. The product reduces object storage costs, speeds up transfer, and enables data reuse.