Capacity optimization
Encyclopedia
Capacity optimization is a general term for technologies used to improve storage utilization by shrinking stored data. The primary technologies used for capacity optimization are deduplication
and data compression
. These solutions are delivered as software or hardware solution, integrated with existing storage systems or delivered as standalone products. Deduplication algorithms look for redundancy in sequences of bytes across comparison windows. Typically using cryptographic hash functions as identifiers of unique sequences, sequences are compared to the history of other such sequences, and where possible, the first uniquely stored version of a sequence is referenced rather than stored again. Different solutions use different methods for selecting data windows, from 4KB blocks to full-file comparisons known as Single Instance Storage or SIS.
Capacity optimization generally refers to the use of this kind of technology in a storage system. An example of this kind of system is the Venti file system in the Plan9 open source OS. There are also implementations in networking (especially Wide Area networking), where they are sometimes called bandwidth optimization or WAN Optimization
technologies.
Commercial implementations of capacity optimization are most often found in backup/recovery storage, where storage of iterating versions of backups day to day creates an opportunity for reduction in space using this approach. The term was first used widely in 2005.
Deduplication
The term deduplication refers generally to eliminating duplicate or redundant information.* Data deduplication, in computer storage, refers to the elimination of redundant data...
and data compression
Data compression
In computer science and information theory, data compression, source coding or bit-rate reduction is the process of encoding information using fewer bits than the original representation would use....
. These solutions are delivered as software or hardware solution, integrated with existing storage systems or delivered as standalone products. Deduplication algorithms look for redundancy in sequences of bytes across comparison windows. Typically using cryptographic hash functions as identifiers of unique sequences, sequences are compared to the history of other such sequences, and where possible, the first uniquely stored version of a sequence is referenced rather than stored again. Different solutions use different methods for selecting data windows, from 4KB blocks to full-file comparisons known as Single Instance Storage or SIS.
Capacity optimization generally refers to the use of this kind of technology in a storage system. An example of this kind of system is the Venti file system in the Plan9 open source OS. There are also implementations in networking (especially Wide Area networking), where they are sometimes called bandwidth optimization or WAN Optimization
WAN Optimization
WAN optimization is a collection of techniques for increasing data-transfer efficiencies across wide-area networks. In 2008, the WAN optimization market was estimated to be $1 billion , and it will grow to $4.4 billion according to Gartner, a technology research firm.The most common measures of...
technologies.
Commercial implementations of capacity optimization are most often found in backup/recovery storage, where storage of iterating versions of backups day to day creates an opportunity for reduction in space using this approach. The term was first used widely in 2005.