HPL-2009-122 Sparse Indexing: Large Scale, Inline Deduplication Using Sampling and Locality - Lillibridge, Mark; Eshghi, Kave; Bhagwat, Deepavali; Deolalikar, Vinay; Trezise, Greg; Camble, Peter
Keyword(s): deduplication, storage, scaling, locality, inline deduplication, sparse index, chunking
Abstract: We present sparse indexing, a technique that uses sampling and exploits the inherent locality within backup streams to solve for large-scale backup (e.g., hundreds of terabytes) the chunk-lookup disk bottleneck problem that inline, chunk-based deduplication schemes face. The problem is that these sc ...
Full Report