期刊导航

论文摘要

基于重复数据删除的快速文件归档方法

Rapid Archiving Method Based on Data Deduplication

作者:马晓旭(四川大学 计算机学院);刘晓洁(四川大学 计算机学院);胡晓勤(四川大学 计算机学院);赵辉(四川大学 计算机学院)

Author:Ma Xiaoxu(School of Computer Sci.,Sichuan Univ.);Liu Xiaojie(School of Computer Sci.,Sichuan Univ.);Hu Xiaoqin(School of Computer Sci.,Sichuan Univ.);Zhao Hui(School of Computer Sci.,Sichuan Univ.)

收稿日期:2010-10-20          年卷(期)页码:2011,43(5):120-125

期刊名称:工程科学与技术

Journal Name:Advanced Engineering Sciences

关键字:文件归档;重复数据删除;数据指纹;局部性;Winnowing

Key words:file archive;data deduplication;data fingerprint;locality;winnowing

基金项目:国家自然科学基金资助项目(60873246);教育部博士点基金(20070610032);教育部重大项目培育基金(708075)

中文摘要

为了提高归档系统的存储效率及性能,提出了一种基于重复数据删除的快速文件归档方法(RAMBDD),利用文件分块、比较数据块指纹、删除重复数据,实现了文件的数据块级归档。RAMBDD中给出了一种基于winnowing的重复数据删除方法LMCA,它在提高文件冗余检测率的同时也保证了文件分块的效率,并通过使用指纹快速检索方法和局部指纹缓存方法,减少了在查找不存在的数据块指纹时的磁盘读取次数,加速了查找重复数据块的过程。实验结果表明,与传统的文件归档方法相比,本方法大大节省了归档数据的存储空间和网络传输带宽,缩短了归档时间,提高了文件归档的效率。

英文摘要

In order to improve the storage efficiency and performance of archival system, a method of rapid archive based on data deduplication(RAMBDD) was proposed.The archive in block-level was achieved by subdividing the files into chunks, comparing chunk-fingerprints and deleting the duplicate data. In RAMBDD, a novel redundancy elimination algorithm based on winnowing(LMCA) was provided, which ensured the efficiency of file chunking while improving the detection rate of redundant, and through rapid fingerprint indexing method and locality fingerprint caching method, reduced the disk I/Os looking for a non-existent duplicate chunk fingerprint and accelerated the process of finding duplicate chunks. The experiment results indicated that this method can save storage space and bandwidth of network remarkably,decrease archival time and improve efficiency of archive evidently over traditional file archive.

关闭

Copyright © 2020四川大学期刊社 版权所有.

地址:成都市一环路南一段24号

邮编:610065