RefineDedup: efficient deduplication for mobile systems via application-wise learning

With the rapid increase in the size of mobile applications, data deduplication technologies are used to improve the storage capacity of mobile systems. However, existing data deduplication approaches show high memory footprints and performance costs on mobile systems due to the relatively low duplication ratios, which generally range from 10% to 35%, of mobile applications. The highly individualized and evolving data accesses of applications make data deduplication on mobile devices even more challenging. Thus, we propose RefineDedup, an efficient data deduplication method for mobile systems that leverages the application-wise data access behaviors. The basic idea of RefineDedup is to avoid the overhead of useless fingerprints of applications using tiny application-specific learning models. First, RefineDedup uses a forecasting sliding window to collect and analyze data accesses of applications. Then, RefineDedup trains application-specific models for the applications with high data duplication ratios, along with a shared model for the applications with low data duplication ratios. These models are effective in predicting the data access behaviors of applications. We implement RefineDedup on a smartphone running Android 10 and evaluate it with real-world applications. Extensive experimental results show that RefineDedup stores 33.4% fewer fingerprints than FinerDedup, the state-of-the-art data deduplication scheme for mobile systems. With a duplication ratio below 35%, RefineDedup's write bandwidth is on average 12.2% higher than that of FinerDedup.