Data-Aware Partitioning Schema in MapReduce

• 作者: Liang Junjie Liu Qiongni Yin Li Yu Dunhui
• 来源期刊: 国际计算机前沿大会会议论文集
• 学科: 社会科学
• 关键词: CLOUD COMPUTING MAPREDUCE HIGH-DIMENSIONAL DATA Dataaware partitioning
• 年，卷（期）: 2015,（1）
• 页码范围: 28-29
• 页数: 2页
• 分类号: C5

摘要

With the advantages of MapReduce programming model in parallel computing and processing of data and tasks on large-scale clusters, a Dataaware partitioning schema in MapReduce for large-scale high-dimensional data is proposed. It optimizes partition method of data blocks with the same contribution to computation in MapReduce. Using a two-stage data partitioning strategy, the data are uniformly distributed into data blocks by clustering and partitioning. The experiments show that the data-aware partitioning schema is very effective and extensible for improving the query efficiency of highdimensional data.