Efficient Processing of Multi-way Joins Using MapReduce

• 作者: Aili Liu Baoyan Song Linlin Ding Siping Liu Yu Liu
• 来源期刊: 国际计算机前沿大会会议论文集
• 学科: 社会科学
• 关键词: MAPREDUCE multi-way joins SHARING and COORDINATION
• 年，卷（期）: 2015,（1）
• 页码范围: 23-24
• 页数: 2页
• 分类号: C5

摘要

Multi-way join is critical for many big data applications such as data mining and knowledge discovery. Even though lots of research have been devoted to processing multi-way joins using MapReduce, there are still several problems in general to be further improved, such as transferring numerous unpromising intermediate data and lacking of better coordination mechanisms. This work proposes an efficient multi-way joins processing model using MapReduce, named Sharing-Coordination-MapReduce (SC-MapReduce), which has the functions of sharing and coordination. Our SC-MapReduce model can filter the unpromising intermediatedata largely by using the sharing mechanism and optimize the multiple tasks coordination of multi-way joins. Extensive experiments show that the proposed model is efficient, robust and scalable.