SparkSCAN:A Structure Similarity Clustering Algorithm on Spark

• 作者: Jingbin Wang Qijun Zhou
• 来源期刊: 国际计算机前沿大会会议论文集
• 学科: 社会科学
• 关键词: Directed GRAPH CLUSTERING PARALLEL algorithm SparkRDD.Springer
• 年，卷（期）: 2015,（B12）
• 页码范围: 42-44
• 页数: 3页
• 分类号: C5

摘要

The existing directed graph clustering algorithms are born with some problems such as high latency,resource depletion and poor performance of iterative data processing.A distributed parallel algorithm of structure similarity clustering on Spark(SparkSCAN)is proposed to solve these problems:considering the interaction between nodes in the network,the similar structure of nodes are clustered together;Aiming at the large-scale characteristics of directed graphs,a data structure suitable for distributed graph computing is designed,and a distributed parallel clustering algorithm is proposed based on Spark framework,which improves the processing performance on the premise of the accuracy of clustering results.The experimental results show that the SparkSCAN have a good performance,and can effectively deal with the problem of clustering algorithm for large-scale directed graph.Keywords:Directed graph clustering
Parallel algorithm