TransGPerf: Exploiting Transfer Learning for Modeling Distributed Graph Computation Performance

• 作者: Songjie Niu Shimin Chen
• 来源期刊: 计算机科学技术学报（英文版）
• 年，卷（期）: 2021,（4）
• 所属期刊栏目: Special Section on AI4DB and DB4AI
• 页码范围: 778-791
• 页数: 14页
• 语种: 英文

摘要

It is challenging to model the performance of distributed graph computation. Explicit formulation cannot easily capture the diversified factors and complex interactions in the system. Statistical learning methods require a large number of training samples to generate an accurate prediction model. However, it is time-consuming to run the required graph computation tests to obtain the training samples. In this paper, we propose TransGPerf, a transfer learning based solution that can exploit prior knowledge from a source scenario and utilize a manageable amount of training data for modeling the performance of a target graph computation scenario. Experimental results show that our proposed method is capable of generating accurate models for a wide range of graph computation tasks on PowerGraph and GraphX. It outperforms transfer learning methods proposed for other applications in the literature.