A Data Analysis Framework for Earth System Simulation within an <i>In-Situ</i>Infrastructure

• 作者: D. Wang F. Yuan N. Podhorszki X. Luo
• 来源期刊: 电脑和通信(英文)
• 学科: 医学
• 关键词: IN-SITU DATA ANALYSIS Source Code ANALYSIS DATA STAGING ADIOS Earth System Model Machine Learning SciKit-Learn E3SM
• 年，卷（期）: 2017,（14）
• 页码范围: 76-85
• 页数: 10页
• 分类号: R73

摘要

This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e., automated code analysis and instrumentation), in-situ infrastructure (i.e., ADIOS) and big data analysis engines (i.e., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. The in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.