Hadoop + Spark Platform Based on Big Data System Design of Agricultural Product Price Analysis and Prediction by HoltWinters

• 作者: Qingjun Zhang Xiaohui Cheng Yan Zhu Yun Deng
• 来源期刊: 国际计算机前沿大会会议论文集
• 学科: 社会科学
• 关键词: HADOOP SPARK BIG data Analysis and FORECAST of AGRICULTURAL product PRICES Holt-Winters
• 年，卷（期）: 2019,（1）
• 页码范围: 121-123
• 页数: 3页
• 分类号: C

摘要

In the market of agricultural products, the price of agricultural products is affected by production cost, market supply and other factors. In order to obtain the market information of agricultural products, the price fluctuation can be analyzed and predicted. A distributed big data software platform based on Hadoop, Hive and Spark is proposed to analyze and forecast agricultural price data. Firstly, Hadoop, Hive and Spark big data frameworks were built to store the data information of agricultural products crawled into MYSQL. Secondly, the information of agricultural products crawled from MYSQL was exported to a text file, uploaded to HDFS, and mapped to spark SQL database. The data was cleaned and improved by Holt-Winters (three times exponential smoothing method) model to predict the price of agricultural products in the future. The data cleaned by spark SQL was imported and predicted by improved Holt-Winters into MYSQL database. The technologies of pringMVC, Ajax and Echarts were used to visualize the data.