Details

Autor(en) / Beteiligte

• Dong, Lu
• Li, Peng
• Xu, He
• Luo, Baozhou
• Mi, Yu

Titel

Performance Prediction of Spark Based on the Multiple Linear Regression Analysis

Ist Teil von

• Parallel Architecture, Algorithm and Programming, p.70-81

Ort / Verlag

Singapore: Springer Singapore

Link zum Volltext

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• It is crucial to evaluate performance of a cloud platform and determine the main factors influencing the property. Moreover, the analysis results of related performance indicators can be applied to making theoretical predictions about the performance status of the cloud platform. This work mainly focuses on researching the interrelations between the performance indicators based on the Spark technology of the cloud platform and the load performance of the cluster, and furthermore makes effective predictions for the load performance. Firstly, we put forward the analytic frameworks of Spark performance analysis, the specific indicators analysis as well as the prediction models towards the cluster load. Secondly, with respect to the evaluation indicators, we explore the basis for their selections as well as their concrete implications, and then objectively, accurately calculate the correlation formula between the practically produced performance parameters and the load performance of the cluster when the Spark cluster performs the batch applications utilizing the MLR (Multiple Linear Regression) method, and, therefore, determine the main factors impacting the load performance. Finally, we predict the load value utilizing the Spark indicator analysis and the load prediction model. The results indicate that accuracy is up to 92.307%. Consequently, the solution presented in this paper predicts the cluster load value with effetioncy.