Details

Autor(en) / Beteiligte

• Yuxuan Jiang
• Zhe Huang
• Tsang, Danny H. K.

Titel

Towards Max-Min Fair Resource Allocation for Stream Big Data Analytics in Shared Clouds

Ist Teil von

• IEEE transactions on big data, 2018-03, Vol.4 (1), p.130-137

Ort / Verlag

Piscataway: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)

Erscheinungsjahr

2018

Quelle

IEEE Electronic Library (IEL)

Beschreibungen/Notizen

• Distributed stream big data analytics platforms have emerged to tackle the continuously generated data streams. In stream big data analytics, the data processing workflow is abstracted as a directed graph referred to as a topology. Data are read from the storage and processed tuple by tuple, and these processing results are updated dynamically. The performance of a topology is evaluated by its throughput. This paper proposes an efficient resource allocation scheme for a heterogeneous stream big data analytics cluster shared by multiple topologies, in order to achieve max-min fairness in the utilities of the throughput for all the topologies. We first formulate a novel resource allocation problem, which is a mixed 0-1 integer program. The NP-hardness of the problem is rigorously proven. To tackle this problem, we transform the non-convex constraint to several linear constraints using linearization and reformulation techniques. Based on the analysis of the problem-specific structure and characteristics, we propose an approach that iteratively solves the continuous problem with a fixed set of discrete variables optimally, and updates the discrete variables heuristically. Simulations show that our proposed resource allocation scheme remarkably improves the max-min fairness in utilities of the topology throughput, and is low in computational complexity.