Manuscript Number : CSEIT172566
Hybrid Job-Driven Scheduling for Heterogeneous MapReduce Clusters
Authors(2) :-J. Sivarani, T. Subramanyam
It is cost-efficient for a tenant with a limited budget to establish a heterogeneous virtual MapReduce clusters by renting various virtual private servers (VPSs) from a VPS provider. To provide an appropriate scheduling scheme for this type of computing environment, and MapReduce still performs poorly on heterogeneous clusters, we propose in this paper a hybrid job-driven scheduling scheme (JoSS for short) from a tenant perspective. JoSS provide not only job level scheduling, but also Map-task level scheduling and Reduce-task level scheduling; The deployment of MapReduce in data canters and clouds present several challenges, improve data locality for both map-level task and reduce-level task, avoid job starvation and improve job execution performance. Two variations of JoSS-Task and JoSS-Job are further introduced to separately achieve a better map-data locality and a faster task assignment. We conduct extensive experiments to evaluate and compare the two variations (JoSS-T and JoSS-J) with current scheduling algorithms supported by Hadoop. The result shows that the two variations crush the opposite tested algorithms in terms of map and reduce data locality , and network overhead while not acquisition significant overhead. Additionally, the two variations area unit severally appropriate for various MapReduce-workload eventualities and supply the most effective job performance among all tested algorithms.
J. Sivarani MapReduce, Hadoop, Map-task Scheduling, Reduce-task Scheduling, Heterogeneous virtual MapReduce clusters Publication Details Published in : Volume 2 | Issue 5 | September-October 2017 Article Preview
Department of Computer Science, Sri Padmavathi University, Tirupati, India
T. Subramanyam
Asst. Professor, Department of Computer Science Sri Padmavathi University, Tirupati, India
Date of Publication : 2017-10-31
License: This work is licensed under a Creative Commons Attribution 4.0 International License.
Page(s) : 330-341
Manuscript Number : CSEIT172566
Publisher : Technoscience Academy