TY  - JOUR
TI  - Hybrid Job-Driven Scheduling for Heterogeneous MapReduce Clusters
AU  - J. Sivarani
AU  -  T. Subramanyam
JO  - International Journal of Scientific Research in Computer Science, Engineering and Information Technology
PB  - Technoscience Academy
DA  - 2017/10/31
PY  - 2017
DO  - https://doi.org/10.32628/IJSRCSEIT
UR  - https://ijsrcseit.com/CSEIT172566
VL  - 2
IS  - 5
SP  - 330	
EP  - 341
AB  - It is cost-efficient for a tenant with a limited budget to establish a heterogeneous virtual MapReduce clusters by renting various virtual private servers (VPSs) from a VPS provider. To provide an appropriate scheduling scheme for this type of computing environment, and MapReduce still performs poorly on heterogeneous clusters, we propose in this paper a hybrid job-driven scheduling scheme (JoSS for short) from a tenant perspective. JoSS provide not only job level scheduling, but also Map-task level scheduling and Reduce-task level scheduling; The deployment of MapReduce in data canters and clouds present several challenges, improve data locality for both map-level task and reduce-level task, avoid job starvation and improve job execution performance. Two variations of JoSS-Task and JoSS-Job are further introduced to separately achieve a better map-data locality and a faster task assignment. We conduct extensive experiments to evaluate and compare the two variations (JoSS-T and JoSS-J) with current scheduling algorithms supported by Hadoop. The result shows that the two variations crush the opposite tested algorithms in terms of map and reduce data locality , and network overhead while not acquisition significant overhead. Additionally, the two variations area unit severally appropriate for various MapReduce-workload eventualities and supply the most effective job performance among all tested algorithms.