A job response time prediction method for production Grid computing environments

Goyeneche, A. 2010. A job response time prediction method for production Grid computing environments. PhD thesis University of Westminster School of Electronics and Computer Science

TitleA job response time prediction method for production Grid computing environments
TypePhD thesis
AuthorsGoyeneche, A.
Abstract

A major obstacle to the widespread adoption of Grid Computing in both the scientific

community and industry sector is the difficulty of knowing in advance a job submission running

cost that can be used to plan a correct allocation of resources.

Traditional distributed computing solutions take advantage of homogeneous and open

environments to propose prediction methods that use a detailed analysis of the hardware and

software components. However, production Grid computing environments, which are large and

use a complex and dynamic set of resources, present a different challenge. In Grid computing

the source code of applications, programme libraries, and third-party software are not always

available. In addition, Grid security policies may not agree to run hardware or software analysis

tools to generate Grid components models.

The objective of this research is the prediction of a job response time in production Grid

computing environments. The solution is inspired by the concept of predicting future Grid

behaviours based on previous experiences learned from heterogeneous Grid workload trace

data. The research objective was selected with the aim of improving the Grid resource usability

and the administration of Grid environments. The predicted data can be used to allocate

resources in advance and inform forecasted finishing time and running costs before submission.

The proposed Grid Computing Response Time Prediction (GRTP) method implements

several internal stages where the workload traces are mined to produce a response time

prediction for a given job. In addition, the GRTP method assesses the predicted result against

the actual target job’s response time to inference information that is used to tune the methods

setting parameters.

The GRTP method was implemented and tested using a cross-validation technique to assess

how the proposed solution generalises to independent data sets. The training set was taken from

the Grid environment DAS (Distributed ASCI Supercomputer). The two testing sets were taken

from AuverGrid and Grid5000 Grid environments

Three consecutive tests assuming stable jobs, unstable jobs, and using a job type method to

select the most appropriate prediction function were carried out. The tests offered a significant

increase in prediction performance for data mining based methods applied in Grid computing

environments. For instance, in Grid5000 the GRTP method answered 77 percent of job

prediction requests with an error of less than 10 percent. While in the same environment, the most effective and accurate method using workload traces was only able to predict 32 percent of

the cases within the same range of error.

The GRTP method was able to handle unexpected changes in resources and services which

affect the job response time trends and was able to adapt to new scenarios. The tests showed

that the proposed GRTP method is capable of predicting job response time requests and it also

improves the prediction quality when compared to other current solutions.

Year2010
FileHector_Ariel_GOYENECHE_NEW.pdf
Publication dates
Completed2010

Related outputs

Extracting performance hints for grid users using data mining techniques: a case study in the NGS
Goyeneche, A., Guim, F., Rodero, I., Terstyanszky, G. and Corbalan, J. 2007. Extracting performance hints for grid users using data mining techniques: a case study in the NGS. Mediterranean Journal of Computers & Networks. 3 (2), pp. 52-61.

Improving Grid computing performance prediction using weighted templates
Goyeneche, A., Terstyanszky, G., Delaitre, T. and Winter, S. 2007. Improving Grid computing performance prediction using weighted templates. in: Cox, S.J. (ed.) Proceedings of the UK e-Science All Hands Meeting 2007, Nottingham UK, 10th - 13th September Edinburgh National e-Science Centre. pp. 361-368

Executing parameter study workflows in the P-Grade portal
Sipos, G., Goyeneche, A., Kiss, T. and Kacsuk, P. 2006. Executing parameter study workflows in the P-Grade portal. CoreGRID Workshop on Grid Systems, Tools and Environments in Conjunction with GRIDS@work: CoreGRID Conference, Grid Plugtests and Contest. Sophia-Antipolis, France 01 Dec 2006

User friendly environment to grid enabled legacy codes
Kacsuk, P., Kiss, T., Delaitre, T., Goyeneche, A., Winter, S., Terstyanszky, G., Farkas, Z., Nemeth, C. and Boczko, T. 2006. User friendly environment to grid enabled legacy codes. in: Di Martino, B., Dongarra, J., Hoisie, A., Yang, L.T. and Zima, H. (ed.) Engineering the grid: status and perspective Valencia, California, USA ASP Press. pp. 205-217

GEMLCA: running legacy code applications as grid services
Delaitre, T., Kiss, T., Goyeneche, A., Terstyanszky, G., Winter, S. and Kacsuk, P. 2005. GEMLCA: running legacy code applications as grid services. Journal of Grid Computing. 3 (1-2), pp. 75-90.

Security mechanisms for legacy code applications in GT3 environment
Terstyanszky, G., Delaitre, T., Goyeneche, A., Kiss, T., Sajadah, K., Winter, S. and Kacsuk, P. 2005. Security mechanisms for legacy code applications in GT3 environment. in: 13th Euromicro Conference on Parallel, Distributed, and Network-Based Processing proceedings: Lugano, Switzerland, February 9-11, 2005 Los Alamitos, USA IEEE . pp. 220-226

Experiences with deploying legacy code applications as grid services using GEMLCA
Goyeneche, A., Kiss, T., Terstyanszky, G., Kecskemeti, G., Delaitre, T., Kacsuk, P. and Winter, S. 2005. Experiences with deploying legacy code applications as grid services using GEMLCA. in: Sloot, P.M.A., Hoekstra, A.G., Priol, T. and Reinefeld, A. (ed.) Advances in grid computing (EGC 2005): European Grid Conference, Amsterdam, The Netherlands, February 14-16, 2005: revised selected papers Berlin, Germany Springer.

GEMLCA: running legacy code applications as grid services
Delaitre, T., Kiss, T., Goyeneche, A., Terstyanszky, G., Winter, S. and Kacsuk, P. 2005. GEMLCA: running legacy code applications as grid services. CoreGRID.

Deploying application on a GT3 Grid
Kiss, T., Delaitre, T., Goyeneche, A., Winter, S., Kacsuk, P., Terstyanszky, G., Igbe, D., Maselino, P., Sajadah, K. and Weingarten, N. 2004. Deploying application on a GT3 Grid. London, UK University of Westminster.

High-level grid application environment to use legacy codes as OGSA grid services
Kacsuk, P., Goyeneche, A., Delaitre, T., Kiss, T., Farkas, Z. and Boczko, T. 2004. High-level grid application environment to use legacy codes as OGSA grid services. in: Buyya, R. (ed.) Fifth IEEE/ACM International Workshop on Grid Computing: Pittsburgh, Pennsylvania, 8 November, 2004 Las Alamitos, USA IEEE . pp. 428-435

A high-level grid application environment to grid-enable legacy code
Kacsuk, P., Farkas, Z., Boczko, T., Kiss, T., Goyeneche, A. and Delaitre, T. 2004. A high-level grid application environment to grid-enable legacy code. ERCIM News. 59, pp. 44-45.

An approach to managing quality of services and performance in grid computing
Goyeneche, A., Terstyanszky, G. and Winter, S. 2004. An approach to managing quality of services and performance in grid computing. MicroCAD 2004 International Scientific Conference. Miskolc, Hungary 18-19 Mar 2004

Publishing and executing parallel legacy code using an OGSI grid service
Delaitre, T., Goyeneche, A., Kiss, T. and Winter, S. 2004. Publishing and executing parallel legacy code using an OGSI grid service. in: Lagana, A. (ed.) Computational science and its applications: ICCSA 2004, international conference, Assisi, Italy, May 14-17, 2004, proceedings Heidelberg, Germany Springer.

Experiences with publishing and executing parallel legacy code using an OGSI grid service
Delaitre, T., Goyeneche, A., Kiss, T., Terstyanszky, G., Winter, S., Kacsuk, P., Igbe, D., Maselino, P., Sajadah, K. and Weingarten, N. 2004. Experiences with publishing and executing parallel legacy code using an OGSI grid service. in: Proceedings of the UK E-Science All Hands Meeting, 31st Aug - 3rd Sep, 2004, Nottingham, UK EPSRC. pp. 999-1002

Traffic simulation in P-Grade as a grid service
Delaitre, T., Goyeneche, A., Kiss, T., Terstyanszky, G., Weingarten, N., Maselino, P., Gourgoulis, A. and Winter, S. 2004. Traffic simulation in P-Grade as a grid service. in: Zoltan, J., Kacsuk, P. and Kranzlmuller, D. (ed.) Distributed and parallel systems: cluster and grid computing New York, USA Springer.

GEMLCA: grid execution management for legacy code architecture design
Delaitre, T., Goyeneche, A., Kacsuk, P., Kiss, T., Terstyanszky, G. and Winter, S. 2004. GEMLCA: grid execution management for legacy code architecture design. in: Steinmetz, R., Mauthe, A. and Crnkovic, I. (ed.) 30th Euromicro Conference proceedings: 31 August-3 September, 2004, Rennes, France IEEE . pp. 477-483

Permalink - https://westminsterresearch.westminster.ac.uk/item/905yw/a-job-response-time-prediction-method-for-production-grid-computing-environments


Share this
Tweet
Email