Articles
Tool Support for Distributed Workflow Management with Task Clustering
Authors:
Ayesh Weerasinghe,
University of Moratuwa, LK
About Ayesh
Department of Computer Science & Engineering
Kalana Wijethunga,
University of Moratuwa, LK
About Kalana
Department of Computer Science & Engineering
Randika Jayasekara ,
University of Moratuwa, LK
About Randika
Department of Computer Science & Engineering
Indika Perera,
University of Moratuwa, LK
About Indika
Department of Computer Science & Engineering
Anuradha Wickramarachchi
Australian National University, AU
Abstract
When in need for executing complex sets of interrelated calculations on High-Performance Computing (HPC) environments the obvious choice is to use scientific workflows. As workload management software do not support the execution of interrelated tasks, workflow management systems have been introduced to execute workflows on HPC environments. Recently, a new distributed architectural model that offers dynamic workflow execution capabilities to workflow management systems is introduced. It executes workflows on a per-task basis. While this approach facilitates dynamic workflows, it adds a considerable overhead to workflows substantially increasing their makespans. As most workflows are static, task-wise execution of workflows degrades the performance of most workflows. In this paper, we introduce a distributed workflow management system, SwarmForm that introduces task clustering to the new architectural model. SwarmForm is open source and offers better performance than existing distributed workflow management systems by clustering workflow tasks to reduce overheads while allowing the users to choose between task-wise and cluster-wise execution of workflows depending on the workflow nature. The paper proves that SwarmForm enables the use of all the features introduced with the new architectural model while providing better makespans for scientific workflows.
How to Cite:
Weerasinghe, A., Wijethunga, K., Jayasekara, R., Perera, I. and Wickramarachchi, A., 2021. Tool Support for Distributed Workflow Management with Task Clustering. International Journal on Advances in ICT for Emerging Regions (ICTer), 14(2), pp.1–8. DOI: http://doi.org/10.4038/icter.v14i2.7223
Published on
30 Mar 2021.
Peer Reviewed
Downloads
