Start Submission Become a Reviewer

Reading: Tool Support for Distributed Workflow Management with Task Clustering

Download

A- A+
Alt. Display

Articles

Tool Support for Distributed Workflow Management with Task Clustering

Authors:

Ayesh Weerasinghe,

University of Moratuwa, LK
About Ayesh
Department of Computer Science & Engineering
X close

Kalana Wijethunga,

University of Moratuwa, LK
About Kalana
Department of Computer Science & Engineering
X close

Randika Jayasekara ,

University of Moratuwa, LK
About Randika
Department of Computer Science & Engineering
X close

Indika Perera,

University of Moratuwa, LK
About Indika
Department of Computer Science & Engineering
X close

Anuradha Wickramarachchi

Australian National University, AU
X close

Abstract

When in need for executing complex sets of interrelated calculations on High-Performance Computing (HPC) environments the obvious choice is to use scientific workflows. As workload management software do not support the execution of interrelated tasks, workflow management systems have been introduced to execute workflows on HPC environments. Recently, a new distributed architectural model that offers dynamic workflow execution capabilities to workflow management systems is introduced. It executes workflows on a per-task basis. While this approach facilitates dynamic workflows, it adds a considerable overhead to workflows substantially increasing their makespans. As most workflows are static, task-wise execution of workflows degrades the performance of most workflows. In this paper, we introduce a distributed workflow management system, SwarmForm that introduces task clustering to the new architectural model. SwarmForm is open source and offers better performance than existing distributed workflow management systems by clustering workflow tasks to reduce overheads while allowing the users to choose between task-wise and cluster-wise execution of workflows depending on the workflow nature. The paper proves that SwarmForm enables the use of all the features introduced with the new architectural model while providing better makespans for scientific workflows.
How to Cite: Weerasinghe, A., Wijethunga, K., Jayasekara, R., Perera, I. and Wickramarachchi, A., 2021. Tool Support for Distributed Workflow Management with Task Clustering. International Journal on Advances in ICT for Emerging Regions (ICTer), 14(2), pp.1–8. DOI: http://doi.org/10.4038/icter.v14i2.7223
Published on 30 Mar 2021.
Peer Reviewed

Downloads

  • PDF (EN)

    comments powered by Disqus