Policy based data placement in distributed systems

Scientific domains are increasingly adopting workflow systems to automate and manage large distributed applications. Workflow Management Systems (WMS) manage overall scheduling and monitoring of both compute and data placement jobs for such applications. The management of data placement jobs in WMS provide the overall context for the problems addressed in this thesis. ❧ This thesis starts by automating data placement for scientific applications based on user provided data-placement policies and proceeds to interface a WMS with a policy based data placement service (PDPS). It provides a solution for the lack of testing data in workflow science by developing a Synthetic Directed Acyclic Graph Generator (SDAG) and using synthetic workflows generated by it in a case study. ❧ This thesis relies on actual software development and experimental analysis for both major research contributions. Experimental results using existing workflows prove the immediate benefit of PDPS for mid-sized virtual organizations. Results for SDAG demonstrate its usefulness in the design and development of future WMS.