Migrating data and application between Kubernetes edge servers

Edge devices are nearer to users than the cloud and can compute data and return results to users faster than the cloud, making them more suitable for applications that need to respond in real-time. Deep neural network (DNN) models are increasingly used in edge devices such as smartphones. Examples of such applications are a translation application, image recognition, etc. Since the edge devices have resource limitations, they might have issues when running the DNN models, which need powerful hardware. We generally trade off accuracy with less computation by limiting the number of parameters and layers of the DNN models to fit appropriately with the underlying edge resources. Furthermore, the edge environment also has many issues with network latency, scalability, and energy efficiency. Data migration and applications can mitigate such limitations. Also, migration can minimize edge latency when users move from one place to another and for load balancing. To ease mobility in a limited resource edge environment, we could use container orchestration. The popular container orchestration software is Kubernetes. Containers are encapsulated inside a pod within a Kubernetes cluster. Kubernetes does not officially support migrating pods. To migrate a pod on the Kubernetes, we need to checkpoint the state of an application running at the source and resume this state again at the destination. Each DNN model needs data for training and inference. Since we migrate DNN models, we need to migrate the dataset as well. There is no existing research studying migrating the active state of the DNN models. Therefore, this research focuses on study the migration of DNN datasets and applications between Kubernetes nodes. In our research, we studied the applicability of existing checkpointing mechanisms (CRIU and DMTCP) with DNN applications. We found that CRIU can be used to checkpoint containers running DNN models effectively. Hence, we decided to adopt the MigratingPod API, which integrates CRIU with Kubernetes as an enabling mechanism to migrate DNN applications. To migrate DNN models between worker nodes in a Kubernetes cluster, we created another API based on the MigratingPod API to make it more convenient for the migration; our API firstsaves, the states of DNN models, migrates them to the destination, and restore them from their previous states. Second, we created a new middleware according to the previously proposed framework to dynamically transfer DNN datasets from one host to another host in a Kubernetes cluster using TCP parallel connection. The middleware was integrated as a part of our API. The proposed API was evaluated in terms of effectiveness and efficiency in an experiment testbed for edge environments. The proposed API can migrate DNN active states and, as a result, this allows for faster recovery and saves training time by 10 to 73 percent. Migrating a dataset between Kubernetes workers using the Kubernetes’s persistent volume with kubectl cp is generally suitable and efficient. However, when network latency is high (which is common for edge environments), using our middleware approach with a feedback controller to migrate data daptively in parallel can speed up total migration time when compared to using the Kubernetes' persistent volume approach alone up to 51.74 percent.