Abstract | Today, cloud computing is a well-established paradigm that enjoys a moderate level of popularity among Science Gateways and Virtual Research Environments. Historically, Science Gateways have empowered researchers with traditional onpremise High-Performance Computing (HPC), but recent years have seen many turning to the cloud to fill that space. One often cited cloud complaint is the distance that data must travel. Sensors and input devices generate data locally in the lab, but even for a basic calculation, all those bits must make the costly journey to a data centre hundreds of miles away. The proliferation of the Internet of Things, though, is quickly changing this. As microchips decrease in size, the compute power of those sensors and input devices is increasing, along with every other compute or networking device in the lab. These devices, at the edge of the network, can now be exploited for less intensive computations, such as pre-processing or analytics. This creates an ideal opportunity for Science Gateways. As well as connecting research communities to the cloud, they can bring edges into the mix and get the advantages of both. In that, lies a considerable challenge though. Edge devices come in all shapes and sizes – or, from a computing point of view, all operating systems and system architectures. Many such devices are connected by Wi-Fi, so connection issues are common and unpredictable. Once deployed, communications between edge and cloud must be secured. DevOps engineers are solving these issues with OCIcompliant containers (Docker), configuration management (Ansible®), container orchestrators (Kubernetes), and edgeenabling add-ons (k0s, k3s, KubeEdge or microk8s). But configuring a container orchestration cluster in the cloud takes expert knowledge and effort, and navigating the landscape of edge add-ons takes even more. These are lengthy, manual processes, not suited to a Science Gateway administrator. This abstract proposes a generic solution to the problem of edge connectivity by abstracting away this complexity and automating the process. Users bring their containers and edge devices, and Kubernetes, Ansible and KubeEdge or k3s do the rest, with some help from MiCADO, a dynamic cloud orchestrator. Figure 1 shows this process. An Ansible Playbook creates a Kubernetes cluster in the cloud, deploying an appropriate Kubernetes edge add-on in the process. Then, a user provides a description of their application (container images and configuration) and edge nodes (connection details). MiCADO supports these descriptions in TOSCA, but higher-level tools can be used to generate TOSCA from descriptive metadata. Actioning this description triggers Kubernetes and a second Ansible Playbook, which create the connection to the edge and schedule containers appropriately across edge and cloud. In MiCADO, the TOSCASubmitter is responsible for these triggers. A KubernetesAdaptor transpiles the relevant container descriptions to Kubernetes Manifests and executes them. An AnsibleAdaptor, newly created for connecting edges on the fly,automates configuration and execution of the Playbook. This approach is being used for a Science Gateway in the PITHIA-NRF Project, and for an Industry Gateway in the DIGITbrain Project. It supports a simpler route to edge computing that can be exploited by Science Gateways for efficient computing along the cloud-to-edge continuum. |
---|