TL;DR: A brand-new integration allows Kubernetes to orchestrate Singularity containers for compute-driven workloads. The enabling components are an OCI-compliant runtime in Singularity alongside a beta release Singularity Container Runtime Interface (CRI).
It is now possible to orchestrate Singularity containers via Kubernetes. We start here by sharing a demonstration based upon a use case from Deep Learning. In the remainder of the post, a technical overview of this native integration is provided.
Example: An Object-Classification Service
In our documentation, Sylabs software engineer Sasha Yakovtseva includes an object-classification example that makes use of TensorFlow as the framework for Deep Learning. Although the example is sophisticated enough to result in a client-server application that makes use of GPUs, the configuration specifics are contained within a single YAML file. The configuration can be applied via Kubernetes as follows:
$ kubectl apply -f darkflow.yaml
Ultimately, Sasha explains in the documentation how proper functioning of this service can be verified.
Each of the components that comprise this object-classification service have been containerized via Singularity. Based upon the specifics contained within the YAML file (above), all containers are orchestrated via Kubernetes.
Taken together, Sasha has implemented a representative example of the application of Deep Learning to the task of object classification that makes use of the most widely used framework in TensorFlow. Not restricted to ‘just’ Deep Learning, you’ll find that she’s included other examples here that involve cats, books, and databases. Because reproducibility is a fundamental reason for the popularity of containers, with Kubernetes and the Singularity components described below, you can easily explore these examples for yourself.
Less obvious, at this point, is the fact that all of these examples make use of a recently developed integration between Singularity containers and Kubernetes. It is a technical overview of this integration that concerns us for the remainder of this post.
In developing an initial integration between Singularity and Kubernetes, this Sylabs’ lead project opted for compliance with the Open Containers Initiative (OCI) perspective for image and runtime specification. To achieve a universal integration between Singularity and Kubernetes, it became evident that two major efforts required attention:
- Implementation of an OCI-compliant runtime in Singularity
- Implementation of a Container Runtime Interface (CRI) for Singularity
In late 2016, a CRI was introduced into Kubernetes 1.5. Since this time, a number of CRI’s have been implemented and refined; in fact, Red Hat championed CRI-O has just this week been accepted as an incubation-level hosted project by the Cloud Native Computing Foundation (CNCF). Although Kubernetes has advanced significantly to the version 1.14 release, the CRI remains the same from an architectural perspective – a perspective shared originally through the lower branch of the schematic below.
From this lower branch, it is evident that any OCI-compliant container runtime interfaces with Kubernetes as the client (using a kubelet at the node level) via a server “… over Unix sockets using the gRPC framework …” to quote from the original 2016 blog post. CRI-O, alluded to above, would provide a representative example of an existing implementation.
Compliance with open standards is extremely appealing – especially from a user’s, and ultimately a customer’s, perspective. For example by ensuring implementations conform to a lowest common denominator, in this case for container image and runtime specifications, users can exercise choice when it comes to implementations, and rest assured they are making future-proofed investments.
In the case of Singularity however, and based upon requirements emerging from compute-driven use cases, the Singularity Image Format (SIF) has necessarily developed along a completely independent trajectory from what is today embodied within the OCI image specification. As an extensible format however, OCI compliance is addressable via SIF; in fact, SIF files encapsulate the ‘raw materials’ necessary for instantiating an OCI runtime as of version 3.1 of Singularity. With the addition of a CRI for Singularity containers, the matter of an integration with Kubernetes can be realized in practice. The introduction of this CRI and modifications to support an OCI compliant runtime are considered in further detail below.
As the upper branch of the figure above implies, an OCI-runtime compliant integration between Singularity and Kubernetes can be achieved by:
- Enhancing the Singularity core to support an OCI-compliant runtime – the green rounded rectangle (upper branch)
- Encapsulating the ‘raw materials’ necessary for instantiating an OCI runtime in SIF files – the grey rounded rectangles to the right side if the schematic (upper branch)
As of version 3.1 of Singularity, OCI runtime relevant enhancements to the Singularity core are evident through the introduction of a new command group for Singularity:
The introduction of an OCI compliant runtime for Singularity comprises the most-significant contribution of the version 3.1.0 release. Directly from Singularity, a new command group allows users to execute typical OCI workflows – from creating an isolated container with an OCI bundle, to all phases of runtime execution and eventual dismantling of the environment.
And when it comes to SIF itself:
By encapsulating support for the OCI runtime in the native format for Singularity containers, mounted Singularity Image Format (SIF) files can provide the file system bundles and metadata required to bootstrap, use, and manage an OCI compliant [container].
This new command group has been documented here. Use of
$ sudo singularity oci mount ./busybox_latest.sif /var/tmp/busybox
$ sudo ls -la /var/tmp/busybox
drwx------ 4 root root 4096 Apr 4 14:30 .
drwxrwxrwt 4 root root 4096 Apr 4 14:30 ..
-rw-rw-rw- 1 root root 9879 Apr 4 14:30 config.json
drwx------ 4 root root 4096 Apr 4 14:30 overlay
drwx------ 1 root root 4096 Apr 4 14:30 rootfs
In other words, this content conforms with the filesystem bundle expectations of the OCI-runtime specification. The example here makes use of a container for BusyBox; however, SIF’s ability to encapsulate OCI runtime ‘raw materials’ is of generic applicability – i.e., this applies to any SIF file as of Singularity 3.1.
When it comes to compliance with the specification of the OCI runtime, a quantitative assessment is available; in the case of Singularity:
Because Singularity’s support for a fully compliant OCI runtime is made explicit through successful performance on a suite of validation tests, users can rest assured that they have adopted a standards-compliant container runtime offering.
Notably, changes to the implementation of SIF were not required. This serves as a powerful means for validating its extensibility as a format for representing container runtimes.
Sylabs’ software engineer Cedric Clerget served as the lead developer on this and related efforts – including collaboration with Sasha Yakovtseva on Singularity CRI.
Singularity CRI Beta Release
Singularity CRI has progressed rapidly since we first announced it in November 2018; in fact, Sasha tagged the alpha release about a month ago. As the object-classification example at the outset here seems to indicate, the plugin that enables device support now includes GPUs. Because Singularity CRI is now feature/functionality complete, and garnering interest and feedback from the community, we’re announcing here that it is now tagged beta!
As noted above, and conveyed via the architectural schematic (also above), Singularity CRI is implemented as a gRPC server. Situated between Kubernetes (as proxied on a per-node basis by kubelets) and the OCI-compliant runtime recently added to the Singularity core, Singularity CRI mediates all interactions. The result achieved is a native integration between Singularity and Kubernetes.
As is the case for our implementation of an OCI runtime, compliance of Singularity CRI is a matter of continuous validation:
With [Continuous Integration] in place, the validity of the Singularity CRI implementation will be assessed on an ongoing basis – literally, with each [pull request]. With this almost real time feedback, and as development becomes increasingly DevOps, all stakeholders are kept apprised as to the continuously validated quality of this integration …
… is now passing  of 74 validation tests. If you’re interested in the details, by checking here, you’ll be able to identify the final [three] tests that remain outstanding. Of these, one … will be addressed with our next release of SIF – the Singularity Image Format employed to containerize runtimes for use in Singularity. The final two tests that still require some attention only apply to those making use of SELinux – tests we’ll continue to work on with the assistance of the open source community.
Although numerous options exist, we understand that the availability of a fully baked Kubernetes environment may not be in everyone’s wheelhouse; for this reason, we’re particularly excited about Sykube:
If you’d prefer to bypass the need to build from source, you can jump to the Sykube part of our documentation here. Inspired by Minikube, Sykube allows you to run a Kubernetes (K8s) container cluster locally that includes the Singularity runtime and Singularity CRI; of course, we are distributing ready to use Sykube as a Singularity container via the Sylabs Cloud Container Library. Armed with Sykube, you can test the Singularity CRI with an arbitrary number of nodes, and even deploy a K8s environment that exists emphermerally in a temporary filesystem. Once deployed, this Sykube based environment can be employed to work through the feline use case example alluded to above.
With Sykube, the onramp for ‘local’ development of Kubernetes orchestrated Singularity containers has never been so gradual and inviting!
Next Steps …
Implementation of an OCI-compliant runtime in the Singularity core in tandem with Singularity CRI delivers a native integration between Singularity and Kubernetes – a codependence made clear through Singularity CRI’s requirement of Singularity with OCI runtime support as an installation prerequisite. As demonstrated at the outset, this combination today enables sophisticated client-server use cases in Deep Learning – e.g., for object classification via TensorFlow.
Despite the significance embodied by a native integration between Singularity and Kubernetes, we regard this as more of a milestone representing progress, as opposed to an end in itself. Even though this integration unlocks the potential for even more complex use cases involving streamed workloads, real-time analysis, and data pipelining into compute-focused services, we know the Singularity user, developer, and provider community has unmet requirements in mind … and we couldn’t be more excited!
To reiterate the closing from our initial post on Singularity CRI:
We anticipate many invaluable contributions from the Singularity Community, together with those from the open source communities that surround Kubernetes. Our collective efforts will accelerate progress and ensure that this integration exceeds the requirements and expectations of all involved. We urge you to get involved by joining our Community!