An open source system for automating deployment, scaling, and operations of applications.

Tuesday, May 24, 2016

Hypernetes: Bringing Security and Multi-tenancy to Kubernetes

Today’s guest post is written by Harry Zhang and Pengfei Ni, engineers at HyperHQ, describing a new hypervisor based container called HyperContainer

While many developers and security professionals are comfortable with Linux containers as an effective boundary, many users need a stronger degree of isolation, particularly for those running in a multi-tenant environment. Sadly, today, those users are forced to run their containers inside virtual machines, even one VM per container.

Unfortunately, this results in the loss of many of the benefits of a cloud-native deployment: slow startup time of VMs; a memory tax for every container; low utilization resulting in wasting resources.

In this post, we will introduce HyperContainer, a hypervisor based container and see how it naturally fits into the Kubernetes design, and enables users to serve their customers directly with virtualized containers, instead of wrapping them inside of full blown VMs.

HyperContainer

HyperContainer is a hypervisor-based container, which allows you to launch Docker images with standard hypervisors (KVM, Xen, etc.). As an open-source project, HyperContainer consists of an OCI compatible runtime implementation, named runV, and a management daemon named hyperd. The idea behind HyperContainer is quite straightforward: to combine the best of both virtualization and container.

We can consider containers as two parts (as Kubernetes does). The first part is the container runtime, where HyperContainer uses virtualization to achieve execution isolation and resource limitation instead of namespaces and cgroups. The second part is the application data, where HyperContainer leverages Docker images. So in HyperContainer, virtualization technology makes it possible to build a fully isolated sandbox with an independent guest kernel (so things like `top` and /proc all work), but from developer’s view, it’s portable and behaves like a standard container.

HyperContainer as Pod

The interesting part of HyperContainer is not only that it is secure enough for multi-tenant environments (such as a public cloud), but also how well it fits into the Kubernetes philosophy.

One of the most important concepts in Kubernetes is Pods. The design of Pods is a lesson learned (Borg paper section 8.1) from real world workloads, where in many cases people want an atomic scheduling unit composed of multiple containers (please check this example for further information). In the context of Linux containers, a Pod wraps and encapsulates several containers into a logical group. But in HyperContainer, the hypervisor serves as a natural boundary, and Pods are introduced as first-class objects:


HyperContainer wraps a Pod of light-weight application containers and exposes the container interface at Pod level. Inside the Pod, a minimalist Linux kernel called HyperKernel is booted. This HyperKernel is built with a tiny Init service called HyperStart. It will act as the PID 1 process and creates the Pod, setup Mount namespace, and launch apps from the loaded images. 

This model works nicely with Kubernetes. The integration of HyperContainer with Kubernetes, as we indicated in the title, is what makes up the Hypernetes project. 

Hypernetes

One of the best parts of Kubernetes is that it is designed to support multiple container runtimes, meaning users are not locked-in to a single vendor. We are very pleased to announce that we have already begun working with the Kubernetes team to integrate HyperContainer into Kubernetes upstream. This integration involves:
  1. container runtime optimizing and refactoring
  2. new client-server mode runtime interface
  3. containerd integration to support runV
The OCI standard and kubelet’s multiple runtime architecture make this integration much easier even though HyperContainer is not based on Linux container technology stack.

On the other hand, in order to run HyperContainers in multi-tenant environment, we also created a new network plugin and modified an existing volume plugin. Since Hypernetes runs Pod as their own VMs, it can make use of your existing IaaS layer technologies for multi-tenant network and persistent volumes. The current Hypernetes implementation uses standard Openstack components.

Below we go into further details about how all those above are implemented.

Identity and Authentication

In Hypernetes we chose Keystone to manage different tenants and perform identification and authentication for tenants during any administrative operation. Since Keystone comes from the OpenStack ecosystem, it works seamlessly with the network and storage plugins we used in Hypernetes.

Multi-tenant Network Model

For a multi-tenant container cluster, each tenant needs to have strong network isolation from each other tenant. In Hypernetes, each tenant has its own Network. Instead of configuring a new network using OpenStack, which is complex, with Hypernetes, you just create a Network object like below.
apiVersion: v1
kind: Network
metadata:
 name: net1
spec:
 tenantID: 065f210a2ca9442aad898ab129426350
 subnets:
   subnet1:
     cidr: 192.168.0.0/24
     gateway: 192.168.0.1

Note that the tenantID is supplied by Keystone. This yaml will automatically create a new Neutron network with a default router and a subnet 192.168.0.0/24. 

A Network controller will be responsible for the life-cycle management of any Network instance created by the user. This Network can be assigned to one or more Namespaces, and any Pods belonging to the same Network can reach each other directly through IP address.

apiVersion: v1
kind: Namespace
metadata:
 name: ns1
spec:
 network: net1

If a Namespace does not have a Network spec, it will use the default Kubernetes network model instead, including the default kube-proxy. So if a user creates a Pod in a Namespace with an associated Network, Hypernetes will follow the Kubernetes Network Plugin Model to set up a Neutron network for this Pod. Here is a high level example:

A Hypernetes Network Workflow.png


Hypernetes uses a standalone gRPC handler named kubestack to translate the Kubernetes Pod request into the Neutron network API. Moreover, kubestack is also responsible for handling another important networking feature: a multi-tenant Service proxy.

In a multi-tenant environment, the default iptables-based kube-proxy can not reach the individual Pods, because they are isolated into different networks. Instead, Hypernetes uses a built-in HAproxy in every HyperContainer as the portal. This HAproxy will proxy all the Service instances in the namespace of that Pod. Kube-proxy will be responsible for updating these backend servers by following the standard OnServiceUpdate and OnEndpointsUpdate processes, so that users will not notice any difference. A downside of this method is that HAproxy has to listen to some specific ports which may conflicts with user’s containers.That’s why we are planning to use LVS to replace this proxy in the next release.

With the help of the Neutron based network plugin, the Hypernetes Service is able to provide an OpenStack load balancer, just like how the “external” load balancer does on GCE. When user creates a Service with external IPs, an OpenStack load balancer will be created and endpoints will be automatically updated through the kubestack workflow above.

Persistent Storage

When considering storage, we are actually building a tenant-aware persistent volume in Kubernetes. The reason we decided not to use existing Cinder volume plugin of Kubernetes is that its model does not work in the virtualization case. Specifically:

The Cinder volume plugin requires OpenStack as the Kubernetes provider.
The OpenStack provider will find on which VM the target Pod is running on
Cinder volume plugin will mount a Cinder volume to a path inside the host VM of Kubernetes.
The kubelet will bind mount this path as a volume into containers of target Pod.

But in Hypernetes, things become much simpler. Thanks to the physical boundary of Pods, HyperContainer can mount Cinder volumes directly as block devices into Pods, just like a normal VM. This mechanism eliminates extra time to query Nova to find out the VM of target Pod in the existing Cinder volume workflow listed above.

The current implementation of the Cinder plugin in Hypernetes is based on Ceph RBD backend, and it works the same as all other Kubernetes volume plugins, one just needs to remember to create the Cinder volume (referenced by volumeID below) beforehand.

apiVersion: v1
kind: Pod
metadata:
 name: nginx
 labels:
   app: nginx
spec:
 containers:
 - name: nginx
   image: nginx
   ports:
   - containerPort: 80
   volumeMounts:
   - name: nginx-persistent-storage
     mountPath: /var/lib/nginx
 volumes:
 - name: nginx-persistent-storage
   cinder:
     volumeID: 651b2a7b-683e-47e1-bdd6-e3c62e8f91c0
     fsType: ext4


So when the user provides a Pod yaml with a Cinder volume, Hypernetes will check if kubelet is using the Hyper container runtime. If so, the Cinder volume can be mounted directly to the Pod without any extra path mapping. Then the volume metadata will be passed to the Kubelet RunPod process as part of HyperContainer spec. Done!

Thanks to the plugin model of Kubernetes network and volume, we can easily build our own solutions above for HyperContainer though it is essentially different from the traditional Linux container. We also plan to propose these solutions to Kubernetes upstream by following the CNI model and volume plugin standard after the runtime integration is completed.
We believe all of these open source projects are important components of the container ecosystem, and their growth depends greatly on the open source spirit and technical vision of the Kubernetes team. 

Conclusion

This post introduces some of the technical details about HyperContainer and the Hypernetes project. We hope that people will be interested in this new category of secure container and its integration with Kubernetes. If you are looking to try out Hypernetes and HyperContainer, we have just announced the public beta of our new secure container cloud service (Hyper_), which is built on these technologies. But even if you are running on-premise, we believe that Hypernetes and HyperContainer will let you run Kubernetes in a more secure way.


~Harry Zhang and Pengfei Ni, engineers at HyperHQ

Tuesday, May 3, 2016

CoreOS Fest 2016: CoreOS and Kubernetes Community meet in Berlin (& San Francisco)

CoreOS Fest 2016 will bring together the container and open source distributed systems community, including many thought leaders in the Kubernetes space. It is the second annual CoreOS community conference, held for the first time in Berlin on May 9th and 10th. CoreOS believes Kubernetes is the container orchestration component to deliver GIFEE (Google’s Infrastructure for Everyone Else).

At this year’s CoreOS Fest, there are tracks dedicated to Kubernetes where you’ll hear about various topics ranging from Kubernetes performance and scalability, continuous delivery and Kubernetes, rktnetes, stackanetes and more. In addition, there will be a variety of talks, from introductory workshops to deep-dives into all things containers and related software.

Don’t miss these great speaker sessions at the conference in Berlin:


If you can’t make it to Berlin, Kubernetes is also a focal point at the CoreOS Fest San Francisco satellite event, a one day event dedicated to CoreOS and Kubernetes. In fact, Tim Hockin, senior staff engineer at Google and one of the creators of Kubernetes, will be kicking off the day with a keynote dedicated to Kubernetes updates.

San Francisco sessions dedicated to Kubernetes include:

  • Tim Hockin’s keynote address, Senior Staff Engineer at Google
  • When rkt meets Kubernetes: a troubleshooting tale by Loris Degioanni, CEO of Sysdig
  • rktnetes: what's new with container runtimes and Kubernetes by Derek Gonyeo, Software Engineer at CoreOS
  • Magical Security Sprinkles: Secure, Resilient Microservices on CoreOS and Kubernetes by Oliver Gould, CTO of Buoyant

Kubernetes Workshop in SF: Getting Started with Kubernetes, hosted at Google San Francisco office (345 Spear St - 7th floor) by Google Developer Program Engineers Carter Morgan and Bill Prin on Tuesday May 10th from 9:00am to 1:00pm, lunch will be served afterwards. Limited seats, please RSVP for free here.

Get your tickets:

Learn more at: coreos.com/fest/ and on Twitter @CoreOSFest #CoreOSFest


-- Sarah Novotny, Kubernetes Community Manager

Friday, April 22, 2016

Introducing the Kubernetes OpenStack Special Interest Group

Editor’s note: This week we’re featuring Kubernetes Special Interest Groups; Today’s post is by the SIG-OpenStack team about their mission to facilitate ideas between the OpenStack and Kubernetes communities. 

The community around the Kubernetes project includes a number of Special Interest Groups (SIGs) for the purposes of facilitating focused discussions relating to important subtopics between interested contributors. Today we would like to highlight the Kubernetes OpenStack SIG focused on the interaction between Kubernetes and OpenStack, the Open Source cloud computing platform.

There are two high level scenarios that are being discussed in the SIG:

  • Using Kubernetes to manage containerized workloads running on top of OpenStack
  • Using Kubernetes to manage containerized OpenStack services themselves

In both cases the intent is to help facilitate the inter-pollination of ideas between the growing Kubernetes and OpenStack communities. The OpenStack community itself includes a number of projects broadly aimed at assisting with both of these use cases including:

  • Kolla - Provides OpenStack service containers and deployment tooling for operating OpenStack clouds.
  • Kuryr - Provides bridges between container networking/storage framework models and OpenStack infrastructure services.
  • Magnum - Provides containers as a service for OpenStack.
  • Murano - Provides an Application Catalog service for OpenStack including support for Kubernetes itself, and for containerized applications, managed by Kubernetes.

There are also a number of example templates available to assist with using the OpenStack Orchestration service (Heat) to deploy and configure either Kubernetes itself or offerings built around Kubernetes such as OpenShift. While each of these approaches has their own pros and cons the common theme is the ability, or potential ability, to use Kubernetes and where available leverage deeper integration between it and the OpenStack services themselves. 

Current SIG participants represent a broad array of organizations including but not limited to: CoreOS, eBay, GoDaddy, Google, IBM, Intel, Mirantis, OpenStack Foundation, Rackspace, Red Hat, Romana, Solinea, VMware. 

The SIG is currently working on collating information about these approaches to help Kubernetes users navigate the OpenStack ecosystem along with feedback on which approaches to the requirements presented work best for operators. 

Kubernetes at OpenStack Summit Austin

The OpenStack Summit is in Austin from April 25th to 29th and is packed with sessions related to containers and container management using Kubernetes. If you plan on joining us in Austin you can review the schedule online where you will find a number of sessions, both in the form of presentations and hands on workshops, relating to Kubernetes and containerization at large. Folks from the Kubernetes OpenStack SIG are particularly keen to get the thoughts of operators in the “Ops: Containers on OpenStack” and “Ops: OpenStack in Containers” working sessions.

Kubernetes community experts will also be on hand in the Container Expert Lounge to answer your burning questions. You can find the lounge on the 4th floor of the Austin Convention Center.

Follow @kubernetesio and #OpenStackSummit to keep up with the latest updates on Kubernetes at OpenStack Summit throughout the week.

Connect With Us

If you’re interested in Kubernetes and OpenStack, there are several ways to participate:


-- Steve Gordon, Principal Product Manager at Red Hat, and Ihor Dvoretskyi, OpenStack Operations Engineer at Mirantis

Wednesday, April 20, 2016

SIG-UI: the place for building awesome user interfaces for Kubernetes

Editor’s note: This week we’re featuring Kubernetes Special Interest Groups; Today’s post is by the SIG-UI team describing their mission and showing the cool projects they work on.

Kubernetes has been handling production workloads for a long time now (see case studies). It runs on public, private and hybrid clouds as well as bare metal. It can handle all types of workloads (web serving, batch and mixed) and enable zero-downtime rolling updates. It abstracts service discovery, load balancing and storage so that applications running on Kubernetes aren’t restricted to a specific cloud provider or environment.

The abundance of features that Kubernetes offers is fantastic, but implementing a user-friendly, easy-to-use user interface is quite challenging. How shall all the features be presented to users? How can we gradually expose the Kubernetes concepts to newcomers, while empowering experts? There are lots of other challenges like these that we’d like to solve. This is why we created a special interest group for Kubernetes user interfaces.

Meet SIG-UI: the place for building awesome user interfaces for Kubernetes
The SIG UI mission is simple: we want to radically improve the user experience of all Kubernetes graphical user interfaces. Our goal is to craft UIs that are used by devs, ops and resource managers across their various environments, that are simultaneously intuitive enough for newcomers to Kubernetes to understand and use.

SIG UI members have been independently working on a variety of UIs for Kubernetes. So far, the projects we’ve seen have been either custom internal tools coupled to their company workflows, or specialized API frontends. We have realized that there is a need for a universal UI that can be used standalone or be a standard base for custom vendors. That’s how we started the Dashboard UI project. Version 1.0 has been recently released and is included with Kubernetes as a cluster addon. The Dashboard project was recently featured in a talk at KubeCon EU, and we have ambitious plans for the future!
Dashboard UI v1.0 home screen showing applications running in a Kubernetes cluster.

Since the initial release of the Dashboard UI we have been thinking hard about what to do next and what users of UIs for Kubernetes think about our plans. We’ve had many internal discussions on this topic, but most importantly, reached out directly to our users. We created a questionnaire asking a few demographic questions as well as questions for prioritizing use cases. We received more than 200 responses from a wide spectrum of user types, which in turn helped to shape the Dashboard UI’s current roadmap. Our members from LiveWyer summarised the results in a nice infographic

Connect with us


We believe that collaboration is the key to SIG UI success, so we invite everyone to connect with us. Whether you’re a Kubernetes user who wants to provide feedback, develop your own UIs, or simply want to collaborate on the Dashboard UI project, feel free to get in touch. There are many ways you can contact us:

-- Piotr Bryk, Software Engineer, Google




Tuesday, April 19, 2016

SIG-ClusterOps: Promote operability and interoperability of Kubernetes clusters

Editor’s note: This week we’re featuring Kubernetes Special Interest Groups; Today’s post is by the SIG-ClusterOps team whose mission is to promote operability and interoperability of Kubernetes clusters -- to listen, help & escalate.

We think Kubernetes is an awesome way to run applications at scale! Unfortunately, there's a bootstrapping problem: we need good ways to build secure & reliable scale environments around Kubernetes. While some parts of the platform administration leverage the platform (cool!), there are fundamental operational topics that need to be addressed and questions (like upgrade and conformance) that need to be answered.

Enter Cluster Ops SIG – the community members who work under the platform to keep it running.

Our objective for Cluster Ops is to be a person-to-person community first, and a source of opinions, documentation, tests and scripts second. That means we dedicate significant time and attention to simply comparing notes about what is working and discussing real operations. Those interactions give us data to form opinions. It also means we can use real-world experiences to inform the project.

We aim to become the forum for operational review and feedback about the project. For Kubernetes to succeed, operators need to have a significant voice in the project by weekly participation and collecting survey data. We're not trying to create a single opinion about ops, but we do want to create a coordinated resource for collecting operational feedback for the project. As a single recognized group, operators are more accessible and have a bigger impact.

What about real world deliverables?

We've got plans for tangible results too. We’re already driving toward concrete deliverables like reference architectures, tool catalogs, community deployment notes and conformance testing. Cluster Ops wants to become the clearing house for operational resources. We're going to do it based on real world experience and battle tested deployments.

Connect with us.

Cluster Ops can be hard work – don't do it alone. We're here to listen, to help when we can and escalate when we can't. Join the conversation at:


The Cluster Ops Special Interest Group meets weekly at 13:00PT on Thursdays, you can join us via the video hangout and see latest meeting notes for agendas and topics covered.

--Rob Hirschfeld, CEO, RackN 


Monday, April 18, 2016

SIG-Networking: Kubernetes Network Policy APIs Coming in 1.3

Editor’s note: This week we’re featuring Kubernetes Special Interest Groups; Today’s post is by the Network-SIG team describing network policy APIs coming in 1.3 - policies for security, isolation and multi-tenancy.

The Kubernetes network SIG has been meeting regularly since late last year to work on bringing network policy to Kubernetes and we’re starting to see the results of this effort.

One problem many users have is that the open access network policy of Kubernetes is not suitable for applications that need more precise control over the traffic that accesses a pod or service. Today, this could be a multi-tier application where traffic is only allowed from a tier’s neighbor. But as new Cloud Native applications are built by composing microservices, the ability to control traffic as it flows among these services becomes even more critical.

In most IaaS environments (both public and private) this kind of control is provided by allowing VMs to join a ‘security group’ where traffic to members of the group is defined by a network policy or Access Control List (ACL) and enforced by a network packet filter.

The Network SIG started the effort by identifying specific use case scenarios that require basic network isolation for enhanced security. Getting the API right for these simple and common use cases is important because they are also the basis for the more sophisticated network policies necessary for multi-tenancy within Kubernetes.

From these scenarios several possible approaches were considered and a minimal policy specification was defined. The basic idea is that if isolation were enabled on a per namespace basis, then specific pods would be selected where specific traffic types would be allowed.

The simplest way to quickly support this experimental API is in the form of a ThirdPartyResource extension to the API Server, which is possible today in Kubernetes 1.2.

If you’re not familiar with how this works, the Kubernetes API can be extended by defining ThirdPartyResources that create a new API endpoint at a specified URL.

third-party-res-def.yaml

kind: ThirdPartyResource
apiVersion: extensions/v1beta1
metadata:
 name: network-policy.net.alpha.kubernetes.io
description: "Network policy specification"
versions:
- name: v1alpha1


$kubectl create -f third-party-res-def.yaml


This will create an API endpoint (one for each namespace):


/net.alpha.kubernetes.io/v1alpha1/namespace/default/networkpolicys/


Third party network controllers can now listen on these endpoints and react as necessary when resources are created, modified or deleted. Note: With the upcoming release of Kubernetes 1.3 - when the Network Policy API is released in beta form - there will be no need to create a ThirdPartyResource API endpoint as shown above. 

Network isolation is off by default so that all pods can communicate as they normally do. However, it’s important to know that once network isolation is enabled, all traffic to all pods, in all namespaces is blocked, which means that enabling isolation is going to change the behavior of your pods

Network isolation is enabled by defining the network-isolation annotation on namespaces as shown below:


net.alpha.kubernetes.io/network-isolation: [ on | off ]

Once network isolation is enabled, explicit network policies must be applied to enable pod communication.

A policy specification can be applied to a namespace to define the details of the policy as shown below:


POST /apis/net.alpha.kubernetes.io/v1alpha1/namespaces/tenant-a/networkpolicys/

{
"kind": "NetworkPolicy",
"metadata": {
"name": "pol1"
},
"spec": {
"allowIncoming": {
"from": [
{ "pods": { "segment": "frontend" } }
],
"toPorts": [
{ "port": 80, "protocol": "TCP" }
]
},
"podSelector": { "segment": "backend" }
}
}

In this example, the ‘tenant-a’ namespace would get policy ‘pol1’ applied as indicated. Specifically, pods with the segment label ‘backend’ would allow TCP traffic on port 80 from pods with the segment label ‘frontend’ to be received.

Today, Romana, OpenShift, OpenContrail and Calico support network policies applied to namespaces and pods. Cisco and VMware are working on implementations as well. Both Romana and Calico demonstrated these capabilities with Kubernetes 1.2 recently at KubeCon. You can watch their presentations here: Romana (slides), Calico (slides). 

How does it work?

Each solution has their their own specific implementation details. Today, they rely on some kind of on-host enforcement mechanism, but future implementations could also be built that apply policy on a hypervisor, or even directly by the network itself. 

External policy control software (specifics vary across implementations) will watch the new API endpoint for pods being created and/or new policies being applied. When an event occurs that requires policy configuration, the listener will recognize the change and a controller will respond by configuring the interface and applying the policy.  The diagram below shows an API listener and policy controller responding to updates by applying a network policy locally via a host agent. The network interface on the pods is configured by a CNI plugin on the host (not shown).

controller.jpg


If you’ve been holding back on developing applications with Kubernetes because of network isolation and/or security concerns, these new network policies go a long way to providing the control you need. No need to wait until Kubernetes 1.3 since network policy is available now as an experimental API enabled as a ThirdPartyResource.

If you’re interested in Kubernetes and networking, there are several ways to participate - join us at:

The Networking “Special Interest Group,” which meets bi-weekly at 3pm (15h00) Pacific Time at SIG-Networking hangout

--Chris Marino, Co-Founder, Pani Networks

Friday, April 15, 2016

How to deploy secure, auditable, and reproducible Kubernetes clusters on AWS

Today’s guest post is written by Colin Hom, infrastructure engineer at CoreOS, the company delivering Google’s Infrastructure for Everyone Else (#GIFEE) and running the world's containers securely on CoreOS Linux, Tectonic and Quay.

Join us at CoreOS Fest Berlin, the Open Source Distributed Systems Conference, and learn more about CoreOS and Kubernetes. 

At CoreOS, we're all about deploying Kubernetes in production at scale. Today we are excited to share a tool that makes deploying Kubernetes on Amazon Web Services (AWS) a breeze. Kube-aws is a tool for deploying auditable and reproducible Kubernetes clusters to AWS, currently used by CoreOS to spin up production clusters.

Today you might be putting the Kubernetes components together in a more manual way. With this helpful tool, Kubernetes is delivered in a streamlined package to save time, minimize interdependencies and quickly create production-ready deployments.

A simple templating system is leveraged to generate cluster configuration as a set of declarative configuration templates that can be version controlled, audited and re-deployed. Since the entirety of the provisioning is by AWS CloudFormation and cloud-init, there’s no need for external configuration management tools on your end. Batteries included!

To skip the talk and go straight to the project, check out the latest release of kube-aws, which supports Kubernetes 1.2.x. To get your cluster running, check out the documentation.

Why kube-aws? Security, auditability and reproducibility

Kube-aws is designed with three central goals in mind.

Secure: TLS assets are encrypted via the AWS Key Management Service (KMS) before being embedded in the CloudFormation JSON. By managing IAM policy for the KMS key independently, an operator can decouple operational access to the CloudFormation stack from access to the TLS secrets.

Auditable: kube-aws is built around the concept of cluster assets. These configuration and credential assets represent the complete description of the cluster. Since KMS is used to encrypt TLS assets, you can feel free to check your unencrypted stack JSON into version control as well! 

Reproducible: The --export option packs your parameterized cluster definition into a single JSON file which defines a CloudFormation stack. This file can be version controlled and submitted directly to the CloudFormation API via existing deployment tooling, if desired. 

How to get started with kube-aws

On top of this foundation, kube-aws implements features that make Kubernetes deployments on AWS easier to manage and more flexible. Here are some examples.

Route53 Integration: Kube-aws can manage your cluster DNS records as part of the provisioning process.

cluster.yaml

externalDNSName: my-cluster.kubernetes.coreos.com
createRecordSet: true
hostedZone: kubernetes.coreos.com
recordSetTTL: 300

Existing VPC Support: Deploy your cluster to an existing VPC.

cluster.yaml

vpcId: vpc-xxxxx
routeTableId: rtb-xxxxx

Validation: Kube-aws supports validation of cloud-init and CloudFormation definitions, along with any external resources that the cluster stack will integrate with. For example, here’s a cloud-config with a misspelled parameter:

userdata/cloud-config-worker

#cloud-config
coreos:
 flannel:
   interrface: $private_ipv4
   etcd_endpoints: {{ .ETCDEndpoints }}

$ kube-aws validate

> Validating UserData...
   Error: cloud-config validation errors:
   UserDataWorker: line 4: warning: unrecognized key "interrface"

To get started, check out the kube-aws documentation.

Future Work

As always, the goal with kube-aws is to make deployments that are production ready. While we use kube-aws in production on AWS today, this project is pre-1.0 and there are a number of areas in which kube-aws needs to evolve.

Fault tolerance: At CoreOS we believe Kubernetes on AWS is a potent platform for fault-tolerant and self-healing deployments. In the upcoming weeks, kube-aws will be rising to a new challenge: surviving the Chaos Monkey – control plane and all!

Zero-downtime updates: Updating CoreOS nodes and Kubernetes components can be done without downtime and without interdependency with the correct instance replacement strategy.

A github issue tracks the work towards this goal. We look forward to seeing you get involved with the project by filing issues or contributing directly.

Learn more about Kubernetes and meet the community at CoreOS Fest Berlin - May 9-10, 2016

– Colin Hom, infrastructure engineer, CoreOS