Teleport 16: Advancing Infrastructure Defense in Depth with Device Trust, MFA, and VNET
Jul 25
Register Today
Teleport logo

Teleport Blog - Open Source Serverless Solutions for Kubernetes - Feb 21, 2019

Open Source Serverless Solutions for Kubernetes

by Abraham Ingersoll

The 800-pound Lambda vs Open Source Kubernetes Appliances

In a YC Startup School interview late last year, YC Software Engineer, Kyle Corbitt, quizzed Amazon's CTO, Werner Vogels, about containers and Kubernetes.

Visibly bewildered by 44+ minutes live on stage, Werner wandered into the woods of AWS's container offerings, then suddenly pivoted: "The thing with containers is, it almost brings you back to pre-cloud days." He pressed, "even though containers are a great developer abstraction, customers need to do a lot of work. Nobody cares about containers running underneath, it's just a tax you have to pay."

Ouch! Who poured GPLv3 creamer into his morning coffee? :-)

All Open Source Kubernetes Serverless Roads Lead to Knative

Of course, Werner is talking about his own book here. AWS has half-heartedly committed to Kubernetes as a service not only because it's "Google's Infrastructure For Everyone Else" (GIFEE) but also because they see it as a distraction to the next, more impactful, sticky and easily billable stage, "serverless." So while AWS continues its Mr. Softy strategy of extend, embrace and extinguish with Lambda, Google is doubling down on its Kubernetes masterstroke with the upcoming "Knative."

k8s ninjas becoming
k8s ninjas becoming

Serverless on Kubernetes, From The Beginning

Where does one even start with Serverless on Kubernetes? Those playing along in their own datacenters have invariably come across the CNCF's "Cloud Native" landscape. Besides the "trail map" guiding container-curious pilgrims, there's a logo soup PDF (and corresponding single page SPDY/laptop battery benchmarking tool), that aims to guide devotees down the CNCF's "recommended path through the cloud native landscape." The CNCF now even sports a "Serverless Landscape." If you dial the dropdown nobs and drill into "installable" platforms that are open source, you'll land on a small handful of serious "Serverless on Kubernetes" contenders.

serverless on kubernetes 2019 logos
serverless on kubernetes 2019 logos

Let's skim through each open source serverless offering, starting chronologically based on date of inception.

Apache OpenWhisk, a Versatile and Industry-Strength Serverless Solution

The first open source serverless platform on the map looks sharp as an incubating ASF project, dripping wet in fresh IBM Blue.

OpenWhisk StatDetail
InceptionFeb 2016 (First public commit)
Velocity2,300+ commits, 240+ watchers, 3,700+ stars, 700+ forks, 1,200+ slack members (only 160+ in #kubernetes)
LicenseApache License 2.0
StatusDecent production usage (basis of IBM Cloud Functions, plus an Adobe product)
LOC~64K lines (platform only), written in Scala
Primary authorsRodric Rabbah, Markus Thömmes, James Dubee, Carlos Santana, Christian Bickel, Perry Cheng. Chetan Mehrotra, Tyson Norris, several being from from IBM and Adobe. (Perry and Rodric have since started a pure serverless cloud,
Install processmultiple deploy targets including k8s
Key features"production ready", industrial design
Security/Multi-tenancySerious, boiling down to containers (linux cgroups and namespaces)

OpenWhisk stands out as the first legit serverless/event-based architecture released as open source by a big vendor. It came out of IBM Research teams based on Yorktown, NY, is written in Scala and looks very, very serious. It even has a project wiki -- hosted on Confluence. It's hard to say precisely how much uptake OpenWhisk has in commercial environments, but it seems like IBM will happily sell you "cloud functions" based on OpenWhisk. It's even been donated to the Apache Foundation. And while it can be used on top of Kubernetes, an original author suggests that it's built for "scale to large deployments, not just tire kicking." Quite interesting, especially if you're a java shop and the thought of using containers or kubernetes APIs (namespaces) as a multi-tenancy boundary gives your CISO nightmares.

Fission, The First True Scotsman Kubernetes Serverless Platform

Next off on the open source serverless race was the first contender wholly premised upon Kubernetes, a project called "Fission" from Platform9.

Fission StatDetail
InceptionAugust 2016
Velocity3,900+ stars, 340+ forks, 140+ watchers, 340+ forks, 800+ slack members
LicenseApache License 2.0
StatusVersion 1.0
LOC~25K lines, golang
Primary authorsSoam Vasani (VMWare/Platform 9, SF), now Ta-Ching Chen (VMView, Taiwan). Also Vishal Biyani (InfraCloud, Pune India)
Install processhelm-based w/clean quickstart
Key FeaturesEase of use, commercial support?
Security/Multi-tenancysoft multi-tenancy at namespace level, slightly looser than K8s itself?

Fission appears to mostly have been built by a small handful of engineers, who along with Platform9's co-founders, cut their teeth slinging virtualization tech at VMWare. Of the options, Fission has some strong buzzword game and a decidedly painless quickstart installation process. It boasts sub-100ms "cold start" by pre-warming of dynamic loaders, live reload, record/replay, canary deployments and prometheus metrics integration. If notable VMWare alumni powering an OpenStack+K8s startup is your thing, and you can dedicate an entire cluster to exclusively running Fission workloads, this isn't one to miss.

Kubeless, the Early Pioneer of Using Kubernetes APIs for Serverless

Quick on the heels of Fission came the aptly named "Kubeless," which is notable for being an early visionary along the path toward Kubernetes Custom Resource Definitions. (What's a CRD or CRD+custom controller? A K8s extension mechanism that leverages the base Kubernetes cluster constructs for higher-level features. Or, the thing powering Istio -- besides Envoy.)

InceptionNovember 2016
Velocity960+ commits, 170+ watchers, 4,000+ stars, 400+ forks, 350+ slack members (#kubeless on K8s Slack)
LicenseApache 2.0
Statusruntimes are "stable", but .. maintenance mode? Will someone pick this up?
LOC12K, golang
Primary authorsTuna Ng (departed, now Lead Blockchain Engineer at TomoChain), Andres Martinez Gotor (Bitnami), Sebastien Goasguen (ex-Bitnami, now TriggerMesh)
Install processYAMLs and curls
Key featuresCLI compliant with AWS Lambda CLI, based on core K8s constructs
Security/Multi-TenancyCRD based, so internal auth relies on K8s APIs/namespaces/RBAC. External auth based on HTTP headers.

What's perplexing about Kubeless is that while still maintained and with some semblance of end-user interest, its creators have all absconded to other projects. The original lead is now a "Blockchain Engineer", and another key leader has founded a new serverless-leaning startup called TriggerMesh (which itself boasts being the driving force between an interesting serverless integration within GitLab). It's hard to say where this one is going, but if you're already busy writing your own controllers and swimming in CRDs, Kubeless could be an interesting proposition. What's not to like about 12K lines of golang "meant as a proof of concept of the K8s APIs"?

OpenFaaS, Simple Serverless on Docker Kubernetes

OpenFaaS is utterly fascinating. It's the only contender boasting a license other than Apache 2.0, it's extremely community-centric, added Kubernetes support in mid-2017 after originally targeting Docker Swarm, and is deliciously lean.

OpenFaaS StatDetail
InceptionDecember 2016
Velocity3,850 commits, 450+ watchers, 15K+ stars, 1600+ forks, 1,200+ slack members
StatusLots of logos on the end-user page, including a few notable big names
LOC~5K, golang, plus more spread out in other repos
Primary authorAlex Ellis (10+ years at ADP in the UK, now VMWare), now a long tail of 180+ community contributors
Install processYAML templated with helm
Key featuresSimplicity! An AWS-SNS trigger system, incoming Istio integration, CRD support, a REST API, ARM support (32 and 64-bit), a "Function+Template Store" and an "OpenFaaS Cloud"
Security/Multi-TenancyAll containers are non-root, including an option for a readonly filesystem. Also sports dedicated namespaces, a malleable K8s RBAC role and "on-by-default auth." (+OAuth 2.0 authz with GitLab or GitHub on OpenFaaS Cloud)

Besides being the brainchild of a dedicated single engineer who has firmly established a surrounding community, OpenFaaS is notable for its simplicity, tight codebase and pitch perfect messaging -- "Serverless Functions Made Simple", "one-click install", "auto-scales as demand increases including to zero." Sweet?! Its maintainers added a Kubernetes intro to the docs in July 2017 and if you're still not a K8s shop, these claims may leave a nice halo: "only serverless solution that can integrate natively with both Kubernetes and Docker Swarm", "driven by values", "community-centric", "160+ contributors now vs 40 a year ago." Recently the project's faas-netes operator was even re-built almost entirely by Stefan Prodan of Weaveworks, enabling tighter integration with Kubernetes.

One slight hitch -- don't mistakenly count OpenFaaS out based on its "Docker Captain" origins (like your author did!): Mr. Ellis will gingerly set the record straight. And if you tick the box on the Google Docs survey that gates the OpenFaaS Slack ecosystem, he'll even automatically invite you to a community contributor meeting. Fascinating.

Knative, aka All Your OSS Serverless (And Ingress) Belong to Us

Last but not least we have Knative. Had you gone in reverse chronological order you would have stopped right here. This is the sweet mamba jamba of anti-lambda competitors. If you weren't paying close attention during Google Cloud's NEXT conference in the middle of 2018, this was easy to miss. Get with the times! Knative is coming, and it's dragging Istio along with it.

Knative StatDetail
InceptionJan 2018
Velocity2400+ commits, 183+ watchers, 2100+ stars, 490+ forks, 1,200+ Slack members (notably NOT #knative on K8s Slack -- different ecosystem)
LicenseApache 2.0
Statusv0.4.0 (alpha, beta?)
LOCserving: ~87K, eventing ~25K
Primary authorsTons of Googlers, "~300 contributors from ~48 diff companies."
Install processA "yeasty" set of Isitio CRDs, then just clean yamls
Key featuresIstio, finally a use case! Woot! Seriously -- automatic scale up/down, transparent build, user-space telemetry, revisioning, traffic splitting, etc..
Security/Multi-TenancyInherits Istio? Reasonable to expect they get this right
NotableGoogle! Solicited T-Mobile to build a store locator prototype on Knative pre-launch. Talk here:

Word on the street is that Google has more than 90 engineers dedicated to building Knative, and that it's the basis of deep upcoming "cloud function" integration between GCP and GKE. If you're brave, you can even request to participate in a beta program where getting knative is as simple as clicking a button in the Google Cloud console.

The quotes from those close to the project speak for themselves. Oren Teich, "All of them are re-platforming onto Knative" And a long-time Googler in the Knative announcement HN thread: "These are early days of course, but given that the goal is to codify the commonalities (the 80% we all do roughly the same anyway) and to improve customer workload portability overall, I hope to see new products built using Knative, and existing products re-base on Knative as well." In the same thread, a Senior Software Engineer from Pivotal noted "I think of FaaS as a PaaS with some extra features (scale-to-zero being the most-noticeable)."

Teleport cybersecurity blog posts and tech news

Every other week we'll send a newsletter with the latest cybersecurity news and Teleport updates.

Other Open Source Serverless Platforms

Astute observers of the #serverless space will note that there are many others -- Fn from Oracle, Riff via Pivotal, Dispatch from VMWare, Galactic Fog, Nuclio, Virtual Kubelet (seriously, considered serverless?), PipelineAI, Nuclio, and probably more. Sorry, TLDR, and at a glance it appears that most of them are quickly getting out of the way of Knative.

What about hosted serverless? Google Cloud Functions, Huawei FunctionStage, Cloudflare Workers, Azure Functions, Serverless(.com) Sure, go feed that meter while you build real business value. I'm getting back to my YAML.


Teleport Newsletter

Stay up-to-date with the newest Teleport releases by subscribing to our monthly updates.


Subscribe to our newsletter

PAM / Teleport