kploy

kploy, the opinionated Kubernetes deployment system for appops

v0.10
GitHub

Why bother?

Are you an appops? Do you want to run microservices-style Kubernetes applications in a simple and convenient way? Is the kubectl interface too low-level and feels sorta cumbersome to use? If you found yourself answering one of the above questions with a Yes, then read on.

The basic idea of kploy is to deploy and manage an app based on a collection of manifest files, such as Kubernetes service or replication controller manifests. With kploy you simply set up your app deployment descriptor and use convention over configuration in order to manage your app across the entire lifecycle.

For a quick overview, check out the walkthrough example and have a look at the following screen cast:

Install

You'll need Python 2.6 or above and then you can install kploy simply like so:

git clone https://github.com/kubernauts/kploy.git && cd kploy

kploy assumes that you have access to a Kubernetes cluster. It does not help you in setting one up.

Note that if you're experiencing authentication issues you can use kubectl proxy --port=8080 & along with apiserver: http://localhost:8080 in the Kployfile.

Prepare your deployment

Use the init command to create the required scaffolding:

$ ./kploy init
================================================================================

OK, I've set up the Kployfile deployment file and created necessary directories.
Now edit the deployment file and copy manifests into the respective directories.

$ ls -al
drwxr-xr-x  10 mhausenblas  staff   748B 13 Dec 17:41 .
drwxr-xr-x  23 mhausenblas  staff   816B  5 Dec 07:44 ..
-rw-r--r--   1 mhausenblas  staff    85B 13 Dec 17:41 Kployfile
drwxr-xr-x   2 mhausenblas  staff    68B 13 Dec 17:41 rcs
drwxr-xr-x   2 mhausenblas  staff    68B 13 Dec 17:41 services

$ cat Kployfile
apiserver: http://localhost:8080
author: CHANGE_ME
cache_remotes: False
name: CHANGE_ME
namespace: default
source: CHANGE_ME

In the above Kployfile, change at least the apiserver field to point it to the API Server of your Kubernetes cluster.

Alternatively, you can create an app from an existing archive. By convention the ZIP archive must be called app.kploy in order to be recognized when you execute the init command:

$ ls -al
drwxr-xr-x  8 mhausenblas  staff   714B 22 Dec 13:28 .
drwxr-xr-x  6 mhausenblas  staff   238B 21 Dec 13:58 ..
-rw-r--r--  1 mhausenblas  staff   1.0K 22 Dec 12:10 app.kploy

$ ./kploy init -v
2015-12-22T01:40:49 Detected archive /Users/mhausenblas/Documents/repos/kubernauts/kploy/app.kploy
================================================================================

OK, I've set up the deployment from archive.
You can now validate it with `kploy dryrun`

$ ls -al
drwxr-xr-x  8 mhausenblas  staff   714B 22 Dec 13:40 .
drwxr-xr-x  6 mhausenblas  staff   238B 21 Dec 13:58 ..
-rw-r--r--  1 mhausenblas  staff   174B 22 Dec 13:40 Kployfile
-rw-r--r--  1 mhausenblas  staff   1.0K 22 Dec 12:10 app.kploy
drwxr-xr-x  2 mhausenblas  staff   102B 22 Dec 13:40 rcs
drwxr-xr-x  2 mhausenblas  staff   102B 22 Dec 13:40 services

Deploy your app

To validate your deployment use the dryrun command:

$ ./kploy dryrun
Validating application `myns/simple_app` ...

  CHECK: Is the Kubernetes cluster up & running and accessible via `http://52.10.201.177/service/kubernetes`?
  \o/ ... I found 1 node(s) to deploy your wonderful app onto.

  CHECK: Are there RC and service manifests available around here?
         I found 1 RC manifest(s) in /Users/mhausenblas/Documents/repos/mhausenblas/kploy/rcs
         I found 1 service manifest(s) in /Users/mhausenblas/Documents/repos/mhausenblas/kploy/services
  \o/ ... I found both RC and service manifests to deploy your wonderful app!
================================================================================

OK, we're looking good! You're ready to deploy your app with `kploy run` now :)

Looks fine, so to actually deploy your app, do:

$ ./kploy run
2015-12-14T10:34:45 From /Users/mhausenblas/Documents/repos/mhausenblas/kploy/services/webserver-svc.yaml I created the service "webserver-svc" at /api/v1/namespaces/default/services/webserver-svc
2015-12-14T10:34:46 From /Users/mhausenblas/Documents/repos/mhausenblas/kploy/rcs/nginx-webserver-rc.yaml I created the RC "webserver-rc" at /api/v1/namespaces/default/replicationcontrollers/webserver-rc
================================================================================

OK, I've deployed `simple_app`.
Use `kploy list` to check how it's doing.

There you go, you just deployed an app on Kubernetes, with a single command. Well done!

You can also use URLs rather than manifests themselves: kploy will download these so called remotes and via cache_remotes in the Kployfile you can control when this should take place. The `cache_remotes` flag only applies to the `run` command, that is, all other commands such as `dryrun` or `list` will always work with the cached manifests.

One use case for remotes is to benefit from charts (of the helm project), like so:

$ echo https://raw.githubusercontent.com/helm/charts/master/elasticsearch/manifests/elasticsearch-rc.yaml > rcs/elasticsearch-rc.yaml.url

Note that when kploy run is executed, the resources are marked with the additional label guard=pyk. This label allows us to identify resources (RCs and services) that are managed by kploy. Do not manually remove them.

If you need to pass confidential data such as passwords or API keys to one of your services, just let kploy handle it automagically via its env mechanism: read more about it in the Secrets section of the env doc.

Manage your app

To see how your app is doing, use the list command. All services and RCs of the app will be listed, along with their status (online means it’s deployed and running) and their resource URL:

$ ./kploy list
Resources of app `myns/simple_app`:
NAME           MANIFEST                     TYPE     STATUS    URL
webserver-svc  services/webserver-svc.yaml  service  online   http://ma.dcos.ca1.mesosphere.com/service/kubernetes/api/v1/namespaces/default/services/webserver-svc
webserver-rc   rcs/nginx-webserver-rc.yaml  RC       online   http://ma.dcos.ca1.mesosphere.com/service/kubernetes/api/v1/namespaces/default/replicationcontrollers/webserver-rc

Hint: if you want to learn about any of the supported commands, simply add an explain before the command, for example:

$ ./kploy explain list
list:
    Lists apps and their status.

If you want to learn more about how your app uses the cluster, use the stats command:

$ ./kploy stats
Runtime stats for app `myns/simple_app`:

[Your app's pods]

NAME                HOST             STATUS    URL
webserver-rc-29pxj  167.114.218.157  Running   http://ma.dcos.ca1.mesosphere.com/service/kubernetes/api/v1/namespaces/default/pods/webserver-rc-29pxj

================================================================================
[Nodes used by your app]

IP               HOST OS                CONTAINER RUNTIME    CAPACITY (PODS, CPU, MEM)    URL
167.114.218.157  CentOS Linux 7 (Core)  docker://1.8.2       40, 40, 256708Mi             http://ma.dcos.ca1.mesosphere.com/service/kubernetes/api/v1/nodes/167.114.218.157

================================================================================

Scale

To scale one of the RCs of your app, use the scale command as described in the Scale section of the dev & ops doc.

Debugging

Should you need to do some troubleshooting on the Pod-level, use the debug command as described in the Debugging section of the dev & ops doc.

Tear down your app

To tear down your app, use the destroy command, for example:

$ ./kploy destroy
2015-12-14T10:23:42 Deleted resource /api/v1/namespaces/default/services/webserver-svc
2015-12-14T10:23:42 Deleted resource /api/v1/namespaces/default/replicationcontrollers/webserver-rc
================================================================================

OK, I've destroyed `myns/simple_app`

Export your app

To share your app, you can make a snapshot of your app using the export command. This creates an archive called app.kploy that you can use in turn with the init command to create the app in a different location.

$ ./kploy export -v
2015-12-22T01:47:18 Exporting app based on content from /Users/mhausenblas/Documents/repos/kubernauts/kploy
Adding content of app `myns/simple_app` to app.kploy

Experimental

You can store a snapshot of your app using the push command and restore it using pull. What happens under the cover is that kploy exports your app (just as with the export command) and then stores the archive in the kploy application registry.

$ ./kploy push
Adding content of app `default/kploy2` to /Users/mhausenblas/Documents/repos/kubernauts/kploy/.app.kploy
Using https://github.com/kubernauts/kploy as the app's workspace
================================================================================

OK, I've successfully pushed the app archive to the registry:
http://registry.kploy.net/api/v1/app/bca0965d2335ab87d806a2982b132adb80d35a42d815a5bb4d9fe27aa81a3a66?workspace=https://github.com/kubernauts/kploy

To list the available app(s), use the `kploy pull` command.

To restore the app, use pull. First you'd usually list available versions:

$ ./kploy pull
Using https://github.com/kubernauts/kploy as the app's workspace
TIMESTAMP                 ID                                                                  SIZE
2016-01-04T12:43:13.545Z  7983b3dd7b38f2fb82d195eed7f78dcc1ac37de84a31f4bdf305bdb9d7e019f2    1017
2016-01-04T12:43:07.917Z  bca0965d2335ab87d806a2982b132adb80d35a42d815a5bb4d9fe27aa81a3a66    1017
2016-01-04T12:42:52.455Z  e832dbb5f60ad653033edf7b5fa5a573d43322a79c74841019344892b49c8303    1017
2016-01-04T12:41:12.557Z  adea1abd8d0b03a024e927c978b54010c5315d1e5c4f694b51561b10c5a6c2a2    1016
================================================================================

OK, I've successfully pulled from the registry.

You can now `kploy pull $ID` to download and init an app.

WARNING: a `kploy pull $ID` will overwrite whatever you had locally.

You'd then use pull again to pull a certain versions (note: most recent is at the top of the list):

$ ./kploy pull adea1abd8d0b03a024e927c978b54010c5315d1e5c4f694b51561b10c5a6c2a2
Using https://github.com/kubernauts/kploy as the app's workspace
================================================================================

OK, I've set up the app from archive.
You can now validate it with `kploy dryrun`

Note that in order for push/pull to work you MUST have the source field in the Kployfile set to either a GitHub username URL or project URL. For example, https://github.com/mhausenblas and https://github.com/kubernauts/kploy are valid.