Frankenetes TLS

· 312 words · 2 minute read
view gist on GitHub

Frankenetes TLS overview 🔗

  1. Define what certs I want in the tls directory

    • ca-config.json defines my Certificate Authority
    • Individual -csr.json files represent individual certs / Kubernetes user accounts
    • All certs are signed by the CA, which allows the k8s api server to trust their authenticity
    • CN= corresponds to a k8s user
    • O= corresponds to a k8s group. There can be multiple

  2. I use az storage file upload-batch to put my configuration & create-certs.sh script in an Azure File share

  3. I use cfssl to create certs in an Azure Container Instance

    • Calls to ACI are here
    • This is a horrible idea - production certs should not be generated on other peoples’ machines in an online environment
    • I’m mounting the Azure File share to launch the script, read certificate requests, and drop public/private keys back to the share

  4. I download the .pem files from the Azure File share

  5. I create kubeconfigs for each user. This includes core component roles (controller manager, scheduler) and nodes (virtual kubelet)

    • Some of these have magic permissions because of their names - details here
    • I use some of these later to assign ClusterRoleBindings to

  6. For each control plane role (api server, controller manager, scheduler) and node, I create an Azure File share & upload the certs & kubeconfig as needed

  7. When I launch each control plane role, I mountthe corresponding Azure File share and reference the cert/kubeconfig as needed in the startup arguments

  8. Locally, I use the admin.kubeconfig. I’ve said that I am in the system:masters group in the cert, so I can do whatever I want

  9. I can also verify that things are working as expected by authenticating via the client certificates with curl or etcdctl. Ex:

# list everything in etcd
ETCDCTL_API=3 ~/temp/etcd328/etcd-v3.2.8-linux-amd64/etcdctl --endpoints=https://frankenetes-etcd.eastus.azurecontainer.io:2379 --cacert=output/ca.pem --cert=output/etcd.pem --key=output/etcd-key.pem get / --prefix=true

# get the version from the k8s api server
curl --cacert output/ca.pem --cert output/admin.pem --key output/admin-key.pem https://frankenetes-apiserver.eastus.azurecontainer.io:6443/version
gist