How to Create and Use Secrets in Kubernetes

This tutorial gives you practical experience in creating and using secrets in Kubernetes, including passwords, OAuth tokens, ssh keys, and certificates.

By Alwyn Botha, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

This tutorial gives you practical experience in creating and using secrets in Kubernetes. The Kubernetes functionality discussed here is suitable for all kinds of secrets: passwords, OAuth tokens, ssh keys, certificates, and so on.

Prerequisites

This tutorial is written using Windows 10, kubectl, and minikube. If you have a similar setup you are good to go.

Important Note for Windows Users

All files you create must have Unix (LF) line endings, not Windows ( CR LF ) endings. Read your editor documentation on how to change it to default to LF line endings. There should also be some EOL conversion menu option somewhere to convert to LF.

If you are using a server then you need a running Kubernetes cluster, and your kubectl command-line tool must be able to access your cluster. If you run kubectl get nodes and it shows a list of nodes - even just one - you are good to go.

You need about 2 weeks of basic beginner exposure to Kubernetes to follow these tutorials.

If you have zero Kubernetes experience I suggest you spend a few days reading at https://kubernetes.io/docs/concepts/ Unfortunately these are not step by step how-to tutorials, mostly in-depth theory with snippets assuming you know how to apply it. The https://kubernetes.io/docs/tasks/ are step by step tutorials, but you need to be quite familiar with the theory and concepts.

You need not be a theory expert to follow this tutorial, we will walk you thorough some practical experience of applying Kubernetes.

Creating a Secret: kubectl Create Secret

Most applications need a database. Most databases need a username and password. Let's create a simple secret that includes a username and password. This is a two step process:

Create files containing our secrets,
Use kubectl create secret generic to create a Kubernetes secret object.

Username and password are called keys in this context. ( When using these secrets later you will see how keys are used. )

The simplest way to create both those keys and their secret content is using files.

Run the following at your shell:

echo -n 'dbadmin' > ./username.txt
echo -n 'db3344@@pwd' > ./password.txt

The -n flag means echo should not output the trailing newline. We want only those text as-is in those files.

Note: I deliberately picked a simple password so that when we decode the secret we will instantly see it shows the correct value. This is not recommended for actual use.

Now we use this as input for Kubernetes to create a Kubernetes secret object.

kubectl create secret generic my-db-secret-from-files --from-file=./username.txt --from-file=./password.txt

secret "my-db-secret-from-files" created

The kubectl create secret generic command is used to create secrets using files as input.

I wanted myDBsecret as secret name, but secret names cannot contain upper case characters:

kubectl create secret generic myDBsecret --from-file=./username.txt --from-file=./password.txt

Error I got:

The Secret "myDBsecret" is invalid: metadata.name: Invalid value: "myDBsecret": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is [a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)

Let's investigate what Kubernetes created:

kubectl get secrets

NAME                  TYPE                                  DATA   AGE
default-token-gs2wt   kubernetes.io/service-account-token   3      4d23h
my-db-secret-from-files          Opaque                                2      32s

That first default token gets created automatically upon a Kubernetes install. Next we see our secret listed.

You can get more detail about our secret:

kubectl describe secrets/my-db-secret-from-files

Name:         my-db-secret-from-files
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
password.txt:  26 bytes
username.txt:  18 bytes

Nothing really awesome useful shown: the secret my-db-secret-from-files contains both the username and password.

You have now successfully created your first Kubernetes secret object - ready for use by any Pod.

Using Secrets

Creating secrets was a two step process. Using secrets requires just one step: accessing the secret via volume mounts.

Let's create a simple Pod using our secrets:

nano mySecretPod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: my-secret-pod
spec:
  containers:
  - name: my-secret-pod
    image: alpine
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: my-volume
      mountPath: "/etc/secret"
      readOnly: true
  volumes:
  - name: my-volume
    secret:
      secretName: my-db-secret-from-files

( My approach: create Pod that has access to secrets then explain the theory - referring to this working example. )

Create the Pod:

kubectl create -f mySecretPod.yaml 
   
pod/my-secret-pod created

Let Kubernetes describe all the detail it has about our Pod:

kubectl describe -f mySecretPod.yaml 
   
Name:               my-secret-pod
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               minikube/10.0.2.15
Start Time:         Mon, 31 Dec 2018 10:36:14 +0200
Labels:             <none>
Annotations:        <none>
Status:             Running
IP:                 172.17.0.5
Containers:
  my-secret-pod:
    Container ID:  docker://28cbd33320d5a4d309615cc7d0a4c970bdc22b9f4edd84c7c8019dfa7d9153ff
    Image:         alpine
    Image ID:      docker-pullable://alpine@sha256:46e71df1e5191ab8b8034c5189e325258ec44ea739bba1e5645cff83c9048ff1
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/ash
      -ec
      while :; do echo '.'; sleep 15 ; done
    State:          Running
      Started:      Mon, 31 Dec 2018 10:36:15 +0200
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /etc/secret from my-volume (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-gs2wt (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  my-volume:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  my-db-secret-from-files
    Optional:    false
  default-token-gs2wt:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-gs2wt
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  10s   default-scheduler  Successfully assigned default/my-secret-pod to minikube
  Normal  Pulled     9s    kubelet, minikube  Container image "alpine" already present on machine
  Normal  Created    9s    kubelet, minikube  Created container
  Normal  Started    9s    kubelet, minikube  Started container

That is too much detail for our purposes.

Let's focus on the secret functionality :

( I even removed the - serviceaccount from default-token - for clarity )

mySecretPod.yaml with irrelevant stuff removed:::
   
Name:               my-secret-pod
Containers:
  my-secret-pod:
    Image:         alpine
    Mounts:
      /etc/secret from my-volume (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  my-volume:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  my-db-secret-from-files
    Optional:    false
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  10s   default-scheduler  Successfully assigned default/my-secret-pod to minikube
  Normal  Pulled     9s    kubelet, minikube  Container image "alpine" already present on machine
  Normal  Created    9s    kubelet, minikube  Created container
  Normal  Started    9s    kubelet, minikube  Started container

We see our Pod in perfect health:

Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True

Our secrets are mounted at /etc/secret read-only (ro)

    Mounts:
      /etc/secret from my-volume (ro)

my-volume contains our secret: my-db-secret-from-files

Volumes:
  my-volume:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  my-db-secret-from-files

Let's enter our Pod and investigate if we can access our secrets:

( Works exactly like docker exec )

kubectl exec my-secret-pod -i -t -- /bin/ash

/ # ls /etc/secret/
password.txt  username.txt

/ # cat /etc/secret/password.txt
db3344@@pwd

/ # cat /etc/secret/username.txt
dbadmin
 / # exit

Both our secrets are mounted at /etc/secret - and the decoded content is as expected: password is correct.

Our secrets are available as plain unencrypted text inside our Pod.

You are now familiar with the basics of creating secrets in files as well as how to access it in a Pod.

/etc/secret/password.txt and /etc/secret/username.txt are only mounted in RAM - those secrets are never written to any disk when Pods are using it.

Let's now review our Pod spec to understand how secrets are accessed in Pods.

nano mySecretPod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: my-secret-pod
spec:
  containers:
  - name: my-secret-pod
    image: alpine
    imagePullPolicy: IfNotPresent
    
    tty: true
    stdin: true
       
    command: ["/bin/ash", "-ec", "while :; do echo '.'; sleep 15 ; done"]

    volumeMounts:
    - name: my-volume
      mountPath: "/etc/secret"
      readOnly: true
      
  volumes:
  - name: my-volume
    secret:
      secretName: my-db-secret-from-files

.stdin : true Keep stdin open on the container in the pod
tty stands for TeleTYpewriter
.tty : true Allocates a TTY / terminal console for each container in the pod

These 2 fields enable us to exec / ssh into a Pod.

The while loop and echos .command just keep the container in the Pod busy doing something. Otherwise the container will immediately just shut down since it has no work to do.

The .volumes field right at the bottom defines my-volume as a volume containing our secret my-db-secret-from-files .

This .secretName must match the secret name we created using:

kubectl create secret generic my-db-secret-from-files --from-file=./username.txt --from-file=./password.txt

Now we have my-volume containing my-db-secret-from-files . We must mount it to be able to use it. Similar to Linux volumes: you must mount it to be able to use it.

The .volumeMounts must refer to the exact volume we wish to mount ... my-volume

The .mountPath specifies the directory path where we wish to access our secrets: "/etc/secret"

You must change your Pod application to access the secret at that .mountPath .

Alternatively specify .mountPath to be exactly where your Pod application expects to get the secret from .

The ls /etc/secret/ command proves our secret got mounted where we expected it.

/ # ls /etc/secret/
password.txt  username.txt

Spend your time reading the Pod spec and this explanation below twice or even three times. The better you understand this most simple case the better you will able to understand all the other cases below.

See the value of the logic of this tutorial: first we create a Pod with access to a secret. Then we kubectl exec into it to run ls /etc/secret/ . If we now read the theory of why those color-coded lines exist in the YAML file, we can see in our running Pod how everything fits together.

You do not need to be able to recreate it all from a blank screen, you JUST need to be able to read and understand the Pod spec detail lines.

Define Key Names Different from Filenames

When you used

kubectl create secret generic my-db-secret-from-files --from-file=./username.txt --from-file=./password.txt

Kubernetes creates secrets with the keys equal to the base filename ( filename before the dot ): username and password.

If your key names need to be different from the filenames, this is the syntax:

=user= and =pass= defines the keys.

kubectl create secret generic my-db-secret-new-key --from-file=<strong>user</strong>=./username.txt --from-file=pass=./password.txt

Now user and pass are they keys for my-db-secret-new-key .

Run command to create my-db-secret-new-key .

kubectl create secret generic my-db-secret-new-key --from-file=user=./username.txt --from-file=pass=./password.txt

secret/my-db-secret-new-key created

We need to delete our existing Pod, so that we can create a new Pod using this new secret.

kubectl delete -f mySecretPod.yaml --force --grace-period=0

warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "my-secret-pod" force deleted

Note: I use --force --grace-period=0 in these tutorials to immediately stop Pods. Otherwise it will take 30 seconds for Pod to gracefully shut down. Using --force in production is a sure way to cause data corruption.

Edit manifest ( our YAML file with desired spec for our Pod ). Note very last line ... our new secret.

nano mySecretPod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: my-secret-pod
spec:
  containers:
  - name: my-secret-pod
    image: alpine
    imagePullPolicy: IfNotPresent
    
    tty: true
    stdin: true
       
    command: ["/bin/ash", "-ec", "while :; do echo '.'; sleep 15 ; done"]

    volumeMounts:
    - name: my-volume
      mountPath: "/etc/secret"
      readOnly: true
      
  volumes:
  - name: my-volume
    secret:
      secretName: my-db-secret-new-key

Create our Pod.

kubectl create -f mySecretPod.yaml

pod/my-secret-pod created

Describe the secret. Note our new 4 character keys at bottom.

kubectl describe secret/my-db-secret-new-key

Name:         my-db-secret-new-key
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
<strong>pass</strong>:  26 bytes
<strong>user</strong>:  18 bytes

Enter Pod, ls the directory where our secrets are mounted. Note our new 4 character keys.

kubectl exec my-secret-pod -i -t -- /bin/ash

/ # ls /etc/secret
pass  user
/ # exit

Demo complete, delete our Pod.

kubectl delete -f mySecretPod.yaml --force --grace-period=0

warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.

Files Containing Secrets in a Folder

Run the following at your shell:

mkdir secret-folder

echo -n 'dbadmin' > ./secret-folder/username.txt
echo -n 'db3344@@pwd' > ./secret-folder/password.txt

We now have a folder with 2 files containing our secrets.

Run command to create my-db-secret-demo .

kubectl create secret generic my-db-secret-demo --from-file=./secret-folder

secret/my-db-secret-demo created

Note that --from-file specifies our folder: secret-folder

List all the secrets we have so far:

kubectl get secret

NAME                   TYPE                                  DATA   AGE
default-token-gs2wt    kubernetes.io/service-account-token   3      5d2h
my-db-secret           Opaque                                2      129m
my-db-secret-demo      Opaque                                2      23s
my-db-secret-new-key   Opaque                                2      26m

kubectl describe secrets/my-db-secret-demo

Name:         my-db-secret-demo
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
password.txt:  26 bytes
username.txt:  18 bytes

This output is identical to the secret we created earlier using file NAMES directly.

Creating secrets using folders is convenient if your Pod-containerized application needs many secrets - just specify folder name and all the files inside it will be used to create a mega-secret package with a list of secrets.

Use key=val Pairs to Create a Secret

Another way to create a secret is to have key=value pairs in a file.

nano env-file-demo

key1=abc
key2=123
usera=test
usera-password=demo

Run command to create my-db-secret-env-file .

Note the --from-env-file below

kubectl create secret generic my-db-secret-env-file --from-env-file=./env-file-demo

secret/my-db-secret-env-file created

Describe the secret:

kubectl describe secrets/my-db-secret-env-file

Name:         my-db-secret-env-file
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
usera-password:  4 bytes
key1:            3 bytes
key2:            3 bytes
usera:           4 bytes

Create a Secret from a Literal

Note: This is the most insecure way to create a secret.

The problem with this approach is that the secret is now visible in plain text in your bash history.

To create a secret using a literal value use this syntax.

kubectl create secret generic my-literal-secret --from-literal=litkey1=insecure1 --from-literal=litkey2=passtheword

secret/my-literal-secret created

kubectl describe secret/my-literal-secret 
   
Name:         my-literal-secret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
litkey1:  9 bytes
litkey2:  11 bytes

Exercises

Following a step by step tutorial is easy. At work you will not have step by step instructions: you will have existing Pods that need minor mods.

These exercises simulate real life: modify existing Pods.

For your own benefit actually do these exercises. This is the ONLY work-relevant experience you get since anyone can follow a tutorial via cut and paste and still not really understand anything.

modify every example given here: remove one of the keys,
modify every example given here: add one new key,
mount the secrets at a different directory - do at least one such exercise,
modify some existing YAML files during your experiments,
create new copies of YAML files - give all objects new names in this new YAML files.

kubectl exec into your Pods to check your work.

Clean Up

Get a list of your secrets :

kubectl get secrets

NAME                           TYPE                                  DATA   AGE
secret/default-token-gs2wt     kubernetes.io/service-account-token   3      6d4h
secret/my-db-secret            Opaque                                2      28h
secret/my-db-secret-demo       Opaque                                2      26h
secret/my-db-secret-env-file   Opaque                                4      25h
secret/my-db-secret-new-key    Opaque                                2      26h
secret/my-literal-secret       Opaque                                2      6h7m

Delete the secrets ... you may have more to delete.

kubectl delete secret/my-db-secret
kubectl delete secret/my-db-secret-demo
kubectl delete secret/my-db-secret-env-file
kubectl delete secret/my-db-secret-new-key
kubectl delete secret/my-literal-secret

Run kubectl get secrets again to check you deleted it all.

Run kubectl get po to get list of Pods.

Delete ones you do not need anymore :

For example :

kubectl delete -f mySecretPod.yaml --force --grace-period=0

Community

How to Create and Use Secrets in Kubernetes

Prerequisites

Important Note for Windows Users

Creating a Secret: kubectl Create Secret

Using Secrets

Define Key Names Different from Filenames

Files Containing Secrets in a Folder

Use key=val Pairs to Create a Secret

Create a Secret from a Literal

Exercises

Clean Up

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

Container Service for Kubernetes

Container Registry

ECS(Elastic Compute Service)