Distribute Credentials Securely Using Secrets
This page shows how to securely inject sensitive data, such as passwords and encryption keys, into Pods.
Before you begin
You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:
Convert your secret data to a base-64 representation
Suppose you want to have two pieces of secret data: a username my-app and a password
39528$vdg7Jb. First, use a base64 encoding tool to convert your username and password to a base64 representation. Here's an example using the commonly available base64 program:
echo -n 'my-app' | base64
echo -n '39528$vdg7Jb' | base64
The output shows that the base-64 representation of your username is bXktYXBw,
and the base-64 representation of your password is Mzk1MjgkdmRnN0pi.
Create a Secret
Here is a configuration file you can use to create a Secret that holds your username and password:
apiVersion: v1
kind: Secret
metadata:
name: test-secret
data:
username: bXktYXBw
password: Mzk1MjgkdmRnN0pi
Create the Secret
kubectl apply -f https://k8s.io/examples/pods/inject/secret.yamlView information about the Secret:
kubectl get secret test-secretOutput:
NAME TYPE DATA AGE test-secret Opaque 2 1mView more detailed information about the Secret:
kubectl describe secret test-secretOutput:
Name: test-secret Namespace: default Labels: <none> Annotations: <none> Type: Opaque Data ==== password: 13 bytes username: 7 bytes
Create a Secret directly with kubectl
If you want to skip the Base64 encoding step, you can create the
same Secret using the kubectl create secret command. For example:
kubectl create secret generic test-secret --from-literal='username=my-app' --from-literal='password=39528$vdg7Jb'
This is more convenient. The detailed approach shown earlier runs through each step explicitly to demonstrate what is happening.
Create a Pod that has access to the secret data through a Volume
Here is a configuration file you can use to create a Pod:
apiVersion: v1
kind: Pod
metadata:
name: secret-test-pod
spec:
containers:
- name: test-container
image: nginx
volumeMounts:
# name must match the volume name below
- name: secret-volume
mountPath: /etc/secret-volume
# The secret data is exposed to Containers in the Pod through a Volume.
volumes:
- name: secret-volume
secret:
secretName: test-secret
Create the Pod:
kubectl apply -f https://k8s.io/examples/pods/inject/secret-pod.yamlVerify that your Pod is running:
kubectl get pod secret-test-podOutput:
NAME READY STATUS RESTARTS AGE secret-test-pod 1/1 Running 0 42mGet a shell into the Container that is running in your Pod:
kubectl exec -i -t secret-test-pod -- /bin/bashThe secret data is exposed to the Container through a Volume mounted under
/etc/secret-volume.In your shell, list the files in the
/etc/secret-volumedirectory:# Run this in the shell inside the container ls /etc/secret-volumeThe output shows two files, one for each piece of secret data:
password usernameIn your shell, display the contents of the
usernameandpasswordfiles:# Run this in the shell inside the container echo "$( cat /etc/secret-volume/username )" echo "$( cat /etc/secret-volume/password )"The output is your username and password:
my-app 39528$vdg7Jb
Define container environment variables using Secret data
Define a container environment variable with data from a single Secret
Define an environment variable as a key-value pair in a Secret:
kubectl create secret generic backend-user --from-literal=backend-username='backend-admin'Assign the
backend-usernamevalue defined in the Secret to theSECRET_USERNAMEenvironment variable in the Pod specification.apiVersion: v1 kind: Pod metadata: name: env-single-secret spec: containers: - name: envars-test-container image: nginx env: - name: SECRET_USERNAME valueFrom: secretKeyRef: name: backend-user key: backend-usernameCreate the Pod:
kubectl create -f https://k8s.io/examples/pods/inject/pod-single-secret-env-variable.yamlIn your shell, display the content of
SECRET_USERNAMEcontainer environment variablekubectl exec -i -t env-single-secret -- /bin/sh -c 'echo $SECRET_USERNAME'The output is
backend-admin
Define container environment variables with data from multiple Secrets
As with the previous example, create the Secrets first.
kubectl create secret generic backend-user --from-literal=backend-username='backend-admin' kubectl create secret generic db-user --from-literal=db-username='db-admin'Define the environment variables in the Pod specification.
apiVersion: v1 kind: Pod metadata: name: envvars-multiple-secrets spec: containers: - name: envars-test-container image: nginx env: - name: BACKEND_USERNAME valueFrom: secretKeyRef: name: backend-user key: backend-username - name: DB_USERNAME valueFrom: secretKeyRef: name: db-user key: db-usernameCreate the Pod:
kubectl create -f https://k8s.io/examples/pods/inject/pod-multiple-secret-env-variable.yamlIn your shell, display the container environment variables
kubectl exec -i -t envvars-multiple-secrets -- /bin/sh -c 'env | grep _USERNAME'The output is
DB_USERNAME=db-admin BACKEND_USERNAME=backend-admin
Configure all key-value pairs in a Secret as container environment variables
Create a Secret containing multiple key-value pairs
kubectl create secret generic test-secret --from-literal=username='my-app' --from-literal=password='39528$vdg7Jb'Use envFrom to define all of the Secret's data as container environment variables. The key from the Secret becomes the environment variable name in the Pod.
apiVersion: v1 kind: Pod metadata: name: envfrom-secret spec: containers: - name: envars-test-container image: nginx envFrom: - secretRef: name: test-secretCreate the Pod:
kubectl create -f https://k8s.io/examples/pods/inject/pod-secret-envFrom.yamlIn your shell, display
usernameandpasswordcontainer environment variableskubectl exec -i -t envfrom-secret -- /bin/sh -c 'echo "username: $username\npassword: $password\n"'The output is
username: my-app password: 39528$vdg7Jb
Pods consuming Secrets with prod / test credentials
This example illustrates a Pod which consumes a secret containing production credentials and another Pod which consumes a secret with test environment credentials.
You can create a kustomization.yaml with a secretGenerator field or run
kubectl create secret.
kubectl create secret generic prod-db-secret --from-literal=username=produser --from-literal=password=Y4nys7f11
The output is similar to:
secret "prod-db-secret" created
You can also create a secret for test environment credentials.
kubectl create secret generic test-db-secret --from-literal=username=testuser --from-literal=password=iluvtests
The output is similar to:
secret "test-db-secret" created
Special characters such as $, \, *, =, and ! will be interpreted by your
shell and require escaping.
In most shells, the easiest way to escape the password is to surround it with single quotes (').
For example, if your actual password is S!B\*d$zDsb=, you should execute the command this way:
kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password='S!B\*d$zDsb='
You do not need to escape special characters in passwords from files (--from-file).
Now make the Pods:
cat <<EOF > pod.yaml
apiVersion: v1
kind: List
items:
- kind: Pod
apiVersion: v1
metadata:
name: prod-db-client-pod
labels:
name: prod-db-client
spec:
volumes:
- name: secret-volume
secret:
secretName: prod-db-secret
containers:
- name: db-client-container
image: myClientImage
volumeMounts:
- name: secret-volume
readOnly: true
mountPath: "/etc/secret-volume"
- kind: Pod
apiVersion: v1
metadata:
name: test-db-client-pod
labels:
name: test-db-client
spec:
volumes:
- name: secret-volume
secret:
secretName: test-db-secret
containers:
- name: db-client-container
image: myClientImage
volumeMounts:
- name: secret-volume
readOnly: true
mountPath: "/etc/secret-volume"
EOF
Add the pods to the same kustomization.yaml:
cat <<EOF >> kustomization.yaml
resources:
- pod.yaml
EOF
Apply all those objects on the API server by running:
kubectl apply -k .
Both containers will have the following files present on their filesystems with the values for each container's environment:
/etc/secret-volume/username
/etc/secret-volume/password
Note how the specs for the two Pods differ only in one field; this facilitates creating Pods with different capabilities from a common Pod template.
You could further simplify the base Pod specification by using two service accounts:
prod-userwith theprod-db-secrettest-userwith thetest-db-secret
The Pod specification is shortened to:
apiVersion: v1
kind: Pod
metadata:
name: prod-db-client-pod
labels:
name: prod-db-client
spec:
serviceAccount: prod-db-client
containers:
- name: db-client-container
image: myClientImage
Secret visible to one container in a Pod
Consider a program that needs to handle HTTP requests, do some complex business logic, and then sign some messages with an HMAC. Because it has complex application logic, there might be an unnoticed remote file reading exploit in the server, which could expose the private key to an attacker.
This could be divided into two processes in two containers: a frontend container which handles user interaction and business logic, but which cannot see the private key; and a signer container that can see the private key, and responds to simple signing requests from the frontend (for example, over localhost networking).
With this partitioned approach, an attacker now has to trick the application server into doing something rather arbitrary, which may be harder than getting it to read a file.