Volumes

This document describes the current state of Volumes in kubernetes. Familiarity with pods is suggested.

A Volume is a directory, possibly with some data in it, which is accessible to a Container. Kubernetes Volumes are similar to but not the same as Docker Volumes.

A Pod specifies which Volumes its containers need in its spec.volumes property.

A process in a Container sees a filesystem view composed from two sources: a single Docker image and zero or more Volumes. A Docker image is at the root of the file hierarchy. Any Volumes are mounted at points on the Docker image; Volumes do not mount on other Volumes and do not have hard links to other Volumes. Each container in the Pod independently specifies where on its image to mount each Volume. This is specified in each container's VolumeMounts property.

Types of Volumes

Kubernetes currently supports multiple types of Volumes: emptyDir, gcePersistentDisk, awsElasticBlockStore, gitRepo, secret, nfs, iscsi, glusterfs, persistentVolumeClaim, rbd. The community welcomes additional contributions.

Selected volume types are described below.

EmptyDir

An EmptyDir volume is created when a Pod is bound to a Node. It is initially empty, when the first Container command starts. Containers in the same pod can all read and write the same files in the EmptyDir volume. When a Pod is unbound, the data in the EmptyDir is deleted forever.

Some uses for an EmptyDir are:

• scratch space, such as for a disk-based mergesort or checkpointing a long computation,
• a directory that a content-manager container fills with data while a webserver container serves the data.

Currently, the user cannot control what kind of media is used for an EmptyDir (see also the section Resources, below). If the Kubelet is configured to use a disk drive, then all EmptyDir volumes will be created on that disk drive. In the future, it is expected that Pods can control whether the EmptyDir is on a disk drive, SSD, or tmpfs.

HostPath

A Volume with a HostPath property allows access to files on the current node.

Some uses for a HostPath are:

• running a container that needs access to Docker internals; use a HostPath of /var/lib/docker.
• running cAdvisor in a container; use a HostPath of /dev/cgroups.

Watch out when using this type of volume, because:

• pods with identical configuration (such as created from a podTemplate) may behave differently on different nodes due to different files on different nodes.
• When Kubernetes adds resource-aware scheduling, as is planned, it will not be able to account for resources used by a HostPath.

GCEPersistentDisk

Important: You must create a PD using gcloud or the GCE API before you can use it

A Volume with a GCEPersistentDisk property allows access to files on a Google Compute Engine (GCE) Persistent Disk.

There are some restrictions when using a GCEPersistentDisk:

• the nodes (what the kubelet runs on) need to be GCE VMs
• those VMs need to be in the same GCE project and zone as the PD
• avoid creating multiple pods that use the same Volume if any mount it read/write.
  • if a pod P already mounts a volume read/write, and a second pod Q attempts to use the volume, regardless of if it tries to use it read-only or read/write, Q will fail.
  • if a pod P already mounts a volume read-only, and a second pod Q attempts to use the volume read/write, Q will fail.
  • replication controllers with replicas > 1 can only be created for pods that use read-only mounts.

Creating a PD

Before you can use a GCE PD with a pod, you need to create it.

gcloud compute disks create --size=500GB --zone=us-central1-a my-data-disk

GCE PD Example configuration:

apiVersion: v1
kind: Pod
metadata:
  name: testpd
spec:
  containers:
  - image: kubernetes/pause
    name: testcontainer
    volumeMounts:
    - mountPath: /testpd
      name: testvolume
  volumes:
  - name: testvolume
    # This GCE PD must already exist. 
    gcePersistentDisk:
      pdName: test
      fsType: ext4

AWSElasticBlockStore

Important: You must create an EBS volume using aws ec2 create-volume or the AWS API before you can use it

A Volume with an awsElasticBlockStore property allows access to files on a AWS EBS volume

There are some restrictions when using an awsElasticBlockStore volume:

• the nodes (what the kubelet runs on) need to be AWS EC2 instances
• those instances need to be in the same region and availability-zone as the EBS volume
• EBS only supports a single EC2 instance mounting a volume

Creating an EBS volume

Before you can use a EBS volume with a pod, you need to create it.

aws ec2 create-volume --availability-zone eu-west-1a --size 10 --volume-type gp2

Make sure the zone matches the zone you brought up your cluster in. (And also check that the size and EBS volume type are suitable for your use!)

AWS EBS Example configuration:

apiVersion: v1
kind: Pod
metadata:
  name: testpd
spec:
  containers:
  - image: kubernetes/pause
    name: testcontainer
    volumeMounts:
    - mountPath: /testpd
      name: testvolume
  volumes:
  - name: testvolume
    # This AWS EBS volume must already exist.
    awsElasticBlockStore:
      volumeID: aws://<availability-zone>/<volume-id>
      fsType: ext4

(Note: the syntax of volumeID is currently awkward; #10181 fixes it)

NFS

Kubernetes NFS volumes allow an existing NFS share to be made available to containers within a pod.

See the NFS Pod examples section for more details. For example, nfs-web-pod.yaml demonstrates how to specify the usage of an NFS volume within a pod. In this example one can see that a volumeMount called "nfs" is being mounted onto /var/www/html in the container "web". The volume "nfs" is defined as type nfs, with the NFS server serving from nfs-server.default.kube.local and exporting directory / as the share. The mount being created in this example is not read only.

Secrets

Secret volumes are used to pass sensitive information, such as passwords, to pods that mount these volumes. Secrets are described here.

Resources

The storage media (Disk, SSD, or memory) of an EmptyDir volume is determined by the media of the filesystem holding the kubelet root dir (typically /var/lib/kubelet). There is no limit on how much space an EmptyDir or HostPath volume can consume, and no isolation between containers or between pods.

In the future, we expect that EmptyDir and HostPath volumes will be able to request a certain amount of space using a compute resource specification, and to select the type of media to use, for clusters that have several media types.