Backup and Restore

Before you begin, make sure you have the following prerequisites to confirm readiness and gather required credentials:

To install the Velero server, create a velero-values.yaml file that configures the backup storage location (S3 bucket and region), the snapshot location (EBS snapshot region), the required feature flags (for example, CSI snapshots), and the AWS credentials and plugin settings. Then deploy Velero with the official Helm chart into the velero namespace. Copy the YAML configuration into a file, update the values as described, and then run the Helm install commands.

configuration:
  backupStorageLocation:
  - bucket: velero-example-bucket  
    config:
      region: us-east-2            
    name: default
    provider: aws
  volumeSnapshotLocation:
  - config:
      region: us-east-2          
    name: default
    provider: aws
  features: EnableCSI
credentials:
  extraEnvVars: {}
  secretContents:
    cloud: |
      [default]
      aws_access_key_id=XXXXXXXXXXX     
      aws_secret_access_key=XXXXXXX    
initContainers:
- image: velero/velero-plugin-for-aws:v1.10.0
  imagePullPolicy: IfNotPresent
  name: velero-plugin-for-aws
  volumeMounts:
  - mountPath: /target
    name: plugins
deployNodeAgent: true

helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
helm upgrade --install velero vmware-tanzu/velero --namespace velero --create-namespace -f velero-values.yaml

kubectl -n velero get pods,deploy,ds,cm,secret

In this step, you install the Velero command-line tool on your workstation so you can initiate and manage backup or restore operations against the Velero server running in the cluster.

This is done by downloading the appropriate Velero release tarball for your OS, extracting it, and placing the Velero binary into a directory on your PATH, typically /usr/local/bin for Ubuntu or Debian, then validating the installation with velero help. See Install the CLI for detailed instructions for other OS types.

curl -L -o /tmp/velero.tar.gz https://github.com/vmware-tanzu/velero/releases/download/v1.14.0/velero-v1.14.0-linux-amd64.tar.gz 
tar -C /tmp -xvf /tmp/velero.tar.gz
mv /tmp/velero-v1.14.0-linux-amd64/velero /usr/local/bin/velero
chmod +x /usr/local/bin/velero
velero --help

In this section, you can interact by running Velero backup create to capture Kubernetes resources for a target namespace into the configured AWS S3 bucket, and Velero restore create to rehydrate those resources back into the cluster from a selected backup. It also includes the basic commands to check backup or restore status and review logs so you can confirm the operation completed successfully.

velero backup create nginx-example-bkp --include-namespaces nginx-example

velero restore create --from-backup nginx-example-bkp

Make sure you have the following prerequisites and update the cluster or tooling where needed before moving on.

It provisions an in-cluster, S3-compatible object store that Velero or OADP uses as the backup target where backup metadata and volume backup data are uploaded. The installation is performed using the MinIO Helm chart. You add the MinIO Helm repository, create a minio-values.yaml file to define the deployment mode, and then run helm upgrade --install to deploy MinIO into a dedicated minio namespace.

On OpenShift, you need to grant the MinIO service account the required Security Context Constraints (SCC) to run successfully. Finally, use the MinIO client (mc) inside the pod to create the Velero bucket and a dedicated Velero user with read/write permissions.

helm repo add minio https://charts.min.io/
helm repo update

mode: standalone
persistence:
  enabled: true
  size: 50Gi
resources:
  requests:
    memory: 512Mi
    cpu: 250m
rootUser: "minioadmin"
rootPassword: "minioadmin"
service:
  type: ClusterIP
replicas: 1
console:
  enabled: false

helm upgrade --install minio minio/minio \
  -n minio --create-namespace \
  -f minio-values.yaml \
  --version 5.4.0

oc adm policy add-scc-to-user anyuid -z minio-sa -n minio
oc adm policy who-can use scc anyuid

kubectl exec -it deploy/minio -n minio -- bash
# Inside pod:
mc alias set localminio http://minio:9000 minioadmin minioadmin
mc mb localminio/velero
mc admin user add localminio velero-user vP@assWoRd
mc admin policy attach localminio readwrite --user velero-user
mc ls localminio

Configuring cloud credentials enables Velero to authenticate to the MinIO object storage in order to write backups and read them back during restores. You do this by creating a Kubernetes Secret named cloud-credentials.yaml in the OADP namespace, containing an AWS-style credentials file.

apiVersion: v1
kind: Secret
metadata:
  name: cloud-credentials
  namespace: openshift-adp
stringData:
  cloud: |
    [default]
    aws_access_key_id = velero-user
    aws_secret_access_key = vP@assWoRd

The following table defines each field of the above configuration:

Run the command to apply the secret to the OADP namespace so that OADP/Velero can use these credentials to access the MinIO S3 bucket for backups and restores.

kubectl apply -f cloud-credentials.yaml

The deployment of the DPA custom resource lets OADP configure Velero with the correct settings. This defines Velero’s configuration, plugins, and backup location.

Create a dpa.yaml file in the OADP namespace with your Velero configuration:

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: velero-dpa
  namespace: openshift-adp
spec:
  configuration:
    nodeAgent:
      enable: true
      uploaderType: kopia
    velero:
      disableFsBackup: false
      defaultPlugins:
        - openshift
        - aws
        - kubevirt
  backupLocations:
    - velero:
        provider: aws
        objectStorage:
          bucket: velero
          prefix: backups
        config:
          region: minio
          s3ForcePathStyle: "true"
          s3Url: http://minio.minio.svc.cluster.local:9000
        credential:
          name: cloud-credentials
          key: cloud
        default: true

The following table defines each field of the above configuration:

Run the following command to apply, and then verify that the Velero/OADP pods are running:

kubectl apply -f dpa.yaml
kubectl get pods -n openshift-adp

The Velero configuration uses the information to know where backups are stored (bucket/prefix) and how to connect to that storage. The BSL points Velero to the MinIO S3-compatible service and the velero bucket, so backups and restore artifacts are consistently written to the correct location.

Create a tuned BSL for faster uploads, backup-storage-location.yaml.

apiVersion: velero.io/v1
kind: BackupStorageLocation
metadata:
  name: velero-bsl
  namespace: openshift-adp
spec:
  provider: aws
  objectStorage:
    bucket: velero
    prefix: backups
  config:
    region: minio
    s3ForcePathStyle: 'true'
    s3Url: 'http://minio.minio.svc.cluster.local:9000'
    maxParallel: "10"
    maxRetries: "5"
    maxRetriesBackoff: "2s"
    partSize: "104857600"
  credential:
    name: cloud-credentials
    key: cloud
  default: true

The following table defines each field of the above configuration:

After applying the backup-storage-location.yaml, you can reference it by name, for example, velero-bsl, in backup specs through the storageLocation to ensure backups use the tuned settings. Run the following command:

kubectl apply -f backup-storage-location.yaml

Creating a backup is the step where you define what data and Kubernetes resources Velero should capture and how the backup should be executed and retained.

Create a backup.yaml, adjust names/namespaces:

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: virtana-io-backup-16sep-01
  namespace: openshift-adp
spec:
  csiSnapshotTimeout: 120m0s
  defaultVolumesToFsBackup: true
  includedNamespaces:
    - virtana-io
  itemOperationTimeout: 10h0m0s
  snapshotMoveData: false
  storageLocation: velero-bsl
  ttl: 72h0m0s

The following table defines each field of the above configuration:

Run the following command to apply the backup resource to start the Velero backup job, then list backups in the openshift-adp namespace to confirm it was created and to monitor its status.

kubectl apply -f backup.yaml
velero backup get -n openshift-adp`

In this step, you use Velero through OADP to recreate Kubernetes/OpenShift resources and recover persistent data from a previously completed backup stored in MinIO.

Create a restore.yaml that points to the desired backupName, optionally tunes restore behavior:

apiVersion: velero.io/v1
kind: Restore
metadata:
  name: virtana-io-restore-16sep-01
  namespace: openshift-adp
spec:
  uploaderConfig:
    parallelFilesDownload: 16
  includedNamespaces:
    - virtana-io
  backupName: virtana-io-backup-16sep-01
  restorePVs: true
  restoreVolumes: true
  itemOperationTimeout: 12h

The following table defines each field of the above configuration:

Run the following command to apply restore.yaml and monitor progress and confirm completion using the Velero CLI.

kubectl apply -f restore.yaml
velero restore get -n openshift-adp

In this step, confirm that your Velero/OADP backup and restore actually succeeded at both the Velero control-plane level and at the application level. Validate the resources and application behavior in the restored namespace.

velero backup get -n openshift-adp
velero backup describe virtana-io-backup-16sep-01 -n openshift-adp

velero restore get -n openshift-adp
velero restore describe virtana-io-restore-16sep-01 -n openshift-adp

Before you begin, ensure your environment meets the following requirements:

MinIO is a high-performance, S3-compatible object storage system that serves as the backend where Velero stores all backup data. Installing MinIO within your cluster gives you a self-hosted, lightweight storage target.

helm repo add minio https://charts.min.io/
helm repo update

mode: standalone
persistence:
  enabled: true
  size: 50Gi
resources:
  requests:
    memory: 512Mi
    cpu: 250m
rootUser: "minioadmin"
rootPassword: "minioadmin"
service:
  type: ClusterIP
replicas: 1
console:
  enabled: false

helm upgrade --install minio minio/minio \
  -n minio --create-namespace \
  -f minio-values.yaml \
  --version 5.4.0

kubectl exec -it deploy/minio -n minio -- bash
# Inside pod:
mc alias set localminio http://minio:9000 minioadmin minioadmin
mc mb localminio/velero
mc admin user add localminio velero-user vP@assWoRd
mc admin policy attach localminio readwrite --user velero-user
mc ls localminio

Configuring cloud credentials enables Velero to authenticate to the MinIO object storage in order to write backups and read them back during restores. Create a Kubernetes Secret named cloud-credentials.yaml containing the access key and secret key in the AWS credential format that Velero expects.

apiVersion: v1
kind: Secret
metadata:
  name: cloud-credentials
  namespace: velero
stringData:
  cloud: |
    [default]
    aws_access_key_id = velero-user
    aws_secret_access_key = vP@assWoRd

The following table defines each field of the above configuration:

Run the command to apply the cloud-credentials.yaml so that Velero can use these credentials to access the MinIO S3 bucket for backups and restores.

kubectl apply -f cloud-credentials.yaml

This step creates a pod that runs the Velero CLI, which you use to execute backup and restore commands. The pod mounts the cloud credentials secret so Velero can authenticate with MinIO.

Create the velero-client.yaml configuration file:

apiVersion: v1
kind: Pod
metadata:
  name: velero-client
  namespace: velero
spec:
  containers:
  - name: velero
    image: velero/velero:v1.12.0
    command: ["/bin/sh"]
    args: ["-c", "sleep infinity"]
    volumeMounts:
    - name: cloud-credentials
      mountPath: /credentials
      readOnly: true
  volumes:
  - name: cloud-credentials
    secret:
      secretName: cloud-credentials

The following table defines each field of the above configuration:

Run the command to apply the manifest to deploy the Velero client pod, which will mount the cloud credentials and remain running for you to execute backup and restore commands.

kubectl apply -f velero-client.yaml

The Velero configuration uses the information to know where backups are stored and how to connect to that storage. The BSL points Velero to the MinIO S3-compatible service and the velero bucket, so backups and restore artifacts are consistently written to the correct location.

Create a tuned BSL for faster uploads, backup-storage-location.yaml.

apiVersion: velero.io/v1
kind: BackupStorageLocation
metadata:
  name: default
  namespace: velero
spec:
  provider: aws
  objectStorage:
    bucket: velero
    prefix: backups
  config:
    region: minio
    s3ForcePathStyle: 'true'
    s3Url: 'http://minio.minio.svc.cluster.local:9000'
    maxParallel: "10"
    maxRetries: "5"
    maxRetriesBackoff: "2s"
    partSize: "104857600"
  credential:
    name: cloud-credentials
    key: cloud
  default: true

The following table defines each field of the above configuration:

Run the command to apply the manifest, registering MinIO as Velero's default backup storage location with the configured connection and performance settings.

kubectl apply -f backup-storage-location.yaml

Creating a backup is the step where you define what data and Kubernetes resources Velero should capture and how the backup should be executed and retained.

Create a backup:

velero backup create virtana-io-backup-16sep-01 \
  --include-namespaces virtana-io \
  --ttl 72h

Adjust parameters as required. See Velero Backup Reference for more details.

Run the command to check the backup status:

velero backup get
velero backup describe virtana-io-backup-16sep-01

When you need to recover workloads, Velero can restore all resources from a previously created backup. This recreates the Kubernetes objects and restores persistent volume data.

Run the command to restore from backup:

velero restore create virtana-io-restore-16sep-01 \
  --from-backup virtana-io-backup-16sep-01

Adjust parameters as required. See Velero Restore Reference for more details.

Run the command to check the restore status:

velero restore get
velero restore describe virtana-io-restore-16sep-01

After a backup or restore operation, verify that it completed successfully by checking the status of both the Velero operations and the restored workloads.

Run the command to check the backup and restore status:

velero backup get
-and-
velero restore get

As a final step, confirm that all pods, persistent volume claims, and services in the restored namespace are running as expected.

Run the following command to verify the workload:

kubectl get pods -n virtana-io
kubectl get pvc -n virtana-io
kubectl get svc -n virtana-io