Velero Life cycle commands – Backup and restore scenarios

Velero is an open source tool to safely backup and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes.

In this guide we will learn some velero lifecycle commands. Please check out How to install Velero for backups using GCP provider before proceeding.

# Performing Backups

Create Backup

1
2
3
4
5
# velero backup create [backup name] [options]

# Examples
velero backup create backup1 --include-cluster-resources=true --ordered-resources 'pods=ns1/pod1,ns1/pod2;persistentvolumes=pv4,pv8' --include-namespaces=ns1
velero backup create backup2 --ordered-resources 'statefulsets=ns1/sts1,ns1/sts0' --include-namespaces=ns1

# Specify Backup Orders of Resources of Specific Kind

To backup resources of specific Kind in a specific order, use option -ordered-resources to specify a mapping Kinds to an ordered list of specific resources of that Kind. Resource names are separated by commas and their names are in format namespace/resourcename. For cluster scope resource, simply use resource name. Key-value pairs in the mapping are separated by semi-colon. Kind name is in plural form.

1
2
velero backup create backupName --include-cluster-resources=true --ordered-resources 'pods=ns1/pod1,ns1/pod2;persistentvolumes=pv4,pv8' --include-namespaces=ns1
velero backup create backupName --ordered-resources 'statefulsets=ns1/sts1,ns1/sts0' --include-namespaces=ns1

Other examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# yearly backup on Jan 1st, ttl 10 years + 2 day
velero schedule create "full-yearly" --schedule "0 0 1 1 *" --volume-snapshot-locations default --ttl 87720h0m0s s

# monthly backup on the 1st day of the month, ttl 53 weeks
velero schedule create "full-monthly" --schedule "0 1 1 * *" --volume-snapshot-locations default --ttl 8904h0m0s

# weekly backup on sunday, ttl 8 weeks
velero schedule create "full-weekly" --schedule "0 2 * * 0" --volume-snapshot-locations default --ttl 1344h0m0s

# daily backup, ttl 14 days
velero schedule create "full-daily" --schedule "0 6 * * *" --volume-snapshot-locations default --ttl 336h0m0s

# Adding a backup location to a cluster

first add a new secret with the credentials (see above)

1
kubectl create secret generic -n velero bsl-credentials --from-file=gcp=./credentials-velero

use the new secret to create the new backup location (in this case add mnp-staging backup location to the current cluster)

1
velero backup-location create mnp-staging --provider gcp --bucket citizix-backups --prefix comms-live --credential=bsl-credentials=gcp

to show all backup locations for the current cluster:

1
velero backup-location get

# Delete backups

1
2
3
4
velero backup delete [backup name]

# Examples
velero backup delete backup1

# List backups

1
velero backup get

# Get logs

1
2
3
4
velero backup logs [backup name]

# Examples
velero backup logs backup1

# Schedule backup

As a great tool to create backup, you can create a schedule to automate it! Depending your project and your needs, it can be an hourly one or a daily one!

You can use a CRON or the annotation @every. The two following example will create a backup every 6 hours.

1
2
3
4
5
6
# velero schedule create [schedule name] --schedule="[schedule]" [options]

# Examples
velero schedule create test1 --schedule="0 */6 * * *"

velero schedule create test2 --schedule="@every 6h"

Once you create the scheduled backup, you can then trigger it manually using the velero backup command.

1
velero backup create --from-schedule example-schedule

This command will immediately trigger a new backup based on your template for example-schedule. This will not affect the backup schedule, and another backup will trigger at the scheduled time.

# Delete backup schedules

Deletes schedule

1
velero delete schedule test1

# List Schedules

1
velero get schedules

# Recovery - restore from backups

From Backup

To restore from a backup.

1
2
3
4
5
6
velero restore create [Name of the restore] --from-backup [Name of the backup] [options]

velero restore create restore1 --from-backup backup1

# Create a restore with a default name ("backup1-<timestamp>") from backup "backup1"
velero restore create --from-backup backup1

From Schedule

To restore from the last backup of a schedule.

1
2
3
4
velero restore create [Name of the restore] --from-schedule [Name of the backup] [options]

# As from a backup, if you don't specify a restore name, one will be generated
velero restore create --from-schedule schedule-1

# List Restores

To list all the restore which have been done.

1
velero restore get

# Describe restores

Allow you to get more informations from specific restores.

1
2
3
4
velero restore describe [Restore name 1] [Restore name 2] ...

# Example
velero restore describe restore1 restore2

# Check Restore Logs

To get the logs of a specific restore. Useful for troubleshooting.

1
2
3
4
velero restore logs [Restore name 1]

# Example
velero restore logs restore1

# Exclude specific resources from backup

To exclude a specific resource from all your backups, you can add the label velero.io/exclude-from-backup=true.

1
2
3
# kubectl label -n  <ITEM_NAMESPACE>  <RESOURCE>/ <NAME> velero.io/exclude-from-backup=true

kubectl label -n [namespace] [resource]/[name] velero.io/exclude-from-backup=true

# Examples

# Basic example (without PersistentVolumes)

Create this file as nginx-base.yaml in the current directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
---
apiVersion: v1
kind: Namespace
metadata:
  name: nginx-example
  labels:
    app: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: nginx-example
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:latest
        name: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: my-nginx
  namespace: nginx-example
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Start the sample nginx app:

1
kubectl apply -f nginx-base.yaml

Create a backup:

1
velero backup create nginx-backup --include-namespaces nginx-example

Simulate a disaster:

1
kubectl delete namespaces nginx-example

Wait for the namespace to be deleted.

Restore your lost resources:

1
velero restore create --from-backup nginx-backup

# Snapshot example (with PersistentVolumes)

NOTE: For Azure, you must run Kubernetes version 1.7.2 or later to support PV snapshotting of managed disks.

Create this file as nginx-with-pv.yaml in the current directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
---
apiVersion: v1
kind: Namespace
metadata:
  name: nginx-example
  labels:
    app: nginx
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nginx-logs
  namespace: nginx-example
  labels:
    app: nginx
spec:
  # Optional:
  # storageClassName:  <YOUR_STORAGE_CLASS_NAME>
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Mi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: nginx-example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
      annotations:
        pre.hook.backup.velero.io/container: fsfreeze
        pre.hook.backup.velero.io/command: '["/sbin/fsfreeze", "--freeze", "/var/log/nginx"]'
        post.hook.backup.velero.io/container: fsfreeze
        post.hook.backup.velero.io/command: '["/sbin/fsfreeze", "--unfreeze", "/var/log/nginx"]'
    spec:
      volumes:
        - name: nginx-logs
          persistentVolumeClaim:
           claimName: nginx-logs
      containers:
      - image: nginx:1.17.6
        name: nginx
        ports:
        - containerPort: 80
        volumeMounts:
          - mountPath: "/var/log/nginx"
            name: nginx-logs
            readOnly: false
      - image: ubuntu:bionic
        name: fsfreeze
        securityContext:
          privileged: true
        volumeMounts:
          - mountPath: "/var/log/nginx"
            name: nginx-logs
            readOnly: false
        command:
          - "/bin/bash"
          - "-c"
          - "sleep infinity"
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: my-nginx
  namespace: nginx-example
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Start the sample nginx app:

1
kubectl apply -f nginx-with-pv.yaml

Create a backup with PV snapshotting:

1
velero backup create nginx-backup --include-namespaces nginx-example

Simulate a disaster:

1
kubectl delete namespaces nginx-example

Because the default reclaim policy for dynamically-provisioned PVs is Delete, these commands should trigger your cloud provider to delete the disk that backs the PV. Deletion is asynchronous, so this may take some time. Before continuing to the next step, check your cloud provider to confirm that the disk no longer exists.

Restore your lost resources:

1
velero restore create --from-backup nginx-backup