Skip to content

Backups and snapshots

Backups

The research data is backed up nightly to tape and kept for 3 months, for the purpose of disaster recovery. If you would like to recover old or lost files within the last 3 months, raise a support ticket with us with a list of filenames and last known access times.

However, we take nightly snapshots of the whole file system, and if the file exists on an overnight snapshot, you can recover it yourself quickly as shown in the snapshots section.

What is backed up:

  • User home directories /data/home
  • Shared project storage /data/

What is NOT backed up

  • auto deleting scratch space/gpfs/scratch

The backup schedule consists of daily incremental backups which are retained for 3 months. This is currently handled by Storage Protect (previously TSM) to build a full backup. The tapes are kept geographically separate to our cluster.

Snapshots

Snapshots are taken daily and hold a copy of the entire storage at the point they were taken. All files exist in the snapshots with exactly the same permissions as they did in the original, which means you can recover files from snapshots yourself. These snapshots are kept for a week.

Snapshot structure

Snapshots can be found under any directory shared between nodes in the cluster (scratch, home or group storage). They are found in a directory called .snapshots. They are normally taken at 23:00 each day and kept for 7 days. These locations contain a dated directory for each available snapshot.

These directories are named by date (any additional ones are there for administration purposes, such as taking tape backups). Below these directories is the same structure as the original directory. Hence the snapshot of the the file lost_file can be found at .snapshots/YYYY-M-D/lost_file. Directories are also mirrored at higher levels so the entire directory structures can be recovered under /data/.snapshots

These directories are read-only; they can not be changed.

A full list of available snapshots for home directories and group shares under /data can be gathered with the mmlssnapshot gpfsWork command, as shown below:

$ mmlssnapshot gpfsWork
Snapshots in file system gpfsWork:
Directory   SnapId    Status  Created                   ExpirationTime            Fileset
2026-5-22   2738      Valid   Fri May 22 23:00:02 2026  Fri May 22 23:00:02 2026
2026-5-23   2740      Valid   Sat May 23 23:00:04 2026  Sat May 23 23:00:04 2026
2026-5-24   2742      Valid   Sun May 24 23:00:02 2026  Sun May 24 23:00:02 2026
2026-5-25   2744      Valid   Mon May 25 23:00:03 2026  Mon May 25 23:00:03 2026
2026-5-26   2746      Valid   Tue May 26 23:00:03 2026  Tue May 26 23:00:03 2026
2026-5-27   2748      Valid   Wed May 27 23:00:10 2026  Wed May 27 23:00:10 2026
2026-5-28   2750      Valid   Thu May 28 23:00:03 2026  Thu May 28 23:00:03 2026

Scratch snapshots and any shares under /gpfs can be gathered with the mmlssnapshot gpfsFlash command.

Recovering files from snapshots

To recover a file from the snapshot, simply copy it somewhere outside the .snapshots directory. For example:

cp .snapshots/<date>/lost_file recovered_data/