Full Raw Disk Backup
Donate
1 to 1 disk backup
Introduction[edit]
TODO: what is a full raw disk backup
It is not possible to backup an operating system installed on an internal disk such as for example Qubes OS while that operating system is running from that operating system.
The advantage of a full raw disk backup is that it should be bootable either by:
- disconnecting the internal disk and booting from the external backup disk, OR
- changing the disk ids (undocumented) (of either the internal disk or external disk after the backup), OR
- using a different computer with compatible hardware to boot from the external disk.
Unfortunately without either above method any attempt to boot the external drive after the backup might likely lead to actually boot the internal drive because these share the same disk ids after the raw disk backup.
Prerequisite Knowledge[edit]
- How to install an operating system (Debian) on a external USB drive.
Prerequisite Exercises[edit]
echo Output Redirection Essentials[edit]
This should be exercised in a safe environment such as in a disposable VM.
1. Learn the syntax.
Do not run the following command.
>- the greater-than sign - means the output file to write to.- echo "input content" > path-to-file-name
2. Exercise the redirect to file redirection.
echo "test output" > test-output-file
2. View the created file.
cat test-output-file
3. Sample printout.
test output
4. Exercise more.
Based on above example, keep changing the file content and output file names until this becomes clear.
PipeViewer Read-Only Essentials[edit]
This should be exercised in a safe environment such as in a disposable VM.
1. Create a new file such as for example:
Open file file-input in a text editor of your choice as a regular, non-root user.
If you are using a graphical environment, run. mousepad file-input
If you are using a terminal, run. nano file-input
2. Add some contents such as for example:
test input
3. Save and close the file:
4. Learn the syntax.
Do not run the following command.
<- the less-than sign - means the input file to read from.- pv < path-to-file-name-to-read-from
5. Test reading from the file.
pv < file-input
6. Read the output.
Sample output.
test input 11.0 B 0:00:00 [34.9KiB/s] [================================================================>] 100%
7. Exercise more.
Based on above example, keep changing the input file names until this becomes clear.
PipeViewer Read-Write Essentials[edit]
This should be exercised in a safe environment such as in a disposable VM.
1. Create a new file such as for example:
Open file file-input in a text editor of your choice as a regular, non-root user.
If you are using a graphical environment, run. mousepad file-input
If you are using a terminal, run. nano file-input
2. Add some contents such as for example:
test input
3. Save and close the file.
4. Learn the syntax.
Do not run the following command.
<- the less-than sign - means the input file to read from.>- the greater-than sign - means the output file to write to.- pv < path-to-file-name-to-read-from > path-to-file-name-to-write-to
5. Test reading from an input file and redirecting the output to another output file.
pv < file-input > file-output
6. Read the output.
Sample output.
11.0 B 0:00:00 [ 149KiB/s] [================================================================>] 100%
7. Compare the files.
View the two different files. These should look the same.
cat file-input
cat file-output
8. Exercise more.
Based on above example, keep changing the input files and output files until this becomes clear.
Prerequisites[edit]
- An installed operating system (such as for example Qubes OS) on an (internal) disk.
- An operating system installed on an external disk, most likely USB such as for example Debian.
- A separate external backup disk where the backup should be stored.
Information Gathering[edit]
Qubes Users Recommendations[edit]
For educational purposes, it is useful to run gparted and gnome-disks from the Qubes installation which should backed up as full raw disk backup.
Tested in Qubes R4.0 only. In later Qubes versions with untrusted storage domain, these instructions might need changes.
gparted is a disk partitioning tool which will be used as an easy way to find out how many hard drives the system has and what size they have. gnome-disks is a similar utility.
Disconnect any unneeded storage devices.
In dom0, install gparted and gnome-disks.
sudo qubes-dom0-update gparted gnome-disk-utility
In dom0, run gparted.
sudo --set-home gparted
Check the arrow down button below the X (which would close the window) if there are multiple disks.
In dom0, run gnome-disks.
gnome-disks
Make notes. For example.
Note: Modify these notes according to what can be seen in gparted and gnome-disks.
/dev/sda (476.91 GiB)
Obviously easiest if there is only 1 disk. Assuming there is only 1 disk.
Recommendations[edit]
1. Exercise with completely different test hardware.
It is recommended against to apply this procedure with production hardware if doing this procedure for the first time.
Since data loss is possible if making a mistake during the raw disk backup procedure, it is recommended to exercise the procedure with completely different hardware. Such as a second computer as well as an external boot drive and backup disk that does not contain any important data.
2. Creating a backup date note file.
After booting from the internal disk (which should be backed up).
Create a text file with a small explanation for yourself "Today is day x with date y and time z prior backup number 1."
This will later be handy when doing a restoration test.
3. USB boot operating system should have a graphical diff viewer such as meld installed as well as the following tools.
After booting the operating system from USB.
Install meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue. To accomplish that, the following steps A. to D. need to be done.
A. Update the package lists.
sudo apt update
B. Upgrade the system.
sudo apt full-upgrade
C. Install the meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue package.
Using apt command line parameter --no-install-recommends is in most cases optional.
sudo apt install --no-install-recommends meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue
D. Done.
The procedure of installing meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue is complete.
Backup High Level Overview[edit]
1. Boot from external USB drive.
2. Find out the device paths of the internal drive and the USB boot drive.
3. Find out the device path of the USB backup drive.
4. Use pv to read from the internal drive and to write to the USB backup drive.
5. Restoration test. (Optional but highly recommended.)
Backup Instructions[edit]
In Linux, unfortunately device names and device paths are non-deterministic, unpredictable, might change with kernel versions and operating system upgrades.
A raw backup with the pv can lead to data loss if used incorrectly as pv is a very powerful tool.
For example, sda is a device name and /dev/sda is a device path. Other device path examples are dev/sdb, dev/sdc.
The actual pv command is not very difficult but the device paths need to be carefully determined before starting the backup, otherwise data loss is at risk.
1. Boot from external USB disk.
2. Do not attach any other disks at this time.
If any other disks are already attached, remove them for now for simplicity.
3. Write output of fdisk to file "old".
sudo fdisk -l > old
4. Information gathering with a few hard drive utilities.
Using lshw.
sudo lshw -class disk
Using hwinfo.
sudo hwinfo --disk
See what disks are currently attached with an alternative tool such as gnome-disks as well.
Try:
gnome-disks
Try gsmartcontrol.
gsmartcontrol-root
See what disks are currently attached with an alternative tool such as gparted as well.
Try:
- Start
gpartedfrom start menu if it can be found there. - Try with pkexec if that works: /usr/sbin/gparted
- Or try with lxsudo if that works: lxsudo /usr/sbin/gparted
4. Attach another disk, the external USB disk which should be used for the backup.
5. Write output of disk to file "new".
sudo fdisk -l > new
6. Compare file "old" with file "new" using "diff".
diff old new
7. Compare file "old" with file "new" using "diff".
meld old new
8. View contents of file /etc/fstab.
cat /etc/fstab
Example printout.
# /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # # <file system> <mount point> <type> <options> <dump> <pass> /dev/mapper/debian--vg-root / ext4 errors=remount-ro 0 1 # /boot was on /dev/sdb1 during installation UUID=86983f37-38db-401f-889a-bc93d83a3be4 /boot ext2 defaults 0 2 #/dev/mapper/debian--vg-swap_1 none swap sw 0 0
Watch out for the UUID= field. In above example it is 86983f37-38db-401f-889a-bc93d83a3be4.
8. View output of the blkid.
For informational purposes only.
sudo blkid
9. Confirm the device path of the boot device.
Compare output of file /etc/fstab with blkid command.
For example. Note: replace UUID with the actual UUID from /etc/fstab.
sudo blkid | grep 86983f37-38db-401f-889a-bc93d83a3be4
Sample printout.
/dev/sdb1: UUID="86983f37-38db-401f-889a-bc93d83a3be4" TYPE="ext4" PARTUUID="decf0fe9-01"
In above example, the device path of the boot disk is /dev/sdb. Not /dev/sdb1. The 1 means partition number 1. When making raw disk backups of the full disk, partition numbers must be omitted. Otherwise it would just be a partition backup. In this case, the backup would be unbootable and couldn't be easily restored.
10. Make some notes such as.
Qubes internal disk 476.91 GiB Debian external boot disk 931.42 GiB Backup external disk 931.41 GiB
11. Note the device paths.
Write down the device paths of the internal disk, the external USB boot disk and the external USB backup disk with help according the above instructions.
For the author of this wiki page, /dev/sda was the (Qubes) internal disk, /dev/sdb the (Debian) USB boot disk and /dev/sdc the USB backup drive. This might be different for readers!
12. Note that device paths can change after reboot.
Note that device paths are unfortunately unstable. It happened to the author of this wiki page that /dev/sda and /dev/sdb paths were swapped. It is therefore required to re-identify device paths every time the computer is rebooted.
13. Become root. [1]
sudo su
14. Backup.
Explanation:
<- the less-than sign - means the input device to read from>- the greater-than sign - means the output device to write to.
Syntax:
pv < /dev/xxx > /dev/yyy
Note:
- Replace
/dev/xxxwith the actual device path of the drive which should be backed up. - Replace
/dev/yyywith the actual device path of the drive where the backup should be stored.
Examples:
Notice: DATA LOSS POSSIBLE if used incorrectly! Do not use this without prior verification of the device paths!
pv < /dev/sda > /dev/sdc
pv < /dev/nvme0n1 > /dev/sdc
The backup might take a long time.
15. Check exit code.
echo $?
Expected output if success.
0
16. Switch back to normal user.
If the user was previously running sudo su, it should now be undone. Switch back to normal user by running the following command.
exit
17. Done.
Backup is complete.
18. Restoration test.
Without restoration test, it's unclear if the backup could be restored in case needed.
Backup Verification[edit]
verification without progress meter[edit]
size=$(sudo blockdev --getsize64 /dev/nvme0n1)
sudo cmp --bytes=$size /dev/nvme0n1 /dev/sdb
verification with progress meter[edit]
Small script with progress meter. Untested.
original_disk and backup_disk need to be adjusted.
#!/bin/bash set -x set -e set -o pipefail original_disk=/dev/nvme0n1 backup_disk=/dev/sdb test -e "$original_disk" test -e "$backup_disk" ## Get the size of the original disk in bytes. original_size=$(blockdev --getsize64 "$original_disk") ## Using cp with --bytes="$original_size" to ensure that the comparison stops ## at the size of the original disk, even if the backup disk is larger. ## This way, the comparison is not confused if the backup_disk is larger. pv "$original_disk" | cmp --bytes="$original_size" /dev/stdin "$backup_disk" ## Alternative. Untested. #ddrescue --verbose --no-scrape --size="$original_size" "$original_disk" "$backup_disk" ~/logfile
Restoration Test High Level Overview[edit]
Before a restoration test can be performed, a backup is required. Once a backup has been created, attempt the following restoration test instructions.
1. Boot from internal USB drive.
2. Create a (or update) a backup date note file.
Create a text file with a small explanation for yourself "This is prior restoration test. The current date is replace-actual-date. The current time is actual-time. This note should no longer exist after the restoration test."
3. Shut down.
4. Boot from external USB drive.
5. Find out the device paths of the internal drive and the USB boot drive.
6. Find out the device path of the USB backup drive.
7. Use pv to read from the USB backup drive and to write to the internal drive. (Vice versa backup procedure.)
8. Shut down.
9. Boot from internal disk to test if the restoration was successful.
10. Check the backup date note file which was created in step 2.
The note "This is prior restoration test." is now expected to be gone. If the note still exists, then restoration has not been successful. Make sure step 2, the update of the backup date note file has not been previously skipped.
11. Done.
Restoration test has been completed.
Restoration Example[edit]
WARNING: UNTESTED
NOTE: Modify for your needs. See above.
size=$(sudo blockdev --getsize64 /dev/nvme0n1)
sudo su
pv -pterb -s "$size" /dev/sdb > /dev/nvme0n1
Qubes Specific[edit]
Alternatives[edit]
Potential alternatives, untested by the author of this wiki page.
https://www.tecmint.com/linux-disk-cloning-tools/
Footnotes[edit]
- ↑
Using sudo is unfortunately not possible. At least not without complicating the following
pvcommand. Becoming root is required because shell redirection are a bash feature, not a sudo feature. These are set up before becoming sudo. Alternatively, it would be possible to write a small script, pasting the command there and executing that script using sudo. - ↑
- https://unix.stackexchange.com/questions/433443/ssd-backup-image
- It is unfortunately not possible to compare. Running
diff /dev/sda /dev/sdcfor backup verification is not possible because of different disk sizes.
- https://unix.stackexchange.com/questions/433443/ssd-backup-image
Unfinished: This wiki is a work in progress. Please do not report broken links until this notice is removed, use Search Engines First and contribute improving this wiki.
A secure by default operating system with the latest security research in place.
At Kicksecure we value freedom and trust, supporting Open Source and Freedom Software
2012-2024 ENCRYPTED SUPPORT LP
We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 12 year success story and maybe DONATE!