Housekeeping: Debian backup (raspberry).

Recently i was messing around with my Raspberry Pi and i totally screwed my system.

After restoring my system to a functional level i decided to do some housekeeping chores.

By housekeeping i mean doing some backups and getting prepared for restoring them.

I'm writing this post mainly as a future reference. All of the contents are explained elsewhere (i mean i dont claim their authorship) and are public knowledge.

First get yourself plenty of storage, i would advise to get an external (independently powered) hard disk, but if you have a big enough sd card the process will be pretty similar.

I really like using the dd command (on wikipedia they say it stands for disk destroyer or delete data, that would really make sense). Its a powerfull low level command. On the first place it is not a good idea to copy a live system with dd so you should start by starting from another partition to do the backup. That is why i decided i needed a multiboot system. Noobs and berryboot really mess the filesystem so i decided to have multiple system partitions and boot from them chosing with the cmdline.txt file.

First we will need to have two (or maybe three) equally sized partitions (at the end of the post i'll explain how to overcome this). I really advise to format the partitions on ext4 filesystem. I haven't tried using btrfs (or any other fancy fs out there) but any intensive use on ntfs is really slow for the little Pi. I will call your partitions sda1, sda2 and sda3 (if you have a multipartition sd card you will have mmcblk0p2, mmcblk0p3, mmcblk0p4). Remember your boot partition will always be mmcblk0p1 We will assume sda1 and sda2 are equally sized partitions (roughly 5 gigs) and sda3 is a much bigger storage partition.

On a linux PC (use a live one if you dont have a linux installed on your PC) first copy your system partition sda1 over sda2. For this i use gparted graphical interface but you could use dd as we explain latter.

Now you have two system partitions and you could start any of them just by editing the cmdline.txt file on /boot. 

BEWARE of a few catches: 

• before you can start from sda2 you first need to mount it and edit fstab file so that the root partition is sda2 and not sda1 (no need to touch anything else there).
• When the system boots (before it connects to the internet, if that is even possible) the system time will get loaded from a file callled /etc/fake-hwclock.data. Your system might detect that your partitions have been mounted sometime in the future. The Pi will think it is due to filesystem corruption and will stall until you confirm that you want to do a diskcheck. If you dont have a keyboard (called running headless) you are in trouble. The solution is to manually change fake-hwclock.data to sometime in the future. If the problem has already appeared you will have to unplug your hdd (my pi and its hdd are 10 mins deep under the TV) and do it in your ext4 capable pc (i mean linux) or plug an external keyboard (my boss looks at me funny when he sees me stealing keyboards from work).

So at this point you have two equally capable partitions. For the price of hdds these days i would advise having three system partitions. In my case i have: 

1. Stable system (sda1).
2. Service (for backups and stuff) (sda2)
3. Test (for installing potentially harmfull or system-breaking stuff) (sda3)

Ok. now you have to boot from your service partition and start with the backup fun. So you have a sda2 mounted as a root partition. Your storage partition (sda3) is mounted in /media/whatever (could even mount a network disk but performance most probably be sluggish).
To copy your partition you use:

sudo dd if=/dev/sda1 of=media/whatever/image.img conv=sync, noerror

This will copy (bit by bit) whatever is in your sda1 partition, conv=sync, noerror means that any error reading a bit will be written with the null value so disk structure is kept.

 If it is the first time you do this you might want to stop it after a minute or so with Ctrl+C to see what is the speed so you can calculate how long is it going to take.

It took a little bit north of 15 min to copy my 5GB partition. Since it is a bit by bit copy you will get a 5GB file (the size of your partition) so you might want to pass it through gzip:

sudo dd if=/dev/sda1 conv=nosync,noerror | gzip -c > /media/whatever/image.img.gz

This took 90 mins (about the length of a movie) and gave me a 2GB file.

But here I remembered something. When you delete a file the information is still there, files are just removed from file indexes. So the bits are still there when you dd the disk and feed it to gzip you are feeding all those deleted files. What can be done? Well we can restore the system to a maximum order (reduce entropy, nice word: entropy. Use it whenever you can to make you look smarter). So we create a file composed of all zeros to fill up your disk:

first mount the partition that you want to fill with zeros. The mount point has to be created first (if it wasn't: sudo mkdir /media/test):

sudo mount /dev/sda1 /media/test


dd if =/dev/zero of=delete.me bs=8M

This creates a  file made up of zeros. When it stops because disk is full you delete the recently created file:

sudo rm delete.me

unmount the partition and repeat the dd (wash, rinse and repeat...). Hey create a file and then delete it just to reduce entropy (for those worried for the laws of thermodynamics know that to reduce your hdds entropy universes entropy has been increased.

sudo umount /dev/sda1

sudo dd if=/dev/sda1 conv=nosync,noerror | gzip -c > /media/whatever/image.img.gz

This time it took only an hour and it generated a 1.1 GB file.

Now that we are at it:

If you have some nasty secrets that you deleted you know anyone can undelete them. If you cover your tracks with zeros not everyone can undelete them, but CIA and NSA can. so if you cover your files with zeros (delete) and repeat with ones (up to 8 times) you can be fairly sure no one will be able to reccover your dirty secrets.

You think this is useless: Well... some politicians in my country don't know this and are in jail because of it.

OK we have a backup on a .img.gz file what can we do with it? 

Imagine you have messed up, deleted or otherwise rendered unusable your system.  you take your sd card to your PC, change your root partition to sda2 and boot normally. Then you have to restore from your .img.gz file:

gzip -dc /media/whatever/image.img.gz | dd of=/dev/sda1

If it wasn't zipped: 

dd if =/media/whatever/image.img of=/dev/sda1

I would recommend that you try that your backup is working on one of your unused partitions (sda2, sda3) before you really need it.

Hey, I want a file from my backup but i don't want to go through all the hussle of copying the whole partition and boot off of it:

No prob there:

mount -t ext4 -o loop /media/whatever/image.img /media/test

But you said that I should compress my file.

Ok, afaik you shoul decompress you file and then mount it.

If you messed up your sda3 (test) partition and want to restore it you might want to copy sda1 over sda3:

sudo dd if=/dev/sda1 of=/dev/sda3 conv=sync, noerror 

Before we leave it:

What happens if you have a 5 gig partition and you want to copy it over your 4 gig card (of course all the data must fit, i mean more than 1 gig free space on the 5gig one). Dont worry, it can be done.

You have to resize your partition:

First check the filesystem for errors:

e2fsck -f

Then resize the filesystem. Remember the partition holds your partition:

resize2fs /dev/sda3 3.5GB

Willl yield a 3.5 GB filesystem. You can make a direct copy to the smaller partition or to a file.

After the copy you could resize your filesystem:

resize2fs /dev/sda3 

This will swell your filesystem to the size of its partition. 

If you wanted to shrink the partition you would have to shrink the fs (like we just saw) and then use fdisk:

sudo fdisk -l

lists the partitions

sudo fdisk /dev/sda 

will get you in fdisk. You have several options:

• m help.
• d delete partition
• n create new partition

To shrink (or enlarge a partition you'll have to delete it and recreate it with the aproppiate size.

As a last bonus, to create a filesystem on a newly created partition use:

 

mkfs.ext4 -L Disklabel /dev/sda3

So what if you want to try an image meant to be flashed on your SD card:

First you'll have to check it with fdisk:

fdisk -l /media/whatever/image.img