# [Howto] Creation of a Stage 5 archive

## d2_racing

Last Update : 2009-02-04

 Howto Creation of a Stage 5

Table of Contents 

 Context 

 Differences between the Stage 4 and the Stage 5 

 Creation of a Stage 5 

  Creation of a Stage 5 Standard

 Creation of a Stage 5 Custom

 Advantages & Inconveniences 

Possible Options

Copy your  Stage 5 archive 

Restoration of a Stage 5 

Restoration of a Stage 5 Standard

Restoration of a Stage 5 Custom

1) Context  

Hello everyone, because of a awesome crash after a restoration of a Stage 4 archive, I decided to write this howto for those who don't wont problems.

The Stage 4 works perfectly in general, but if you are lucky like me, it will crash your Gentoo  :Evil or Very Mad: 

So, if you have a Gensplash, a /mnt directory, you will have a problem with the Stage 4, because he doesn't backup these directory by default.

Here's my  configuration :

```

#

# Sample boot menu configuration file

#

# Boot automatically after 30 secs.

timeout 5

# By default, boot the first entry.

default 1

splashimage=(hd0,8)/boot/grub/splash.xpm.gz

title=GentooLinux-2.6.18-r3

root=(hd0,8)

kernel /boot/kernel-2.6.18-gentoo-r3 root=/dev/sda9 video=vesafb:mtrr,ywrap,1024x768-32@85 splash=silent,fadein,theme:livecd-2006.1 quiet CONSOLE=/dev/tty1

initrd (hd0,8)/boot/fbsplash-livecd-2006.1

#kernel /boot/kernel-2.6.18-gentoo-r3 root=/dev/sda9 video=radeonfb:mtrr,ywrap,1024x768-32@75

title=WindowsXP

rootnoverify (hd0,0)

makeactive

chainloader +1

title=GentooLinux-2.6.18-r2

root=(hd0,8)

kernel /boot/kernel-2.6.18-gentoo-r2 root=/dev/sda9 video=radeonfb:mtrr,ywrap,1024x768-32@75 

```

```

# This file is edited by fstab-sync - see 'man fstab-sync' for details

# /etc/fstab: static file system information.

# $Header: /home/cvsroot/gentoo-src/rc-scripts/etc/fstab,v 1.14 2003/10/13 20:03:38 azarah Exp $

#

# noatime turns off atimes for increased performance (atimes normally aren't

# needed; notail increases performance of ReiserFS (at the expense of storage

# efficiency).  It's safe to drop the noatime options if you want and to

# switch between notail and tail freely.

# <fs>             <mountpoint>    <type>     <opts>            <dump/pass>

/dev/sda9      /      ext3      defaults,noatime   0 1

/dev/sda10      none      swap      sw         0 0

none         /proc      proc      defaults      0 0   

none         /dev/shm   tmpfs      nodev,nosuid,noexec   0 0

/dev/cdroms/cdrom0   /mnt/cdrom0   auto      noauto,user      0 0

/dev/cdroms/cdrom1   /mnt/cdrom1   auto      noauto,user      0 0

/dev/sda8      /mnt/pont   vfat      defaults,rw,user,umask=0 0 0

/dev/sda1      /mnt/win_c   ntfs      defaults,ro,user,nls=iso8859-1,nls=utf8,umask=0 0 0

/dev/sda5      /mnt/win_d   ntfs      defaults,ro,user,nls=iso8859-1,nls=utf8,umask=0 0 0   

/dev/sda6      /mnt/win_e   ntfs      defaults,ro,user,nls=iso8859-1,nls=utf8,umask=0 0 0

/dev/sda7      /mnt/win_f   ntfs      defaults,ro,user,nls=iso8859-1,nls=utf8,umask=0 0 0

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.

#/dev/BOOT      /boot      ext2      noauto,noatime      1 1

#/dev/ROOT      /      xfs      noatime         0 0

#/dev/SWAP      none      swap      sw         0 0

#/dev/cdroms/cdrom0   /mnt/cdrom   iso9660      noauto,ro      0 0

#/dev/fd0      /mnt/floppy   auto      noauto         0 0

#

# NOTE: The next line is critical for boot!

#none         /proc      proc      defaults      0 0

#

# glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for

# POSIX shared memory (shm_open, shm_unlink).

# (tmpfs is a dynamically expandable/shrinkable ramdisk, and will

#  use almost no memory if not populated with files)

# Adding the following line to /etc/fstab should take care of this:

#

#none         /dev/shm   tmpfs      defaults      0 0

#

#/dev/hdd                /media/cdrom            auto    user,exec,noauto,managed 0 0

#/dev/hdc                /media/cdrecorder       auto    user,exec,noauto,managed 0 0

#/dev/floppy/0           /media/floppy           auto    user,exec,noauto,managed 0 0

#/dev/sdb1               /media/sdb1             auto    user,exec,noauto,managed 0 0 

```

So, because of a catastrophic restoration, I decided to create a Stage 5 with the command tar and this will give a valid Stage 5 and a user will be able to restore in no time

and it will boot perfectly without any errors.

For the record, I will list the problems that I had after a restore of Stage 4 backup:

-A lot of messages about udev.rule, UDEV-event #.

-The script didn't backup /mnt, so in my case, I had a warning about the FSTAB.

-For a unknown reason, my Gensplash didn't start, because of a missing /dev/TTY1...so my Gensplash was disable. 

2) Differences between the Stage 4 and the Stage 5 

First of all, you can find the excellent  Stage 4 script here : http://blinkeye.ch/mediawiki/index.php/GNU/Linux_System_Backup_Script_%28stage4%29

The Stage 4, is a script that backup only the necessary directories to have a minimal functionable Gentoo.So, the Stage 4 doesn't take a ghost of a partition.Also, the Stage 4 never backup the directory /dev and if you have a Gensplash, you won't have /dev/console and /dev/tty1, and it will crash your Gensplash.

The Stage 5, is only uses the tar command, so you can do what you want.In fact, you can take a ghost of a partition or only take a backup of certain directory.

Like the Stage 4, the Stage 5 have a integrity check and the end of the process.

Finally, I wrote the Stage 5 for that purpose and I choosed the next number for the Stage.

3) Creation of  Stage 5  

Creation of a Stage 5 Standard

```

# su -

# rm /usr/portage/distfiles/*

# mkdir /mnt/backup

# mount -o bind / /mnt/backup

# cd /mnt/backup

# tar --exclude  stage5.tar.bz2 -cvvjpf stage5.tar.bz2 .

# time bzip2 -tv stage5.tar.bz2 

# umount /mnt/backup 

```

By the way, I saw a little bug inside my current box. 

If you have a /boot, before starting your tar command, double check inside /mnt/back/boot if you have something. 

If not, run this command : 

```
# mount -o bind  /boot /mnt/backup/boot 
```

The bind command is only working for a specific partition at once. 

So, I had a / for sda8 and a /boot for sda7 for example.

The "." after the .bz2 is very important  :Smile: 

The bzip2 command will check the integrity of the archive. 

The creation of the .tar.bz2 takes some time, so be patient. 

The Stage 5 act like Ghost or Acronis True Image.In fact, you can backup a complete partition.With this stage, you can backup a complete HDD and transfer  it to a new one without any problems.The technique can be use by administrator, because in case of a crash, they can restore only the partition that failed.

Creation of a Stage 5 Custom

The Stage 5 custom, is a Stage 5 with parameters.

Advantages & Inconveniences 

 This can backup a Gentoo installation or a complete partition and it can be use to backup only certain directories.

 It will be much faster, because it doesn't backup all the directories.

 You have to be very careful when you exclude directories, because it can crash your restoration, because you may have remove some files or directories that were essential for your Gentoo.

Possible options

Here a example of a command that exclude /tmp,/var/tmp & /usr/portage/distfiles 

 Note that you don't have to put / before the directory, because you have binded the / 

```

# tar --exclude tmp --exclude var/tmp --exclude usr/portage/distfiles --exclude stage5.tar.bz2 -cvvjpf stage5.tar.bz2 .

```

Also, certain users wont to have a Stage 5 that contain only the minimum directory, so for that here an example :

```

# rm -rf /usr/portage/distfiles/* && rm -rf /var/tmp/portage/* && rm -rf /var/log/portage/* 

# tar cvvjpf --exclude=dev --exclude=proc --exclude=sys --exclude=tmp --exclude=stage5.tar.bz2  stage5.tar.bz2 .

```

But, with this command, you will have to do some extra command in the restoration, because you will have to recreate certain directories.

```

# mkdir {dev,proc,tmp,sys} && chmod 1777 tmp && cd dev && MAKEDEV generic

```

Also, many users wont to backup only their /home directory.So here's the command :

```

# mount -o bind /home/$user /mnt/backup

# cd /mnt/backup

# tar --exclude stage5.tar.bz2 -cvvjpf stage5.tar.bz2 .

```

You can also backup everything except the /home directory :

```

# mount -o bind / /mnt/backup

# cd /mnt/backup

# tar --exclude stage5.tar.bz2 --exclude home/$user -cvvjpf stage5.tar.bz2 .

# time bzip2 -tv stage5.tar.bz2 

# umount /mnt/backup

# mount -o bind /home/$user /mnt/backup

# cd /mnt/backup

# tar --exclude stagehome.tar.bz2 -cvvjpf stagehome.tar.bz2 .

# time bzip2 -tv stagehome.tar.bz2 

# umount /mnt/backup

# mkdir /mnt/backup/full

# cd /

# cp stage5.tar.bz2 /mnt/backup/full/

# cp /home/$user/stagehome.tar.bz2 /mnt/backup/full

# tar -cvf stagefull.tar stage5.tar.bz2 stagehome.tar.bz2

```

4)Copy your  Stage 5 archive 

Now, you can burn the .tar.bz2 with K3B or you can leave this image on a separate partition. 

5)Restoration of a Stage 5 

Restoration of a Stage 5 Standard

The first thing to do is to boot with the Gentoo LiveCD and use the docache option.This will be useful, because we need

to umount the LiveCD

Boot with the Gentoo LiveCD. 

```

#gentoo docache

```

Now you have to format the destination partition 

```

# mke2fs -j /dev/sda9 dans mon cas.

#umount /mnt/cdrom

```

Remove the Gentoo LiveCD and put on your Stage 5 DVD backup. 

```

#mount /dev/cdrom /mnt/cdrom

#mount -t ext3 /dev/sda9 /mnt/gentoo dans mon cas

# cd /mnt/cdrom

# ls -la

# tar xjvpf /mnt/cdrom/stage5.tar.bz2 -C /mnt/gentoo/

# mount -t proc none /mnt/gentoo/proc

# mount -o bind /dev /mnt/gentoo/dev

# chroot /mnt/gentoo /bin/bash

# env-update

# source /etc/profile

# emerge --sync

# exit

# cd /

# umount /mnt/cdrom

# umount /mnt/gentoo/dev

# umount /mnt/gentoo/proc

# umount /mnt/gentoo

# reboot

```

 Now you will have a Gentoo back on track

Restoration of a Stage 5 Custom

The procedure of restoration of a Stage 5 Custom is almost the same as the Stage 5 Standard except some details. 

But, you will have to insert your own command to be sure that everything is ok, because at the tar command you will have to adapt your command.

Exemple :

```

#gentoo docache 

# mke2fs -j /dev/sda9 dans mon cas.

#umount /mnt/cdrom 

#mount /dev/cdrom /mnt/cdrom

#mount -t ext3 /dev/sda9 /mnt/gentoo dans mon cas

# cd /mnt/cdrom

# ls -la 

# tar xjvpf /mnt/cdrom/stage5.tar.bz2 -C /mnt/gentoo/

# mkdir {dev,proc,tmp,sys} 

# chmod 1777 tmp 

# cd dev 

# MAKEDEV generic 

# mount -t proc none /mnt/gentoo/proc 

# mount -o bind /dev /mnt/gentoo/dev

# chroot /mnt/gentoo /bin/bash

# env-update

# source /etc/profile

# emerge --sync

# exit

# cd /

# umount /mnt/cdrom

# umount /mnt/gentoo/dev

# umount /mnt/gentoo/proc

# umount /mnt/gentoo

# reboot

```

Last edited by d2_racing on Wed Feb 04, 2009 12:26 pm; edited 4 times in total

----------

## d2_racing

Any comments ???

----------

## Conan

Stage4 works just fine... the only difference is you are bindmounting to get the static devnodes, whereas they are excluded in a stage4 because most of /dev is dynamic (and therefore a waste of time to add.)  You could achieve the same results by copyng /dev/null and /dev/console from the livecd, or mknoding them.  The lack of these two devices is what caused you problems

----------

## d2_racing

So, why the Stage 4 doesn't copy these 2 files ???

----------

## Conan

because the wiki hasn't been updated... you could do -X stage4.excl --include /dev/null --include /dev/console I think, and that'd work just fine.

----------

## likewhoa

 *Conan wrote:*   

> Stage4 works just fine... the only difference is you are bindmounting to get the static devnodes, whereas they are excluded in a stage4 because most of /dev is dynamic (and therefore a waste of time to add.)  You could achieve the same results by copyng /dev/null and /dev/console from the livecd, or mknoding them.  The lack of these two devices is what caused you problems

 

you don't even have to copy /dev/null or /dev/console from livecd, what you do is mount your partition as normal but skip the 'mount -o bind /dev /mnt/gentoo/dev' part, then you just

cd into /dev and type 'MAKEDEV generic' and it will generate the require nodes for you  :Smile: 

----------

## hirakendu

frankly, i dont understand why so much effort. i do understand that the intricacy is only because it is being done from inside the working gentoo? what i do is boot using a live cd and simply do tarring along with gzip/bzip2/lzma on the root partition. and a simple extraction on a freshly formatted drive is enough for restoring.

so, for creating image :

$ cd /gentoo/   ( /gentoo/ being the mounted root partition containing bin, boot, usr, etc after booting with a livecd.)

$ tar -cvps --atime-preserve * | lzma e -si -so > /data/gentoo-img.tar.lzma (for a lzma archive)

(or)

$ tar -cvzps --atime-preserve -f /data/gentoo-img.tar.gz * (for gzip archive)

and for restoring from image :

$ mkreiserfs /dev/sdaX

$ mount /dev/sdaX /gentoo

$ cat /data/gentoo-img.tar.lzma | lzma d -si -so | tar -xps --atime-preserve -C /gentoo/ ( from lzma archive)

(or)

$ tar -xvzps --atime-preserve -f /data/gentoo-img.tar.gz -C /gentoo/ (from gzip archive)

and after that, the usual (re)installation of grub and editing some configs (fstab, grub.conf, net , hostname, xorg.conf) to one's liking, just as is done in case of stage3.

btw, i created an archive of a system buit with cflags "o2, prescott" for cpu's with sse3 +. and the image works fine on all machines, inlcuding athlon 64x2's. (yeah all those performance loss fundaes dont work - its 'the' same - atleast with tests like openssl and superpi. ~infinite uptimes, no crashes, stable , fast and responsive.) 

my system is about 5.0gb and it takes 15 minutes with gzip and 1hour,30mins with lzma to compress. while decompressing, gzip takes 15 minutes while lzma takes 7 minutes (flat!). btw, gzip compresses it to about 1.6 gb while lzma does it to about 1.1 gb. (this is on a athlon 64x2 3800+ with sata drive @ 67 MB/s (hdparm -tT) and a 16x dvd drive.)

hirakendu.

----------

## likewhoa

there is no need to perform the tar up from a livecd.

----------

## hirakendu

yes, i understand that its not necessary, but if its as simple as that from a livecd (and we dont mind rebooting the machine for a 'small' downtime of about 15 minutes for gzip or 2 hrs for lzma) rather than a painstaking script from the running system, i go for the livecd one. anyway, just a poor man's solution, never mind.

----------

## steveL

So does that work is simply cd / and run those cmds likewhoa? What about all the /dev stuff?

I just want a simple way to backup my existing gentoo before I transfer it to a new HDD. I'm guessing the Stage4 is the way to do it?

----------

## d2_racing

Yes, use theses commands and you will have a Stage 5 archive and you will be able to transfert to your new harddrive with less problems then a Stage 4.

----------

## Conan

Your "less problems" is just talking out of your hind end.

----------

## steveL

Let's not get rude here, yeah? Simply saying `that's not true' and (hopefully) giving reasons is better imo.

I still don't see how: tar -cvzps --atime-preserve -f /data/gentoo-img.tar.gz *

..avoids /dev and /proc for example.

Thanks.

----------

## Earthwings

 *Conan wrote:*   

> Your "less problems" is just talking out of your hind end.

 Conan, second warning from me now. Please improve your language or it will end in a ban sooner or later.

----------

## steveL

 *Earthwings wrote:*   

> Please improve your language or it will end in a ban sooner or later.

 Now why can't you guys moderate the dev list and bugzilla? That would make things a lot nicer  :Razz: 

----------

## onelife151

Thanks for posting the link for the stage 4 backup and your detailed explanation of your exotic back up strategy. I want to thank you to all of the gentoo community. Even though we have out bouts here and there, we are all here to learn and share information is this slighly less microsoft owned world.

----------

## d2_racing

 *onelife151 wrote:*   

> Thanks for posting the link for the stage 4 backup and your detailed explanation of your exotic back up strategy. I want to thank you to all of the gentoo community. Even though we have out bouts here and there, we are all here to learn and share information is this slighly less microsoft owned world.

 

No problem  :Smile: 

----------

## alexdu

 *d2_racing wrote:*   

>  Now you will have a Gentoo back on track. 
> 
> 

 

Hi.

I don't understand   :Confused: 

if someone needs to make/restore a *nix snapshot why do not use simple low level coping:

```
# cp /dev/hda1 ...

; or even

# cp /dev/hda ...

; later mount LiveCD and do back

# cp ... /dev/hda1

; or

# cp ... /dev/hda

```

you can pipe it any way you want - tar, scp, bzip...

Also, playing with tools like resize_reiserfs you can shrink or expand your partition. 

You can also change the physical order of restored partitions  - all you need is to fix /etc/fstab after restore. (and may be somewhere else if you use  raw hard disk access somewhere - it's unusual case)

If you want to change something in the saved binary image, you can mount it as a loop device: mount -o loop file... mountpoint.... , while it is a single partition and  not compressed or splitted.

----------

## d2_racing

 *alexdu wrote:*   

>  *d2_racing wrote:*    Now you will have a Gentoo back on track. 
> 
>  
> 
> Hi.
> ...

 

Yes, it's only an another methode  :Smile: 

It's what I love the most about Linux and Unix, you can choose your own path and you will arrive at destination every time  :Smile: 

----------

## steveL

 *alexdu wrote:*   

> f someone needs to make/restore a *nix snapshot why do not use simple low level coping:
> 
> ```
> # cp /dev/hda1 ...
> 
> ...

 Are you saying I can backup the whole of my system (on /dev/hdb) with:

# cp /dev/hda ~/backup/today/ *Quote:*   

> 
> 
> you can pipe it any way you want - tar, scp, bzip...
> 
> Also, playing with tools like resize_reiserfs you can shrink or expand your partition. 
> ...

 

Can you give examples of these?

----------

## d2_racing

 *steveL wrote:*   

>  *alexdu wrote:*   f someone needs to make/restore a *nix snapshot why do not use simple low level coping:
> 
> ```
> # cp /dev/hda1 ...
> 
> ...

 

Yes, send an example with the tar or something else plz  :Smile: 

----------

## likewhoa

 *steveL wrote:*   

> So does that work is simply cd / and run those cmds likewhoa? What about all the /dev stuff?
> 
> I just want a simple way to backup my existing gentoo before I transfer it to a new HDD. I'm guessing the Stage4 is the way to do it?

 

my method to backup/clone a system for transfering to another system of the same -march is simple..

```
tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --file stage4.tar.bz2 /
```

then simply scp that tarball to your new system,.. untar it, then simply cd into the /mnt/gentoo/dev directory and run 

```
MAKEDEV generic
```

 this will create the necessary nodes. after mount proc & dev then finally chroot to it. that's it; you should know the rest.  :Smile: 

there is really no need to tar/copy /dev,/sys,/proc and you should also delete files from /tmp,/usr/portage/distfiles,/var/tmp/portage, & /var/log/portage to keep the tarball small.

Hope this helps. This method is what i use to setup systems of the same -march and it works well.

----------

## d2_racing

 *likewhoa wrote:*   

> 
> 
> there is really no need to tar/copy /dev,/sys,/proc and you should also delete files from /tmp,/usr/portage/distfiles,/var/tmp/portage, & /var/log/portage to keep the tarball small.
> 
> Hope this helps. This method is what i use to setup systems of the same -march and it works well.

 

I don't have any problem to exclude what you have said, but the /dev is necessary, because if you have a framebuffer it will crash because of the messing node in /dev.

----------

## likewhoa

 *d2_racing wrote:*   

>  *likewhoa wrote:*   
> 
> there is really no need to tar/copy /dev,/sys,/proc and you should also delete files from /tmp,/usr/portage/distfiles,/var/tmp/portage, & /var/log/portage to keep the tarball small.
> 
> Hope this helps. This method is what i use to setup systems of the same -march and it works well. 
> ...

 

While it's ok to copy /sys & /proc you still don't have to and it's only optional as these folders will be regenerated on the new system. I stick to excluding them from my backups.

----------

## nth10sd

pardon my ignorance, i've been away awhile but what is stage 5?

stage 1 = everything incl. bootstrap

stage 2 = no bootstrap

stage 3 = precompiled binaries

stage 4 = ?

stage 5 = ??

Is it something to do with liveCDs?

----------

## d2_racing

 *nth10sd wrote:*   

> pardon my ignorance, i've been away awhile but what is stage 5?
> 
> stage 1 = everything incl. bootstrap
> 
> stage 2 = no bootstrap
> ...

 

Stage 4 is a backup stage from Blinkeye.

Stage 5 is an another methode to backup a existing installation.

I choose the next number to create this script.

----------

## nth10sd

ok thanks for the info!

----------

## d2_racing

No problem  :Smile: 

----------

## steveL

 *likewhoa wrote:*   

> my method to backup/clone a system for transfering to another system of the same -march is simple..
> 
> ```
> tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --file stage4.tar.bz2 /
> ```
> ...

 

Thanks for the info; you say we know the rest, but I'm not looking to do a new install, but copy to a new hard drive. I'm guessing that I do this from a live disk then set up grub and I'm good to go?

Is it ok to do this from a running system (I'm thinking of temp files in the home directory.) Also, it seems like this would be the definitive line?: 

```
tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --exclude=/tmp --exclude=/var/tmp --include=/dev/null --include=/dev/console --file stage4.tar.bz2 /
```

----------

## d2_racing

I think  :Smile: 

----------

## likewhoa

 *steveL wrote:*   

>  *likewhoa wrote:*   my method to backup/clone a system for transfering to another system of the same -march is simple..
> 
> ```
> tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --file stage4.tar.bz2 /
> ```
> ...

 

```

rm -rf /usr/portage/distfiles/* && rm -rf /var/tmp/portage/* && rm -rf /var/log/portage/* && tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --exclude=/tmp --file /stage4.tar.bz2 /

```

is more like it. then to restore boot with livecd then do the following steps after you have mounted the partition to /mnt/gentoo

```

cd /mnt/gentoo && tar xvjpf stage4.tar.bz2 && mkdir {dev,proc,tmp,sys} && chmod 1777 tmp && cd dev && MAKEDEV generic && mount -t proc none /mnt/gentoo/proc && mount -o bind /dev /mnt/gentoo/dev 

```

then just copy /etc/resolv.conf if needed them chroot to it, setup grub etc.. /sys will be regenerated for you, just add it to fstab.

----------

## d2_racing

I rewritted the howto, so if you have any comments or update that you want, just post  :Smile: 

----------

## likewhoa

thanks for the guide, a stage4/5 is a great way to clone a Gentoo server and rapidly deploy it on a server with the same arch.

----------

## alexdu

 *d2_racing wrote:*   

> 
> 
> Yes, send an example with the tar or something else plz 

 

Yeah, your are right, tar is not useful for raw disk images  :Smile: 

I guess

```
cat /dev/hda | bzip2 | ssh user@host "cat > /home/user/hda.backup.010107.bzip"
```

could be an example?

----------

## dR0PS

 *likewhoa wrote:*   

>  *steveL wrote:*    *likewhoa wrote:*   my method to backup/clone a system for transfering to another system of the same -march is simple..
> 
> ```
> tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --file stage4.tar.bz2 /
> ```
> ...

 

for me i got some headache while untarring with these tar options

```
cd /mnt/gentoo && tar xvjpf stage4.tar.bz2
```

 i had to use

```
cd /mnt/gentoo && tar xvjpfP stage4.tar.bz2
```

Note the capital P

elsewhise all softlinks that point to sth. with '/' at the target would get lost (pointing to nirvana)

.... so i believe thats important !!!!!

pls, correct if i'm wrong on here but at least only these options worked for me.

dR0PS

----------

## d2_racing

It's the first time that I heard about this P problem...

----------

## d2_racing

I will check this out tomorrow  :Smile: 

----------

## d2_racing

For my configuration, I don't need to add the P....

What kind of encoding do you use ?

----------

## Dieter@be

Why didn't you just edit the variables in the stage 4 script?

The values in there are meant to be defaults and users are able to add and remove directories as they wish

----------

## d2_racing

Because the Stage 4 doesn't backup a complete partition.

I don't want to include each and every directories in the Stage 4 script.

----------

## Dieter@be

Are you sure about that?

Of course it does not "backup partitions" (or file systems, which is probably what you mean?) but it recursively backups all the directories (mountpoints) you want if you just configure some variables.

 *Quote:*   

> 
> 
> $default_include_folders: folders which need to be included in the backup recursively for a minimal working system.
> 
> $custom_include_list: directories which are not imperative for a working system but which may be desirable to be also saved by your backup interactively (like /home or /usr/src/)
> ...

 

Unless the *_exclude_{list,pattern} variables match stuff you need, it should back up everything you want, or am I missing something?

http://blinkeye.ch/mediawiki/index.php/GNU/Linux_System_Backup_Script_%28stage4%29#How_to_customise

Also, backing up complete file systems makes a backup less convenient imho (less flexibility, fragmented state of file systems, ...) Especially since creating file systems is so easy to do (you could even script it  :Wink:  ) edit: hmm after studying your code a bit more you also need to create the new file system. I honestly don't see the use.

----------

## zxy

 *alexdu wrote:*   

>  *d2_racing wrote:*    Now you will have a Gentoo back on track. 
> 
>  
> 
> Hi.
> ...

 

Simple cp won't preserve atimes for example (and some other things too), and some software uses that. So when you restore the data you'll find yourself in some problems. 

I once read a nice document about this on the internet, but (by murphy) I can't find it now.  Maybe more luck later...

EDIT: Indeed...  here is the link describing (also) the possible problems with different ways of backing up unix systems: http://www.halfgaar.net/backing-up-unix

----------

## d2_racing

Thanks for the article  :Smile: 

----------

## Roman_Gruber

Hi

 *Quote:*   

> The Stage 5 act like Ghost or Acronis True Image.In fact, you can backup a complete partition.With this stage, you can backup a complete HDD and transfer it to a new one without any problems.The technique can be use by administrator, because in case of a crash, they can restore only the partition that failed. 

 

As I have seen Acronis True Image is  mentioned.

Does this script really reconstruct everything?

Does this script restore on a new individual harddisk my windo**s, my ntfs-data-partition, my root partition with ext3, my boot partition with ext2?

What does this script with partioning? Assume i want to restore it to a smaller/bigger hdd? Do I have to use gparted before? Is the MBR proper restored? Is grub proper restored?

Thanks, Roman

----------

## d2_racing

Ok, the Stage 5 is use to backup a *nix parittion.

So everything from ext2,ext3,ReiserFS etc... can do the job.

But, I didn't test for a FAT32 partition.

Also, you cannot bakcup a NTFS partition with the Stage 5 script and the MBR.

But, Acronis True Image can backup a HDD that constain a NTFS,EXT3,EXT2 and also a MBR without any troubles.

----------

## slycordinator

1) Why do "rm /usr/portage/distfiles/*"? Instead of removing your distfiles, just have that directory excluded.

2) The major reason that you created this script is because stage4 didn't (previously) copy /dev/console for you and currently doesn't copy /dev/tty1 for you. On the other hand, copying /dev/tty1 can be done with stage4 by adding it to the default include list.

----------

## likewhoa

for a succesfull boot you only need /dev/{console,null} /dev/tty is not mandatory to have, and udev would generate that anyways for you after the init sequence begins.

----------

## slycordinator

 *likewhoa wrote:*   

> for a succesfull boot you only need /dev/{console,null} /dev/tty is not mandatory to have, and udev would generate that anyways for you after the init sequence begins.

 

But apparently it is needed for gensplash.

----------

## likewhoa

agreed.

----------

## slycordinator

My point, though, was that a new backup method is unnecessary to get what he wanted.

----------

## cfgauss

 *likewhoa wrote:*   

> 
> 
> my method to backup/clone a system for transfering to another system of the same -march is simple..
> 
> ```
> ...

 

I tried this and have trouble after chroot when I boot into the new system. udev complains

```
Letting udev process events !!
```

 and then e2fsck complains it can't find /dev/hda3 and it throws me into root for a manual fsck. When I look into /dev, I can't find hda3. All that's there is 

```
console core fd null pts shm stderr stdin stdout tty1
```

.

Any pointers will be welcomed.

----------

## likewhoa

did you recompile your base system?

make sure to compile the kernel and check if you have the require IDE/SATA drivers.

----------

## cfgauss

 *likewhoa wrote:*   

> did you recompile your base system?
> 
> make sure to compile the kernel and check if you have the require IDE/SATA drivers.

 

I compiled the kernel and rebooted and got the same result. dmesg shows that the hard disk, hda, was detected and prints out the correct partition. But /dev still has only generic nodes and, in particular, hda isn't in there. When I try to mount /boot in the root shell that e2fsck throws me into, mount correctly complains that there is no /dev/hda1.

When do these device nodes get created? Would it work if my source backup included /dev?

Thanks.

[EDIT] Re-emerged udev and all is well. Thanks. [/EDIT]

----------

## likewhoa

 *cfgauss wrote:*   

>  *likewhoa wrote:*   did you recompile your base system?
> 
> make sure to compile the kernel and check if you have the require IDE/SATA drivers. 
> 
> I compiled the kernel and rebooted and got the same result. dmesg shows that the hard disk, hda, was detected and prints out the correct partition. But /dev still has only generic nodes and, in particular, hda isn't in there. When I try to mount /boot in the root shell that e2fsck throws me into, mount correctly complains that there is no /dev/hda1.
> ...

 

that's what i meant by did you "emerge your base system". that includes udev.

----------

## Kollin

thank you dear d2_racing your guide worked for me   :Wink: 

----------

## d2_racing

By the way, I saw a little bug inside my current box.

If you have a /boot, before starting your tar command, double check inside /mnt/back/boot if you have something.

If not, run this command :

```

# mount -o bind  /boot /mnt/backup/boot

```

The bind command is only working for a specific partition at once.

So, I had a / for sda8 and a /boot for sda7.

----------

## scouter389

Here is how I keep my bare metal backup of my laptop when i transferred my data to a new drive: 

 WARNING!! WARNING!! WARNING!! This command if mistyped can HOSE the host system! Always triple check the command before pressing enter!

this does require a lot of space on the host system.

I moved my laptop sata drive to my desktop and ran 

```
 dd if=/dev/sda of=/home/jawall/laptop.bare.img bs=512 
```

after the data was copied I disconnected the old drive and hooked up my new drive and ran 

```
 dd if=/home/jawall/laptop.bare.img of=/dev/sda bs=512 
```

This command will take several hours to run so don't get impatient over no status updates it IS working. 

To explain the command: dd is a command that does binary copying quite well

```
if=<filename>
```

 is the source file whether it is a device node or image file

```
of=<filename>
```

 is the output file 

```
bs=512
```

 tells dd to copy the hard drive sector by sector

for more ways to use dd run 

```
man dd
```

----------

## d2_racing

This backup the whole HDD and this will recreate the same partition from your old installation.

But if you want to be a little bit more flexible, then the Stage 5 can recover any backup partition from any size.

In fact, I had to backup a 10 Gig of data from /, and then I untar it on a brand new 100 Gig / without any warning or else.

DD is awesome to take an exact copy of a partition or a HDD.

----------

## minor_prophets

 *Quote:*   

> 
> 
> The Stage 4 works perfectly in general, but if you are lucky like me, it will crash your Gentoo 
> 
> 

 

d2_racing,

I'm a little unclear on your statement here.  How is it that Stage 4 crashed your Gentoo?  Are you implying that running the script destroyed your functional Gentoo system?  I've never had any problems with running the script myself.   :Shocked: 

----------

## d2_racing

No, I mean if you restore that Stage, if you have a Framebuffer and other stuff like Gensplash, the Stage 4 doesn't backup at all the /dev directory and other stuff.

With the Stage 5, you don't have this kind of problem.

----------

## xmaes

thanks it was usefull  :Smile: 

Just two questions

```
# tar --exclude  stage5.tar.bz2 -cvvjpf stage5.tar.bz2 . 
```

Why is "." so important and why 2 "v" in the tar command.

In your pdf on your website for the stage 5, there are 2 mistakes.

 *Quote:*   

> L'option bind est fonctionne seulement pour une partition a la fois.

 

and the command is missing a "-"

 *Quote:*   

> # tar -exclude stage5.tar.bz2 -cvvjpf stage5.tar.bz2 .

 

btw when i copy and paste the command i get 

 *Quote:*   

> # tar {exclude stage5.tar.bz2 -cvvjpf stage5.tar.bz2 .

 

not the minus signs.

Xavier

----------

## d2_racing

 *xmaes wrote:*   

> 
> 
> ```
> # tar --exclude  stage5.tar.bz2 -cvvjpf stage5.tar.bz2 . 
> ```
> ...

 

The . is important, because it will tar the current directory, so basically the /mnbt/backup

And the extra v makes the tar command more verbose.

----------

## d2_racing

 *xmaes wrote:*   

> 
> 
> btw when i copy and paste the command i get 
> 
>  *Quote:*   # tar {exclude stage5.tar.bz2 -cvvjpf stage5.tar.bz2 . 
> ...

 

Hi, can you post this :

```

# locale

$ locale

```

Have you the UTF-8 support, because I don't have your bug right now.

----------

## xmaes

here you go 

 *Quote:*   

> # locale
> 
> LANG=en_US.utf8
> 
> LC_CTYPE="en_US.utf8"
> ...

 

 *Quote:*   

> $ locale
> 
> LANG=en_US.utf8
> 
> LC_CTYPE="en_US.utf8"
> ...

 

----------

## Kollin

 *d2_racing wrote:*   

> By the way, I saw a little bug inside my current box.
> 
> If you have a /boot, before starting your tar command, double check inside /mnt/back/boot if you have something.
> 
> If not, run this command :
> ...

 

Can you add this line to the main guide please   :Rolling Eyes: 

----------

## SlashBeast

Imho backup should be from remounted rootfs

 *Quote:*   

> mkdir /tmp/real_rootfs
> 
> mount /dev/root /tmp/real_rootfs

 

remounted rootfs have not mounted nothing, empty /proc and /sys. /dev with static files, nothing mounted in /mnt or /media.

 *Quote:*   

> mizore /tmp # mkdir real_rootfs
> 
> mizore /tmp # mount /dev/root real_rootfs 
> 
> mizore /tmp # ls -l /tmp/real_rootfs/dev | wc -l
> ...

 

----------

## d2_racing

 *Kollin wrote:*   

>  *d2_racing wrote:*   By the way, I saw a little bug inside my current box.
> 
> If you have a /boot, before starting your tar command, double check inside /mnt/back/boot if you have something.
> 
> If not, run this command :
> ...

 

Done  :Razz: 

----------

## Kollin

 *d2_racing wrote:*   

> 
> 
> Done 

 

Thank you   :Very Happy: 

----------

## Snake

I have a problem extracting backup. A few days ago I didn't have any problems, but today I just can't extract bz2 file, some files extract but after some time I get error, something about reiserfs_priv (it is reiserfs partition). Could be the reason that I am using different boot CD that I used a few days ago when extracting went smoothly?

----------

## d2_racing

Hi, did you try the latest SystemRescueCD ?

I always use this one when I backup or restore my box.

----------

## Snake

Hi, I am using Gentoo minimal cd to boot, filesystem is made with mkreiserfs. The problem is that a few days ago there were no problems with extraction on the same computer. This is error:

```
tar: ./.reiserfs_priv: Cannot mkdir: Permission denied

tar: Error exit delayed from prevoius errors

```

Edit: it is the same with SystemRescueCD. As I understand reiserfs_priv is hidden folder with filesystem files, which can't be modified. What I can't understand is why I didn't have any problems the first time with the same bz2 file on the same partition .

----------

## LoSeR_5150

Thank you d2_racing your guide worked great for me to re-do my raid array from raid1 to raid0 ... I somehow managed to miss my /boot directory (must have forgot to bind it) but installing grub/memtest86 and re-compiling a kernel was super easy.  Too me 300min to tar/bzip2 45Gb and 100min to restore.  Much better than a full re-install... Thanks again.

----------

## d2_racing

No problem, thanks for your comment  :Razz: 

----------

## pingpongboss

Am I right in assuming that this method of backing up would work in any other distribution?

----------

## d2_racing

In fact, it's universal, I used this method to backup my FreeBSD 7.2 and everything is working #1.

It's the swiss knife for backup  :Razz: 

----------

## slycordinator

 *d2_racing wrote:*   

> No, I mean if you restore that Stage, if you have a Framebuffer and other stuff like Gensplash, the Stage 4 doesn't backup at all the /dev directory and other stuff.
> 
> With the Stage 5, you don't have this kind of problem.

 Not to beat a dead horse (since this got discussed in another thread) but it seems that backing up the entire /dev to get a couple nodes is overkill.

Also, the stage4 guide tells you how to create all the basic /dev nodes you'd need, including the tty1 that gensplash uses (just this one is marked optional since only gensplash uses will need it.

----------

## d2_racing

In fact, I know that too, but for someone who doesn't want to take extra care, then it's the simpliest way  :Razz: 

----------

## slycordinator

Touché, though it would seem like a good idea to remove the unneeded nodes from the archive (and that'll likely be a significant space-saver since nearly ever node is unneeded for the archive).

----------

## timofonic

Any hopes?

----------

