# how to ghost a disk[SOLVED]

## jserink

Hi All:

I bought an Akai ANB-9000 eePC type of thing for my kids, 359 singapore dollars, ~US$200. Had Linpus on it and I have installed gentoo over top of that. I've got everything working on it and its proved to be an exceppent little gamin machine for the kids so I bought another one. Exactly the same thing and I'd like to ghost the first machine over to the next without having to go through the entire gentoo install process. I have 3 partitions, boot, swap and root.

Any ideas on how to do this over the ethernet?

Cheers,

JohnLast edited by jserink on Fri Mar 05, 2010 2:08 pm; edited 1 time in total

----------

## Jaglover

sys-block/partimage

Haven't used it for a while but IIRC it is even network-aware.

----------

## d2_racing

partiimage can do the job.

----------

## Mad Merlin

```
dd if=/dev/sda | ssh newbox 'dd of=/dev/sda'
```

----------

## d2_racing

I'm gonna try that one one day  :Razz: 

----------

## jserink

 *Mad Merlin wrote:*   

> 
> 
> ```
> dd if=/dev/sda | ssh newbox 'dd of=/dev/sda'
> ```
> ...

 

this looks conceptually funky and straight forward.

Seems I'll I'd need to do is fire up the gentoo startup disk on the target, get ssh running and on the host just run that command.

the only downside is that it will copy every byte from the host to the target, 160Gbytes worth.

Whereas part image "should" only copy used areas of the disk which should make the amount copied much smaller.

cheers,

john

----------

## bbgermany

As an addition, you could use netcat as well. See this page for information: http://compsoc.dur.ac.uk/~djw/tarpipe.html

At first, you need to create the partitions on the new host. Afterwards you just need to restore your bootloader. This is a quite fast solution (did this several times).

bb

----------

## Mad Merlin

 *jserink wrote:*   

>  *Mad Merlin wrote:*   
> 
> ```
> dd if=/dev/sda | ssh newbox 'dd of=/dev/sda'
> ```
> ...

 

If you want to spend more human time and less computer time, you can create (and mount) the partitions first and then tar:

```
cd /mnt/gentoo; tar cp . | ssh newbox 'tar xvp -C /mnt/gentoo'
```

Edit: You'll also need to install the bootloader on the new machine as well.

----------

## mikegpitt

I've never tried dd over an ssh connection, so I can't attest to its reliability.  If you choose that method please do post back and let us know how it went.

One tip, if you want to speed up dd, change the block size...  I regularly do this to clone disks on the same machine:

```
dd if=/dev/sda of=/dev/sdb bs=4096
```

I've not noticed any speed boost going above 4096, but this will cut the clone time from hours to probably around .5 hours (depending on machine speed).

Another thing you might want to try to save some time, is set up your partitions, make a tarball of your current install, transfer the tarball to the new machine and unpack, and set up grub.  It won't be a disk clone, but it will have the same effect.

----------

## d2_racing

 *mikegpitt wrote:*   

> I've never tried dd over an ssh connection, so I can't attest to its reliability.  If you choose that method please do post back and let us know how it went.

 

In fact, it's the first time that I see that too.

----------

## NeddySeagoon

mikegpitt, jserink

dd over ssh works as described but as has ben said, it copies every byte.

You can also use tar over ssh to just copy the files. Don't bother with the -j or -z options to tar. Its faster to transfer uncompresed dtata than run the compression at both ends.

This page will get you started.

Edit

I made an image of my laptop with tar over ssh.

Note to would be crackers ... its a P3 laptop and the image is served by a SPARC system.  Also /etc/shadow is fake.

----------

## d2_racing

Thanks for the link Roy, ssh copy , it's the geek way to copy data over a lan or the internet  :Razz: 

----------

## LesCoke

I have used tar over ssh; I can say it works very well.  But I have noticed that using -j or -z compression can be faster in cases where you have files that are very compressible.

Les

----------

## NeddySeagoon

LesCoke,

I expect it depends on the CPUs at each end ... when I tested -j wiyj my laptop image, the server was a 450MHz k6-2.

Doing the encryption for ssh and the decompression for tar -j was just too much for it.

----------

## Paczesiowa

use live cds (or usb distro) on both machines, then do

```
cat /dev/zero > /mnt/gentoo/null_file; rm /mnt/gentoo/null_file
```

after that, use usual solution with "dd if= | gzip | nc -> | nc -l | zcat | dd of="

----------

## LesCoke

Okay, a k6-2@450, I can believe.  I had experience with a 350 MHz version and I swear, I had a 40 MHz 386-SX2 that would out run it for some tasks.  When booting windows 98SE that K6-2 350, would seem to go dead for almost 5 minutes while initializing the network stack.

The systems I most recently ran tar over ssh were both PIII class, one was a dual 800 MHz, the other had a single 1GHz.  I had a 500 G barracuda drive start showing pending sectors, and needed to copy my data off so I could write patterns to the drive in an attempt to force smart to replace the bad sectors.  The dual PIII machine is my daily use machine, so I needed to do the copy in the background and still be able to do what I could with the one drive down.  The drive in question is in a removable drive bay, and my other machine had a similar bay, so I installed the spare drive in a suitable tray, put the faltering drive in the second machine, and the new drive in my main machine.  I prepared the partition; and launched a live distro on the second machine, mounted, and launched the tar over ssh back into my main machine.

I also questioned if compression would help or hinder, so I first tried without, and I could see based on the drive activity that it was not getting the usual transfer rate, so I stopped it and tried (-j) compression, much better.  I didn't really dig into where the bottleneck was without the compression, but I have in the past played with piping one tar into another when duplicating partitions on the same machine.  In the same machine case (my dual PIII), using compression was also faster.   Perhaps not as fast as a dd from one drive to another, but a pure duplicate was not what I was after, I wanted each file copied one at a time to the new partition, to effect a poor man's defrag.

The reason compression causes a faster transfer is two fold:

when dealing with typical data, compression yields an average 50% reduction in size.  Text documents yield closer to 90%.

long streams of identical data, get sent in short hand ( write XX N times ).

Pipes in memory buffers, or socket connections over a LAN have finite bandwidth, Less data, equals less time, provided you have a CPU fast enough to do the compression and still keep the communications path saturated.  In my case; the barracuda drives claim a sustained rate of roughly 70 Megabytes / second; my LAN only 100 Megabits or 10 Megabytes / second.  I routinely see scp achieving rates very near saturation on these machines.  Both machines are maxed out on RAM, so I have plenty for buffering.

Les

----------

## jserink

Right, first off, thank you to Mad Merlin for the suggestion of using dd with ssh, that is absolutely brilliant.

I had some problems with this whole process due to a bad PSU and then with the fact that the Gentoo Live CD on USB hangs on this particular platform after about 4 hours so I could never get it finished(at work on an IEI mini ITX AMD board the gentoo live USB runs for days on end, go figure). I put knoppix on USB, booted, changed the root pw, started ssh, and it ran fine.

I also consulted http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/ which is about the best practical page I've ever seen on dd, well worth a bookmark.

First I did this as proof of concept using the gentoo live CD/USB on the target:

dd if=/dev/sda | ssh 192.168.3.5 'dd of=/dev/sda' bs=1500 count=666667

as it would only take a few minutes over 100Mbps Etherent.

Rebooted the target and sure enough, grub came up just like the host after which the target fell over as I had only copied ~1G of the 160G disk. here is the session:

kids kids # dd if=/dev/sda | ssh 192.168.3.5 'dd of=/dev/sda' bs=1500 count=666667

The authenticity of host '192.168.3.5 (192.168.3.5)' can't be established.

RSA key fingerprint is b5:03:51:97:44:bc:fc:b8:64:f9:fd:83:f9:6c:96:a1.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.3.5' (RSA) to the list of known hosts.

Password: 

641837+24830 records in

641837+24830 records out

993808788 bytes (994 MB) copied, 147.78 s, 6.7 MB/s

and here is the session that actually worked:

kids kids # dd if=/dev/sda | ssh 192.168.1.20 'dd of=/dev/sda' bs=8192 conv=notrunc

root@192.168.1.20's password: 

312581808+0 records in

312581808+0 records out

160041885696 bytes (160 GB) copied, 15876.1 s, 10.1 MB/s

as you can see it took about 4.5 hours. Over 1G Etherent this would have taken only 30 minutes so its something worth looking at. The trinity Rescue kit (http://trinityhome.org) has a utility where it uses multicast so that you could ghost multiple machines in one go say for a university or school. Worth having a look at.

This technique is so powerful that at work I am now using it to ghost windows boxes we have in racks rather reinstalling everytime. Just need to run a utility to change the windoze serial number to that on the sticker of the particular machine.

SATA to SATA is of course much faster than over ethernet and is recommended if possible.

Thanx everyone for all the help and suggestions. This is what i love about the Gentoo forum, you just learn and learn.

Cheers,

john

In then end, what

----------

## bobspencer123

you could also try  clonezilla  for an open source norton ghost like alternative.

----------

## jserink

 *bobspencer123 wrote:*   

> you could also try  clonezilla  for an open source norton ghost like alternative.

 

I looked at clonezilla but decided against as well as a few others.

Upon reflection I could not understand why I should need to install and entire package to replace a single line Linux command....

Didn't make much sense.

Cheers,

john

----------

## gsoe

 *Quote:*   

> I looked at clonezilla but decided against as well as a few others.
> 
> Upon reflection I could not understand why I should need to install and entire package to replace a single line Linux command.... 

 

You don't have to. It runs off a bootable CD on the client machine. You only have to install it if you're going to setup a server to ghost multiple machines simultaneously.

----------

## ppain

 *Mad Merlin wrote:*   

> 
> 
> ```
> dd if=/dev/sda | ssh newbox 'dd of=/dev/sda'
> ```
> ...

 

Sorry - i know this is an old thread to dig up, but will 

```
dd if=/dev/sda dd of=./[ext hdd]BACKUP.img
```

 write over all the data in the ./ HDD? Is there any way to do a copy and still keep the other files in your external drive there? Or will it all be overwritten/wiped?

(This is assuming that i want to create a BACKUP.img on an external hdd with data that i want to keep.)

----------

## jserink

 *ppain wrote:*   

>  *Mad Merlin wrote:*   
> 
> ```
> dd if=/dev/sda | ssh newbox 'dd of=/dev/sda'
> ```
> ...

 

if you want to make sure not to harm any existing files, write to mounted partition so that the file system is running things. Like this for example:

dd if=/dev/sda dd of=/mnt/mydisk/BACKUP.img

dd with write the output file to /mnt/mydisk/BACKUP.img. make sure mydisk has enough space else this will fail.

Cheers,

john

----------

