# Cloning HDD...

## Endo_

I've been reading up on this cause i have 2 computer which i want to have linux, but dont wanna have to build them both (yea, im a lazy bastard lol). Most guides say to use the "dd" and have identical hardware, so i had an idea and i wanna know if its a good one or not. Some people might feel the urge to say "RTFM and try it out!" ... but id rather ask and see if its good/bad then have to pay for a new drive :p 

First boot into a LiveCD / Rescue disk on both comps so hdd arnt loaded.

Create Partitions on the SLAVE big enough for the files.

Next start ssh on 'SLAVE' and connect from 'MASTER'

Then basicly use "cp" to copy all the files from /dev/hda over... except the swap since that would be a waste of bandwidth :p

so basicly cp /dev/hda1 /dev/hda1  (sorry, but i dont know the EXACT commands and stuff to use for ssh since ive never really used it ;P)

would this approach fix the needs-to-be-identical factor? would it work at all? lol

----------

## u2mike

I've never tried it myself. But simply copying everything over should work fine. Just make sure the kernel is compiled with support for the hardware on both machines. Also, make sure your cflags will work with both processors. Otherwise your good to go. Once you've copied everything you will need to boot of the livecd and edit the fstab on the second pc, then run lilo/grub on the to ensure it boots properly.

Another tip, make sure you use -pr (ie "cp -pr /bla /bla") the p makes sure the permissions are the same, and the r makes it recursive   :Wink: 

Also, the you can't copy files via ssh, you will need to setup a nfs share and mount it that way. Although the easyer way to do it would be to take the hardrive out of the second and put it in the first. It will be easyer to copy and MUCH faster. Once your done you just swap it back and edit the fstab and run lilo, simple as that.

----------

## jdgill0

Actually I have "copied" an installation before. The nice thing about Linux vs Windows is such a thing IS possible.

-------------------------------------------------------------------------------------

Boot the new system with the livecd.

Boot old system normally.

Partition the new drive however you want.

Mount the partitions of both drives on their respective computers.

(I assume you can get networking up and running on both systems.)

You want to copy everything but /proc and /dev.

However do create those folders on the new system.

Use scp for the copying, be sure to use the -rp flags to do recursive directory copying and preserve permissions of the files.

Grab a stage3 tarball for the new system (if you don't have it handy already).

Copy the stage3 to the new system's / directory:

tar xvjpf stage3* ./dev

(/proc will be filled on bootup, no need to mess with it, just make sure you create the folder /proc)

Once copying is done, you will need to setup the boot loader on the new system, Follow gentoo docs on chrooting into the new system and setting up the boot loader. Also, now is the time to reconfigure the kernel if you need to, along with anyother specifics that need changing for the new system.

-------------------------------------------------------------------------------------

Alternative is to boot old system with livecd too, then you can scp /dev as well.

----------

## Kope

maybe it's just me, but I really don't trust cp for cloning disks. Granted, the Gnu cp is probably just fine for this purpose, but it's not precisely universal as it relies on Gnu specific extentions, and I'd hate to learn that way to do it and then find myself on a box that didn't use Gnu's cp and be stuck trying to learn how to do it.

This, on the other hand, should work on any unix system 

cd /old/home && find . -print0 | cpio -pdm /new/home

I think the only thing that command line COULD break is if you have files with newline characters in the name . .. but if you do you should be shot on general principles anyway.

----------

## Endo_

Well i was saying to use both systems on a livecd so that /dev and /proc cant be copied since the drive isnt loaded... 

i assumed since dd was able to ssh data to another drive on the network that it could also be done with cp, my bad. 

 *Kope wrote:*   

> 
> 
> cd /old/home && find . -print0 | cpio -pdm /new/home
> 
> 

 

not to sure what this is doing, i will research the command a bit more later tonight. Thanks for the input guys. 

PS. the hardware is almost the exact same for both machines so all compiled stuff SHOULD work  :Wink: 

----------

## jdgill0

Although it sounds like you want something else, I thought I would throw this tip out:

One way to copy a drive is to use RAID ..... You don't need to be able to boot the system ....... install BOTH drives on the raid controller ... on bootup enter into the RAID BIOS setup program, then tell it to MIRROR the first drive onto the second drive ... Now I can't remember if I had to go back and install grub to the master boot record or not (seems like I did not have to, since I believe RAID makes a complete low level copy of the drive) ... Only thing is, you have to have RAID  :Smile: 

----------

## flickerfly

I found this guide that uses dd and netcat, but it doesn't look like netcat is on the LiveCD. Maybe that's something that could change.

http://www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html

Is there something else that would work just the same as netcat that is on the cd?

----------

## jdgill0

That's a very nice guide, thanks flickerfly.

I have a question though. Within the guide the author shows copying the MBR (master boot record) by copying the first 446 bytes of the drive.

Quoting from http://www.geocities.com/rlcomp_1999/procedures/mbr.html

 *Quote:*   

> Note: An MBR has 3 parts: 
> 
> (000000 - 0001bd) The boot code which is 446 bytes long
> 
> (0001be - 0001fd) The partition table which is 64 bytes long
> ...

 

Doesn't this suggest you should copy the first 512 bytes instead of 446?

One other point about cloning NTFS partitions, does this mean you could actually copy a MS Windows installation and it work aftwards? If so, this has to be the easiest/cheapest method of cloning MS Windows installations. MS Windows admins should take notice! Normally cloning a MS Windows installation under MS Windows itself would require commercial software like Ghost.

----------

## flickerfly

I have successfully run the command 'dd of=/dev/hda if=/dev/hdb' on a win XP HD using the Gentoo LiveCD and have had no problems with it at all. It does take awhile though because it copies the entire disk, not just valid data. I believe Ghost is faster.

I have yet to get this working via NICs like the guide I posted does, but I should have that taken care of by the end of the day if I can find a boot CD with both netcat and dd on it.

Oh, the boot sector is copied as part of /dev/hda directly onto the disk. I don't think you have to worry about it. If you are only copying that, I don't know the answer.

----------

## xbmodder

no u re partion ur other disk then boot up normally

lets say original is hda

backup is hdb

fdisk /dev/hdb

u know the drill partion it then mkefs

mkdir /mnt/hdb

mount /dev/hdb3 /dev/hdb

su -

(type ur root password)

cp -rf / /mnt/hdb/

mount /dev/hdb1 /mnt/hdb/boot

mount -t proc none /mnt/hdb/proc

chroot /mnt/hdb

env-update

source /etc/profile

remake kernel 

install bootloader ur good to go  :Smile: 

----------

## flickerfly

I managed to put netcat on a usb stick and make the guide I posted work. My exact steps are at the link below. I don't know that it'll actually boot when I'm done, but it's looking good so far.

http://josiah.ritchietribe.net/pmwiki/index.php/Tech/RemoteCloning4Gentoo

Edit: This resulted in a kernel panic because my root directory has a messed up superblock, but the kernel did boot until that point which caused a kernel panic.

At least my /boot partition came across ok.

Edit (X2): After running fsck on my root partition, everything seems to be fine. In retrospect, perhaps I should not have copied a running system, but it's working now.

----------

## Mikerosoft

I was about to try the same thing with two of my systems.  I have me office workstation setup with Gentoo and everything I need.  Rather than trynig out the 2004.2 CD's I thought if setup a spare hard drive and used Partimage to image everything over to the second drive, that I *should* be able to get the second machine to boot.  I have built a kernel with support for all the hardware on both machines.  Will this work or should I just bite the bullet and build the second system from scratch? Thanks.  :Smile: 

----------

## flickerfly

Depends on if you compiled the stuff with compatible CFLAGS, I think. I'm not an expert in that area. I've only done it on identical machines. If they are close and you already have it booting, I'd think you should be able to pull it off.

----------

## Mikerosoft

The system booted for me but not after a bit of trouble.  I booted a 2004.2 live CD and updated /etc/fstab with the new mounting information for the partition scheme of the new drive.  Next I installed grub to the MBR and after rebooting I got a kernel panic, no init found.  I forgot to update grub.conf root partition.  It works great now, and it was much easier and faster then building another stage one system.  

I recommend that anybody wanting to try this read through the install handbook again.  With the handbook and this forum it is all you could ever need.  The Gentoo community really is the greatest.  Thanks everyone!

----------

## jdgill0

 *flickerfly wrote:*   

> ... In retrospect, perhaps I should not have copied a running system, but it's working now.

 

I know you can not copy certain directories with cp, cpio, rsync, etc while the system is running. But, I was expecting to be able to use dd on the running system. I could see where you might want zero activity while running dd. Did you come to a final conclusion that you should not use dd on a running system?

----------

## jdgill0

 *flickerfly wrote:*   

> ... In retrospect, perhaps I should not have copied a running system, but it's working now.

 

I know you can not copy certain directories with cp, cpio, rsync, etc while the system is running. But, I was expecting to be able to use dd on the running system. I could see where you might want zero activity while running dd. Did you come to a final conclusion that you should not use dd on a running system?

----------

## flickerfly

Yeah, but the problems that ocurred were all recoverable using fsck. My system being copied was almost entirely inactive. I hadn't tried it on a non-running system. If someone does I"d like to hear a quick report to comfirm/deny this.

----------

## gi1242

Well my wonderfull hard disk developed badblocks  :Smile: . I had to copy my instalation over to another hard disk. I did it painlessly with

```
cp -xRp / /mnt/hdb3

```

Ofcourse if you have a seperate boot / usr partition(s), you should do the same for each of those partitions. Install grub in your mbr of the new hard disk (info grub, or use the Gentoo Handbook) and you're good to go. I wanted to replace my old hard disk, so I just switched the jumpers. But if you want to move the hard disk to a different machine, that should be fine too.

If you mount ext2 on a DVD+RW / CD-RW (dvd+rw-tools), then you can directly copy your hard disk to the DVD-RW instead (if there's enough space) and then transfer it to as many machines as you want. The only downside is that you have to reinstall grub on every machine. I unfortunately don't know a way to copy the MBR over. But I'm just a n00b. I'm sure there are a few gurus here that can enlighten us  :Smile: 

GI

----------

## jdgill0

Has anyone tried using UDPCast? ... see: http://www.udpcast.linux.lu/ .... I discovered this tool while looking at a Linux distro that claimed its existance for copying/imaging installations: http://www.halfling.org/phrealon/ ... it happens that udpcast is an ebuild.

From the UDPCast website:

 *Quote:*   

> UDPcast is a file transfer tool that can send data simultaneously to many destinations on a LAN. This can for instance be used to install entire classrooms of PC's at once. The advantage of UDPcast over using other methods (nfs, ftp, whatever) is that UDPcast uses Ethernet's multicast abilities: it won't take longer to install 15 machines than it would to install just 2.

 

----------

## flickerfly

 *jdgill0 wrote:*   

> Has anyone tried using UDPCast? ... see: http://www.udpcast.linux.lu/ .... I discovered this tool while looking at a Linux distro that claimed its existance for copying/imaging installations: http://www.halfling.org/phrealon/ ... it happens that udpcast is an ebuild.
> 
> From the UDPCast website:
> 
>  *Quote:*   UDPcast is a file transfer tool that can send data simultaneously to many destinations on a LAN. This can for instance be used to install entire classrooms of PC's at once. The advantage of UDPcast over using other methods (nfs, ftp, whatever) is that UDPcast uses Ethernet's multicast abilities: it won't take longer to install 15 machines than it would to install just 2. 

 

That looks very cool. If you get a chance to work with it, I'd like to hear how it went. Does it require images made by itself specially or will any tar.gz file do the trick or what? Is it possible to boot the machines with a floppy for this?

----------

## jdgill0

Not yet messed with UDPCast ... but I did come up with an "easy" method of getting netcat onto a livecd.

The following method was actually easier when the livecds used zisofs instead of squashfs ... squashfs requires kernel support for the squashfs filesystem, unlike zisofs ... zisofs-tools has a nice uncompress method to uncompress the zisofs image on the livecd, whereas to access the contents of the livecd.squashfs image you need to actually mount the image to a loop device  :Sad: .

NOTE: before beginning you will need squashfs support in your kernel ... I had to updgrade from Linux 2.6.7 to Linux 2.6.8 (I use gentoo-dev-sources) in order to get compatibility between the kernel's squashfs and the livecd's livecd.squashfs ... you will need a Gentoo LiveCD iso ... I use install-x86-minimal-2004.2.iso ... and of course you will need loop device support in your kernel.

I will use 3 tmp directories, use whatever you like:

```
mkdir  /tmp1  /tmp2  /tmp3

emerge squashfs-tools  ... at this time, you should get v2.0

emerge netcat

mount -o loop install-x86-minimal-2004.2.iso /tmp1 ... read-only

cd /tmp1

mount -o loop livecd.squashfs /tmp2 ... read-only

cd /tmp2

tar czf /tmp3/os.tgz *  ... tar overcame problem I had copying dev folder

cd /tmp3

tar xzf os.tgz

rm os.tgz

cp /usr/bin/nc usr/bin/

mksquashfs  *  livecd.squashfs

umount  /tmp2

cp -a  /tmp1/*  /tmp2/

cd  /tmp2

rm  livecd.squashfs  isolinux/boot.cat

cp  /tmp3/livecd.squashfs ./

mkisofs -pad -l -r -J -v -V "LIVECD" -no-emul-boot -boot-load-size 4 \

        -boot-info-table -b isolinux/isolinux.bin -c isolinux/boot.cat \

        -hide-rr-moved -o livecd.iso ./

```

You now have a new livecd iso image, livecd.iso, that contains netcat.Last edited by jdgill0 on Mon Aug 30, 2004 1:59 pm; edited 1 time in total

----------

## papal_authority

flickerfly:

Your dd of=/dev/hda if=/dev/hdb method of drive cloning sounds great. A question though, does it copy the MBR and the partition table? I'm looking to make an exact duplicate of several drives. This would be awesome if the drive can be blank. In the past I've always used fdisk, then newfs, mount and copy each partition. To copy I've used:

```

   # cd /OLD_DIR_ON_ORIGINAL_HD

   # tar cvf - . | ( cd /NEW_DIR_ON_NEW_HD; tar -xvf - )

```

With alot of partitions, this is a PITA. Your way sounds slick  :Very Happy: 

----------

## flickerfly

 *papal_authority wrote:*   

> flickerfly:
> 
> Your dd of=/dev/hda if=/dev/hdb method of drive cloning sounds great. A question though, does it copy the MBR and the partition table? I'm looking to make an exact duplicate of several drives. This would be awesome if the drive can be blank. In the past I've always used fdisk, then newfs, mount and copy each partition. To copy I've used:
> 
> ```
> ...

 

lol, it's hardly "my" way. I just heard talk about it and figured it out. To answer your question. You shouldn't need to do any MBR stuff at all because it literally copies the 1001010101001 from the disk to the same spot on the new one. It doesn't understand file structures etc. A warning, if you copy a running system you might have to run fsck after install, but that would be it. No extra stuff needed. It is very slick, but another caveat, it is very slow.

----------

## jdgill0

UDPCast:

Well I have given udpcast a test drive ... unfortunately I don't have spare hardware at this time to test using dd to literally copy a disk image from one computer to the next, but I did use udpcast to send a file from one computer to the next, and I did test dd dumping it to stdout.

```
emerge udpcast
```

This will give you two binaries, udp-sender and udp-receiver ... I emerged on both of my computers, set my shorewall rules to allow udp over port 9000 (the default, but changable).

Scenario:  I will transfer a text file from computer A to computer B.

On computer B:

```
udp-receiver | cat >> /file.txt
```

It will then wait for a sender signal.

On comptuer A:

```
udp-sender --interface eth1 -f file.txt
```

My local lan is on eth1, udp-sender defaults to eth0

At this point, I am given a message on computer A to press any key to start transfer ... upon hitting any key, the receiver computers will know.

NOTE: computer B (the receiver computers) will also have a message, after starting the sender, to hit any key to start transfer ... as I only have two computers, I don't know if the transfer will start for all receivers if the transfer is initiated from a receiver computer.

So, I hit a key on computer A and the transfer was broadcast over my lan and received by computer B ... The file transferred just fine!

Now I did try the following test to see what dd would do:

Computer A

```
dd if=/dev/hda2 | udp-sender --interface eth1
```

Computer B

```
upd-receiver 
```

Since upd-receiver defaults to stdout, I DID see the transfer take place, so I am assuming you could

```
upd-receiver | dd of=/dev/hdx
```

You could follow my guide above to add updcast (sender,receiver) to a couple livecd's ... or follow flickerfly's usb stick method

If you have mutliple machines you would like to copy to, just start udp-receiver on each machine ... when you start upd-sender, ALL receiver machines will get the transfer simultaneously! ... very cool in my opinion.

If you did not wish to broadcast to the entire network, udp-sender does have options to limit to a single machine transfer.

----------

## flickerfly

Thanks jdgillo. That looks like it could be a very useful tool to add to my toolbox.

----------

