# Hard disk duplication

## tecknojunky

Tight space?  Need to transfer your system onto a new bigger hard drive?

These 5 easy and safe steps will transfer the data on the current hard drive onto a new bigger one.  The last partition will hold the newly aquired free space.

I drawn this from my own setup which is based upon the x86 installation howto from Gentoo.  My hard drive was partition this way:

- /dev/hda1 : ext2 boot partition

- /dev/hda2 : swap partition

- /dev/hda3 : reiserfs root partition.

Also, I have a dvd-rom on /dev/hdb and the new disk was the primary drive on the second controler: /dev/hdc.  Where is located the new disk is irrelevant.  Just be sure you know that you are accessing the new drive and not another disk that hold crucial data!!!

These steps insure that your hda is untouched and will serve as a backup until you are sure the new drive is fonctionnal.

Prerequisites: make sure you have the tools for each of the file systems on the drive.  Also, a partition program is essential (fdisk, cfdisk, ...).

1- Tranfer the data.

```
# dd if=/dev/hda of=/dev/hdc
```

This copies integraly the whole drive onto the new one, including the MBR and partition table.  It can take a very long time depending on bus speed and drive size.

2- Fix the partition table.

```
# cfdisk /dev/hdc
```

You can use any disk partition program you wish.

You will see that on the new drive, you'll have the same partition table with additionnal free space trailing it.  

Make the boot partitition bootable if it is not so.

Delete the last partition and recreate it using the whole free space.  Write the partition table.  This only change the partition table, not the data in the partition.

3- Resize the last partition.

Depending on the type of file system you had, use the proper tool to resize it.  I had reiserfs:

```
# resize_reiserfs /dev/hdc3
```

4- Check file system integrity

```
# reiserfsck --fix-fixable /dev/hdc3

# e2fsck /dev/hdc1
```

5- Swap the drive and reboot.

Working?  Congrat, you succefully duplicated your disk.

This is based on my setup.  I'm guessing other setups may fail with this.  If so, post your unsuccess here (with preferably a fix).

----------

## nsahoo

how did you know that I needed this tutorial ?  :Wink: 

thanks.

----------

## nsahoo

 *tecknojunky wrote:*   

> 
> 
> 5- Swap the drive and reboot.
> 
> 

 

do you mean i'll have to change the jumper settings (primary/secondary/cable) of the hard disks?

----------

## nsahoo

 *tecknojunky wrote:*   

> 
> 
> 1- Tranfer the data.
> 
> ```
> ...

 

Does it copy the free space in /dev/hda too ?

----------

## hertog

You seem to have missed this excelent scipt:

https://forums.gentoo.org/viewtopic.php?t=28123&highlight=backup

----------

## nsahoo

i finally did it .. woohooo ..    :Very Happy: 

----------

## hawson

I have to take issue with a few thing in this method of duplicating disks.  Having done numerous disk replications and migrations, I have to say that following the instructions above is a Bad Idea.

First of all, using 'dd' to copy data is fine ONLY when the hard disks are absolutely identical.  Minor disk revisions don't cut it; they must be exactly the same.

Your suggestion of using  *Quote:*   

> 
> 
> ```
> dd if=/dev/hda of=/dev/hdc
> ```
> ...

  is incorrect, except in the case of identical hardware.  As mentioned in an earlier post, this will also copy the partition table, so in this caseyou don't need the extra step of partitioning the new disk (see below for more details...).  The biggest problem with this method is that it will not properly configure the new disk if it has a different geometry.  For example, given that hda is 40GB and hdc is 80GB, running 

```
dd if=/dev/hda of=/dev/hdc
```

 will copy only the 40GB of data on the source disk, and leave the remaining 40GB untouched, unformated, and unusable.  Furthermore, since the disk geometry is necessarily different, I would be wary of the filesystem on the news disk.  Essentially, using this method gives you an under-utilized, potentially unstable new hard disk.

On a performance note, the default settings for 'dd' are to use 512byte blocks for data transfer.  This is going to take a loooong time on any file larger than a few MBs.  Adding the 'bs=' option will help.  Something like: 

```
dd if=/dev/hda of=/dev/hdc bs=2048k
```

 will speed things up quite a bit.  There is some arcane magic that goes into finding the optimal block size, but some large number usually suffices (not really, but you can look it up on Google yourself   :Very Happy: )

When I have to clone disks, I usually do something like this (assuming source is hda, dest is hdc):

Ideally, boot off of a floppy disk or CDROM.  You want the source disk to completely quiescent while you do this.  Tomsrtbt works quite well for this, as would Knoppix, or a Gentoo CD.  You can also do this from single user mode, after making absolutely sure you have shutdown everything you possibly can (all you need running, really, is a shell).

Take a look at the source partition table for reference:

```
fdisk -l /dev/hda
```

Parititon the new disk to taste.  If you are shrinking any partitions on the new disk, make sure that you don't inadvertently make any of them too small to hold the source data.

```
fdisk /dev/hdc
```

Mount the new paritions so they are in the desired arrangement.  Given something like this for the sake of example:  hda1 == /boot, hda2 == /, hda3 == /var, hda5 == /home, hda5 == swap, and the partitions on   hdc are in the same arrangement and mount points, I would do the following:

```
mkdir /mnt/newdisk

mount /dev/hdc2 /mnt/newdisk

mkdir /mnt/newdisk/boot

mkdir /mnt/newdisk/var

mkdir /mnt/newdisk/home

mount /dev/hdc1 /mnt/newdisk/boot

mount /dev/hdc3 /mnt/newdisk/var

mount /dev/hdc5 /mnt/newdisk/home

```

You should now be ready to actually copy the data.  I use tar, but cpio or would work just as well (if not better).  The one snag here is that you need to copy only stuff on the source disk, you don't want to recursively copy the data in the /mnt/newdisk directory, nor things like /proc that don't really exist  :Wink: .  These commands usually work:

```
cd /

tar cpf - bin boot dev etc home lib opt root sbin tmp usr var | ( cd /mnt/newdisk;   tar xpvf -)
```

This will take a long time, and every file copied should scroll by.  Be sure that you use the 'p' flag to tar, lest you loose all of your file ownership information.  Needless to say, this must be done as root.

If you use devfs, then you can omit the /dev/ directory from the tar invocation.  You should also add any other top-level directories that are needed, and remove any that may not be approriate (such as /tmp and /var/tmp for example).

A nice side effect of using tar/cpio instead of dd is that it should defragment the disk somewhat.  :Cool: 

At this point, you should create any directories that you omitted in the last step, as well as any other initialization steps that need to be done--such as running 'mkswap' on your swap partitions.

```
cd /mnt/newdisk

mkdir /proc

mkswap /dev/hdc5
```

I'm paranoid, so I reinstall the boot loader at this point.  When you do so, make sure that you reinstall to hdc, and not hda.  Check the LILO/GRUB docs for specifics on this.

Finally, unmount the new disk, poweroff, swap the disks, and start everything up.  With some luck, it will work the first time.  If you are feeling adventurous, you can play clever tricks with kernel root= options, /etc/fstab to boot off of /dev/hdc before you swap disks.  However, if you are willing to try that, you don't need my help.  :Smile: .

There are a *LOT* of scripts out there that take care of this for you, and I'd recommend looking at one of them if you don't feel comfortable with doing it manually.  If you have to do a lot of hard disks, say for a computer lab, check out Dolly.  It creates a "ring" (of sorts) between all of the hosts sending data around.  SourceBox --> HostA --> HostB -->HostC --> .... --> SourceBox.  It's quite nice once it gets going.  You can even use it to clone non-Linux systems.

Good luck!

----------

## nsahoo

 *hawson wrote:*   

> 
> 
> The biggest problem with this method is that it will not properly configure the new disk if it has a different geometry.  For example, given that hda is 40GB and hdc is 80GB, running 
> 
> ```
> ...

 

We are deleting the last partition and making a new partition in its place including the free space..  or else we can just create a new partition using the free space. whats wrong with that ?

Can you please clarify why the disk is going to be unstable ?

----------

## hawson

 *Quote:*   

> We are deleting the last partition and making a new partition in its place including the free space.. or else we can just create a new partition using the free space. whats wrong with that ? 

 

My apologies, I missed that part.  However, I'm not sure it matters...see below.

 *Quote:*   

> Can you please clarify why the disk is going to be unstable ?

 

Well, based on doing the following on different sized drives:

```
dd if=/dev/hda of=/dev/hdc
```

That can potentially screw things up if the hardware doesn't have the same geometry.  

When you have a larger destination drive than source disk, you will only copy as much is on the source drive.  Let's say you have a 20GB source disk, and 60GB destination disk.  The 'dd' command will only copy 20GB to the destination disk, leaving some 40GB uninitalized and unused.  If you do manage to get a valid partition table out of this, you will have to manually create new partitions to make use of the remaining space (which, to be fair, is what you are planning on doing).  I think you can get overlapping partitions if you do this as well--again, not something you want to do unless you really know what you are doing.

If the source disk is larger than the destination disk (60GB copied to 20GB for example), you have more insidious problems.  The 'dd' command will likely complete without error, although if you check the 'XX records in / XX records out' message, you might notice a discrepancy (I know I sure wouldn't).  In this case, the first 20GB will be copied, but the remaining 40GB of raw disk space will be silently dropped.  It is quite possible to use a a drive built like this...for a short while.  Sooner or later you are going to try and access something that exists beyond that that 20GB limit, and there will lots of interesting kernel errors in your logs.  :Smile:   Bear in mind that most filesystems spread their data out across the available space for various reasons (performance, fragmentation avoidance, etc).  Thus, even though you may only have 5GB of data stored on that 60GB partition, it will likely be spread out over all of the availabe 60GB of space.

(FWIW, I've tested both of these, and neither has worked.  The smaller --> larger copy gave me a screwed up partition table, and the larger --> smaller copy first silently lost data, then threw a ton of errors.  However, I didn't expect either to actually work, nor should they.  :Smile: )

Furthermore, since the drive geometries are going to be different, the partition boundries are going to be completely out of whack, and may not even make sense for the new drive.  I will admit to not knowing the hard-core details for partition tables, but I believe that partitions are defined as offsets in terms of either cylinders or sectors, relative to the geometry of a particular disk.  So, 100 cylinders on one disk usually is not the same as 100 cylinders on a second disk.  Thus, when you copy the partition table using 'dd', it's not going to work.  So, even if you do nuke and remake the last partition on the disk (hdc3), it may work, but the rest likely will not, again, because the geometries won't match the partition table.

It's also a bad idea to fdisk partitions with data on them (no, I don't trust partition resizers very much).

Anyhow, there are better ways to clone disks than 'dd' for different hardware.  One final advantage to using tar or cpio is that you will only copy what you care about, and not gigabytes of empty space.  5GB used of a 20GB partition is going to waste 3/4 of the time shoving around bits you don't care about in the first place.

So, all of that said, I have used 

```
dd if=/dev/hda | nc 192.168.1.1 1234
```

 and 

```
nc -l -p 1234 | dd of=/dev/hda
```

 to shove disk images over the network though.   :Smile: 

----------

## Garbz

Hey can i get some input on how i would do this to a windows NTFS / FAT32 drive.

Yesterday i was put into a partiton where we are going to upgrade dad's 10gb drive to a shiny new 40gb (well not new but shiny all the same  :Laughing:  )

Anyway the layout is:

/dev/hda1   NTFS   4gb

/dev/hda2   FAT32  6gb

/dev/hdc     Just been lowleveled.

The problem is i don't want to keep the partition sizes.  Is it even possible to end up with:

/dev/hdc1   NTFS 7gb

/dev/hdc2   FAT32 13gb   without re-formatting ?  I hate installing windows, and there sure as well isn't any easy way to back the bastard up.

----------

## really

I agree that is is bad to just dd a whole disk onto another one. Youll be copying the unnecesary 0's too and wont have that little defragment usual cp'ing whould cause.

I did it like this...

Old drive, 80gig. New drive 160gig.

Becouse i am going to use the 80 drive for the os and my home, and the 160 gig one only for mp3s movies and such things.

I creaed four primary partitions with cfdisk on it, sized, 20, 23, 40 and 80 gigs.

On my old drive i had music in the home directory under /home/listen which was about 60gigs and also contained lots of other files, well my user dir which is about 4gigs, the home partition also contained /home/ftp which was took about 20gigs itself...

making a filesystem on the hdb1-4 partitions (the 160gig one)

and creating /warez/music /warez/motion /warez/motion/cartoons /warez/other

mounting hda1 to music hdb2 to motion and so on, as you like.

Then just simply enough, cp -Rpi /home/listen/* /warez/music/ and so on...

I also cp'ed my user dir to teh otehr one (for defragment purposes and testing speed, then cp back over.

Did the same thing for /usr and others...

Works perfectly.

----------

## really

I agree that is is bad to just dd a whole disk onto another one. Youll be copying the unnecesary 0's too and wont have that little defragment usual cp'ing whould cause.

I did it like this...

Old drive, 80gig. New drive 160gig.

Becouse i am going to use the 80 drive for the os and my home, and the 160 gig one only for mp3s movies and such things.

I creaed four primary partitions with cfdisk on it, sized, 20, 23, 40 and 80 gigs.

On my old drive i had music in the home directory under /home/listen which was about 60gigs and also contained lots of other files, well my user dir which is about 4gigs, the home partition also contained /home/ftp which was took about 20gigs itself...

making a filesystem on the hdb1-4 partitions (the 160gig one)

and creating /warez/music /warez/motion /warez/motion/cartoons /warez/other

mounting hda1 to music hdb2 to motion and so on, as you like.

Then just simply enough, cp -Rpi /home/listen/* /warez/music/ and so on...

I also cp'ed my user dir to teh otehr one (for defragment purposes and testing speed, then cp back over.

Did the same thing for /usr and others...

Works perfectly.

----------

## karnesky

 *Garbz wrote:*   

> Hey can i get some input on how i would do this to a windows NTFS / FAT32 drive.

 Easiest way is to use partimage, which is in portage, on knoppix, and on the sysrescuecd.  Partimage is a Norton Ghost-like program that works with all of the major file systems.  It isn't always the best method of disk cloning, but it is usually better than dd & is almost always easy.

----------

## veezi

 *Garbz wrote:*   

> Hey can i get some input on how i would do this to a windows NTFS / FAT32 drive.
> 
> Yesterday i was put into a partiton where we are going to upgrade dad's 10gb drive to a shiny new 40gb (well not new but shiny all the same  )
> 
> Anyway the layout is:
> ...

 

The ntfsprogs package might be useful to you. Particularly the 'ntfsclone' and 'ntfsresize' tools.

NOTE: Where did you get that image under your name from? Look cool  :Smile: 

Cheers,

----------

## Garbz

hate to be a spoil sport but after 11 months i've actually installed my drives and got the computer up and running   :Rolling Eyes: 

----------

## Garbz

hate to be a spoil sport but after 11 months i've actually installed my drives and got the computer up and running   :Rolling Eyes: 

----------

## veezi

Oooooops  :Embarassed:  I did it again

Hey, since you're still alive after 11 months  :Very Happy: , What about your forum image? I like it, where did you get it from?  

Now, you aren't going to wait 11 months before answering that just to teeze me right?!  :Very Happy: 

Cheers,

----------

## Garbz

lol

all my forum images in various forums are from Azumanga Daioh anime characters.

I've used Chio and Osaka since my sister rekons they are just like me.  Chio is smart, and osaka has her head in the clouds.

I've basically cropped and resized wallpapers. Google them to find some there's some really super ones out there.

----------

