# hd copy with dd

## thecooptoo

HD is failing so i want to  make a copy of it onto a new HD 

ie

 *Quote:*   

>  dd  if=/dev/hda of=/dev/hdb

 

how long should this take ( 40 GB HD)

can I do this on a still-running  machine  ( ie our router) ?

are ther any better ways of doing it ?

eg partition the new HD and copy the partitions separately

----------

## frostschutz

It's better to copy the partitions, since it's easier to create partitions of the same size, rather than getting same-sized HDDs. Besides, this way you can choose what to copy first (personal data partition before system data partition), and what not to copy at all (swap partition).

If the source device is failing, it's better to add error conversion options to dd:

```
dd conv=noerror,sync if=/dev/hda1 of=/dev/hdb1
```

How long it should take is hard to say for a dying disk... the one I had could still read 80% of it's data correctly, but at slow speed only (2-3MB/s instead of 30MB/s). So it can take very long. You can have dd output the progress using the USR1 signal (I think it was) though.

----------

## thecooptoo

```
grenada ~ # fdisk /dev/hda

The number of cylinders for this disk is set to 1245.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/hda: 10.2 GB, 10248118272 bytes

255 heads, 63 sectors/track, 1245 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System

/dev/hda1               1           7       56196   83  Linux

/dev/hda2               8          70      506047+  82  Linux swap / Solaris

/dev/hda3              71        1245     9438187+  83  Linux

Command (m for help): q

grenada ~ # fdisk /dev/hdb

The number of cylinders for this disk is set to 1027.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/hdb: 8455 MB, 8455200768 bytes

255 heads, 63 sectors/track, 1027 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System

/dev/hdb1               1           7       56196   83  Linux

/dev/hdb2               8          70      506047+  82  Linux swap / Solaris

/dev/hdb3              71        1245     9438187+  83  Linux

Command (m for help): q

grenada ~ # dd                                                                                        
```

but ive got 

```
grenada ~ # df /dev/hda1

Filesystem           1K-blocks      Used Available Use% Mounted on

udev                    257980        52    257928   1% /dev

grenada ~ # df /dev/hda2

Filesystem           1K-blocks      Used Available Use% Mounted on

udev                    257980        52    257928   1% /dev

grenada ~ # df /dev/hda3

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/hda3              9288792   3043492   5773392  35% /

grenada ~ #                                          
```

Im only using 35 % of the failing disk.

I assume /dev/hd?1 &2  are the same size 

How do I copy the contents of /dev/hda3 to /dev/hdb3 ( which I assume are different sizes

----------

## frostschutz

I think it's actually better to copy the whole partition, rather than the files. This is because when copying a partition, the hard disk doesn't have to do any seeking - it can just keep reading data on and on. When copying files, it always first has to seek to the beginning of the file before it can start reading it. This is a lot of unnecessary extra work for a dying hard disk... What's more, copying files works only on the file system layer, which may not be able to handle disk errors as "smoothly" as a raw stream reader like dd does.

If you still want to copy files rather than partition for some reason, a common way would be to use plain old cp -a or tar -p.

----------

## thecooptoo

```

grenada ~ # dd if=/dev/hda1  of=/dev/hdb1

112392+0 records in

112392+0 records out

grenada ~ # dd if=/dev/hda2 of=/dev/hdb2

dd: writing to `/dev/hdb2': No space left on device

3+0 records in

2+0 records out

grenada ~ # dd if=/dev/hda3  of=/dev/hdb3

dd: writing to `/dev/hdb3': No space left on device

515345+0 records in

515344+0 records out

grenada ~ #

```

----------

## morodoch

You could always try partimage, booting from a rescue CD with both hard drives attached.

http://www.partimage.org/

I use the system rescue cd:

http://www.sysresccd.org/

I used to use knoppix for such things, but the sysresccd is based on gentoo, and has a closer match to the gentoo install for things like lvm and such.

----------

## thecooptoo

I was going to try and cp the stuff across.

```

grenada ~ # mount -t ext3 /dev/hdb3 /mnt/newdisk/

mount: /dev/hdb3 is not a block device (maybe try `-o loop'?)

grenada ~ #

       
```

and then just to check 

```

grenada ~ # mke2fs -j /dev/hdb3

mke2fs 1.38 (30-Jun-2005)

/dev/hdb3 is not a block special device.

grenada ~ #                                

```

----------

## morodoch

What does 

```
ls -l /dev/hdb3
```

 show?

----------

## thecooptoo

```

grenada ~ # ls -l /dev/hdb3

-rw-r--r--  1 root root 263856128 Dec 10 15:26 /dev/hdb3

grenada ~ #                                                         
```

----------

## morodoch

I see what your problem is - hdb3 is just a file & that's why you ran out of space.

It should be a block device (with a "b" in the first column).

You've copied your partition to a normal file in /dev.

I'm not sure why there is no device file there! what does ls -l /dev/hdb* show?

----------

## thecooptoo

```
grenada ~ # ls -l /dev/hdb*

brw-rw----  1 root disk     3, 64 Dec 10 12:12 /dev/hdb

brw-rw----  1 root disk     3, 65 Dec 10 12:12 /dev/hdb1

brw-rw----  1 root disk     3, 66 Dec 10 12:12 /dev/hdb2

-rw-r--r--  1 root root 263856128 Dec 10 15:26 /dev/hdb3

brw-rw----  1 root disk     3, 69 Dec 10 12:12 /dev/hdb5

brw-rw----  1 root disk     3, 70 Dec 10 12:12 /dev/hdb6

brw-rw----  1 root disk     3, 71 Dec 10 12:12 /dev/hdb7

grenada ~ #                                                         
```

but i dont get this - see the output of fdisk /dev/hdb above

----------

## morodoch

 *Quote:*   

>  see the output of fdisk /dev/hdb above

 

That's what was confusing me - /dev/hdb3 should be a block special file, but it's just a plain old file. You didn't accidentally rm it at any time, did you? That's the only thing I can think of.

To re-create it, run this command:

```
rm /dev/hdb3

mknod /dev/hdb3 b 3 67
```

An ls -l /dev/hdb* should show something like this:

```
ls -l /dev/hdb*

brw-rw----  1 root disk 3, 64 Mar  4  2005 /dev/hdb

brw-rw----  1 root disk 3, 65 Mar  4  2005 /dev/hdb1

brw-rw----  1 root disk 3, 66 Mar  4  2005 /dev/hdb2

brw-rw----  1 root disk 3, 67 Mar  4  2005 /dev/hdb3

brw-rw----  1 root disk 3, 68 Mar  4  2005 /dev/hdb4

brw-rw----  1 root disk 3, 69 Mar  4  2005 /dev/hdb5

brw-rw----  1 root disk 3, 70 Mar  4  2005 /dev/hdb6

brw-rw----  1 root disk 3, 71 Mar  4  2005 /dev/hdb7
```

Once you have your block device back, you may be back up-and-running with dd.

----------

## thecooptoo

```

grenada ~ # rm /dev/hdb3

grenada ~ # mknod /dev/hdb3 b 3 67

grenada ~ # ls -l /dev/hdb*

brw-rw----  1 root disk 3, 64 Dec 10 12:12 /dev/hdb

brw-rw----  1 root disk 3, 65 Dec 10 12:12 /dev/hdb1

brw-rw----  1 root disk 3, 66 Dec 10 12:12 /dev/hdb2

brw-r--r--  1 root root 3, 67 Dec 10 16:37 /dev/hdb3

brw-rw----  1 root disk 3, 69 Dec 10 12:12 /dev/hdb5

brw-rw----  1 root disk 3, 70 Dec 10 12:12 /dev/hdb6

brw-rw----  1 root disk 3, 71 Dec 10 12:12 /dev/hdb7

grenada ~ # dd if=/dev/hda3  of=/dev/hdb3

dd: opening `/dev/hdb3': No such device or address

grenada ~ #

```

----------

## morodoch

OK, the permissions are a bit wrong, but you should still be able to do your dd as root.

----------

## thecooptoo

```
grenada ~ # whoami

root

grenada ~ # dd if=/dev/hda3  of=/dev/hdb3

dd: opening `/dev/hdb3': No such device or address

grenada ~ #

```

----------

## morodoch

Well that *is* odd - I guess that somehow your kernel may have lost visibility of the device. Does cfdisk still work on hdb? Maybe a reboot is in order...

----------

## thecooptoo

cfdisk /dev/hdb givers a blank screen with this at the bottom of the page 

FATAL ERROR: Bad primary partition 2: Partition ends after end-of-disk

                                               Press any key to exit cfdisk      

but 

```
grenada fern # fdisk /dev/hdb

The number of cylinders for this disk is set to 1027.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/hdb: 8455 MB, 8455200768 bytes

255 heads, 63 sectors/track, 1027 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System

/dev/hdb1               1           7       56196   83  Linux

/dev/hdb2               8          70      506047+  82  Linux swap / Solaris

/dev/hdb3              71        1245     9438187+  83  Linux

Command (m for help):                         
```

----------

## morodoch

How much hassle would it be to recreate the partition table on hdb, and start again? (It is hdb you're copying *to*, isn't it?)

It sounds like your partition table has been corrupted. If you don't mind starting from scratch, I suggest this:

1. zero out your partition table with 

```
dd if=/dev/zero of=/dev/hdb count=2
```

 MAKE SURE YOU USE THE RIGHT DEVICE HERE !   :Shocked: 

2. reboot

3. create your new partitions with cfdisk

4. go back to using dd to copy your data again

----------

## thecooptoo

Ive been wondering whether to just start again 

Thanks for the help . I'll let you know how I get on

----------

## thecooptoo

with the new HD as a slave I deleted the old partitions and  recreated them . cfdisk now reads the new partitions

```
                                                                cfdisk 2.12i

                                                              Disk Drive: /dev/hdb

                                                         Size: 8455200768 bytes, 8455 MB

                                              Heads: 16   Sectors per Track: 63   Cylinders: 16383

       Name                  Flags               Part Type          FS Type                       [Label]                    Size (MB)

 -----------------------------------------------------------------------------------------------------------------------------------------------

       hdb1                                       Primary           Linux ext3                                                   32.52

       hdb2                                       Primary           Linux swap / Solaris                                        512.49

       hdb3                                       Primary           Linux                                                      7910.21

```

but 

```
grenada ~ # ls -l /dev/hd*

brw-rw----  1 root disk     3,  0 Dec 11 12:46 /dev/hda

brw-rw----  1 root disk     3,  1 Dec 11 12:46 /dev/hda1

brw-rw----  1 root disk     3,  2 Dec 11 12:46 /dev/hda2

brw-rw----  1 root disk     3,  3 Dec 11 12:46 /dev/hda3

brw-rw----  1 root disk     3, 64 Dec 11 12:46 /dev/hdb

-rw-r--r--  1 root root  57544704 Dec 11 12:55 /dev/hdb1

-rw-r--r--  1 root root 206323712 Dec 11 12:55 /dev/hdb2

-rw-r--r--  1 root root         0 Dec 11 12:56 /dev/hdb3

grenada ~ #                                    
```

why are the new partitions not being created as block devices ?

----------

## morodoch

This is *really* weird.

Is this what your /dev/hd* looks like after a reboot?

----------

## thecooptoo

reboot sorted it 

```
grenada ~ # ls -l /dev/hd*

brw-rw----  1 root disk 3,  0 Dec 11 20:09 /dev/hda

brw-rw----  1 root disk 3,  1 Dec 11 20:09 /dev/hda1

brw-rw----  1 root disk 3,  2 Dec 11 20:09 /dev/hda2

brw-rw----  1 root disk 3,  3 Dec 11 20:09 /dev/hda3

brw-rw----  1 root disk 3, 64 Dec 11 20:09 /dev/hdb

brw-rw----  1 root disk 3, 65 Dec 11 20:09 /dev/hdb1

brw-rw----  1 root disk 3, 66 Dec 11 20:09 /dev/hdb2

brw-rw----  1 root disk 3, 67 Dec 11 20:09 /dev/hdb3

grenada ~ #

```

when I do 

```
dd conv=noerrors,sync if=/dev/hda2 of=/dev/hdb2
```

 the HD clicks a lot and hangs - I assume this is the partition where the problem is.

Do i need to copy the contents of a swap partition over ?

the new /hdb3 is smaller than the old /hda3 so i assume dd will run out of space 

I assume I have to copy the files across with cp  but when i try that i get 

```
grenada ~ # mount -t ext3 /dev/hdb3 /mnt/newdisk

```

----------

## frostschutz

No, you don't need to copy swap partitions.

----------

## morodoch

You just need to mkswap the new swap partition.

If you can mount both the old partition and the new partition, you should be able to do something like this to copy the files over:

 *Quote:*   

> cp -ax /mnt/oldpartition/. /mnt/newpartition/.

 

----------

## thecooptoo

thanks 

what NOT to do is 

cp -Rp  / /mnt/newdisk

it copies all the symlinked files as well ( I did RTFM but clearly didnt understand it ) 

I'll have another go tomorrow

----------

## morodoch

The -x means "stay in this filesystem", so I think cp -ax will do the trick. It'll copy symlinks as symlinks - it should make your new filesystem identical to the old - permissions, everything.

----------

## thecooptoo

and it doesnt work !

what I did :

partition the new HD with the same FS as the old one 

create new partitions with fdisk

old disk 

 *Quote:*   

> grenada ~ # fdisk /dev/hda
> 
> The number of cylinders for this disk is set to 1245.
> 
> There is nothing wrong with that, but this is larger than 1024,
> ...

 

new disk grenada ~ # fdisk /dev/hdb

 *Quote:*   

> 
> 
> The number of cylinders for this disk is set to 16383.
> 
> There is nothing wrong with that, but this is larger than 1024,
> ...

 

the new partitions arent exactly the same size as the old ones - all a bit bigger, icouldnt get it to partition them to exactly the same size 

I then dd copied existing -> new

as the new hdb3 is smaller than the old hda3  I cp -ax'd the files across.( dd runs out of space !)

ive reset the jumper on the new HD to master, disconnected the old one, plugged the cable into the new HD and on boting it the BIOS doesnt detect it.

however, when its mounted as a slave  its detected by dmesg

```
grenada ~ # dmesg |grep hdb

    ide0: BM-DMA at 0xe000-0xe007, BIOS settings: hda:DMA, hdb:DMA

hdb: WDC AC28400R, ATA DISK drive

hdb: max request size: 128KiB

hdb: 16514064 sectors (8455 MB) w/512KiB Cache, CHS=16383/16/63, UDMA(33)

hdb: cache flushes not supported

 hdb: hdb1 hdb2 hdb3

grenada ~ #

```

so what has gone wrong ?

----------

## morodoch

 *Quote:*   

> ive reset the jumper on the new HD to master, disconnected the old one, plugged the cable into the new HD and on boting it the BIOS doesnt detect it. 

 

Do you mean the BIOS can't see it at all, or just can't boot from it? If it can see it, but not boot from it, the most likely reason is that you haven't copied the MBR across, or run grub/lilo/... on the new disc?

----------

## thecooptoo

the bios doesn't detect it and thinks there is no HD 

But just dd'ing the boot partition should have copied over grub etc (shouldn't it) ?

----------

## morodoch

 *Quote:*   

> But just dd'ing the boot partition should have copied over grub etc (shouldn't it) ?

 

Yes & no  :Smile: 

Yes it copies over the settings, but you still have to make the MBR boot the setup.

Did you install grub to the MBR first time, or to the install partition?

----------

