# Linux kernel software RAID mdadm: It Really Works!!!

## Lupin_the_3rd

No questions or problems here, just wanted to post the kernel messages from a recent drive failure in my RAID-1 mirror.  Everything worked exactly as it should, in fact, I didn't even realize a drive had failed until I ran `dmesg` to see the kernel messages as part of my routine system check.  This is on my DEC Alpha workstation.

One thing to take away from this, is the benefit that Linux kernel software RAID provides - it protects not only against a complete drive failure - but also against bad sectors - if a sector on one drive becomes unreadable, it recovers the data from the other RAID member(s)!  With a single drive (i.e. without RAID) you will lose the contents of any bad sectors, resulting in data corruption.

Anyhow, enough rambling, here's the log messages I found interesting.  You can see it performing sector redirection, and then eventually booting the failed drive out of the array.  We can all read about the theory of how it's supposed to work, but actually seeing it work is far more interesting:

end_request: I/O error, dev sdb, sector 29367344

md/raid1:md0: sdb1: rescheduling sector 29365296

md/raid1:md0: redirecting sector 29365336 to other mirror: sdc1

md/raid1:md0: redirecting sector 29365328 to other mirror: sdc1

md/raid1:md0: redirecting sector 29365320 to other mirror: sdc1

md/raid1:md0: redirecting sector 29365312 to other mirror: sdc1

md/raid1:md0: redirecting sector 29365304 to other mirror: sdc1

md/raid1:md0: redirecting sector 29365296 to other mirror: sdc1

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 1e 58 00 00 08 00

end_request: I/O error, dev sdb, sector 29367896

md/raid1:md0: sdb1: rescheduling sector 29365848

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 1e 50 00 00 08 00

end_request: I/O error, dev sdb, sector 29367888

md/raid1:md0: sdb1: rescheduling sector 29365840

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 1e 48 00 00 08 00

end_request: I/O error, dev sdb, sector 29367880

md/raid1:md0: sdb1: rescheduling sector 29365832

md/raid1:md0: redirecting sector 29365848 to other mirror: sdc1

md/raid1:md0: redirecting sector 29365840 to other mirror: sdc1

md/raid1:md0: redirecting sector 29365832 to other mirror: sdc1

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 3b c0 00 00 08 00

end_request: I/O error, dev sdb, sector 29375424

md/raid1:md0: sdb1: rescheduling sector 29373376

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 3b a8 00 00 08 00

end_request: I/O error, dev sdb, sector 29375400

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 3b a0 00 00 08 00

end_request: I/O error, dev sdb, sector 29375392

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 3b 98 00 00 08 00

end_request: I/O error, dev sdb, sector 29375384

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 3b 90 00 00 08 00

end_request: I/O error, dev sdb, sector 29375376

md/raid1:md0: redirecting sector 29373376 to other mirror: sdb1

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Read(10): 28 00 01 c0 4d c8 00 00 08 00

end_request: I/O error, dev sdb, sector 29380040

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Write(10): 2a 00 00 80 27 60 00 00 08 00

end_request: I/O error, dev sdb, sector 8398688

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Write(10): 2a 00 00 80 0b e8 00 00 08 00

end_request: I/O error, dev sdb, sector 8391656

sd 1:0:1:0: [sdb] Unhandled error code

sd 1:0:1:0: [sdb]  

Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK

sd 1:0:1:0: [sdb] CDB: 

Write(10): 2a 00 00 80 08 80 00 00 10 00

end_request: I/O error, dev sdb, sector 8390784

md/raid1:md0: Disk failure on sdb1, disabling device.

md/raid1:md0: Operation continuing on 1 devices.

RAID1 conf printout:

 --- wd:1 rd:2

 disk 0, wo:0, o:1, dev:sdc1

 disk 1, wo:1, o:0, dev:sdb1

RAID1 conf printout:

 --- wd:1 rd:2

 disk 0, wo:0, o:1, dev:sdc1

----------

## frostschutz

 *Lupin_the_3rd wrote:*   

> I didn't even realize a drive had failed until I ran `dmesg` to see the kernel messages as part of my routine system check.

 

You should set up mdadm and smartmond to send you mail when anything happens to any drive.

----------

