# unable to mount big (>2TB) disk anymore

## majoron

Hi,

some weeks ago I managed to make my 4 TB raid working (see this).

But after a crash I cannot access it anymore.

I made a gpt partition with ext3 using parted.

When I try to mount I get:

```
mount -t ext3 /dev/sdc1 /mnt/hd

mount: wrong fs type, bad option, bad superblock on /dev/sdc1,

       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try

       dmesg | tail  or so

```

doing dmesg|grep sdc:

```
sd 4:0:0:0: [sdc] Very big device. Trying to use READ CAPACITY(16).

sd 4:0:0:0: [sdc] 7812499456 512-byte hardware sectors (4000000 MB)

sd 4:0:0:0: [sdc] Write Protect is off

sd 4:0:0:0: [sdc] Mode Sense: cb 00 00 08

sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

sd 4:0:0:0: [sdc] Very big device. Trying to use READ CAPACITY(16).

sd 4:0:0:0: [sdc] 7812499456 512-byte hardware sectors (4000000 MB)

sd 4:0:0:0: [sdc] Write Protect is off

sd 4:0:0:0: [sdc] Mode Sense: cb 00 00 08

sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

 sdc: sdc1

sd 4:0:0:0: [sdc] Attached SCSI disk

VFS: Can't find a valid FAT filesystem on dev sdc1.

VFS: Can't find a valid FAT filesystem on dev sdc1.

VFS: Can't find a valid FAT filesystem on dev sdc1.

VFS: Can't find ext3 filesystem on dev sdc1.

VFS: Can't find ext3 filesystem on dev sdc1.

```

Then I tried this:

```
# e2fsck /dev/sdc1

e2fsck 1.40.9 (27-Apr-2008)

e2fsck: Superblock invalid, trying backup blocks...

e2fsck: Bad magic number in super-block while trying to open /dev/sdc1

The superblock could not be read or does not describe a correct ext2

filesystem.  If the device is valid and it really contains an ext2

filesystem (and not swap or ufs or something else), then the superblock

is corrupt, and you might try running e2fsck with an alternate superblock:

    e2fsck -b 8193 <device>

```

and:

```
# e2fsck -b 8193 /dev/sdc1

e2fsck 1.40.9 (27-Apr-2008)

e2fsck: Device or resource busy while trying to open /dev/sdc1

Filesystem mounted or opened exclusively by another program?

```

But the device is not mounted (as far as umount and df concern).

Does anyone have a wise comment to enlighten me?

TIA

----------

## el-chaote

Hey majoron,

Have you already googled your problem?  :Very Happy:  It looks like this guy had more or less the same problem. I've never heard of this recovery software. First, I would give some other possible steps a try...

Before executing "e2fsck -b 8193 /dev/sdc1", use "lsof /dev/sdc1" to check if there are no other services/programs trying to access the device.

Have you already checked the device itself? "sfdisk -V /dev/sdc"

With "tune2fs -l /dev/sdc1" you'll get more information about the partition. And if you're really sure that you have an ext3 file system, you can re-add a ext3 file system with "tune2fs -j /dev/sdc1". There was a guy with a similar problem. He had an ext3 file system as well and all of sudden, he got the same errors. He found out that the partition was now recognized as ext2 and he just used tune2fs to re-add an ext3 file system. It worked again. However, I don't know if he had lost all his data   :Confused:   :Very Happy: 

These are just a few suggestions... so please, don't blame me if you lose your data  :Very Happy:  If this doesn't matter, use fdisk to rewrite your partition table and mke2fs to reformat the partition.

Good luck  :Wink: 

----------

## majoron

Hi,

thank you for the interest and for the comment.

I maybe did not provide enough information: one very important thing for me is to rescue the data. If this was not an issue here a could start from scratch (as I did weeks ago). Hence re-making the partition (or any other risky activity) is something I don't want to do if I can avoid it.

Now, going to your comments:

 *Quote:*   

> Have you already googled your problem?  It looks like this guy had more or less the same problem. I've never heard of this recovery software. First, I would give some other possible steps a try... 

 

Yes, I tried this. But it looks like it does not behaves well with gpt partitions.

 *Quote:*   

> Before executing "e2fsck -b 8193 /dev/sdc1", use "lsof /dev/sdc1" to check if there are no other services/programs trying to access the device.

 

"lsof |grep sdc1" gives nothing.

 *Quote:*   

> Have you already checked the device itself? "sfdisk -V /dev/sdc" 

 

Again sfdisk cannot manage gpt partitions.

```
# tune2fs -l /dev/sdc1

tune2fs 1.40.9 (27-Apr-2008)

tune2fs: Bad magic number in super-block while trying to open /dev/sdc1

Couldn't find valid filesystem superblock.
```

It looks like the superblock is corrupted, but "e2fsck -b  /dev/sdc1" insists in that the device is been used by some other process (?).

Moreover, I found something interesting; lshw reports the following:

```
              *-storage

                   description: RAID bus controller

                   product: ARC-1120 8-Port PCI-X to SATA RAID Controller

                   vendor: Areca Technology Corp.

                   physical id: e

                   bus info: pci@0000:04:0e.0

                   logical name: scsi4

                   version: 00

                   width: 32 bits

                   clock: 66MHz

                   capabilities: storage pm msi pcix bus_master cap_list

                   configuration: driver=arcmsr latency=32 mingnt=128 module=arcmsr

                 *-disk

                      description: SCSI Disk

                      product: ARC-1120-VOL#00

                      vendor: Areca

                      physical id: 0.0.0

                      bus info: scsi@4:0.0.0

                      logical name: /dev/sdc

                      version: R001

                      serial: 0000004146533666

                      size: 3725GiB (3999GB)

                      capabilities: 10000rpm gpt-1.00 partitioned partitioned:gpt

                      configuration: ansiversion=5 guid=cf80e5bf-31e9-463f-a388-2c2a3d168d50

                    *-volume UNCLAIMED

                         description: EXT3 volume

                         vendor: Linux

                         physical id: 1

                         bus info: scsi@4:0.0.0,1

                         version: 1.0

                         serial: 883796bc-ef4a-4e26-848b-f78ca2a738b7

                         size: 3725GiB

                         capacity: 3725GiB

                         capabilities: journaled extended_attributes large_files huge_files recover ext3 ext2 initialized

                         configuration: created=2008-06-19 00:50:07 filesystem=ext3 modified=2008-06-20 16:38:56 mounted=2008-06-20 16:38:56 state=clean

```

What looks interesting to me is: 1) lshw sees the device as mounted (same as e2fsck), but it is not; and 2) the volume is UNCLAIMED, instead of 1, as it should be.

And rebooting does not help...

Does anyone know how to force umount? I don't see any option in the man page to do that (althought seems that other distros have it).

EDIT: I was reading the spanish manpage; the english one does tell about -l and -f umount options. They do not work though.

Does anyone have any idea to solve this?

I'll keep google-ing, trying to find a solution

In the meantime any help will be welcome  :Very Happy: 

Best

----------

## majoron

More info:

I assume(d) that the superblock is corrupted. Then I need the locations of superblock backups. I got them with:

```
# mke2fs -n /dev/sdc1

mke2fs 1.40.9 (27-Apr-2008)

Warning: 256-byte inodes not usable on older systems

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

134217728 inodes, 536870911 blocks

26843545 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=0

16384 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks:

        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,

        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,

        102400000, 214990848, 512000000

```

Then I tried to repare the file system:

```
# fsck.ext3 -b 32768 -B 4096 /dev/sdc1

e2fsck 1.40.9 (27-Apr-2008)

fsck.ext3: Bad magic number in super-block while trying to open /dev/sdc1

The superblock could not be read or does not describe a correct ext2

filesystem.  If the device is valid and it really contains an ext2

filesystem (and not swap or ufs or something else), then the superblock

is corrupt, and you might try running e2fsck with an alternate superblock:

    e2fsck -b 8193 <device>

```

(it fails alsowith, for instance, 98304 insted of 32768).

???

Best

----------

## el-chaote

Good morning  :Wink: 

To be honest, I haven't heard anything about gpt yet  :Very Happy:  I didn't really know what it is and just gave some advises for ext3 file systems  :Very Happy:  So I've just done some research  :Very Happy:  GPT = GUID Partition Table  :Wink: 

 *Quote:*   

> What looks interesting to me is: 1) lshw sees the device as mounted (same as e2fsck), but it is not; and 2) the volume is UNCLAIMED, instead of 1, as it should be.
> 
> And rebooting does not help... 

 

1) Why do you think the device is mounted? You don't have to mount a device to get these information. I've just run a "lshw" and it gives me similar information about my not-yet-mounted-devices as well. 2) yes, this is a little bit odd... :S And by the way... it should be 0, not 1 *justbeeingasmartass*  :Very Happy: 

Have you tried to run "fsck.ext3 -b 32768 -B 4096 /dev/sdc1" with different superblocks?

----------

## majoron

Hi again, el-chaote

Thank you for the reply.

 *Quote:*   

> 1) Why do you think the device is mounted? 

 

I do not think that the device is mounted.

What I said is that lshw gives me strange information:

When a partition is mounted (in this case sdb6) it says:

```
                 *-logicalvolume:1

                      description: Linux filesystem partition

                      physical id: 6

                      logical name: /dev/sdb6

                      logical name: /home/majoron/emergencias

                      capacity: 17GiB

                      configuration: mount.fstype=ext3 mount.options=rw,nosuid,nodev,noexec,data=ordered state=mounted
```

And then the partition is not shown in df, etc, as it should.

But if I umount it:

```
                 *-logicalvolume:1

                      description: Linux filesystem partition

                      physical id: 6

                      logical name: /dev/sdb6

                      capacity: 17GiB
```

However this is not the behaviour of my sdc1 partition. It is not mounted but both lshw and e2fsck claim that it is!

The question I'm making myself is, where are these programs looking at in order to tell me that the partition is mounted?

 *Quote:*   

> Have you tried to run "fsck.ext3 -b 32768 -B 4096 /dev/sdc1" with different superblocks?

 

Yes, I did. See my previous post.

Regards

----------

## el-chaote

Hey majoron, you're welcome  :Wink: 

What do you mean with:

 *Quote:*   

> And then the partition is not shown in df, etc, as it should. 

 

When you mount /dev/sdb6, it is not shown by running 'df' nor 'mount'?

 *Quote:*   

> What I said is that lshw gives me strange information: 

 

Sorry, I just can't follow you... your 'lshw' doesn't seem too strange to me :s And I just can't see a line which claims that your device should be mounted... Okay... you'll get a lot more information about the device than for your sdb6 partition... but you use another partition table for this disk... (I might be too tired to see your point  :Wink:  )

However, there are two things worrying me... Why is the volume "UNCLAIMED" (instead of 0) and why does this section doesn't show a "logical name:". This might go hand in hand... 

 *Quote:*   

> The question I'm making myself is, where are these programs looking at in order to tell me that the partition is mounted? 

 

As I said, lshw doesn't claim anything (as far as I can see  :Very Happy:  ) But i don't know, why e2fsck says this! Maybe e2fsck is disturbing itself because of a strange fault  :Confused: 

 *Quote:*   

> Yes, I did. See my previous post. 

 

Again, sorry... I have to read more carefully  :Very Happy:   :Very Happy:   :Very Happy: 

----------

## majoron

Hi el-chaote.

Thank you for the quick answer.

What I was saying in my last post is that while sdb6 (a partition without a problem, I mentioned it as an example) behaves as it should from the view point of lshw (lshw says it is mounted when it is, and umounted when it is unmounted), however lshw does not behave as expected with my sdc1 partition (the one I'm unable to mount, the one which motivated this thread). lshw says that the sdc1 partition is mounted, but it is not mounted. And I gave you an example to show you why I think this partition is not mounted: df does not display it (it should if the partition was mounted, as far as I understand).

So if you have a look at my second post in the thread, you'll notice that lshw does indeed claim something concerning sdc1: it says that sdc1 is mounted, while it shouldn't say that.

I hope it is clear now.

Regards

----------

## el-chaote

I went through your output of lshw again and again... but still... i can't see why you think that it's mounted  :Confused: 

I've just used lshw on my machine. I mounted / unmounted my CF and usb stick a couple of times and run lshw. It always showed the same output :s so for me, lshw doesn't tell me if a device is mounted or not. It just gives me information about the device. And it does it with mounted or unmounted devices. The device just has to be plugged in  :Very Happy: 

There's no such line like "Device: mounted" or "mounted: true" in the output of lshw. Can you exactly tell me why you think that lshw tells you that sdc1 is mounted? (eg. "because it gives me information about the device...", "because in lineX it says....")  :Very Happy: 

I still think that your device isn't actually mounted and that this is just mistakenly claimed by e2fsck. 

However, you still have the problem with either a bad superblock or a fu**ed up partition table  :Confused: 

----------

## majoron

Hi again.

 *Quote:*   

> Can you exactly tell me why you think that lshw tells you that sdc1 is mounted?

 

Go, please, to my second post. There I posted the output of lshw. The last line reads:

```
configuration: created=2008-06-19 00:50:07 filesystem=ext3 modified=2008-06-20 16:38:56 mounted=2008-06-20 16:38:56 state=clean
```

I understand that it says it is mounted.

In general, the version of lshw I have says:

```
state=mounted
```

whenever a partition is mounted. Now, yes, lshw doesn't say that in the case of my sdc1 partition, but it says "mounted=...". So I understand there is something wrong there. I'm not sure if the output of lshw is relevant, but that's why I posted it. Maybe someone can see something useful there...

In any case, you are right: the important thing is that I still cannot use the disk!

Regards

----------

## el-chaote

by the way... have you already tried to mount the device with a live cd (eg. knoppix), just to see if the same problems remain? Or to check your disk using a tool like partition magic (I don't no if this tool is able to handle gpt)?

Probably you'll manage to get a usable backup with 'dd'.

----------

## el-chaote

Okay, now I see the point...  :Very Happy: 

I think, "mounted=..." only shows the last time it was successfully mounted. Like "modifed" shows the last time it was (successfully) modified  :Very Happy:  As far as I know, this information is stored in the file system. Therefore it doesn't say if it's mounted right now... Like "modified=" doesn't mean that it is modified right now...  :Confused:  However, I'm not completely sure about this and I couldn't find any further information, so I might be completely wrong...  :Very Happy: 

Have a look here. 

 *Quote:*   

> [...] or UNCLAIMED (no driver has been detected for this node)
> 
> [...] a node is marked as UNCLAIMED if no specific support for it has been loaded (or lshw has been unable to identify the driver)

 

Has anything changed in your system / kernel since the last time you successfully mounted the device?

ADDED:

Have you already compared the time strings? 

   created=2008-06-19 00:50:07

   modified=2008-06-20 16:38:56

   mounted=2008-06-20 16:38:56

If this line would tell you if its mounted, you would have run lshw and e2fs more than two months ago  :Confused:  Or your time setting might just be incorrect  :Wink:   :Very Happy: 

----------

## majoron

Hi,

 *Quote:*   

> by the way... have you already tried to mount the device with a live cd (eg. knoppix), just to see if the same problems remain? Or to check your disk using a tool like partition magic (I don't no if this tool is able to handle gpt)? 

 

yes, I tried with knoppix, but it didn't work. And I do not have partition magic.

```
Probably you'll manage to get a usable backup with 'dd'.
```

This could be a way out, but don't I need to know the superblock info? and btw, don't I need to have a >4 TB disk to dd it into? And I don't have it...

Regards

----------

## el-chaote

Hey, did you see my last post and did you have a look at the link?  :Very Happy: 

I'm playing around with e2fsck, trying to get a corrupted filesystem on a CF. Just to try if dd will still work.  :Very Happy:  I'll tell you as soon as I know  :Very Happy: 

While playing, I came across your error message given by e2fsck  :Very Happy:  I did a "mke2fs -n /dev/..." to get the superblocks on my test partition. However I used a total other one with e2fsck, just to see what will happen...

```
mke2fs -n /dev/sdb3

[...]

Superblock backups stored on blocks: 

   8193
```

and then....

```
e2fsck -b 123 /dev/sdb3

e2fsck 1.40.8 (13-Mar-2008)

e2fsck: Device or resource busy while trying to open /dev/sdb3

Filesystem mounted or opened exclusively by another program?
```

And it was definitely not mounted!  :Confused:   A e2fsck with the superblock 8193 worked perfectly  :Confused: 

----------

## majoron

Hi.

 *Quote:*   

> Has anything changed in your system / kernel since the last time you successfully mounted the device?
> 
> 

 

There happened a crash and since then I'm not able to mount the partition; but I did not change anything important in the system.

 *Quote:*   

> Have you already compared the time strings?
> 
> created=2008-06-19 00:50:07
> 
> modified=2008-06-20 16:38:56
> ...

 

My point about mentioning lshw output is the following: if the partition is not mounted, why lshw displays such information? Maybe I'm wrong, but a normal umounted partition looks (under lshw) like I posted above:

```
                 *-logicalvolume:1

                      description: Linux filesystem partition

                      physical id: 6

                      logical name: /dev/sdb6

                      capacity: 17GiB
```

or at least this is what happen in my system...

That must mean that there is still some residual information somewhere about the last time I mounted sdc1. Maybe this information should have been removed under a normal halt of the system, but due to the crash it could well happen that this wasn't the case. And now this is causing confusion.

So, what is wrong with this partition?

I understand that the partition is corrupted. Then why can I not run e2fsck on it? and how can I get that e2fsck runs on this partition?

Several things I tried:

I tried 

```
e2fsck -c -c -v /dev/sdc1
```

 but does not work: it says

```
e2fsck 1.40.9 (27-Apr-2008)

e2fsck: Superblock invalid, trying backup blocks...

e2fsck: Bad magic number in super-block while trying to open /dev/sdc1

The superblock could not be read or does not describe a correct ext2

filesystem.  If the device is valid and it really contains an ext2

filesystem (and not swap or ufs or something else), then the superblock

is corrupt, and you might try running e2fsck with an alternate superblock:

    e2fsck -b 8193 <device>

```

Of course, "e2fsck -b xxxx /dev/sdc1" keeps saying that the device is busy.

I read that some people had a similar problem and they solved with blkid; so I run

```
blkid /dev/scd1
```

but it does nothing. I'm still unable to mount it

"fuser /dev/sdc1" does not say a word

I used strace to see what is doing e2fsck to tell me that the device is busy, but I didn't see anything useful only that the device is busy:

```
...

open("/dev/sdc1", O_RDWR|O_EXCL|O_LARGEFILE) = -1 EBUSY (Device or resource busy)

open("/dev/sdc1", O_RDWR|O_EXCL|O_LARGEFILE) = -1 EBUSY (Device or resource busy)

write(2, "e2fsck", 6e2fsck)                   = 6

write(2, ": ", 2: )                       = 2

write(2, "Device or resource busy", 23Device or resource busy) = 23

write(2, " ", 1 )                        = 1

write(2, "while trying to open /dev/sdc1", 30while trying to open /dev/sdc1) = 30

write(2, "\r\n", 2

)                     = 2

write(1, "Filesystem mounted or opened exc"..., 61Filesystem mounted or opened exclusively by another program?

) = 61

exit_group(8)                           = ?

```

 *Quote:*   

> While playing, I came across your error message given by e2fsck  I did a "mke2fs -n /dev/..." to get the superblocks on my test partition. However I used a total other one with e2fsck, just to see what will happen... 

 

I appreciate your effort simulating my problem, but I'm skeptical; this looks a very general response from e2fsck (from what I've seen in google). Still, if you find something out, I will thank you.

Regards

----------

## el-chaote

You could try to rebuild your ext3 filesystem using "mke2fs -S". But it looks a little bit risky from what the man page says  :Confused: 

 *Quote:*   

> Write  superblock  and  group  descriptors only.  This is useful if all of the superblock and backup superblocks are corrupted, and a last-
> 
>               ditch recovery method is desired.  It causes mke2fs to reinitialize the superblock and group descriptors, while not touching the inode  ta‐
> 
>               ble  and  the  block  and inode bitmaps.  The e2fsck program should be run immediately after this option is used, and there is no guarantee
> ...

 

The other option would be a backup using dd which might even work with a corrupted superblock. Unfortunately, you will get one (4TB big) image :S 

I'm not very familiar with RAID systems... What about backing up each hard disk separately? Making an image of the filesystem for each hard disk, formating each hard disk and coping back the images? But will this work with a hardware raid controller? :S

By the way, have you tried ALL mentioned superblock backups (given by mke2fs -n) with e2fsck -b xxxx?

----------

## majoron

Hi. Thank you again for the answer.

 *Quote:*   

> You could try to rebuild your ext3 filesystem using "mke2fs -S". But it looks a little bit risky from what the man page says 

 

Thank you; I didn't know this option. And yes it looks rsiky, but if I don't find a better solution I'll have to do that...

 *Quote:*   

> The other option would be a backup using dd which might even work with a corrupted superblock. Unfortunately, you will get one (4TB big) image :S
> 
> 

 

And I don't have a place to store it...

 *Quote:*   

> By the way, have you tried ALL mentioned superblock backups (given by mke2fs -n) with e2fsck -b xxxx?

 

Right  :Wink:  None worked. All return exit code number 8, saying the same as I wrote above.

Regards

----------

## el-chaote

 *Quote:*   

> And I don't have a place to store it... 

 

What about splitting the images by using the parameters 'count' and 'skip'? (see man page of dd) Then you could compress the images... (just another idea  :Wink:  )

----------

