# RAID: mdadm gives totally wrong Array Size after "mdadm -C"

## chia3

Hello!

I've recently run 'mdadm --create' to create a new software RAID (RAID5) with 6 discs of 500GB each.

Now mdadm tells me the Array Size is 301.52GB!!!!!!!!!!

Can anyone please help me with this; it's giving me a hardache.   :Confused: 

---

More info:

Ran fdisk on all disks before mdadm-creation; type fd (Linux AutoRAID)

Ran following for mdadm-creation:

```
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=6 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf
```

Made XFS filesystem on /dev/md0:

```
mkfs.xfs -f /dev/md0
```

Mounted /dev/md0, ran 'df -h': discovered the problem.

Ran 2 mdadm-commands to see what could be wrong:

```

# mdadm --detail /dev/md0

/dev/md0:

        Version : 00.90.03

  Creation Time : Thu Sep  7 04:39:12 2006

     Raid Level : raid5

     Array Size : 294448832 (280.81 GiB 301.52 GB)

    Device Size : 488386496 (465.76 GiB 500.11 GB)

   Raid Devices : 6

  Total Devices : 6

Preferred Minor : 0

    Persistence : Superblock is persistent

    Update Time : Thu Sep  7 12:32:23 2006

          State : clean

 Active Devices : 6

Working Devices : 6

 Failed Devices : 0

  Spare Devices : 0

         Layout : left-symmetric

     Chunk Size : 64K

           UUID : 3323d609:8557c420:db6f0f01:6f10325d

         Events : 0.14

    Number   Major   Minor   RaidDevice State

       0       8        0        0      active sync   /dev/sda

       1       8       16        1      active sync   /dev/sdb

       2       8       32        2      active sync   /dev/sdc

       3       8       48        3      active sync   /dev/sdd

       4       8       64        4      active sync   /dev/sde

       5       8       80        5      active sync   /dev/sdf

```

```

# mdadm -E /dev/sda

/dev/sda:

          Magic : a92b4efc

        Version : 00.90.00

           UUID : 3323d609:8557c420:db6f0f01:6f10325d

  Creation Time : Thu Sep  7 04:39:12 2006

     Raid Level : raid5

    Device Size : 488386496 (465.76 GiB 500.11 GB)

     Array Size : 2441932480 (2328.81 GiB 2500.54 GB)

   Raid Devices : 6

  Total Devices : 6

Preferred Minor : 0

    Update Time : Thu Sep  7 12:32:23 2006

          State : clean

 Active Devices : 6

Working Devices : 6

 Failed Devices : 0

  Spare Devices : 0

       Checksum : 5342d7d3 - correct

         Events : 0.14

         Layout : left-symmetric

     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State

this     0       8        0        0      active sync   /dev/sda

   0     0       8        0        0      active sync   /dev/sda

   1     1       8       16        1      active sync   /dev/sdb

   2     2       8       32        2      active sync   /dev/sdc

   3     3       8       48        3      active sync   /dev/sdd

   4     4       8       64        4      active sync   /dev/sde

   5     5       8       80        5      active sync   /dev/sdf

```

One strange thing is that 'mdadm -E /dev/sda' shows the right Array Size, while 'mdadm --detail /dev/md0' shows the wrong Array Size...

----------

## kmj0377

Did df -h show 301GB as well?

And did you type /dev/sd[a-f]X instead of /dev/sd[a-f] where X is the partition number on the create line?  That's the only thing I can think of that would be the problem.  Also, has it finished syncing?  Do:

```
watch -n 1 'cat /proc/mdstat'
```

----------

## chia3

Yes, it has finished syncing:

```
#cat /proc/mdstat

md0 : active raid5 sdf[5] sde[4] sdd[3] sdc[2] sdb[1] sda[0]

      2441932480 blocks level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]

```

And, yes, 'df -h' also shows 301GB.

But now I'll try recreating my array with /dev/sd[abcdef]1 instead. Did that on my other array som time ago, and that worked without problems. Could that be the solution for this problem?

----------

## chia3

Nah, it wasn't...

I now ran:

```
# mdadm --create --verbose /dev/md0 --level=5 --raid-devices=6 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
```

...and then checked 'mdadm --detail /dev/md0' again, and again it showed 301GB as Array Space. Of course, this last check was done while mdadm was in the beginning of syncing the new array, but this shouldn't matter.

I suspected the problem would be too many large discs(?), and ran instead:

```
# mdadm --create --verbose /dev/md0 --level=5 --raid-devices=5 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
```

This looked much better in the output of 'mdadm --detail /dev/md0': now it showed the right amount of about 2000GB for Array Space.

Seems like the problem is adding a 6th disk of 500GB.

I really don't know where the problem is: I fear it is a kernel issue, but hope it is a limitation of mdadm. If anyone know, please share your wisdom!

Anyway, now I've started mkraid (raidtools) to once again try making the raid array of 6 x 500GB discs. I really hope raidtools would do the trick! I will come back with updates.

In the meantime; please let me know if you know of any solution to my problem with mdadm (or kernel or whatever)!

Best regards,

chia3

----------

## chia3

Finally found the solution now:

Same problem occured with raidtools.

Solution: had to compile LBD (large block device) into kernel. Then retried creating raid array with mdadm. This works lik a charm; Array Size shows the right value of a little less than 2500GB.

Found out that LDB was needed for larger block devices (i.e. raid volumes) than 2TB.

----------

