# [SOLVED] how do I get around the 2TB limit on disk devices?

## humbletech99

I've got a 4TB raid array which is showing up as follows:

```
8    16 2147483647 sdb

8    32 2147483647 sdc

8    48   99470337 sdd

```

How do I get the kernel to see this as 1 large volume?

----------

## zenlunatic

Wow can I have some space?

----------

## lonex

There's a kernel option somewhere. I can't tell you where, because I'm not on a Linux system at the moment, sorry.

----------

## humbletech99

I've already got the option "Support For Large Block Devices" compiled into the kernel which is supposed to allow larger than 2TB according to it's kernel help.

The raid card is a 3ware 9xxx sata raid controller. Perhaps it's a limitation of the 3ware driver or something?

----------

## drdope

 *humbletech99 wrote:*   

> The raid card is a 3ware 9xxx sata raid controller. Perhaps it's a limitation of the 3ware driver or something?

 

Check if auto carving is enabled on the 3Ware Controller:

 *Quote:*   

> Auto carving allows logical units larger than 2 TB to be divided into multiple units for legacy operating systems that are limited to a maximum of 2 TB per logical unit

 

--> http://www.3ware.com/products/serial_ata2-9000.asp

----------

## humbletech99

I found that the first time and disabled it in the 3ware bios but it still shows up as only 2TB devices in linux, despite double checking and recompiling the kernel to make 100% sure I have the "Support for Large Block Devices" compiled directly into the kernel. Rebooted many times but still the same, it shows up as 2TB, 2TB and 100gb.

----------

## Ast0r

 *drdope wrote:*   

>  *humbletech99 wrote:*   The raid card is a 3ware 9xxx sata raid controller. Perhaps it's a limitation of the 3ware driver or something? 
> 
> Check if auto carving is enabled on the 3Ware Controller:
> 
>  *Quote:*   Auto carving allows logical units larger than 2 TB to be divided into multiple units for legacy operating systems that are limited to a maximum of 2 TB per logical unit 
> ...

 

Yeah, there's a utility on their website that will allow you to manage the card and change this setting. The default on the 9xxx series is 2TB.

----------

## humbletech99

I disabled this feature so I should be able to see the whole array as sdb but it still splits it up, this is driving me nuts...

----------

## Janne Pikkarainen

I don't know about 3ware, but couple of years back I had an unpleasant opportunity to meet some Promise UltraTrak box with >3 TB. The box was conforming to SCSI-II standard, which itself limits a single block device maximum size to be 2 TB. That's why it was necessary to split the space to be two smaller partitions.

If this is the situation for you too, then you might consider using LVM2 or EVMS for joining those physical drives to be one big volume group from the Linux point of view. That way it's like a one huge partition to you.

----------

## humbletech99

I'm not sure if Sata has such a limitation but theorectically it shouldn't matter since the card is responsible for joining the devices together form one large partition, and this is beyond the point of where the protocol is used on the cable.

I'm reluctant to use lvm or evms on a server, I don't even want to stripe them in raid, it's just ugly and messy and will slow things down even more.

I gotta get this working, otherwise I'll have to put up with 2 separate arrays or something...

----------

## drdope

 *humbletech99 wrote:*   

> I found that the first time and disabled it in the 3ware bios but it still shows up as only 2TB devices in linux, despite double checking and recompiling the kernel to make 100% sure I have the "Support for Large Block Devices" compiled directly into the kernel. Rebooted many times but still the same, it shows up as 2TB, 2TB and 100gb.

 

Does this mean it was enabled at the time the array was created and disabled later?

Have you tried starting from scratch?

--> Disabling the auto carving & building a kernel with "Support for Large Block Devices" and creating the array afterwards (just be sure)?

----------

## humbletech99

yes, I did that last night and it seems to have worked!

Now I've got one huge sdb device "4394437632 sdb", I just need to find out how to partition it and format it! Any help would be appreciated since fdisk doesn't work to partition it...

----------

## Janne Pikkarainen

fdisk is limited to 2TB, cfdisk handles bigger partitions.

----------

## humbletech99

I've just tried cfdisk but it doesn't work, doing cfdisk just shows sda and doing cfdisk /dev/sdb gives the error:

```
cfdisk /dev/sdb

FATAL ERROR: Cannot get disk size

Press any key to exit cfdisk

```

----------

## humbletech99

parted works with gpt which seems to be the only way to handle this large a device. unfortunately I can't badblocks the partition since I get the error:

```
badblocks -wv /dev/sdb1

badblocks: File too large while trying to determine device size
```

oh well, I'm trying to figure out the best way to filesystem the thingy now, see https://forums.gentoo.org/viewtopic-t-463713.html

----------

## drdope

3ware recomends using parted:

--> http://www.3ware.com/KB/article.aspx?id=11920

edit: 1 minute late  :Sad: 

----------

## humbletech99

that's okay, this is turning out to be much more problematic than I had anticipated... see

https://forums.gentoo.org/viewtopic-t-463742.html (> 2TB in 32-bit mode?)

https://forums.gentoo.org/viewtopic-t-463740.html (files > 2GB in ext3?)

https://forums.gentoo.org/viewtopic-t-463713.html (ext3 on 4TB array?)

https://forums.gentoo.org/viewtopic-t-463747.html (Rebuild system from 32-bit to 64-bit?)

What have I gotten myself into by trying to go over this 2TB thing...

----------

## t35t0r

this is actually a rather stupid problem of not using "long long" variables. Please complain to the author of fdisk and blockdev:

Copyright (C) 1992  A. V. Le Blanc (LeBlanc@mcc.ac.uk)

or the 

Maintainer: Adrian Bunk <bunk@stusta.de>

Or get someone else who understands the code in the util-linux package.

<wyvern> # blockdev --getsize /dev/sdb                                              (30)

<wyvern> BLKGETSIZE: File too large

<t35t0r> wyvern, hey did you try blockdev --getsize64 /dev/sdb ?

wyvern t35t0r: nope, let me try that

<wyvern> 2500509827072

<wyvern> whee!

this is error 27:

asm-generic/errno-base.h:#define        EFBIG           27      /* File too large */

<t35t0r> so that's a 2.5TB part?

<t35t0r> err disk

<wyvern> yeah

<wyvern> sweet.

<wyvern> /dev/sdb1             2.3T  528K  2.3T   1% /vol/a

<wyvern> /dev/sdc1             2.3T  528K  2.3T   1% /vol/b

<t35t0r> wyvern, what did you do ?

<wyvern> parted + gtp disklabel format seems to work.

<wyvern> dunno why NO ONE seemed to mention that except in one well-hidden page on 3ware's documentation

<wyvern> just 'mklabel gpt' and then mkpart will work fine

Someone mark this as [SOLVED].

----------

## humbletech99

oh, actually I did that and it's all been working for quite some time... must have forgotten to post that back. I'll mark this as solved.

----------

