# mdadm growing raid5 array concerns and reshape speeds

## ali3nx

I've been doing an mdadm reshape on a 4x 1TB sata raid array but i'm really under a deadline as this server i'm working on is down and the entire department needs to access data. I'm curious if it's safe to access the array while a reshape is under way or if it's possible? I tried to start the reshape with the array mounted as rootfs and of course that ended quickly with mdadm politely informing me of the common device in use or busy message. 

I know every ounce of systems admin "best practices" really leans towards "this is just not going to be a good idea genius" but my department manager and due dillgence dictate I make an inquiry under the circumstances that staff have urgent deadlines to meet. I've done my homework on why it's taking so bloody long and it does appear to be that it really does take this long. Its been running for over 14 hours and only 40% complete... If anyone has any safe and very stable tips that might speed up the reshape i'm open to suggestions.

https://raid.wiki.kernel.org/index.php/Growing

Also for future reference if anyone is considering doing a mdadm raid array, you happen to find this post while performing research and your prospective system has the following hardware I would highly advise against it. The Marvell 88SE6145 SATA II is pure garbage even with pata_marvell drivers disabled due to poor bandwidth, shared device resource handlers with the intel ICH7 controller and most importantly no ncq support because the marvell or intel are forced to use the ahci driver for sata mode instead of the sata_intel or sata_marvell drivers which do not work. It's highly amusing an X3220 xeon on an asus server motherboard is only doing 7200K/s reshape right?  :Smile: 

```

livecd ~ # cat /proc/mdstat

Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]

md1 : active raid1 sdc1[2] sda1[0] sdb1[1] sdd1[3]

      112320 blocks [4/4] [UUUU]

      bitmap: 0/14 pages [0KB], 4KB chunk

md0 : active raid5 sdc3[3] sda3[0] sdb3[1] sdd3[2]

      1949085952 blocks super 0.91 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

      [=======>.............]  reshape = 39.9% (389356748/974542976) finish=1419.9min speed=6868K/sec

unused devices: <none>

livecd ~ # mdadm -D /dev/md0

/dev/md0:

        Version : 0.91

  Creation Time : Wed Aug  4 10:53:46 2010

     Raid Level : raid5

     Array Size : 1949085952 (1858.79 GiB 1995.86 GB)

  Used Dev Size : 974542976 (929.40 GiB 997.93 GB)

   Raid Devices : 4

  Total Devices : 4

Preferred Minor : 0

    Persistence : Superblock is persistent

    Update Time : Wed Aug 11 09:40:14 2010

          State : clean, recovering

 Active Devices : 4

Working Devices : 4

 Failed Devices : 0

  Spare Devices : 0

         Layout : left-symmetric

     Chunk Size : 64K

 Reshape Status : 40% complete

  Delta Devices : 1, (3->4)

           UUID : 220b505f:c70cab8c:cb201669:f728008a (local to host livecd)

         Events : 0.88191

    Number   Major   Minor   RaidDevice State

       0       8        3        0      active sync   /dev/sda3

       1       8       19        1      active sync   /dev/sdb3

       2       8       51        2      active sync   /dev/sdd3

       3       8       35        3      active sync   /dev/sdc3

livecd ~ # cat /sys/block/md0/md/sync_speed

7225

livecd ~ # cat /proc/sys/dev/raid/speed_limit_min

1000

livecd ~ # cat /proc/sys/dev/raid/speed_limit_max

200000

livecd ~ # cat /sys/block/md0/md/stripe_cache_active

160

livecd ~ # cat /sys/block/md0/md/stripe_cache_size

256

livecd ~ # cat /proc/cpuinfo |grep model

model           : 15

model name      : Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz

model           : 15

model name      : Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz

model           : 15

model name      : Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz

model           : 15

model name      : Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz

livecd ~ # lspci

00:00.0 Host bridge: Intel Corporation 3200/3210 Chipset DRAM Controller (rev 01)

00:01.0 PCI bridge: Intel Corporation 3200/3210 Chipset Host-Primary PCI Express Bridge (rev 01)

00:1c.0 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 1 (rev 01)

00:1c.1 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 2 (rev 01)

00:1c.2 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 3 (rev 01)

00:1c.3 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 4 (rev 01)

00:1c.4 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 5 (rev 01)

00:1c.5 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 6 (rev 01)

00:1d.0 USB Controller: Intel Corporation N10/ICH7 Family USB UHCI Controller #1 (rev 01)

00:1d.1 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #2 (rev 01)

00:1d.7 USB Controller: Intel Corporation N10/ICH 7 Family USB2 EHCI Controller (rev 01)

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)

00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01)

00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01)

00:1f.2 SATA controller: Intel Corporation N10/ICH7 Family SATA AHCI Controller (rev 01)

00:1f.3 SMBus: Intel Corporation N10/ICH 7 Family SMBus Controller (rev 01)

01:03.0 VGA compatible controller: XGI Technology Inc. (eXtreme Graphics Innovation) Z7/Z9 (XG20 core)

02:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 12)

03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 12)

04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 12)

05:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 12)

06:00.0 RAID bus controller: Marvell Technology Group Ltd. 88SE6145 SATA II PCI-E controller (rev a1)

```

----------

## mbar

Best tip I have:

```
echo 10240 > /sys/block/md0/md/stripe_cache_size 
```

then:

```
echo 100000 > /proc/sys/dev/raid/speed_limit_min 
```

----------

