# [DRBD] Much network transfert than copied data

## loopx

Hello, 

I'm trying DRBD and making some test. I found a strange behavior using these configuration :

- ext3 on DRBD mounted with "sync" option

- DRBD protocol C

- DRBD configuration :

```

# last update by loopx on 20100414 :

#       - update the configuration to be easier with version 8.3

#       - creation of this config

#################################################################

global {

        #don't send statistics through internet ...

        usage-count no;

}

common {

        protocol C;

        syncer {

                #10Mbyte per second

                rate 10M;

                #use compression for bitmap exchange

                use-rle;

        }

}

resource r0 {

        device          /dev/drbd0;

        meta-disk       internal;

        #use "none" for write-after-write (because, we've got battery for our server)

        disk {

                no-disk-barrier;

                no-disk-flushes;

                no-disk-drain;

        }

        on wk02lhe.<domain> {

                address         <ip1>:7789;

                disk            /dev/loop0;

        }

        on wk01lhe.<domain> {

                address         <ip2>:7789;

                disk            /dev/loop0;

        }

}

```

In fact, I'm trying to have the faster replication as possible ; I know there is lost of performance, but I just want to try. 

The first test was made without any synced filesystem ... all was fine : about 10Mb/s through the network.

But now, with "synced" filesystem, I got about 5Mb/s through the network ... but this is not all ... Currently, the transfert over the network is 14Gb ... and when I have a look on the DRBD mounted partition (where I'm currently copying a 10Gb file), I only see a file of 1.7Gb ....

So, why is only 1.7Gb copied on the DRBD (and, probably, replicated on the other node) and the network show me than there is more than 14Gb transfered ? How this is possible since the file to transfert is only 9.6Gb  :Surprised:  ??????????

I have a look on the CPU IO wait and ... it's about 50% of IO wait ... may be it's because of that wait time ? 

I really not understand ... All other test (without "sync" on filesystem) completed the replication when the network has sent 10Gb ... not over ...

Any idea of what's happening ?

----------

## richard.scott

The 'sync' flag will slow down your throughput lots as it forces the data to be flushed to the disk each time something is written.

As your doing this on a mounted drbd device you'll need to get this feedback from the other node before your node will confirm the write.

This will a) increase your bandwidth usage (as you'll be sending data, and getting confirmation back) and b) slow down your write speed.

I would not use the sync option, but use the option in DRBD instead.

----------

## loopx

 *richard.scott wrote:*   

> The 'sync' flag will slow down your throughput lots as it forces the data to be flushed to the disk each time something is written.
> 
> As your doing this on a mounted drbd device you'll need to get this feedback from the other node before your node will confirm the write.
> 
> This will a) increase your bandwidth usage (as you'll be sending data, and getting confirmation back) and b) slow down your write speed.
> ...

 

Thanks you, I will not use EXT3 with "sync" option. Now, I'm looking for DRBD + GFS to make a little cluster R/W and RO  :Wink: 

Should be better   :Cool: 

----------

