# iSCSI "how to"?

## DingbatCA

Does ony one know of a good iSCSI "how to"   Looking to set up a simple test with one server and one client, nothing complex.

----------

## m3_del

I can help you out if you would like, I have been running iSCSI over a HA cluster for a few weeks now in testing.

iSCSI is the easy part

iSCSI uses Initiators to on the client to connect to the "Targets" (data server). It uses a method called "Discovery" once you input the IP of the server in the initiator it looks for all the targets on that server. It treats these targets as block devices that you can appear as local disks. You can then have your way with it! The local server will still be read as a raw disk.

Step 1:

Unmask "echo "sys-block/iscsitarget" >> /etc/portage/package.keywords"

step 2:

emerge iscsitarget

step 3:   [This step may be redundant, I think the modules is auto loaded during the init process, someone please let me know]

add iscsi_trgt to the modules autoload (if you want it to start at boot)    echo "iscsi_trgt" >> /etc/modules.autoload.d/kernel-2.6

Step 4:

create /etc/ietd.conf

will look like this

```
#The part before the colon is a name standard for iscsi with your domain of course. After is your descriptive text

Target iqn.2001-04.org.gentoo:iSCSI.test.disk1

        Lun 0 Path=/dev/hda5,Type=fileio

        MaxConnections          1

        #InitialR2T              Yes

        #ImmediateData           No

        #MaxRecvDataSegmentLength 8192

        #MaxXmitDataSegmentLength 8192

        #MaxBurstLength          262144

        #FirstBurstLength        65536

        #DataPDUInOrder          Yes

        #DataSequenceInOrder     Yes

        #ErrorRecoveryLevel      0

        #HeaderDigest            CRC32C

        #DataDigest              CRC32C

        #Wthreads                8

```

You can have multiple targets in the conf file. The different settings are optional and you can read about them in "man ietd.conf"

Step 5:

to start the server start the init

/etc/init.d/ietd start

step 6:

connect to the target with a iSCSI initiator from a different machine on the network.

This is a basic setup with no authentication. CHAP is the name of the Authentication used in iSCSI read more about that in the ietd.conf manual.

----------

## DingbatCA

Thank you for the help.  Was just starting on the iSCSI stuff and you solved all my bugs.  Now have my 1.2TB NAS directly connected to my app server with a cross over cable and am running iSCSI.

----------

## m3_del

If you would like any help on failover using iscsi let me know

----------

## DingbatCA

iSCSI is online and working, now its time for the tuning.

OK, speed, FASTER!!! 

Lets start with a simple speed test.  

```
 tar cf - . | cat - > /dev/null 
```

Shows that I can move at 29.27MB/sec  But I am unable to get better then 19.52MB/sec accross iSCSI.  

My rig.

App server running 2.0GHz 64-bit CentOS 4.3

NAS (running gentoo) directly connected to the app server with a cross over cable + iSCSI.   When I direct connect the NAS to the network I can get 23.86MB/sec to the disks using NFS, but when I use iSCSI I only get 19.52MB/sec.  What should I mess with to improve my iSCSI speed.  

When I start iSCSI on the NAS (target) i get a few warnings.   Any clue on what config i need to edit?

```

nas ~ # /etc/init.d/ietd start

 * net.core.wmem_max is lower than recommended 1048576

 * net.core.wmem_default is lower than recommended 1048576

 * net.core.rmem_max is lower than recommended 1048576

 * net.core.rmem_default is lower than recommended 1048576

 * net.ipv4.tcp_mem:min is lower than recommended 1048576

 * net.ipv4.tcp_mem:default is lower than recommended 1048576

 * net.ipv4.tcp_mem:max is lower than recommended 1048576

 * net.ipv4.tcp_rmem:min is lower than recommended 1048576

 * net.ipv4.tcp_rmem:default is lower than recommended 1048576

 * net.ipv4.tcp_rmem:max is lower than recommended 1048576

 * net.ipv4.tcp_wmem:min is lower than recommended 1048576

 * net.ipv4.tcp_wmem:default is lower than recommended 1048576

 * net.ipv4.tcp_wmem:max is lower than recommended 1048576

 * Loading iSCSI-Target modules - iscsi_trgt ...                                                                    [ ok ]

 * Starting iSCSI Enterprise Target ...        

```

----------

## Vieri

I'm new to iSCSI and would like to start testing it.

What's the difference between open-iscsi and iscsitarget ?

Which one would you suggest for the "server"/target side?

[EDIT] I believe open-iscsi is an initiator-only project.

[EDIT 2] I just successfully tested iscsitarget. I used an EVMS volume which is particularly interesting especially when wanting to manage snapshots.

----------

## BigBeer

 *m3_del wrote:*   

> If you would like any help on failover using iscsi let me know

 

Can you post some more info on this topic please?

And, anyone for speed tuning?

----------

## Vieri

As far as the man page goes you might want to 

EDIT /etc/ietd.conf:

Target iqn.2006-07.com.domain.host:storage.disk2.INF-BL07

        Lun 0 Path=/dev/evms/root2,Type=fileio

        Alias TEST

        MaxConnections          1

        InitialR2T              No

        ImmediateData           Yes

        MaxRecvDataSegmentLength 262144

        MaxXmitDataSegmentLength 262144

        MaxBurstLength          262144

        FirstBurstLength        262144

        MaxOutstandingR2T       16

        Wthreads                16

On a gigabit lan

EDIT /etc/sysctl.conf:

net.core.rmem_max = 1073741824

net.core.wmem_max = 1073741824

net.ipv4.tcp_rmem = 1048576 16777216 1073741824

net.ipv4.tcp_wmem = 1048576 16770216 1073741824

net.ipv4.tcp_mem = 1048576 16770216 1073741824

# sysctl -p

# /etc/init.d/ietd start

You might also want to:

# ifconfig eth1 txqueuelen 1000  (or whichever eth the iSCSI target is listening on). but this should already be set.

# sysctl w net.core.netdev_max_backlog=1000

But I'm not sure about this.

Any suggestions?

----------

## m3_del

Here is some good reading (failover related). I will try to post more of what I have been working on tomorrow, I am off in 10 minutes  :Wink: 

Neat stuff but not a whole lot of info out there on the web...more just in bits and pieces

b.	DRBD related

i.	http://en.wikipedia.org/wiki/DRBD

ii.	https://services.ibb.gatech.edu/wiki/index.php/Howto:Software:DRBD

iii.	http://linux-ha.org/DRBD

c.	iSCSI related

i.	http://en.wikipedia.org/wiki/Iscsi

ii.	http://iscsitarget.sourceforge.net/

d.	uCarp related

i.	http://www.ucarp.org/project/ucarp

ii.	http://gentoo-wiki.com/HOWTO_Setup_IP_failover_with_UCARP

----------

## richard.scott

 *DingbatCA wrote:*   

> When I direct connect the NAS to the network I can get 23.86MB/sec to the disks using NFS, but when I use iSCSI I only get 19.52MB/sec.  What should I mess with to improve my iSCSI speed.

 

ok, I'm new to iSCSI, but have been using NAS for a while.

Please correct me if I'm wrong, but have you taken into account the overhead of TCP/IP that iSCSI needs? 

From the way I understand things, NAS doesn't use TCP/IP and would therefore be able to squeeze more data into the same space?

----------

