# trouble with Intel X520-da

## bigdweeb

I just replaced an older intel 10G card with the newer X520-DA model.  This is going in my lab, so I didn't buy the Intel SFP+ modules as I'm either going to use existing Cisco 10G modules or twinax cables.  When I boot the server with the card plugged in, the modules don't load properly and the logs complain that I am using non-Intel SFP+ modules.  I have tried defeating this but it isn't working.

Here's what I get when I boot:

```
May  8 14:36:30 labnas kernel: ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.9.15-k

May  8 14:36:30 labnas kernel: ixgbe: Copyright (c) 1999-2012 Intel Corporation.

May  8 14:36:30 labnas kernel: ixgbe 0000:0b:00.0: failed to load because an unsupported SFP+ module type was detected.

May  8 14:36:30 labnas kernel: ixgbe 0000:0b:00.0: Reload the driver after installing a supported module.

May  8 14:36:30 labnas kernel: ixgbe 0000:0b:00.1: failed to load because an unsupported SFP+ module type was detected.

May  8 14:36:30 labnas kernel: ixgbe 0000:0b:00.1: Reload the driver after installing a supported module.
```

Here's what I get from running lspci -vnk

```
0b:00.0 0200: 8086:10fb (rev 01)

        Subsystem: 8086:0003

        Flags: fast devsel, IRQ 18

        Memory at b8c80000 (64-bit, prefetchable) [size=512K]

        I/O ports at 2020 [size=32]

        Memory at b8d04000 (64-bit, prefetchable) [size=16K]

        Capabilities: [40] Power Management version 3

        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+

        Capabilities: [70] MSI-X: Enable- Count=64 Masked-

        Capabilities: [a0] Express Endpoint, MSI 00

        Capabilities: [100] Advanced Error Reporting

        Capabilities: [140] Device Serial Number 90-e2-ba-ff-ff-2d-b5-ac

        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)

        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)

0b:00.1 0200: 8086:10fb (rev 01)

        Subsystem: 8086:0003

        Flags: fast devsel, IRQ 19

        Memory at b8c00000 (64-bit, prefetchable) [size=512K]

        I/O ports at 2000 [size=32]

        Memory at b8d00000 (64-bit, prefetchable) [size=16K]

        Capabilities: [40] Power Management version 3

        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+

        Capabilities: [70] MSI-X: Enable- Count=64 Masked-

        Capabilities: [a0] Express Endpoint, MSI 00

        Capabilities: [100] Advanced Error Reporting

        Capabilities: [140] Device Serial Number 90-e2-ba-ff-ff-2d-b5-ac

        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)

        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
```

Here's what I have in /etc/udev/rules.d/70-persistent-net.rules

```
# PCI device 0x8086:0x10fb (ixgbe)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="90:e2:ba:2d:b5:ac", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x8086:0x10fb (ixgbe)

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="90:e2:ba:2d:b5:ad", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
```

I searched around and according to ixgbe_phy.c, it looks like you can set an option of allow_unsupported_sfp=1 to override the firmware and allow non-Intel hardware.  Here's what I set in /etc/conf.d/modules

 *Quote:*   

> modules="bonding ixgbe"
> 
> module_bonding_args="mode=802.3ad miimon=100"
> 
> module_ixgbe_args="allow_unsupported_sfp=1"

 

When I reboot with this set, it still doesn't work.  If however I do the following:

```
labnas ~ # lsmod

Module                  Size  Used by

nfsd                   86923  11 

exportfs                3339  1 nfsd

bonding                86870  0 

ixgbe                 141198  0 

e1000e                171717  0 

mdio                    3366  1 ixgbe

labnas ~ # rmmod ixgbe

labnas ~ # modprobe ixgbe allow_unsupported_sfp=1
```

The interfaces become available:

```
May  8 14:52:19 labnas kernel: ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.9.15-k

May  8 14:52:19 labnas kernel: ixgbe: Copyright (c) 1999-2012 Intel Corporation.

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0 (unregistered net_device): WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics.  Using untested modules is not supported and may cause unstable operation or damage to the module or the adapter.  Intel Corporation is not responsible for any harm caused by using untested modules.

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 66 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 67 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 68 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 69 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 70 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 71 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 72 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 73 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: irq 74 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: (PCI Express:2.5GT/s:Width x4) 90:e2:ba:2d:b5:ac

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: MAC: 2, PHY: 11, SFP+: 5, PBA No: E68793-005

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: PCI-Express bandwidth available for this card is not sufficient for optimal performance.

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: For optimal performance a x8 PCI-Express slot is required.

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.0: Intel(R) 10 Gigabit Network Connection

May  8 14:52:19 labnas /etc/init.d/net.eth2[2169]: net.eth2: not allowed to be hotplugged

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1 (unregistered net_device): WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics.  Using untested modules is not supported and may cause unstable operation or damage to the module or the adapter.  Intel Corporation is not responsible for any harm caused by using untested modules.

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 75 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 76 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 77 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 78 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 79 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 80 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 81 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 82 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: irq 83 for MSI/MSI-X

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: (PCI Express:2.5GT/s:Width x4) 90:e2:ba:2d:b5:ad

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: MAC: 2, PHY: 11, SFP+: 6, PBA No: E68793-005

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: PCI-Express bandwidth available for this card is not sufficient for optimal performance.

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: For optimal performance a x8 PCI-Express slot is required.

May  8 14:52:19 labnas kernel: ixgbe 0000:0b:00.1: Intel(R) 10 Gigabit Network Connection

May  8 14:52:19 labnas /etc/init.d/net.eth3[2186]: net.eth3: not allowed to be hotplugged
```

Why won't my statement in /etc/conf.d/modules work the same way running modprobe does?

----------

## bigdweeb

still having trouble, any suggestions?

----------

## ProzZerg

I ran into the same problem and I suppose this is because we got the "module autoload" option enabled in the kernel.

After adding "allow_unsupported_sfp=1" to grub.conf it all worked like a charm for me.

So problably two ways of fixing this; Either turn off the autoloading of modules or adding the option to the kernel row in grub.conf

----------

