# [solved] esata hotplug not recognized no device node created

## s|mon

I'm facing problems when powering up my external drives for backup.

For that i use an external bay (icybox/dock) where one can easily switch drives. I used the eSata connector for that to avoid the usb-bridge controller (which would limit drive size).

This worked well the past years. Since some days i no longer get the kernel event entries in dmesg which indicate a new drive is recognized and what was the assigned device node (e.g. /dev/sdh) if i power on the bay+drive.

So the kernel seems not to regonize them any more.

As this was happening with two different external bays and multiple drives i had the suspicion that the controller was not ok.

Interestingly if i have them powered on during boot they are recognized fine and ok till i power them off. After another power-up they are not picked up any more again.

Of course i don't want to reboot my server just to connect in my backup drives.

I now added a additional controller with eSata ports (ASM 1601) - no change - also works during boot but not if powered up during operation.

Furthermore i booted and older kernel (5.6.10 from may and 5.4.6 from dez 19) but also there no luck - also with 5.4.60 which i rebuild based of an old config from 5.3 line.

As everything is recognized fine in bios / when available during boot i'm curious what i can do to narrow it down?

Two controllers, three different eSata cables, multiple drives, two different bays and all was ok some weeks ago (not sure what changed or what could influence at that level)

Next i took the drive & bay and connected it to an sata port with a sata-esata cable on my workstation (which has no esata)

There i could see it detected after issuing

 *Quote:*   

> 
> 
> echo 0 0 0 | tee /sys/class/scsi_host/host*/scan
> 
> 

 

This also does not help on the affected machine.

Also all SATA controllers are in AHCI mode

 *Quote:*   

> lspci -k | grep SATA -A2
> 
> 00:1f.2 SATA controller: Intel Corporation 5 Series/3400 Series Chipset 6 port SATA AHCI Controller (rev 06)
> 
> 	Subsystem: Micro-Star International Co., Ltd. [MSI] 5 Series/3400 Series Chipset 6 port SATA AHCI Controller
> ...

 

In my kernel (5.8.5 gentoo-sources) i have the following active for SATA and AHCI

 *Quote:*   

> 
> 
>  CONFIG_SATA_HOST=y
> 
> CONFIG_SATA_PMP=y
> ...

 

What could be checked? Any powermgt parameters, any specific hotplug enable parameter i'm not aware off?

Where there changes that might require that hotplug capable devices need to be marked somehow?

[edit 1]

after booting with active power on one external disk i did first part of a backup to a disk (ST3000DM001) umounted and suspended via hdparm. Powered off the bay replaced the disk and repowered it.

This time the change was recognized - it seems it still suspected the old disk somehow as there is this model number mismatch showing.

 *Quote:*   

> 
> 
> [ 1303.747281] ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> 
> [ 1303.748862] ata8.00: configured for UDMA/133
> ...

 

Then i did my backup to that disk without problems as well.

I again did umount, put disk to sleep and powered off the bay. Waited 2 min. Then swapped disk and powered on.

Now i see that disk is removed and shows ata8 being detached.

After that it does not pick up anything on that connection any more

 *Quote:*   

> 
> 
>  2599.672085] ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> 
> [ 2602.395455] ata8.00: configured for UDMA/133
> ...

 

If i rescan again with 

"echo 0 0 0 | tee /sys/class/scsi_host/host*/scan"

ata8 is not recognized.

 *Quote:*   

> 
> 
> [ 3034.736534] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> 
> [ 3034.859621] ata1.00: configured for UDMA/100
> ...

 Last edited by s|mon on Tue Sep 15, 2020 12:33 pm; edited 1 time in total

----------

## s|mon

Tested with another cable. No luck. When i plug in on the same bay the usb-cable the disk is recognized and a device node is created for it - but with esata no entry at all.

So it seems to be

- eSata connection works when available during boot till power off for more than some seconds.

- usb works but i can not use it due to the bridge chip being limited to 2TB drives.

I assume it is not a problem with udev as it would still expect at least some entry appearing in dmesg when plugged in, correct?

I'm not aware of any specific hotplug setting in bios - nor would i remember changing sth. like that but i'll double check.

What confuses me most is that it is with more then one drivebay and all are working during boot. Not sure if it can be that they all have the same failure only affecting sth. related to hotplug on different controllers.

Kernel command line is 

 *Quote:*   

> Command line: BOOT_IMAGE=/kernel-5.8.5 root=/dev/sdb2 ro scsi_mod.use_blk_mq=1

 

Which i see no hint on disabling of hotplug?

Any further advice - is there sth. related to kernel tracing one would advice to check (never done that). I would not mind replacing the bay - but given that two differnt bays are not working i fear it might not be the faulting  piece.

----------

## NeddySeagoon

s|mon,

Does sys-apps/rescan-scsi-bus help?

It should not be needed because SATA is hotplug by design.

Some early SATA chipsets had broken hotplug support.

Does your USB bridge really have a 2TB limit?

To get past 137Mb, it must support 48 bit LBA, that's all that matters to the kernel. 

2TiB is a DOS disk label limit.

----------

## s|mon

Hi NeddySeagoon,

On rescan-scsi-bus: unfortunately not - just tested with it. 

The weirdest thing is that i needed not to scan for the past years at all it just worked fine - which hinted to a hardware issue (after i reverted back to older kernels as well). But having tested now 2 different bays (and one very old) i got doubts. 

I get another icybox to test in the next days to test one more.

Regarding the 2 TB limit: I'm not sure if it is 2 TB or what exactly. I just plugged  in a 6 TB disk and the dmesg output states

 *Quote:*   

> [Sep 8 19:31] usb 2-1.1: new high-speed USB device number 6 using ehci-pci
> 
> [  +0,079541] usb 2-1.1: New USB device found, idVendor=13fd, idProduct=1340, bcdDevice= 2.10
> 
> [  +0,000007] usb 2-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> ...

 

which is not matching. So there is some limit in that chip.

----------

## NeddySeagoon

s|mon,

That looks about right for the bottom 32 bits of the 48bit LBA count.

Without the top 16 bits,the block count wraps at 2TiB. 5.46TiB would be about right for a 6TB drive.

Its been around for a while. smartmontools got support for that USB bridge chip in April 2009.

Has the eSATA port got disabled in the BIOS ?

----------

## s|mon

The eSata port in Bios is active and set to AHCI.

Also the additional controller card is shown before grub and if sth. is connected powered on, lists the devices. 

Devices are recognized fine. After power off they of course are removed but with repowering not newly recognized no matter how long i wait.

----------

## Anon-E-moose

If you plug in your device to the esata port after you've booted, can you hear the drive spin up? Or is getting power at all. If the external case has it's own power supply make sure you're getting the right current, hdd's need both 5v and 12v IIRC or the drive won't spin up and be recognized.

----------

## s|mon

Power is ok and drive is spinning. Both trays have external power supplies and are spinning up.

----------

## s|mon

Got another external bay which did not change the situation. Same behaviour.

I got some usb stick with ubuntu 19.10 installation media on it. 

If i boot from there hotplug works fine.

E.g. dmesg shows that when i power up the disk

 *Quote:*   

> [Sep13 14:35] ata11: softreset failed (1st FIS failed)
> 
> [  +0.872012] ata11: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> 
> [  +0.001237] ata11.00: ATA-9: ST3000DM001-1CH166, CC29, max UDMA/133
> ...

 

So i think it can exclude a hardware issue with high confidence for now. 

Next guess would be kernel as it is already not showing entries in dmesg which would speak against userspace  or udev if my knowledge is correct.

What would be odd (but not impossible) is that if i had some missconfiguration in my kernel.config which i applied to all kernel versions and which would only affect hotplug.

Any hints on kernel config or parameters that could influence such things - so only affecting hotplug but not regular detection or operation?

----------

## Anon-E-moose

If ubuntu works, then see what the config looks like on that system.

What is the motherboard?

Edit to add: I suppose the chipset might have been altered with some type quirks applied in the kernel, if nothing else, might check it out, you'll need vendor/product id for searching kernel code.

----------

## s|mon

Mainboard is a: MSI H57M-ED65 (and in addition i added for testing a controller with ASMEDIA ASM1061)

i tried to check config of ubuntu vs mine. Ubuntu config has of course a lot of modules whereas i usually compile things into directly.

Config=y from ubuntu for *ATA*

 *Quote:*   

> 
> 
> CONFIG_ARCH_CLOCKSOURCE_DATA=y
> 
> CONFIG_SOCK_CGROUP_DATA=y
> ...

 

vs mine

 *Quote:*   

> CONFIG_SYSTEM_DATA_VERIFICATION=y
> 
> CONFIG_RELOCATABLE=y
> 
> CONFIG_ATA=y
> ...

 

and in addition loaded modules from ubuntu

 *Quote:*   

> Module                  Size  Used by
> 
> zfs                  3604480  6
> 
> zunicode              331776  1 zfs
> ...

 

from where i do not make out an obvious candidate - will need to wait for a quiet day for testing a new kernel.

Regarding the hint to search for vendor ID quirks i have two questions:

1. Would i find these IDs in the lspci output (e.g. in my top post).

2. And then start with grep on kernel source filtered by quirks or how would one start there (just a rough guide as i have never searched much besides existing diffs on kernel sources)

Thanks!

----------

## Anon-E-moose

lspci -nnk will give you the product/vendor id

It will look something like this

```
05:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)

   Subsystem: ASUSTeK Computer Inc. I211 Gigabit Network Connection [1043:85f0]

   Kernel driver in use: igb

   Kernel modules: igb
```

The 8086:1539 is the vendor/product id, or at least that's what you'd search for.

Me, I'd search on the vendor id first (8086) and then if found look to see if the product id (1539) shows.

----------

## NeddySeagoon

s|mon,

Building on Anon-E-mooses example, put 

```
8086:1539 pci linux
```

into google.

Hopefully, one of the top hits is  Linux Kernel Driver DataBase on cateee.net

The vendor and device ID work with USB too, so you need to tell google if you want USB or PCI results.

Once you have the symbol name, use the search in menuconfig.

----------

## s|mon

lspci for SATA would be

```
00:1f.2 SATA controller [0106]: Intel Corporation 5 Series/3400 Series Chipset 6 port SATA AHCI Controller [8086:3b22] (rev 06)

        Subsystem: Micro-Star International Co., Ltd. [MSI] 5 Series/3400 Series Chipset 6 port SATA AHCI Controller [1462:7587]

        Kernel driver in use: ahci

02:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)

        Subsystem: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:1060]

        Kernel driver in use: ahci

03:00.0 SATA controller [0106]: JMicron Technology Corp. JMB363 SATA/IDE Controller [197b:2363] (rev 03)

        Subsystem: Micro-Star International Co., Ltd. [MSI] JMB363 SATA/IDE Controller [1462:7587]

        Kernel driver in use: ahci
```

That would be the following for "197b:2363 pci linux" for my onboard eSata  which results in 

https://cateee.net/lkddb/web-lkddb/BLK_DEV_JMICRON.html - which is in the deprecated ATA part - gave it a try to test but did not make a difference.

https://cateee.net/lkddb/web-lkddb/PATA_JMICRON.html - was already activated.

For the intel and asm ones i only find the generic sata support https://cateee.net/lkddb/web-lkddb/SATA_AHCI.html

 *Quote:*   

> vendor: 1b21 ("ASMedia Technology Inc."), device: 0612 ("ASM1062 Serial ATA Controller")
> 
> vendor: 8086 ("Intel Corporation"), device: 3b22 ("5 Series/3400 Series Chipset 6 port SATA AHCI Controller")
> 
> 

 

As devices attached are working i guess the config to enable support in general is ok. Question would be how to search for quirks - which are not bound to one chip (as both the JMB and the ASM are affected).

Being not sure where to start that with kernel code i guess i'll try to take a vanilla kernel from the same series as used in the old ubuntu with my config and/or with the generic ubuntu config once and test that. 

Thanks for the support so far. I am still quite confused on what the cause can be. So thanks for any further hints.

Code tags added to preserve formatting -- NeddySeagoon

----------

## Anon-E-moose

If there is code to handle quirks, the file name will usually reflect this.

```
$ find /usr/src/linux/ -print|grep -i quirks

/usr/src/linux/arch/arm/mach-davinci/pdata-quirks.c

/usr/src/linux/arch/arm/mach-omap2/pdata-quirks.c

/usr/src/linux/arch/x86/kernel/early-quirks.c

/usr/src/linux/arch/x86/kernel/platform-quirks.c

/usr/src/linux/arch/x86/kernel/quirks.c

/usr/src/linux/arch/x86/platform/efi/quirks.c

/usr/src/linux/drivers/gpu/drm/drm_panel_orientation_quirks.c

/usr/src/linux/drivers/gpu/drm/i915/display/intel_quirks.c

/usr/src/linux/drivers/gpu/drm/i915/display/intel_quirks.h

/usr/src/linux/drivers/hid/hid-quirks.c

/usr/src/linux/drivers/hid/i2c-hid/i2c-hid-dmi-quirks.c

/usr/src/linux/drivers/iommu/amd/quirks.c

/usr/src/linux/drivers/mmc/core/quirks.h

/usr/src/linux/drivers/pci/quirks.c

/usr/src/linux/drivers/pnp/quirks.c

/usr/src/linux/drivers/scsi/ufs/ufs_quirks.h

/usr/src/linux/drivers/usb/core/quirks.c

/usr/src/linux/drivers/usb/host/pci-quirks.c

/usr/src/linux/drivers/usb/host/pci-quirks.h

/usr/src/linux/drivers/xen/xen-pciback/conf_space_quirks.c

/usr/src/linux/drivers/xen/xen-pciback/conf_space_quirks.h

/usr/src/linux/include/linux/usb/quirks.h

/usr/src/linux/include/config/drm/panel/orientation/quirks.h

/usr/src/linux/include/config/pci/quirks.h

/usr/src/linux/sound/soc/intel/common/soc-intel-quirks.h

/usr/src/linux/sound/usb/mixer_quirks.c

/usr/src/linux/sound/usb/mixer_quirks.h

/usr/src/linux/sound/usb/quirks-table.h

/usr/src/linux/sound/usb/quirks.c

/usr/src/linux/sound/usb/quirks.h
```

I would imagine that ubuntu (that works) is probably doing some type of reset on the esata port, so that unplug/plug works properly. 

You might check and see if there are some udev rules on ubuntu that aren't in gentoo.

----------

## NeddySeagoon

s|mon,

Which chipset is the eSATA port attched to? 

dmesg will tell.

On Gentoo, connect a HDD and boot that way, so we cas see that HDD come up.

On Ubuntu, connect the HDD once its booted.

We need all of dmesg on a pastebin. Comparing the two might be useful too.

For a wild guess 

Try AHCI PLATFORM SUPPORT and PORT MULTIPLIER support. OK, that's two guesses :)

That's from memory, so the kernel symbol names won't be right.

You usually get quirks for free, there is nothing to configure.

-- edit --

Reading between the lines.

```
03:00.0 SATA controller [0106]: JMicron Technology Corp. JMB363 SATA/IDE Controller [197b:2363] (rev 03)

        Subsystem: Micro-Star International Co., Ltd. [MSI] JMB363 SATA/IDE Controller [1462:7587]

        Kernel driver in use: ahci
```

The ahci driver looks wrong.

PATA_JMICRON should drive the PATA and SATA parts of that chip.

Which driver does Ubuntu use there?

----------

## s|mon

Hi, here the two dmesg outputs:

dmesg ubuntu - after two esata devices are powered up

gentoo-kernel-5.8.5 - with both esata device powered since boot

[edit: added 2nd dmesg without legacy ata see remark below]

gentoo-kernel-5-8-5 - with both esata devices powered and no legacy ata

ST3000DM001-1CH1 is on the esata port of the asm chip.

 HGST HDN726060AL  is connected to the onboard chip (JMB363 with AHCI)

Both guesses where already set:

 *Quote:*   

> CONFIG_SATA_AHCI_PLATFORM=y
> 
> CONFIG_SATA_PMP=y
> 
> 

 

regarding your assumption of PATA vs AHCI - there is also a IDE interface for the JMB363 active - need to see if that is using the same driver in ubuntu

 *Quote:*   

> 03:00.0 SATA controller [0106]: JMicron Technology Corp. JMB363 SATA/IDE Controller [197b:2363] (rev 03)
> 
> 	Subsystem: Micro-Star International Co., Ltd. [MSI] JMB363 SATA/IDE Controller [1462:7587]
> 
> 	Kernel driver in use: ahci
> ...

 

i always thought that ahci would be correct for sata devices and IDE would be only for the IDE ports i still have on that board (e.g. optical drives connected).

I now see two obvious differences regarding jmicron - i need to check if this due to me reactivating legacy for BLK_DEV_JMICRON during testing earlier today or what you hinted that it should use pata_jmicron (which is also yes in my kernel - so i guess i need to be sure it is also used?)

 *Quote:*   

> ubuntu
> 
> [    0.917763] scsi host0: pata_jmicron
> 
> [    0.918662] scsi host1: pata_jmicron
> ...

 

 *Quote:*   

> gentoo
> 
> [    0.832473] jmicron 0000:03:00.1: IDE controller (0x197b:0x2363 rev 0x03)
> 
> [    0.832562] jmicron 0000:03:00.1: 100% native mode on irq 17
> ...

 

[edit: tested again without legacy ata - now also gentoo uses pata_jmicron (see 3rd pastebin)]

----------

## s|mon

I found the reason: SATA power management was active due to powertop settings which prevents hotplug to work for eSata or i guess sata in general.

Tested now with disabling all sata related toggles in powertop and once with boot without powertop settings to be called - in both cases all drives/bays where recogniced immediately.

I'm not sure which exact toggle is causing the problem  (pm for controller or link power management or sth else) - will need some time to check that.

Thanks again to everyone for their ideas and suggestions. It really helped me going with the investigations.

----------

