[fedora-arm] Ethernet monopolizing LAN9514 upstream USB bus

[Bernardo Donadio]

Is there a way to stop the Ethernet controller from monopolizing the
upstream USB channel, while in high usage, in detriment of other USB
devices on LAN9514 (Raspberry Pi 2 USB-Hub/Ethernet chip)?

Basically, if you have a USB-attached storage device and start writing
data that comes from the Ethernet into this USB disk, the Ethernet chip
will always have priority on the USB bus and the kernel will never have
the chance to use it to write on the storage device.

As a result, the data incoming from the Ethernet gets stored in the
kernel IO buffer until memory runs out, and then the transfer stops for
a while (as the kernel starting dropping TCP packets and the TCP flow
control algorithm kicks in). On the other hand, if you stop the network
transfer before the buffer is filled, the content on it is immediately
flushed to disk.

This would not be a huge problem if the USB-attached storage devices
didn't have a timeout: after not receiving some keepalive packets from
the system it's attached to, it dies. With my USB disk, at least, you
have to unplug its USB cable and plug it again to convince its
controller that the system is back up.

Of course, this is a even bigger issue on Raspberry if the system's root
is installed on this USB disk.

I'm not sure that this is fixed in the Raspberry Pi Foundation's remix
of Debian, but I'm definitely hitting this bug on Fedora 25 beta with
Linux 4.8.0-0.rc7.git0.1.fc25.armv7hl.

How to reproduce, considering /dev/sda is some USB-attached storage on
the rasp2:
desktop$ dd if=some_big_file bs=8M | ssh ***@rasp2 dd of=/dev/sda bs=8M

Meanwhile, you may watch the kernel buffer's filling up on the rasp2
with htop or free. You may also notice that if you cancel the SSH
transfer before the rasp2 kernel buffer is filled (and also before your
USB disk gives up waiting on the keepalive packet), the content stored
on the buffer is almost immediately flushed to the disk.

How can we fix this?

[Peter Robinson]

On Sun, Oct 23, 2016 at 7:36 AM, Bernardo Donadio

Post by Bernardo Donadio
Is there a way to stop the Ethernet controller from monopolizing the
upstream USB channel, while in high usage, in detriment of other USB devices
on LAN9514 (Raspberry Pi 2 USB-Hub/Ethernet chip)?

Not really. You might try putting it into polling mode as opposed to
the standard interrupt mode but I'm not even sure the usb NICs would
support that.

Post by Bernardo Donadio
Basically, if you have a USB-attached storage device and start writing data
that comes from the Ethernet into this USB disk, the Ethernet chip will
always have priority on the USB bus and the kernel will never have the
chance to use it to write on the storage device.
As a result, the data incoming from the Ethernet gets stored in the kernel
IO buffer until memory runs out, and then the transfer stops for a while (as
the kernel starting dropping TCP packets and the TCP flow control algorithm
kicks in). On the other hand, if you stop the network transfer before the
buffer is filled, the content on it is immediately flushed to disk.
This would not be a huge problem if the USB-attached storage devices didn't
have a timeout: after not receiving some keepalive packets from the system
it's attached to, it dies. With my USB disk, at least, you have to unplug
its USB cable and plug it again to convince its controller that the system
is back up.
Of course, this is a even bigger issue on Raspberry if the system's root is
installed on this USB disk.
I'm not sure that this is fixed in the Raspberry Pi Foundation's remix of
Debian, but I'm definitely hitting this bug on Fedora 25 beta with Linux
4.8.0-0.rc7.git0.1.fc25.armv7hl.

Well I have no interest in doing kernel archaeology on a fork of an
old kernel. we use the upstream kernel for a reason. You probably want
to "dnf upgrade" to the latest 4.8.3 kernel although I doubt it'll
have any effect.

Post by Bernardo Donadio
How to reproduce, considering /dev/sda is some USB-attached storage on the
Meanwhile, you may watch the kernel buffer's filling up on the rasp2 with
htop or free. You may also notice that if you cancel the SSH transfer before
the rasp2 kernel buffer is filled (and also before your USB disk gives up
waiting on the keepalive packet), the content stored on the buffer is almost
immediately flushed to the disk.
How can we fix this?

We probably can't, but ultimately the RPi has a single, some what
buggy, usb interface with a hub attached, it's not designed for high
throughput data IO workloads. You're probably better off getting a
device that's more suited to those sorts of workloads. Even the Orange
Pi series of devices are better in this regard as they have a
dedicated SoC attached NIC and 3 usb root devices.

Peter
_______________________________________________
arm mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to arm-***@lists.fedor

[Bernardo Donadio]

Post by Peter Robinson

Post by Bernardo Donadio
How can we fix this?

We probably can't, but ultimately the RPi has a single, some what
buggy, usb interface with a hub attached, it's not designed for high
throughput data IO workloads. You're probably better off getting a
device that's more suited to those sorts of workloads. Even the Orange
Pi series of devices are better in this regard as they have a
dedicated SoC attached NIC and 3 usb root devices.

Well, ok. I totally understand that this is a somewhat hacky design,
with inherent flaws. Anyway, the caveat is now documented for future
Google lurkers. :)

Thanks.