I’m seeing an issue with a specific combination of circumstances that causes TCP duplicate ACKs to be sent. This happens with a service hosted on Ubuntu Server 20.04.
I have a web service that, among many things, accepts uploads with multipart/form-data. When setting up a new version of the service on a new server I observed slow behavior and eventually landed on the following.
I have two computers to test from. When sending a request to it from a Windows 10 computer, some packets would be ACKed twice or ACKed out of order and then retransmitted. Here’s a screenshot of Wireshark’s TCP window scaling analysis when sending a 25 MB file with random bytes – the window never gets big enough to keep up throughput due to frequent (every 10-20 packets or so) duplicate ACKs or out-of-order ACKs and corresponding retransmissions. It takes around 55 seconds to transfer the file because of this.
When sending a request to it from a macOS 10.15 computer, I don’t see this behavior. Here’s a screenshot of Wireshark’s TCP window scaling analysis – the window quickly grows and all the data is transfered in a handful of seconds the way it should be.
Using identical versions of Chrome and Firefox and also Safari on Mac and (old) Edge on Windows, the behavior is the same – all browsers on each computer are affected by the same (respective) behavior. This seems to me to indicate that there is some sort of TCP parameter tweak needing to happen.
To isolate the web stack I’m using, I wrote a minimal test program in Node.js v12 which accepts uploads with multipart/form-data, and I get identical behavior – bad behavior from Windows and good behavior from Mac. Node.js uses libuv and the other stack is ASP.NET Core using the Kestrel server which is configured with a non-libuv transport, so their networking have nothing in common above the OS’s sockets.
As far as I can tell, both the good behavior and the bad behavior is maintained without modification when reverse proxied through nginx (adding TLS termination).
You might say that this is related to the network on my side. Both computers reside on the same wireless network behind the same router, and the Linux server is hosted in a different country, but I also see similar behavior when I try using computers not from within my network, or through cellular data.
As a final parameter, when hosting the test program or the web service on the Mac and sending a similar upload request from the Windows computer, I get the good, fast behavior. In other words, the Windows computer is not just throttled or riddled with packet loss compared to the Mac – it can work, but somehow when connected to the Linux server it ends up not working. (The Windows computer and Mac post similar scores in bandwidth tests.)
So, all of this seems to tell me that there’s some sort of bad dice roll when Ubuntu 20.04’s TCP settings (or the way they’re configured at my host or affected by my host’s environment) meets the Windows TCP stack’s settings. What could cause these things, and what can I change on the server to hopefully force a good behavior?
(It’s possible this is more of a generic Linux networking question – I’m posting it here because I’m running into it on Ubuntu Server 20.04.)
0 Asked on October 25, 2020 by joe-corneli
0 Asked on October 23, 2020 by vivek-sharma
1 Asked on October 21, 2020 by obe
1 Asked on October 16, 2020 by user2483829
0 Asked on October 16, 2020 by kathya-pau-acua
4 Asked on October 16, 2020 by gaurab-kumar
4 Asked on October 14, 2020 by eslam-karem
0 Asked on October 13, 2020 by susheel
2 Asked on October 13, 2020 by user311751
0 Asked on October 4, 2020
1 Asked on October 3, 2020 by wirawan-purwanto
3 Asked on October 2, 2020 by user132060
0 Asked on October 1, 2020 by diwakar-yadav
0 Asked on September 30, 2020 by sztankay-zsolt
0 Asked on September 27, 2020 by andrew-josephus
1 Asked on September 24, 2020 by kumar-ketkar
1 Asked on September 20, 2020 by thundr_strike
Get help from others!