Swupd Curl - Download error - (92) Stream error in the HTTP/2 framing layer

General Discussion
1

Hi again all,

Fundamental question here. I just did a swupd update on my machine. All seemed to be going well until the post-package-download steps, where I get hundreds of curl download errors and reports of files “incompletely downloaded”, which doesn’t seem to trigger any error handling or even acknowledgement by Clear Linux.

Here’s a walkthough of the situation. This is not the first time this happened, by the way – last time something like this happened, my system was completely bricked.

So, ran the update: sudo swupd update

...
 [100%]

Finishing packs extraction...
    
Statistics for going from version 35780 to version 35940:

    changed bundles   : 178
    new bundles       : 0
    deleted bundles   : 0

    changed files     : 266907
    new files         : 27861
    deleted files     : 5701

Validate downloaded files
 [100%]

Starting download of remaining update content. This may take a while...

All looks great so far, it was fast and easy. But then problems appear:

Starting download of remaining update content. This may take a while...
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - File incompletely downloaded - 'https://cdn.download.clearlinux.org/update/35910/files/3363338bc2c965706ff3564df450404e749d446e5015ae6269edc219da06a2c1.tar'
Error: Curl - File incompletely downloaded - 'https://cdn.download.clearlinux.org/update/35910/files/f35fc976ad15b9d35f40a035517f19167b8b36c0ddd499e044373c67d4f025b8.tar'
Error: Curl - File incompletely downloaded - 'https://cdn.download.clearlinux.org/update/35910/files/bbb535715c92d1ab1f9e04f341270122c407fd9de2c5d460fc5778ce5d35c71d.tar'
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
Error: Curl - Download error - (92) Stream error in the HTTP/2 framing layer
 [ 78%]

And the ticker is still going up (it’s at 81% now and climbing). I’m scared the system is on the way to bricking. How can I clean up this mess? And would it be possible to enhance the update client to better understand when something goes wrong so it can stop digging itself deeper?

I don’t want another broken system on my hands because Clear Linux failed to update itself. This is a server, not a test laptop, so maybe having an option to “play it safe” with updates would be useful – it’s surprising after all these years there isn’t a robust practical solution to download mirror issues with cdn.

[Update]:
The download finished, and I was pleasantly surprised by a new block of activity I hadn’t seen before. The updater was apparently able to recognize something was amiss and had curl try to resume the failures, which seemed to work okay according to the stdout log (I’ll let you know in a subsequent update if the server reboots successfully). One line that stood out was “reducing number of parallel downloads to 1” – what is the default? My server has 272 threads (Knights Landing), so if it is one per thread I’d be in some trouble. :wink:

2
3

Thanks for posting a related issue! However, timeouts are not the case for me as nothing had timed out – everything was indeed extremely rapid and no timeout errors were reported by curl or any other component or logfile.

It is likely there is another issue causing the curl download errors in my case. One possibility may be extreme parallalism – if the download concurrency is set to, e.g. number of cores, it’s likely cdn is rejecting additional connections (I have 272 threads). If it is just a mirror refusing connection, then there may be a more graceful way of handling the problem. What are the chances a download could become corrupt (e.g. bad data is downloaded and resuming doesn’t fix the bad bits)? Is the final checksum tested against?

Thanks again.

4

i don’t know if it caused by cdn, if that the case maybe cloudflare can do better than akamai.

cdn.download.clearlinux.org really slow here, also it seems that per connection has limited around 60-75Mbps bandwith. (maybe related: Swupd insanely slow · Issue #1614 · clearlinux/swupd-client · GitHub)

Personally, i want Cloudflare proxied repository :sweat_smile: