GeezerVison is showing github.com — Slow uploads with HTTP/2 · Issue #9578 · nextcloud/documentation
Slow uploads with HTTP/2 #9578DescriptionHow to use GitHub
- Please use the 👍 reaction to show that you are affected by the same issue.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Summary
When HTTP/2 is enabled on the webserver, uploads are very slow, topping out at slightly over 1MByte/s, despite both client and server offering vastly more bandwidth. Once HTTP2 was disabled, upload speeds were improved by a factor of over x10.
All PHP/DB optimizations are in place, and more. The CPU load on server and client remains low, IO load on the DB and storage remains low. Issue is observed during large file uploads, so DB overhead isn't the issue.
This issue was observed by multiple people, e.g. here, here and here. It seems to affect both nginx and Apache.
Steps to reproduce
- Enable HTTP/2 on webserver
- Upload using a WebDAV client - this test used the Windows 10 client via a mapped network drive
- Get uploadin'
Expected behaviour
>10 Mbyte/s
Actual behaviour
1-2 Mbyte/s
Server configuration
Operating system: Debian Linux 10
Web server: Apache 2.4.38-3+deb10u4
Database: MariaDB 10.3.27-0+deb10u1
PHP version: PHP FPM-CGI 7.3.19-1~deb10u1
Nextcloud version: 19.0.7
Updated from an older Nextcloud/ownCloud or fresh install: Updated
Where did you install Nextcloud from: Included updater
Signing status:
Signing status
No errors have been found.
List of activated apps:
App list
Enabled:
- accessibility: 1.5.0
- activity: 2.12.1
- calendar: 2.0.4
- camerarawpreviews: 0.7.10
- checksum: 0.4.5
- cloud_federation_api: 1.2.0
- comments: 1.9.0
- contacts: 3.4.3
- dav: 1.15.0
- duplicatefinder: 0.0.4
- federatedfilesharing: 1.9.0
- federation: 1.9.0
- files: 1.14.0
- files_3d: 0.3.3
- files_rightclick: 0.16.0
- files_sharing: 1.11.0
- files_trashbin: 1.9.0
- files_versions: 1.12.0
- logreader: 2.4.0
- lookup_server_connector: 1.7.0
- metadata: 0.12.0
- nextcloud_announcements: 1.8.0
- notes: 3.6.4
- notifications: 2.7.0
- oauth2: 1.7.0
- password_policy: 1.9.1
- photos: 1.1.0
- privacy: 1.3.0
- provisioning_api: 1.9.0
- recommendations: 0.7.0
- serverinfo: 1.9.0
- settings: 1.1.0
- sharebymail: 1.9.0
- support: 1.2.1
- survey_client: 1.7.0
- systemtags: 1.9.0
- text: 3.0.1
- theming: 1.10.0
- twofactor_backupcodes: 1.8.0
- updatenotification: 1.9.0
- viewer: 1.3.0
- workflowengine: 2.1.0
Disabled:
- admin_audit
- contactsinteraction
- encryption
- files_external
- files_pdfviewer
- files_videoplayer
- firstrunwizard
- ransomware_protection
- user_ldap
Nextcloud configuration:
Config report
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"***REMOVED SENSITIVE VALUE***"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
"overwritewebroot": "\/",
"htaccess.RewriteBase": "\/",
"dbtype": "mysql",
"version": "19.0.7.1",
"knowledgebaseenabled": false,
"apps_paths": [
{
"path": "\/var\/www\/nextcloud\/apps",
"url": "\/apps",
"writable": true
}
],
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"loglevel": 2,
"mail_smtpmode": "sendmail",
"theme": "",
"maintenance": false,
"trashbin_retention_obligation": "auto",
"memcache.local": "\\OC\\Memcache\\APCu",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"session_lifetime": 86400,
"mysql.utf8mb4": true,
"updater.release.channel": "stable",
"preview_max_x": "2048",
"preview_max_y": "2048",
"preview_max_filesize_image": 100,
"app_install_overwrite": [
"checksum"
]
}
}
Are you using external storage, if yes which one: no
Are you using encryption: no
Are you using an external user-backend, if yes which one: no
Client configuration
Browser: -
Client software: Windows 10 WebClient service (Microsoft-WebDAV-MiniRedir/10.0.18363)
Operating system: Windows 10
Logs
Web server error log
Web server error log
no error logs during upload
Nextcloud log (data/nextcloud.log)
Nextcloud log
no nextcloud log during upload
Browser log
Browser log
Reacted by haarp, Robin, Anthony Penninga, Sashko Todorov, Kowalski Dragon, matrix07012, fuomag9, DoktorHeisenberg, wolfgangschoeffel, Bryan Drewery and 7 more Activityaledr commented on Jun 2, 2021 I see no improvement in disabling h2 and h2c.
1.8 MB/sec maximum.
Apache 2.4.43
PHP 7.4.19
Nextcloud 21.0.2 szaimen commented on Jul 7, 2021 haarp commented on Jul 8, 2021 Author show commentMore actions SimplyCorbett commented on Jul 17, 2021 CarlSchwan commented on May 18, 2022 zijiren233 commented on Oct 17, 2022 For apache, add the H2WindowSize 1048576 parameter
For nginx,add the http2_body_preread_size 1048576; parameter
Both of these can be set as global parameters Reacted by Russell Steele, zijiren, nuk3s, Alexandre D. Rogoski, Persaeus and zzjin Reacted by Alexandre D. Rogoski Reacted by Alexandre D. Rogoski sentient-russ commented on Oct 23, 2022 TY zijiren233! This allowed the same improvement while keeping https end to end encryption. I did the following:
#If your using SSLEngine on in your VirtualHost configuration for nextcloud make sure mod http2 is enabled
sudo a2enmod http2
sudo nano /etc/apache2/mods-enabled/http2.conf
#Add line H2WIndowSize 1048576
<IfModule !mpm_prefork>
Protocols h2 h2c http/1.1
H2WindowSize 1048576
</IfModule>
Reacted by MichaelHeimann Jookia commented on Oct 30, 2022 Just to add the pile of data, we went from 300KiB/s to 6MiB/s by setting client_max_body_size to 5M; A smaller buffer size did not max out the connection. prashant3285 commented on Nov 6, 2022
TY zijiren233! This allowed the same improvement while keeping https end to end encryption. I did the following:
#If your using SSLEngine on in your VirtualHost configuration for nextcloud make sure mod http2 is enabled
sudo a2enmod http2
sudo nano /etc/apache2/mods-enabled/http2.conf
#Add line H2WIndowSize 1048576
<IfModule !mpm_prefork>
Protocols h2 h2c http/1.1
H2WindowSize 1048576
</IfModule>
Thanks @zijiren233 and @sentient-russ for the guidance.
I tried H2WindowSize 1048576 but didnt see much improvement.
My connection upload limit is 3.5 MB/s but I was getting 1 MB/s
So I took trials and finally settled at H2WindowSize 5242880 . Now I am getting 3.5 MB/s as next cloud upload speed.
Also I disabled file chunking as it was causing severe delays in uploading Check Here
sudo -u www-data php var/www/nextcloud/occ config:app:set files max_chunk_size --value 0
Edit:
uploading big files was becoming an issue so moved to chunk size of 50Mb
sudo -u www-data php var/www/nextcloud/occ config:app:set files max_chunk_size --value 5242880 Reacted by Russell Steele, nuk3s and Persaeus rs-development commented on Nov 12, 2022 So does anyone experience issues with downloads or only uploads?
I am using haproxy in front of nginx/fpm. If I have enabled http2 for all connections downloads start of 50% and drop down to 300kB/s. With http2 terminated in haproxy and http/1.1 between haproxy and nginx the download maxes out my 50MBit/s connection. szaimen commented on Nov 12, 2022 Reacted by Alexandre D. Rogoski and MichaelHeimann Reacted by Alexandre D. Rogoski Reacted by Alexandre D. Rogoski szaimen commented on Jan 23, 2023 I'll move this to documentation so that it does not get lost Reacted by Alexandre D. Rogoski Reacted by Alexandre D. Rogoski Reacted by Alexandre D. Rogoski
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|