Enabling HTTP/2 Can Break WebSocket Connections to Backends

[Gerg]

Related Issue: cloudfoundry/routing-release#230

Root Issue

When HTTP/2 is enabled, HAProxy will attempt to bootstrap WebSocket connections via HTTP/2 as defined in RFC 8441.

Since not all backends implement RFC 8441, this can break WebSocket requests that would otherwise work with HTTP/1.1. For example, Golang's HTTP server does not yet support HTTP/2 WebSockets: golang/go#32763.

Impact to Cloud Foundry

For Cloud Foundry, enabling HTTP/2 on HAProxy will break WebSocket request to apps and streaming app logs from Loggregator. This is because the Gorouter (using Golang's HTTP server) does not support WebSockets over HTTP/2.

Possible Fix

One possible workaround would be to (optionally?) configure a separate backend for WebSocket traffic over HTTP/1.1. For example, something like:

# HTTPS Frontend {{{
frontend https-in
    mode http
      bind :443  ssl crt /var/vcap/jobs/haproxy/config/ssl   alpn h2,http/1.1

    http-request del-header X-Forwarded-Client-Cert
    http-request del-header X-SSL-Client
    http-request del-header X-SSL-Client-Session-ID
    http-request del-header X-SSL-Client-Verify
    http-request del-header X-SSL-Client-Subject-DN
    http-request del-header X-SSL-Client-Subject-CN
    http-request del-header X-SSL-Client-Issuer-DN
    http-request del-header X-SSL-Client-NotBefore
    http-request del-header X-SSL-Client-NotAfter


    capture request header Host len 256
    default_backend http-routers
    acl xfp_exists hdr_cnt(X-Forwarded-Proto) gt 0
    acl is_websocket hdr(Upgrade) -i WebSocket          # Add is_websocket ACL looking for upgrade header
    acl is_websocket hdr_beg(Host) -i ws                # is_websocket ACL also looks for ws* scheme
    use_backend http-routers-ws if is_websocket         # If request matches ACL, route to http-routers-ws backend instead
    http-request add-header X-Forwarded-Proto "https" if ! xfp_exists
# }}}

# Default Backend {{{
backend http-routers
    mode http
    balance roundrobin
        errorfile 503 /var/vcap/jobs/haproxy/errorfiles/custom503.http

    server node0 10.244.0.34:443 check inter 1000  ssl verify required ca-file /var/vcap/jobs/haproxy/config/backend-ca-certs.pem alpn h2,http/1.1
# }}}
backend http-routers-ws         # New backend, identical to http-routers, except without alpn
    mode http
    balance roundrobin
        errorfile 503 /var/vcap/jobs/haproxy/errorfiles/custom503.http

    server node0 10.244.0.34:443 check inter 1000  ssl verify required ca-file /var/vcap/jobs/haproxy/config/backend-ca-certs.pem

[46bit]

Thanks @Gerg. I'm working on shipping this in #261

[peterellisjones]

Hi @Gerg we fixed this using your suggested solution in 11.6.0