Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for disabling PQ key exchanges to avoid TLS failures #714

Open
olii opened this issue Oct 11, 2024 · 5 comments
Open

Add support for disabling PQ key exchanges to avoid TLS failures #714

olii opened this issue Oct 11, 2024 · 5 comments
Assignees
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@olii
Copy link

olii commented Oct 11, 2024

Bug Description

In Readme there is an example how to connect to the AlloyDB instance using SOCKS5 proxy:

ALL_PROXY=socks5://localhost:8000 \
HTTPS_PROXY=socks5://localhost:8000 \
    ./alloydb-auth-proxy <INSTANCE_URI>

The above command works in version 1.11.0 but it fails in the 1.11.1 and 1.11.2.

Looking in the commitlog I think that the update to golang 1.23 broke it.

Example code (or command)

No response

Stacktrace

ALL_PROXY=socks5://<proxy_url>:1080 alloydb-auth-proxy --auto-iam-authn "<instance_string>" --run-connection-test
2024/10/11 17:08:33 Authorizing with Application Default Credentials
2024/10/11 17:08:33 [<redacted>] Listening on 127.0.0.1:5432
2024/10/11 17:08:33 The proxy has started successfully and is ready for new connections!
2024/10/11 17:08:33 Connection test started
2024/10/11 17:08:33 Connection test failed
2024/10/11 17:08:33 The proxy has encountered a terminal error: Dial error: handshake failed (instance URI = "<instance_string>"): EOF


### Steps to reproduce?

1. Run the instance of Alloydb in a private VCP
2. Run a socks5 proxy in kubernets with access to the VCP
3. Run the alloydb-auth-proxy with ALL_PROXY set to the socks5 proxy created in step 2


### Environment

1. OS type and version:  Ubuntu 22.04
2. AlloyDB Proxy version: 1.11.1 +
3. Proxy invocation command (for example, `./alloydb-auth-proxy --port 6000 <INSTANCE_URI>`):


### Additional Details

_No response_
@olii olii added the type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. label Oct 11, 2024
@olii olii changed the title alloydb-auth-proxy does not use socks5 proxy from v1.11.1 alloydb-auth-proxy does not use socks5 proxy (ALL_PROXY) from v1.11.1 Oct 11, 2024
@enocom enocom added the priority: p2 Moderately-important priority. Fix may not be included in next release. label Oct 11, 2024
@enocom
Copy link
Member

enocom commented Oct 11, 2024

Thanks for the report @olii.

We've seen another unusual problem with the Cloud SQL Proxy when using Go 1.23. See:

In short, Go 1.23 introduces a new curve in the default TLS config (golang/go#67061) which we've seen to break VPN connections. Your issue looks the same (handshake fails on EOF).

As a workaround, would you mind reporting if this works for you?

GODEBUG=tlskyber=0 \
ALL_PROXY=socks5://localhost:8000 \
HTTPS_PROXY=socks5://localhost:8000 \
    ./alloydb-auth-proxy <INSTANCE_URI>

Meanwhile, we're working on tracking down whether this is something wrong with Go, something we can do in the Proxy, or a common configuration problem we'll have to deal with. Right now, I don't know what the root issue is.

@olii
Copy link
Author

olii commented Oct 12, 2024

I confirm that I can successfully connect to the SOCKS5 proxy by adding this ENV var GODEBUG=tlskyber=0 to the command line.

Thank you.

@enocom
Copy link
Member

enocom commented Oct 13, 2024

Glad to hear it -- we'll be exploring how to remove the need to set GODEBUG meanwhile.

@enocom enocom changed the title alloydb-auth-proxy does not use socks5 proxy (ALL_PROXY) from v1.11.1 TLS Handshake failures with latest version Oct 23, 2024
@enocom enocom changed the title TLS Handshake failures with latest version TLS handshake failures with latest version Oct 23, 2024
@enocom
Copy link
Member

enocom commented Nov 5, 2024

For reference here is a link to the code that sets curve preferences:

https://github.com/golang/go/blob/635c2dce04259f2c84aeac543f0305b3e7c8ed7b/src/crypto/tls/defaults.go#L18-L24

@enocom
Copy link
Member

enocom commented Nov 6, 2024

Talking with some folks on the Go team about this, I've learned this is actually a bug in some TLS servers described in http://tldr.fail/.

Basically, some servers incorrectly assume the TLS ClientHello will always be sent in a single TCP packet. With Post-Quantum (PQ) cryptography, that ClientHello is often larger than a single packet. In Go 1.24, there will be explicit curve IDs for the PQ key exchanges, such that we in the Proxy code can easily disable them. Whether we should surface that as a flag, or continue using GODEBUG=tlskyber=0 depends on how soon Go will remove that debug variable.

I'm going to convert this to a feature request in the meantime and leave it open so we can explore options as the ecosystem (and Go) evolves.

@enocom enocom added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Nov 6, 2024
@enocom enocom changed the title TLS handshake failures with latest version Add support for disabling PQ key exchanges to avoid TLS failures Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants