Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Logging in to Azure CLI access not working (application access) #48522

Open
gabrielcorado opened this issue Nov 6, 2024 · 0 comments · May be fixed by #48550
Open

Logging in to Azure CLI access not working (application access) #48522

gabrielcorado opened this issue Nov 6, 2024 · 0 comments · May be fixed by #48550
Assignees
@greedy52
Labels
application-access bug test-plan-problem Issues which have been surfaced by running the manual release test plan

Comments

@gabrielcorado
Copy link
Contributor

gabrielcorado commented Nov 6, 2024 •
edited
Loading

Expected behavior

When logging in to Azure CLI access, the tsh app login azure-cli --azure-identity xxx command should succeed, and credentials to Azure CLI access should be generated.

Current behavior

The tsh app login command fails, generating no credentials.

Bug details

Debug logs

Teleport 
2024-11-06T14:48:04Z DEBU [PROXY:SER] Connecting to 192.168.97.1:35484 through tunnel. trace.fields:map[cluster:root] reversetunnel/localsite.go:881
2024-11-06T14:48:05Z DEBU [PROXY:SER] Succeeded dialing from: "@web-proxy" to: "@local-node". trace.fields:map[cluster:root] reversetunnel/localsite.go:301
2024-11-06T14:48:05Z DEBU [APP:WEB]   failed to re-sign azure JWT error:[
ERROR REPORT:
Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive
Stack Trace:
        github.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken
        github.com/gravitational/teleport/lib/web/app/transport.go:312 github.com/gravitational/teleport/lib/web/app.(*transport).resignAzureJWTCookie
        github.com/gravitational/teleport/lib/web/app/transport.go:263 github.com/gravitational/teleport/lib/web/app.(*transport).rewriteRequest
        github.com/gravitational/teleport/lib/web/app/transport.go:166 github.com/gravitational/teleport/lib/web/app.(*transport).RoundTrip
        github.com/gravitational/teleport/lib/httplib/reverseproxy/reverse_proxy.go:216 github.com/gravitational/teleport/lib/httplib/reverseproxy.(*roundTripperWithLogger).RoundTrip
        net/http/httputil/reverseproxy.go:481 net/http/httputil.(*ReverseProxy).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:289 github.com/gravitational/teleport/lib/web/app.(*Handler).handleHttp
        github.com/gravitational/teleport/lib/web/app/middleware.go:57 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.func4
        github.com/gravitational/teleport/lib/web/app/middleware.go:108 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.makeHandler.func6
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        github.com/julienschmidt/[email protected]/router.go:460 github.com/julienschmidt/httprouter.(*Router).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:157 github.com/gravitational/teleport/lib/web/app.(*Handler).ServeHTTP
        github.com/gravitational/teleport/lib/web/apiserver.go:431 github.com/gravitational/teleport/lib/web.(*APIHandler).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/internal/ratelimit/ratelimit.go:106 github.com/gravitational/teleport/lib/limiter/internal/ratelimit.(*TokenLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/connlimiter.go:82 github.com/gravitational/teleport/lib/limiter.(*ConnectionsLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/limiter.go:82 github.com/gravitational/teleport/lib/limiter.(*Limiter).ServeHTTP
        github.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        net/http/server.go:3210 net/http.serverHandler.ServeHTTP
        net/http/server.go:2092 net/http.(*conn).serve
        runtime/asm_arm64.s:1223 runtime.goexit
User Message: azure jwt signed by unknown key
        go-jose/go-jose: error in cryptographic primitive] app/transport.go:268
2024-11-06T14:48:05Z DEBU [PROXY:SER] Dialing from: "192.168.97.1:64545" to: "@local-node". trace.fields:map[cluster:root] reversetunnel/localsite.go:295
2024-11-06T14:48:05Z DEBU [PROXY:SER] Tunnel dialing to 825aef96-a930-46e0-80bb-572fc95a96c4.root, client source 192.168.97.1:64545 trace.fields:map[cluster:root] reversetunnel/localsite.go:441
2024-11-06T14:48:05Z DEBU [PROXY:SER] Connecting to 192.168.97.1:35484 through tunnel. trace.fields:map[cluster:root] reversetunnel/localsite.go:881
2024-11-06T14:48:05Z DEBU [PROXY:SER] Succeeded dialing from: "192.168.97.1:64545" to: "@local-node". trace.fields:map[cluster:root] reversetunnel/localsite.go:301
2024-11-06T14:48:05Z INFO [APP:WEB]   Round trip: GET /subscriptions?api-version=2022-12-01, code: 500, duration: 662.020854ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:proxy.teleport.dev reverseproxy/reverse_proxy.go:223
2024-11-06T14:48:05Z INFO  emitting audit event event_type:app.session.chunk fields:map[app_name:azure-cli app_public_addr:azure-cli.proxy.teleport.dev app_uri:cloud://Azure azure_identity:/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourcegroups/database-access/providers/microsoft.managedidentity/userassignedidentities/identity cluster_name:root code:T2008I ei:6.3907146673e+10 event:app.session.chunk namespace:default private_key_policy:none server_id:825aef96-a930-46e0-80bb-572fc95a96c4 server_version:17.0.0-alpha.4 session_chunk_id:798df200-ce28-4e92-a955-6dae8ef2f383 sid:cfca50ebbd12629da7b136d8bf129966a73bcfed8a989aed9562f21de8544ecf time:2024-11-06T14:48:05.907Z trace.component:audit uid:83609883-b9ad-411c-9d56-5a75d9571205 user:alice user_kind:1] events/emitter.go:287
2024-11-06T14:48:05Z DEBU [APP:WEB]   failed to re-sign azure JWT error:[
ERROR REPORT:
Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive
Stack Trace:
        github.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken
        github.com/gravitational/teleport/lib/web/app/transport.go:312 github.com/gravitational/teleport/lib/web/app.(*transport).resignAzureJWTCookie
        github.com/gravitational/teleport/lib/web/app/transport.go:263 github.com/gravitational/teleport/lib/web/app.(*transport).rewriteRequest
        github.com/gravitational/teleport/lib/web/app/transport.go:166 github.com/gravitational/teleport/lib/web/app.(*transport).RoundTrip
        github.com/gravitational/teleport/lib/httplib/reverseproxy/reverse_proxy.go:216 github.com/gravitational/teleport/lib/httplib/reverseproxy.(*roundTripperWithLogger).RoundTrip
        net/http/httputil/reverseproxy.go:481 net/http/httputil.(*ReverseProxy).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:289 github.com/gravitational/teleport/lib/web/app.(*Handler).handleHttp
        github.com/gravitational/teleport/lib/web/app/middleware.go:57 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.func4
        github.com/gravitational/teleport/lib/web/app/middleware.go:108 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.makeHandler.func6
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        github.com/julienschmidt/[email protected]/router.go:460 github.com/julienschmidt/httprouter.(*Router).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:157 github.com/gravitational/teleport/lib/web/app.(*Handler).ServeHTTP
        github.com/gravitational/teleport/lib/web/apiserver.go:431 github.com/gravitational/teleport/lib/web.(*APIHandler).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/internal/ratelimit/ratelimit.go:106 github.com/gravitational/teleport/lib/limiter/internal/ratelimit.(*TokenLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/connlimiter.go:82 github.com/gravitational/teleport/lib/limiter.(*ConnectionsLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/limiter.go:82 github.com/gravitational/teleport/lib/limiter.(*Limiter).ServeHTTP
        github.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        net/http/server.go:3210 net/http.serverHandler.ServeHTTP
        net/http/server.go:2092 net/http.(*conn).serve
        runtime/asm_arm64.s:1223 runtime.goexit
User Message: azure jwt signed by unknown key
        go-jose/go-jose: error in cryptographic primitive] app/transport.go:268
2024-11-06T14:48:05Z INFO [APP:WEB]   Round trip: GET /subscriptions?api-version=2022-12-01, code: 500, duration: 125.260498ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:proxy.teleport.dev reverseproxy/reverse_proxy.go:223
2024-11-06T14:48:07Z DEBU [APP:WEB]   failed to re-sign azure JWT error:[
ERROR REPORT:
Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive
Stack Trace:
        github.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken
        github.com/gravitational/teleport/lib/web/app/transport.go:312 github.com/gravitational/teleport/lib/web/app.(*transport).resignAzureJWTCookie
        github.com/gravitational/teleport/lib/web/app/transport.go:263 github.com/gravitational/teleport/lib/web/app.(*transport).rewriteRequest
        github.com/gravitational/teleport/lib/web/app/transport.go:166 github.com/gravitational/teleport/lib/web/app.(*transport).RoundTrip
        github.com/gravitational/teleport/lib/httplib/reverseproxy/reverse_proxy.go:216 github.com/gravitational/teleport/lib/httplib/reverseproxy.(*roundTripperWithLogger).RoundTrip
        net/http/httputil/reverseproxy.go:481 net/http/httputil.(*ReverseProxy).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:289 github.com/gravitational/teleport/lib/web/app.(*Handler).handleHttp
        github.com/gravitational/teleport/lib/web/app/middleware.go:57 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.func4
        github.com/gravitational/teleport/lib/web/app/middleware.go:108 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.makeHandler.func6
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        github.com/julienschmidt/[email protected]/router.go:460 github.com/julienschmidt/httprouter.(*Router).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:157 github.com/gravitational/teleport/lib/web/app.(*Handler).ServeHTTP
        github.com/gravitational/teleport/lib/web/apiserver.go:431 github.com/gravitational/teleport/lib/web.(*APIHandler).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/internal/ratelimit/ratelimit.go:106 github.com/gravitational/teleport/lib/limiter/internal/ratelimit.(*TokenLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/connlimiter.go:82 github.com/gravitational/teleport/lib/limiter.(*ConnectionsLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/limiter.go:82 github.com/gravitational/teleport/lib/limiter.(*Limiter).ServeHTTP
        github.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        net/http/server.go:3210 net/http.serverHandler.ServeHTTP
        net/http/server.go:2092 net/http.(*conn).serve
        runtime/asm_arm64.s:1223 runtime.goexit
User Message: azure jwt signed by unknown key
        go-jose/go-jose: error in cryptographic primitive] app/transport.go:268
2024-11-06T14:48:07Z INFO [APP:WEB]   Round trip: GET /subscriptions?api-version=2022-12-01, code: 500, duration: 126.476998ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:proxy.teleport.dev reverseproxy/reverse_proxy.go:223
2024-11-06T14:48:10Z DEBU [APP:WEB]   failed to re-sign azure JWT error:[
ERROR REPORT:
Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive
Stack Trace:
        github.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken
        github.com/gravitational/teleport/lib/web/app/transport.go:312 github.com/gravitational/teleport/lib/web/app.(*transport).resignAzureJWTCookie
        github.com/gravitational/teleport/lib/web/app/transport.go:263 github.com/gravitational/teleport/lib/web/app.(*transport).rewriteRequest
        github.com/gravitational/teleport/lib/web/app/transport.go:166 github.com/gravitational/teleport/lib/web/app.(*transport).RoundTrip
        github.com/gravitational/teleport/lib/httplib/reverseproxy/reverse_proxy.go:216 github.com/gravitational/teleport/lib/httplib/reverseproxy.(*roundTripperWithLogger).RoundTrip
        net/http/httputil/reverseproxy.go:481 net/http/httputil.(*ReverseProxy).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:289 github.com/gravitational/teleport/lib/web/app.(*Handler).handleHttp
        github.com/gravitational/teleport/lib/web/app/middleware.go:57 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.func4
        github.com/gravitational/teleport/lib/web/app/middleware.go:108 github.com/gravitational/teleport/lib/web/app.NewHandler.(*Handler).withAuth.makeHandler.func6
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        github.com/julienschmidt/[email protected]/router.go:460 github.com/julienschmidt/httprouter.(*Router).ServeHTTP
        github.com/gravitational/teleport/lib/web/app/handler.go:157 github.com/gravitational/teleport/lib/web/app.(*Handler).ServeHTTP
        github.com/gravitational/teleport/lib/web/apiserver.go:431 github.com/gravitational/teleport/lib/web.(*APIHandler).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/internal/ratelimit/ratelimit.go:106 github.com/gravitational/teleport/lib/limiter/internal/ratelimit.(*TokenLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/connlimiter.go:82 github.com/gravitational/teleport/lib/limiter.(*ConnectionsLimiter).ServeHTTP
        github.com/gravitational/teleport/lib/limiter/limiter.go:82 github.com/gravitational/teleport/lib/limiter.(*Limiter).ServeHTTP
        github.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1
        net/http/server.go:2220 net/http.HandlerFunc.ServeHTTP
        net/http/server.go:3210 net/http.serverHandler.ServeHTTP
        net/http/server.go:2092 net/http.(*conn).serve
        runtime/asm_arm64.s:1223 runtime.goexit
User Message: azure jwt signed by unknown key
        go-jose/go-jose: error in cryptographic primitive] app/transport.go:268
2024-11-06T14:48:11Z INFO [APP:WEB]   Round trip: GET /subscriptions?api-version=2022-12-01, code: 500, duration: 125.885915ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:proxy.teleport.dev reverseproxy/reverse_proxy.go:223
tsh 
2024-11-06T11:48:02-03:00 INFO [CLIENT]    ALPN connection upgrade required for "proxy.teleport.dev:4443": false. client/api.go:866
2024-11-06T11:48:02-03:00 INFO [CLIENT]    no host login given. defaulting to gabrielcorado client/api.go:1210
2024-11-06T11:48:02-03:00 INFO [CLIENT]    [KEY AGENT] Connected to the system agent: "/private/tmp/com.apple.launchd.xcQV1NhlUr/Listeners" client/api.go:4661
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 INFO [KEYAGENT]  Loading SSH key for user "alice" and cluster "root". client/keyagent.go:198
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [TSH]       Azure identity is "/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourcegroups/database-access/providers/microsoft.managedidentity/userassignedidentities/identity" common/app.go:567
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [CLIENT]    MFA not required for access. client/cluster_client.go:564
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [CLIENT]    not using loopback pool for remote proxy addr: proxy.teleport.dev:4443 client/api.go:4616
2024-11-06T11:48:02-03:00 DEBU  Attempting request to Proxy web api method:GET host:proxy.teleport.dev:4443 path:/webapi/ping trace_id:0c4e9ef56329dced9136d2f9b69c45d1 span_id:df2c41e65399886a webclient/webclient.go:131
2024-11-06T11:48:02-03:00 DEBU  ALPN connection upgrade test complete address:proxy.teleport.dev:4443 upgrade_required:false trace_id:0c4e9ef56329dced9136d2f9b69c45d1 span_id:df2c41e65399886a client/alpn_conn_upgrade.go:96
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:02-03:00 DEBU [KEYSTORE]  Adding known host root with proxy proxy.teleport.dev client/trusted_certs_store.go:395
2024-11-06T11:48:02-03:00 DEBU [TSH]       Running automatic az login: tsh --debug az login --identity -u /subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourcegroups/database-access/providers/microsoft.managedidentity/userassignedidentities/identity common/app.go:154
2024-11-06T11:48:03-03:00 INFO [CLIENT]    ALPN connection upgrade required for "proxy.teleport.dev:4443": false. client/api.go:866
2024-11-06T11:48:03-03:00 INFO [CLIENT]    no host login given. defaulting to gabrielcorado client/api.go:1210
2024-11-06T11:48:03-03:00 INFO [CLIENT]    [KEY AGENT] Connected to the system agent: "/private/tmp/com.apple.launchd.xcQV1NhlUr/Listeners" client/api.go:4661
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 INFO [KEYAGENT]  Loading SSH key for user "alice" and cluster "root". client/keyagent.go:198
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-11-07 02:47:58 +0000 UTC". client/client_store.go:123
2024-11-06T11:48:03-03:00 DEBU [CLIENT]    Local CA renewed: valid until 2024-11-07T02:47:58Z [valid for 12h0m0s] client/local_proxy_middleware.go:449
2024-11-06T11:48:03-03:00 INFO [LOCALPROX] Starting HTTP access proxy alpnproxy/local_proxy.go:352
2024-11-06T11:48:03-03:00 DEBU [TSH]       Running command: "/opt/homebrew/bin/az login --identity -u /subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourcegroups/database-access/providers/microsoft.managedidentity/userassignedidentities/identity" common/app_azure.go:184
2024-11-06T11:48:03-03:00 DEBU             Started forwarding request for "raw.githubusercontent.com:443". alpnproxy/forward_proxy.go:368
2024-11-06T11:48:03-03:00 ERRO             Failed to proxy between "127.0.0.1:55184" and "192.168.1.141:55187". error:[
ERROR REPORT:
Original Error: trace.aggregate writeto tcp 127.0.0.1:55184->127.0.0.1:55186: readfrom tcp 192.168.1.141:55187->185.199.110.133:443: read tcp 127.0.0.1:55184->127.0.0.1:55186: read: connection reset by peer
Stack Trace:
        github.com/gravitational/teleport/lib/utils/proxyconn.go:95 github.com/gravitational/teleport/lib/utils.ProxyConn
        github.com/gravitational/teleport/lib/srv/alpnproxy/forward_proxy.go:371 github.com/gravitational/teleport/lib/srv/alpnproxy.startForwardProxy
        github.com/gravitational/teleport/lib/srv/alpnproxy/forward_proxy.go:239 github.com/gravitational/teleport/lib/srv/alpnproxy.(*ForwardToHostHandler).Handle
        github.com/gravitational/teleport/lib/srv/alpnproxy/forward_proxy.go:144 github.com/gravitational/teleport/lib/srv/alpnproxy.(*ForwardProxy).ServeHTTP
        net/http/server.go:3210 net/http.serverHandler.ServeHTTP
        net/http/server.go:2092 net/http.(*conn).serve
        runtime/asm_amd64.s:1700 runtime.goexit
User Message: writeto tcp 127.0.0.1:55184->127.0.0.1:55186: readfrom tcp 192.168.1.141:55187->185.199.110.133:443: read tcp 127.0.0.1:55184->127.0.0.1:55186: read: connection reset by peer] alpnproxy/forward_proxy.go:372
2024-11-06T11:48:03-03:00 DEBU             Stopped forwarding request for "raw.githubusercontent.com:443". alpnproxy/forward_proxy.go:374



2024-11-06T11:48:04-03:00 DEBU             Started forwarding request for "azure-msi.teleport.dev:443". alpnproxy/forward_proxy.go:368
2024-11-06T11:48:04-03:00 DEBU [CA]        Generating TLS certificate common_name:azure-msi.teleport.dev dns_names:[azure-msi.teleport.dev] key_usage:1 not_after:2024-11-07 02:47:58 +0000 UTC tlsca/ca.go:1232
2024-11-06T11:48:04-03:00 INFO [AZURE_MSI] MSI: returning token for identity /subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourcegroups/database-access/providers/microsoft.managedidentity/userassignedidentities/identity alpnproxy/azure_msi_middleware.go:132
2024-11-06T11:48:04-03:00 DEBU             Stopped forwarding request for "azure-msi.teleport.dev:443". alpnproxy/forward_proxy.go:374
2024-11-06T11:48:04-03:00 DEBU             Started forwarding request for "azure-msi.teleport.dev:443". alpnproxy/forward_proxy.go:368
2024-11-06T11:48:04-03:00 INFO [AZURE_MSI] MSI: returning token for identity /subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourcegroups/database-access/providers/microsoft.managedidentity/userassignedidentities/identity alpnproxy/azure_msi_middleware.go:132
2024-11-06T11:48:04-03:00 DEBU             Stopped forwarding request for "azure-msi.teleport.dev:443". alpnproxy/forward_proxy.go:374
2024-11-06T11:48:04-03:00 DEBU             Started forwarding request for "management.azure.com:443". alpnproxy/forward_proxy.go:368
2024-11-06T11:48:04-03:00 DEBU [CA]        Generating TLS certificate common_name:management.azure.com dns_names:[management.azure.com] key_usage:1 not_after:2024-11-07 02:47:58 +0000 UTC tlsca/ca.go:1232
2024-11-06T11:48:05-03:00 WARN [LOCALPROX] "ERROR REPORT: Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive Stack Trace:\n\tgithub.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:275 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).parseAuthHeader\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:237 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).replaceAuthHeaders\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:202 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).prepareForwardRequest\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:155 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:143 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:369 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveSession\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:438 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:685 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/auth/middleware.go:804 github.com/gravitational/teleport/lib/auth.(*Middleware).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:3210 net/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2092 net/http.(*conn).serve\n\truntime/asm_amd64.s:1700 runtime.goexit\n\n\tUser Message: failed to parse Authorization header\n\tgo-jose/go-jose: error in cryptographic primitive" alpnproxy/local_proxy.go:324
2024-11-06T11:48:05-03:00 WARN [LOCALPROX] "ERROR REPORT: Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive Stack Trace:\n\tgithub.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:275 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).parseAuthHeader\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:237 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).replaceAuthHeaders\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:202 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).prepareForwardRequest\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:155 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:143 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:369 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveSession\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:438 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:685 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/auth/middleware.go:804 github.com/gravitational/teleport/lib/auth.(*Middleware).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:3210 net/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2092 net/http.(*conn).serve\n\truntime/asm_amd64.s:1700 runtime.goexit\n\n\tUser Message: failed to parse Authorization header\n\tgo-jose/go-jose: error in cryptographic primitive" alpnproxy/local_proxy.go:324
2024-11-06T11:48:07-03:00 WARN [LOCALPROX] "ERROR REPORT: Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive Stack Trace:\n\tgithub.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:275 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).parseAuthHeader\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:237 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).replaceAuthHeaders\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:202 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).prepareForwardRequest\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:155 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:143 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:369 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveSession\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:438 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:685 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/auth/middleware.go:804 github.com/gravitational/teleport/lib/auth.(*Middleware).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:3210 net/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2092 net/http.(*conn).serve\n\truntime/asm_amd64.s:1700 runtime.goexit\n\n\tUser Message: failed to parse Authorization header\n\tgo-jose/go-jose: error in cryptographic primitive" alpnproxy/local_proxy.go:324
2024-11-06T11:48:11-03:00 WARN [LOCALPROX] "ERROR REPORT: Original Error: *errors.errorString go-jose/go-jose: error in cryptographic primitive Stack Trace:\n\tgithub.com/gravitational/teleport/lib/jwt/jwt.go:621 github.com/gravitational/teleport/lib/jwt.(*Key).VerifyAzureToken\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:275 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).parseAuthHeader\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:237 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).replaceAuthHeaders\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:202 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).prepareForwardRequest\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:155 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/azure/handler.go:143 github.com/gravitational/teleport/lib/srv/app/azure.(*handler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:369 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveSession\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:438 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).serveHTTP\n\tgithub.com/gravitational/teleport/lib/srv/app/connections_handler.go:685 github.com/gravitational/teleport/lib/srv/app.(*ConnectionsHandler).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/auth/middleware.go:804 github.com/gravitational/teleport/lib/auth.(*Middleware).ServeHTTP\n\tgithub.com/gravitational/teleport/lib/httplib/httplib.go:104 github.com/gravitational/teleport/lib/httplib.MakeTracingHandler.func1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:177 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:65 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\tnet/http/server.go:2220 net/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:3210 net/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2092 net/http.(*conn).serve\n\truntime/asm_amd64.s:1700 runtime.goexit\n\n\tUser Message: failed to parse Authorization header\n\tgo-jose/go-jose: error in cryptographic primitive" alpnproxy/local_proxy.go:324

Note: I've tested with older v16 (16.3) versions, and it is working correctly. Using tsh v16 with server v17 also works. (mostly likely a tsh problem)
@gabrielcorado gabrielcorado added application-access bug test-plan-problem Issues which have been surfaced by running the manual release test plan labels Nov 6, 2024
@gabrielcorado gabrielcorado mentioned this issue Nov 6, 2024
Open
@greedy52 greedy52 linked a pull request Nov 6, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@greedy52 greedy52

Labels
application-access bug test-plan-problem Issues which have been surfaced by running the manual release test plan
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Properly propagate private key for azure app gravitational/teleport
2 participants
@gabrielcorado @greedy52