Skip to content

Commit

Permalink
Stop logging client disconnect as a pilot error
Browse files Browse the repository at this point in the history
Right now when Envoy shuts down it sometimes sends an Unavailable error
code instead of Cancelled - see
etcd-io/etcd#10289 for details, but its pretty
complex. The end result is this shows up as "internal error" in the
dashboards, when there is nothing going wrong. Instead, we should filter
these out.

Checking the string directly feels a bit like a hack, but etcd does it
so seems reasonable:
https://github.com/etcd-io/etcd/blob/master/etcdserver/api/v3rpc/util.go#L113
  • Loading branch information
howardjohn committed Jan 2, 2020
1 parent ef68aa5 commit 1c63cc6
Showing 1 changed file with 18 additions and 1 deletion.
19 changes: 18 additions & 1 deletion pilot/pkg/proxy/envoy/v2/ads.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,12 +142,29 @@ func newXdsConnection(peerAddr string, stream DiscoveryStream) *XdsConnection {
}
}

// isExpectedGRPCError checks a gRPC error code and determines whether it is an expected error when
// things are operating normally. This is basically capturing when the client disconnects.
func isExpectedGRPCError(err error) bool {
if err == io.EOF {
return true
}

s := status.Convert(err)
if s.Code() == codes.Canceled {
return true
}
if s.Message() == "client disconnected" {
return true
}
return false
}

func receiveThread(con *XdsConnection, reqChannel chan *xdsapi.DiscoveryRequest, errP *error) {
defer close(reqChannel) // indicates close of the remote side.
for {
req, err := con.stream.Recv()
if err != nil {
if status.Code(err) == codes.Canceled || err == io.EOF {
if isExpectedGRPCError(err) {
con.mu.RLock()
adsLog.Infof("ADS: %q %s terminated %v", con.PeerAddr, con.ConID, err)
con.mu.RUnlock()
Expand Down

0 comments on commit 1c63cc6

Please sign in to comment.