Connection and Channels best practices?

[mgdotson]

The documentation example shows how to create a Client that handles reconnecting the Connection and Channel, however, how would one scale this to multiple threads?

For example, a restful interface that sends out messages based on the endpoint, would each call use the same Client or would each get a new Client with new Client.connection and Client.channel or should they get a new Client.channel with the same Client.connection?

Assuming going to the same queue, changing main slightly for illustration:

func main() {
	queueName := "myqueue"
	addr, _ := os.LookupEnv("MQSERVER")
	queue := New(queueName, addr)
	message := []byte("message")

	ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(time.Second*20))
	defer cancel()

	go Publish(ctx, queue, append(message, []byte(" 1")...))  // should these be "new" queues or just new channels?
	go Publish(ctx, queue, append(message, []byte(" 2")...))
	<-ctx.Done()
}

func Publish(ctx context.Context, queue *Client, message []byte) {
loop:
	for {
		select {
		// Attempt to push a message every 2 seconds
		case <-ctx.Done():
			fmt.Printf("Context Timed Out!\n")
			if err := queue.Close(); err != nil {
				fmt.Printf("Closing failed: %v", err)
			}
			break loop
		case <-time.After(time.Second * 2):
			if err := queue.Push(message); err != nil {
				fmt.Printf("Push failed: %s\n", err)
			} else {
				fmt.Printf("%s\n", message)
			}
		}
	}
}

Perhaps a section on best practices for connections and channels?

[Zerpet]

Generally, the best practice is to have long-lived connections and create new channels as needed. It also important to not use the same channel by multiple threads; instead, use one channel per thread/go-routine.

Another common pitfall is to use the same channel to publish and consume. It's best advised to use separate channels for consumption and publishing, because RabbitMQ may block a publisher channel due to an alarm (e.g. high memory usage), if consumers are using this channel, they will be blocked as well.

I recommend to have a look at the production check list, the apps section: https://rabbitmq.com/production-checklist.html#apps
This covers important topics, like connection/channel churn.

Answering your question, I would have my restful interface get a new channel from the same connection.

If you are building on top of the example, you will have to implement a locking mechanism to avoid races, and you will have to modify slightly the recovery mechanism to recover all known channels after a connection closed event. Note that the example aims to showcase the functions and mechanisms available to handle recovery; it is not production ready, rock-solid code.

restful interface that sends out messages based on the endpoint

Another point for consideration, if this is an HTTP based API, you would benefit greatly from having a connection pool. If you don't want to maintain your own connection pool, you may consider using a proxy like https://github.com/bloomberg/amqpprox or https://github.com/cloudamqp/amqproxy

[mgdotson]

Generally, the best practice is to have long-lived connections and create new channels as needed. It also important to not use the same channel by multiple threads; instead, use one channel per thread/go-routine.

Thanks! This is what I figured with the connections but wasn't really finding that in this document. Perhaps a link to the apps link here? (Tons of documentation on the rabbitmq site and a bit overwhelming)

If you are building on top of the example, you will have to implement a locking mechanism to avoid races, and you will have to modify slightly the recovery mechanism to recover all known channels after a connection closed event. Note that the example aims to showcase the functions and mechanisms available to handle recovery; it is not production ready, rock-solid code.

I'm not sure if a locking mechanism would be necessary. If you pull out the connection and connection recovery into a Connection and then create your Client with a pointer to the Collection, the connection would continue to reconnect as needed while the Client would handle the channel reconnect based on being closed when the connection is closed and re-opened via a client.Connection.Channel() wrapper which would ensure the connection is ready. Would this be the reference to locking?

Perhaps I'm overly simplifying and missing something here?

BTW, Thank you for adding contexts to publishing! Hope the plans are to add contexts to all of the calls (Close, Channel, etc) as that would reduce boilerplate code I currently have in my wrapper library.

[Zerpet]

Perhaps a link to the apps link here? (Tons of documentation on the rabbitmq site and a bit overwhelming)

I'm happy to do that, or to review a contribution.

Would this be the reference to locking?

Perhaps I'm overly simplifying and missing something here?

To be honest, I haven't put much thought as what needs locking. I know the Go race detector screams when you run the examples with go test -v -race -timeout 3m example_client_test.go. You'll need to add a comment like // Output: foobar at the end of the Example* functions to make them test-able. If the race detector is happy with your app, then it should be fine.

BTW, Thank you for adding contexts to publishing! Hope the plans are to add contexts to all of the calls (Close, Channel, etc) as that would reduce boilerplate code I currently have in my wrapper library.

Glad to hear you find this useful! Any feedback on this is welcome, as it's somewhat recent, it was added in v1.4.0. We don't have any plans to add context to other functions. That being said, I think it would be a nice addition to the library. Feel free to open an issue to get the discussion going.