AWS OpenSearch support

[yevhenkainaraideals]

Feature

AWS OpenSearch support

Rationale

More configuration options could increase popularity of technology

Example Scenario

Zipkin server already has support of AWS OpenSearch(https://github.com/openzipkin/zipkin-aws)

[codefromthecrypt]

@reta not asking you to volunteer to action this, but do you have any idea what the gap is? I know you don't work on AWS stuff, just in case you know.

[reta]

@reta not asking you to volunteer to action this, but do you have any idea what the gap is? I know you don't work on AWS stuff, just in case you know.

Sure @codefromthecrypt , AFAIK the AWS OpenSearch managed service does not need any special handling and should be supported already, the only gap I can think of is AWS OpenSearch Serverless that needs special treatment. @yevhenkainaraideals is it something that you had in mind?

[yevhenkainaraideals]

@reta Our context is that we have zipkin + zipkin-dependencies deployed to EKS. And we want to use IAM Role for service account to access AWS resources. In this particular case - AWS SDK is able to go to AWS STS to gather temporary credentials(by assuming role from Kubernetes service account) and later should sign every HTTP request sent to AWS OpenSearch with AWS signature version 4.

In zipkin server there is configuration related to it - https://github.com/openzipkin/zipkin-aws/blob/master/module/src/main/java/zipkin/module/aws/elasticsearch/ZipkinElasticsearchAwsStorageModule.java#L67C34-L67C54 and we are able to bring up zipkin-server that works well with our AWS OpenSearch. For zipkin-dependencies it is not working in any way I tried it. And also I see no code similar to what I see in zipkin-aws to achieve what we want.

I think that for AWS OpenSearch Serverless there will be the same problems from authentication perspective.

As a summary - to start zipkin-server with AWS OpenSearch we need to:

• set the proper URL
• pass AWS_REGION
• pass credentials in any AWS SDK compatible way(like static IAM credentials/instance profile/IAM Role for service account)

And I will be happy to have same possibility here - so we can forget about bad security design with signing proxy in the middle between zipkin-dependencies and AWS OpenSearch Service

[reta]

Thanks @yevhenkainaraideals , got it now, seems like there is generally a gap here with respect to zipkin-dependencies (cloud aware storages), or we are missing something @codefromthecrypt?

[codefromthecrypt]

so I think that in the distant past ES_WAN_ONLY maybe was used for elasticsearch on aws and affected here #45

Some sort of code change here might need to be contributed for AWS improvements as I doubt we'll want to have a second repo like zipkin-aws dealing with spark.

About that code or design of it, it isn't something I can really make time for right now, as it is likely to be 10s of hours over a week or two.