Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API GW Management - 403 #889

Closed
2 tasks done
Dasio opened this issue Nov 11, 2020 · 14 comments
Closed
2 tasks done

API GW Management - 403 #889

Dasio opened this issue Nov 11, 2020 · 14 comments
Assignees
Labels
bug This issue is a bug. closed-for-staleness response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. service-api This issue is due to a problem in a service API, not the SDK implementation.

Comments

@Dasio
Copy link

Dasio commented Nov 11, 2020

Describe the question
I'm upgrading SDK from 0.9 to 0.29 and came to issue when I'm sending data to websocket connection.

My old code which is still working with 0.9

func main() {
	config, err := external.LoadDefaultAWSConfig()
	if err != nil {
		panic(err)
	}
	api = apigatewaymanagementapi.New(config)
	e := aws.ResolveWithEndpointURL("https://ws.mydomain.com/app")
	api.Config.EndpointResolver = e
	req := &apigatewaymanagementapi.PostToConnectionInput{
		ConnectionId: aws.String("V1nqOfKwliACGGg="),
		Data:         []byte(`{"test":"yolo"}`),
	}
	res, err := api.PostToConnectionRequest(req).Send(context.Background())
}

New code with 0.29, I'm getting
operation error ApiGatewayManagementApi: PostToConnection, https response error StatusCode: 403, RequestID: <id>, ForbiddenException:

type Resolver struct {
}

func (r *Resolver) ResolveEndpoint(region string, options apigatewaymanagementapi.EndpointResolverOptions) (aws.Endpoint, error) {
	return aws.Endpoint{
		URL: "https://ws.mydomain.com/app",
		SigningRegion: region,
	}, nil
}

func main() {
	r := &Resolver{}
	
	cfg, err := config.LoadDefaultConfig()
	if err != nil {
		panic(err)
	}
	api := apigatewaymanagementapi.NewFromConfig(cfg, func(o *apigatewaymanagementapi.Options) {
		o.EndpointResolver = r
	})
	res, err := api.PostToConnection(context.Background(), &apigatewaymanagementapi.PostToConnectionInput{
		ConnectionId: aws.String("V1nqOfKwliACGGg="),
		Data:         []byte(`{"test":"yolo"}`),
	})
}

Any idea what I'm doing wrong?
Thanks

@Dasio Dasio added the guidance Question that needs advice or information. label Nov 11, 2020
@KaibaLopez
Copy link
Contributor

Hi @Dasio ,
As an update on this, it looks like there is a bigger problem going on than I expected, for now I believe this is an error on the service side but I'll keep you updated.

@KaibaLopez KaibaLopez added bug This issue is a bug. service-api This issue is due to a problem in a service API, not the SDK implementation. and removed guidance Question that needs advice or information. labels Nov 16, 2020
@jamessouth
Copy link

I also cannot get messaging to clients to work with the Go SDK, v1 or v2. I mostly get 403s but when it doesn't error, I can still see in CloudWatch that the API has not been engaged. I was hopeful that this issue from the JS SDK, which seems to apply to Go as well, would lead to a solution, but the data didn't land anywhere; there was no error, but no json showing up in my client either. Using the awscurl tool (Go version) and the steps outlined here, I was able to connect to my API with wscat and then send data with awscurl, so the API works as expected, it just seems to be the Go SDK that is the problem. Excerpt from my lambda where I want to message connected clients:

package main

import (
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"net/http"
	"os"
	"strings"

	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/apigatewaymanagementapi"
	"github.com/aws/smithy-go"
)

func handler(ctx context.Context, req events.DynamoDBEvent) (events.APIGatewayProxyResponse, error) {

		str := "https://" + apiid + ".execute-api." + rec.AWSRegion + ".amazonaws.com"

		customResolver := aws.EndpointResolverFunc(func(service, region string) (aws.Endpoint, error) {
			if service == apigatewaymanagementapi.ServiceID && region == rec.AWSRegion {
				return aws.Endpoint{
					PartitionID:   "aws",
					URL:           str,
					SigningRegion: rec.AWSRegion,
				}, nil
			}
			return aws.Endpoint{}, fmt.Errorf("unknown endpoint requested")
		})

		cfg, err := config.LoadDefaultConfig(ctx,
			config.WithRegion(rec.AWSRegion),
			config.WithEndpointResolver(customResolver),
		)
		if err != nil {
			fmt.Println("cfg err")
		}

		svc := apigatewaymanagementapi.NewFromConfig(cfg)

		b, err := json.Marshal("{a: 19894, b: 74156}")
		if err != nil {
			fmt.Println("error marshalling", err)
		}
		conn := apigatewaymanagementapi.PostToConnectionInput{
                    ConnectionId: aws.String(item["sk"].String()), 
                    Data: b
                }

		o, e := svc.PostToConnection(ctx, &conn)

		if e != nil {
			// To get any API error
			var apiErr smithy.APIError
			if errors.As(err, &apiErr) {
				fmt.Printf("db error, Code: %v, Message: %v", apiErr.ErrorCode(), apiErr.ErrorMessage())
			}
		}

	return events.APIGatewayProxyResponse{
		StatusCode:        http.StatusOK,
		Headers:           map[string]string{"Content-Type": "application/json"},
		MultiValueHeaders: map[string][]string{},
		Body:              "",
		IsBase64Encoded:   false,
	}, nil
}

func main() {
	lambda.Start(handler)
}

This seems like it should work. Here is a similar Python implementation from SO:

import time
import json
import boto3

def lambda_handler(event, context):

        connection_id = event["requestContext"]["connectionId"]
        domain_name = event["requestContext"]["domainName"]
        stage = event["requestContext"]["stage"]

        message = f'{domain_name}: {connection_id}'.encode('utf-8')
        api_client = boto3.client('apigatewaymanagementapi', endpoint_url = f"https://{domain_name}/{stage}")

        for _ in range(5):
            api_client.post_to_connection(Data=message,
                                                ConnectionId=connection_id)
            time.sleep(5)

    
        response = {'statusCode': 200}
        return response

@skotambkar
Copy link
Contributor

Thanks for providing the code. Looking at this, it might be because some required header not being added with the request.

Can you try adding client logging to your application? The following log the SDK request signature calculation , the HTTP message sent, and the HTTP response received. This will help investigate this issue further.

svc := s3.NewFromConfig(cfg, func(o *s3.Options) {
    o.ClientLogMode = aws.LogSigning  | aws.LogRequest | aws.LogResponseWithBody
})

@jasdel jasdel added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Mar 3, 2021
@github-actions
Copy link

github-actions bot commented Mar 9, 2021

This issue has not received a response in 1 week. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

@github-actions github-actions bot added closing-soon This issue will automatically close in 4 days unless further comments are made. closed-for-staleness and removed closing-soon This issue will automatically close in 4 days unless further comments are made. labels Mar 9, 2021
@jamessouth
Copy link

Just saw your reply @skotambkar, I will investigate and reply soon. Thank you.

@jamessouth
Copy link

jamessouth commented Mar 18, 2021

@skotambkar did you mean s3 or apigatewaymanagementapi? I added this:

                          var buffer bytes.Buffer
	                  logger := logging.NewStandardLogger(&buffer)
		       	logger.Logf(logging.Debug, "time to %s", "log")

			cfg, err := config.LoadDefaultConfig(ctx,
				config.WithRegion(rec.AWSRegion),
				config.WithLogger(logger),
				config.WithEndpointResolver(customResolver),
			)
			if err != nil {
				fmt.Println("cfg err")
			}

			svc := apigatewaymanagementapi.NewFromConfig(cfg, func(o *apigatewaymanagementapi.Options) {
				o.ClientLogMode = aws.LogSigning | aws.LogRequest | aws.LogResponseWithBody
			})

but I don't know where it is logging anything, I don't see anything in CloudWatch that wasn't getting logged before.

@skotambkar
Copy link
Contributor

Should be apigatewaymanagementapi in your case. The logs would be written to the buffer you passed in with your logger.

@jamessouth
Copy link

ok I got some logs:

read 1024 bytes: SDK 2021/03/18 21:35:50 DEBUG time to log
SDK 2021/03/18 21:35:50 DEBUG Request Signature:
---[ CANONICAL STRING ]-----------------------------
POST
/%40connections/cZwkGdmUiYcCIJA%253D
amz-sdk-invocation-id:2e1c062d-c1df-4f74-ad00-f7bc873e6d04
amz-sdk-request:attempt=1; max=3
content-length:22
content-type:application/octet-stream
host:[my-api].execute-api.us-east-2.amazonaws.com
x-amz-date:20210318T213550Z
x-amz-security-token:[long token]
read 1024 bytes: f/LWVWqxpzM9GKDU80xmtH8iOyV4a/gVhUedaGB9iplWvtwxdGOVWtdaPifvUEGjQpjsjejt4MIVqUCd6+NtUsZTTfuu3zOX3oNe32TJfgI2bYlv9Qivf6vgrvD0=
x-amz-user-agent:aws-sdk-go-v2/1.2.0 os/linux lang/go/1.16.2 md/GOOS/linux md/GOARCH/amd64 exec-env/AWS_Lambda_go1.x
amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token;x-amz-user-agent
389090ac633a04444394a5ca1ebd2aa392c1aacbe4df4beecbaccebc253ce38c
---[ STRING TO SIGN ]--------------------------------
AWS4-HMAC-SHA256
20210318T213550Z
20210318/us-east-2/execute-api/aws4_request
4371ec9b9b319a436021be10639c742e61f2ae95d08393db8a2a89bd84969341

@jamessouth
Copy link

SDK 2021/03/18 21:35:51 DEBUG Response
HTTP/2.0 403 Forbidden
Content-Length: 23
Content-Type: application/json
Date: Thu, 18 Mar 2021 21:35:51 GMT
X-Amz-Apigw-Id: cZwkqHlYiYcF8cQ=
X-Amzn-Errortype: ForbiddenException
X-Amzn-Requestid: 171c6bdf-18f8-4782-87db-954ab5edce52
{
"message": "Forbidden"
}

@stephanos
Copy link

I'm encountering the same issue! My JavaScript lambda works fine, my Go one receives a 403.

Added debug logging as suggested:

SDK 2021/03/27 03:16:09 DEBUG Request Signature:
--
---[ CANONICAL STRING  ]-----------------------------
POST
/%40connections/c0567cPsvHcCJ0g%253D
amz-sdk-invocation-id:0bf6f6f7-4639-4572-a491-50c31f5e2942
amz-sdk-request:attempt=1; max=3
content-length:250
content-type:application/octet-stream
host:<my gateway>.execute-api.us-west-2.amazonaws.com
x-amz-date:20210327T031609Z
x-amz-security-token:IQoJb3JpZ2luX2VjENv//////////wEaCXVzLXdlc3QtMiJIMEYCIQDYT5KtvYujyKnzAQFH+4xAqE1xkcpUQobj9g+c4og8sgIhAOEjdn8dWW/6hPQI7TwK2S7MSvxm8xqLVXRVejtAW0mLKtoBCCQQABoMMTUzODUxNzg4OTE4IgzIbeidVNurFjtGNikqtwE21dKiRBVcZE3pMtvp1Drv19FZhkH3B834HZwvXru07e6BOlyhiXaULew2DMACOvNouHtE6cy+6v0cy+ZHZqUuMFevvIXUlQSEFBXLVcHuB6ciH5RBGDAGYW+18jjh8UYL/0dqPnZHYOnHZ2vNeSBEKaogRtPADFlUPonUS6Dr+DjlJt1OfUz+RlSV4ugwjg4J5f2q/eg1OQwGZXXtBLzwZIS4HQsvSBDRQhCuz6Nc6YWPvUd6yQIw+cb6ggY63wGSUQjR/ieZYKXFW3kIu8VvZt/zH1vNcVVG0V16vdCnVkKFvK8Qe3lznx5OMk0+hvl55fQtzHaX3kGWdaCRXxjoWSZKlNZh0q69dphc0qZD0FhOQs6NY3p8Lp4FPttRK/icojyylbj3ycidNkDJxneXbA9jQgWFxkF5hYliBbT6h2lKz3FF0elnVnNTfKi/9AjGdOVLGCBvo8xR/UfgS1P4GNCwKdM2WHuhrP0WLQs3vLs/e0r9FWdVAx07kf88zH0i6ZmGJD29QCCKwMHhqivfA0pR4wtUY7FFGrcs9bK2
x-amz-user-agent:aws-sdk-go-v2/1.2.0 os/linux lang/go/1.16 md/GOOS/linux md/GOARCH/amd64 exec-env/AWS_Lambda_go1.x
amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token;x-amz-user-agent
cdf58331103eb4b67b0104b583527d571e1b306d6ea086fe9a918425a066da35
---[ STRING TO SIGN ]--------------------------------
AWS4-HMAC-SHA256
20210327T031609Z
20210327/us-west-2/execute-api/aws4_request
32d55b70591889f54595437b51034482fe885d71c3ea47df033bec4a5eb963dd
-----------------------------------------------------
SDK 2021/03/27 03:16:09 DEBUG Request
POST /@connections/c0567cPsvHcCJ0g%3D HTTP/1.1
Host: <my gateway>.execute-api.us-west-2.amazonaws.com
User-Agent: aws-sdk-go-v2/1.2.0
Content-Length: 250
Amz-Sdk-Invocation-Id: 0bf6f6f7-4639-4572-a491-50c31f5e2942
Amz-Sdk-Request: attempt=1; max=3
Authorization: AWS4-HMAC-SHA256 Credential=ASIASHUSH7Z3KCIP2N5M/20210327/us-west-2/execute-api/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token;x-amz-user-agent, Signature=2df5073371a9a546f96ba93180f6b3dd38e775f61ecfb344aec94c12d1a8cd14
Content-Type: application/octet-stream
X-Amz-Date: 20210327T031609Z
X-Amz-Security-Token: IQoJb3JpZ2luX2VjENv//////////wEaCXVzLXdlc3QtMiJIMEYCIQDYT5KtvYujyKnzAQFH+4xAqE1xkcpUQobj9g+c4og8sgIhAOEjdn8dWW/6hPQI7TwK2S7MSvxm8xqLVXRVejtAW0mLKtoBCCQQABoMMTUzODUxNzg4OTE4IgzIbeidVNurFjtGNikqtwE21dKiRBVcZE3pMtvp1Drv19FZhkH3B834HZwvXru07e6BOlyhiXaULew2DMACOvNouHtE6cy+6v0cy+ZHZqUuMFevvIXUlQSEFBXLVcHuB6ciH5RBGDAGYW+18jjh8UYL/0dqPnZHYOnHZ2vNeSBEKaogRtPADFlUPonUS6Dr+DjlJt1OfUz+RlSV4ugwjg4J5f2q/eg1OQwGZXXtBLzwZIS4HQsvSBDRQhCuz6Nc6YWPvUd6yQIw+cb6ggY63wGSUQjR/ieZYKXFW3kIu8VvZt/zH1vNcVVG0V16vdCnVkKFvK8Qe3lznx5OMk0+hvl55fQtzHaX3kGWdaCRXxjoWSZKlNZh0q69dphc0qZD0FhOQs6NY3p8Lp4FPttRK/icojyylbj3ycidNkDJxneXbA9jQgWFxkF5hYliBbT6h2lKz3FF0elnVnNTfKi/9AjGdOVLGCBvo8xR/UfgS1P4GNCwKdM2WHuhrP0WLQs3vLs/e0r9FWdVAx07kf88zH0i6ZmGJD29QCCKwMHhqivfA0pR4wtUY7FFGrcs9bK2
X-Amz-User-Agent: aws-sdk-go-v2/1.2.0 os/linux lang/go/1.16 md/GOOS/linux md/GOARCH/amd64 exec-env/AWS_Lambda_go1.x
Accept-Encoding: gzip
SDK 2021/03/27 03:16:10 DEBUG Response
HTTP/2.0 403 Forbidden
Content-Length: 23
Content-Type: application/json
Date: Sat, 27 Mar 2021 03:16:10 GMT
X-Amz-Apigw-Id: c057HEFpPHcFmWw=
X-Amzn-Errortype: ForbiddenException
X-Amzn-Requestid: 14ab997a-531a-4ea7-8268-0d382885abdc
{     "message": "Forbidden" }

@stephanos
Copy link

@jamessouth is this problem still happening for you?

@skotambkar should we re-open this issue since we have two reports of this being an ongoing problem?

@stephanos
Copy link

I upgraded the AWS SDK version today and it is just working now 👌

@jamessouth
Copy link

Thank you @stephanos @skotambkar version 1.2.1 is working for me! 😊👌😁

@benjfield
Copy link

benjfield commented Jan 19, 2023

I seem to be this issue with the current version, did anyone find a fix for this? My response today with apigatewaymanagementapi v1.11.0:

Edit: Please ignore this, it was an issue with accessing api gateway from within a vpc. In case anyone ends up with a similar issue:

https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-vpc-connections/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. closed-for-staleness response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. service-api This issue is due to a problem in a service API, not the SDK implementation.
Projects
None yet
Development

No branches or pull requests

7 participants