Google Cloud - Client-based APIs and quota projects do not work well

[jh-edstart]

As per https://cloud.google.com/docs/quotas/quota-project, some GCP APIs require a "quota" project to be specified.

It appears this should be possible using a custom GCP provider and setting the billingProject property. However, this does not seem to work.

Setting the environment variable GOOGLE_CLOUD_QUOTA_PROJECT does seem to work, but this also affects other resources, so can only be used as a temporary work-around.

Expected behavior

If I understand the API correctly, the following code should work:

import * as gcp from "@pulumi/gcp";

export const customQuotaProvider = new gcp.Provider("gcp-quota", {
    billingProject: "quota-12345",
    project: "my-proj"
});

export function setupInfra() {
    new gcp.orgpolicy.Policy("restrict-service-usage", {
        name: "projects/my-proj/policies/gcp.restrictServiceUsage",
        parent: "projects/my-proj",
        spec: {
            rules: [{
                values: {
                    allowedValues: [
                        "bigquery.googleapis.com",
                    ],
                },
            }],
        },

    }, {
        provider: customQuotaProvider
    });
}

Current behavior

You get an error message:

Diagnostics:
  pulumi:pulumi:Stack (my-stack):
    error: update failed

  gcp:orgpolicy:Policy (restrict-service-usage):
    error: 1 error occurred:
    	* Error creating Policy: failed to create a diff: failed to retrieve Policy resource: googleapi: Error 403: Your application is authenticating by using local Application Default Credentials. The orgpolicy.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project, see https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds .
    Details:
    [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "domain": "googleapis.com",
        "metadata": {
          "consumer": "projects/my-project",
          "service": "orgpolicy.googleapis.com"
        },
        "reason": "SERVICE_DISABLED"
      }
    ]

Steps to reproduce

• Runpulumi up with the code specified above
• This works as a temporary work-around: GOOGLE_CLOUD_QUOTA_PROJECT=quota-12345 pulumi up. However, as stated above, this is not optimal since it will be used for all resources that are part of the current stack.

Context (Environment)

We're trying to use Pulumi to configure GCP service usage restrictions.

Affected feature

[VenelinMartinov]

Hi @jh-edstart, thanks for reporting the issue here and sorry you ran into this!

I was able to repro the issue with the provided code, thank you for that!

Glad you found a workaround for the issue - it is quite unfortunate that the config doesn't work and we currently have no way of supplying an env var to a provider only.

Perhaps you can work around the issue with the quota project being used for other resources by splitting up the program and only specifying it for one of them? Automation API might be useful here too.

[mikhailshilkov]

@VenelinMartinov Do you know if this problem is specific to the particular env var, to the particular provider, or is even broader than that? I'd love to understand the next level of details.

[VenelinMartinov]

I've reproduce the same issue in terraform:

provider "google" {
  project     = "pulumi-development"
  billing_project = "pulumi-development"
}

resource "google_org_policy_policy" "restrict_service_usage" {
  name   = "projects/pulumi-development/policies/gcp.restrictServiceUsage"
  parent = "projects/pulumi-development"

  spec {
    rules {
      values {
        allowed_values = ["bigquery.googleapis.com"]
      }
    }
  }

  provider = google
}

Raised hashicorp/terraform-provider-google#18281 upstream

[VenelinMartinov]

From the upstream issue: hashicorp/terraform-provider-google#18281 (comment)

It seems that the provider needs the userProjectOverride parameter set in order for billingProject to take effect.

There's also a note in the upstream docs about it, which we fail to translate: https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#billing_project