Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using secret path breaks pulumi up #434

Open
istvan-fodor opened this issue Jan 23, 2023 · 4 comments
Open

Using secret path breaks pulumi up #434

istvan-fodor opened this issue Jan 23, 2023 · 4 comments
Labels
area/config kind/bug Some behavior is incorrect or out of spec

Comments

@istvan-fodor
Copy link

What happened?

I created a new project from the yaml template (no code at all, just config files) and set a secret value using the following command:

pulumi config set --path 'my.test.password' secretpassword123 --secret

The resulting config file looks like this:

config:
  test-pulumi-config:my:
    test:
      password:
        secure: AAABAMWO68C8qxo2dfXy4UHPUjIeXx76vbwuawqzWj1BBDYDZO3aMI39S9nWzoUQQw==

When I try to run a pulumi up, I get the following error:

╰─ pulumi up                                                                                                                                                                   ─╯
Previewing update (dev)

View Live: https://app.pulumi.com/starschema/test-pulumi-config/dev/previews/e209230c-d2c4-4b3a-b4f3-e9d1aecf6579

     Type                 Name                    Plan       Info
 +   pulumi:pulumi:Stack  test-pulumi-config-dev  create     1 error


Diagnostics:
  pulumi:pulumi:Stack (test-pulumi-config-dev):
    error: an unhandled error occurred: 1 error occurred:
    	* <nil>: 1 error occurred:
    	* unexpected configuration type 'map[string]interface {}': valid types are string, List<string>, number, List<number>, integer, List<integer>, boolean, List<number>

    ;

Steps to reproduce

  1. Create new project
  2. Set secret value: pulumi config set --path 'my.test.password' secretpassword123 --secret
  3. Run pulumi up
  4. Observe error.

Expected Behavior

pulumi up would run without error.

Actual Behavior

Got error on pulumi up

Output of pulumi about

CLI
Version      3.52.1
Go Version   go1.19.5
Go Compiler  gc

Plugins
NAME  VERSION
yaml  unknown

Host
OS       darwin
Version  13.0
Arch     arm64

This project is written in yaml

Current Stack: starschema/test-pulumi-config/dev

Found no resources associated with dev

Found no pending operations associated with dev

Backend
Name           pulumi.com
URL            https://app.pulumi.com/ifodor
User           ifodor
Organizations  ifodor, starschema

No dependencies found

Pulumi locates its logs in /var/folders/0k/5cb46pm90dv17f7675zv7xlm0000gn/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
@istvan-fodor istvan-fodor added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Jan 23, 2023
@istvan-fodor istvan-fodor changed the title Can't set secret path Using secret path breaks pulumi up Jan 23, 2023
@Frassle Frassle transferred this issue from pulumi/pulumi Jan 24, 2023
@Frassle
Copy link
Member

Frassle commented Jan 24, 2023

This is a current limitation of config with the yaml runtime. It doesn't support objects, I think the following config should work:

config:
  test-pulumi-config:my:
    password:
      secure: AAABAMWO68C8qxo2dfXy4UHPUjIeXx76vbwuawqzWj1BBDYDZO3aMI39S9nWzoUQQw==

@Frassle Frassle added area/config and removed needs-triage Needs attention from the triage team labels Jan 24, 2023
@fuadsaud
Copy link

fuadsaud commented Feb 9, 2023

I'm having the exact same symptom when running a YAML program with the aws:assumeRole.roleArn config. Is there a workaround for this one?

@AaronFriel
Copy link
Contributor

I have a workaround for aws:assumeRole.roleArn, but not for the original issue. @istvan-fodor if you are able to break up your initial object into separate keys, that will work for now.

For @fuadsaud using roleArn we have a workaround. In your Pulumi.yaml declare a config entry and resources entry like so:

config:
  awsRoleArn:
    type: string
    secret: true
resources:
  awsProvider:
    type: pulumi:providers:aws
    properties:
      assumeRole:
        roleArn: ${awsRoleArn}
    defaultProvider: true
    # options:
    #   # if you would like to pin to an AWS version
    #   version:

Then run:

pulumi config set --secret awsRoleArn foobar

The default provider will then be used for all resources in your program. The default provider configured in this way only takes configuration via environment variable (AWS_... env vars) and explicit config, so if you have any config in your Pulumi.stack.yaml like so:

# In the stack config file
config:
  aws:foo: "bar"

You will want to pass them into the explicit provider like so:

resources:
  awsProvider:
    type: pulumi:providers:aws
    properties:
      foo: ${aws:foo}

AaronFriel added a commit that referenced this issue Apr 23, 2023
@AaronFriel
Enable workaround for using objects in config
ebb7453 
The type checker is modified to make property access on 'Any' return 'Any',
which enables workarounds for #434 using programs like below.

Unblocking this sans workaround will require further implementation of these issues to support more complex structured, hierarchical config:
- pulumi/pulumi#1052
- pulumi/pulumi#2307

The workaround program is:

```yaml
name: tmp.0T7TLEvBj8
runtime: yaml
description: A minimal Pulumi YAML program
variables:
  myObject:
    fn::secret:
      fn::std:jsondecode:
        input:
          fn::fromBase64: ${myJSON}
outputs:
  test: ${myObject.result.test.password}
```

In this workaround we:

1. Base64 encode the JSON object we want to use in Pulumi YAML. This is
   necessary because Pulumi will attempt to JSON decode the value of config
   variables into objects on our behalf.

   ```sh
   pulumi config set --secret \
     myJSON \
     $(printf '{ "test": { "password": "secretpassword123" } }' | base64)
   ```

2. Use `fn::fromBase64` to decode that string into its original value.
3. Use `fn::std:jsondecode` to convert that string to an object.
4. Use `fn::secret` to ensure the value is marked as a secret. (Experimentally, this was necessary.)

The code change in the analyzer is necessary to allow indexing into the `Any` type on `${myObject.result}`.
@AaronFriel AaronFriel mentioned this issue Apr 23, 2023
Open
@AaronFriel
Copy link
Contributor

A partial fix for Pulumi YAML unblocking a workaround for @istvan-fodor has been posted here:

Sorry that this isn't an ideal solution @istvan-fodor, definitely something we need to improve upon.

@AaronFriel AaronFriel mentioned this issue Apr 24, 2023
Open
AaronFriel added a commit that referenced this issue Apr 24, 2023
@AaronFriel
Enable workaround for using objects in config
ee9a94e 
The type checker is modified to make property access on 'Any' return 'Any',
which enables workarounds for #434 using programs like below.

Unblocking this sans workaround will require further implementation of these issues to support more complex structured, hierarchical config:
- pulumi/pulumi#1052
- pulumi/pulumi#2307

The workaround program is:

```yaml
name: tmp.0T7TLEvBj8
runtime: yaml
description: A minimal Pulumi YAML program
variables:
  myObject:
    fn::secret:
      fn::std:jsondecode:
        input:
          fn::fromBase64: ${myJSON}
outputs:
  test: ${myObject.result.test.password}
```

In this workaround we:

1. Base64 encode the JSON object we want to use in Pulumi YAML. This is
   necessary because Pulumi will attempt to JSON decode the value of config
   variables into objects on our behalf.

   ```sh
   pulumi config set --secret \
     myJSON \
     $(printf '{ "test": { "password": "secretpassword123" } }' | base64)
   ```

2. Use `fn::fromBase64` to decode that string into its original value.
3. Use `fn::std:jsondecode` to convert that string to an object.
4. Use `fn::secret` to ensure the value is marked as a secret. (Experimentally, this was necessary.)

The code change in the analyzer is necessary to allow indexing into the `Any` type on `${myObject.result}`.
AaronFriel added a commit that referenced this issue Apr 24, 2023
@AaronFriel
Enable workaround for using objects in config
9a2ab4e 
The type checker is modified to make property access on 'Any' return 'Any',
which enables workarounds for #434 using programs like below.

Unblocking this sans workaround will require further implementation of these issues to support more complex structured, hierarchical config:
- pulumi/pulumi#1052
- pulumi/pulumi#2307

The workaround program is:

```yaml
name: tmp.0T7TLEvBj8
runtime: yaml
description: A minimal Pulumi YAML program
variables:
  myObject:
    fn::secret:
      fn::std:jsondecode:
        input:
          fn::fromBase64: ${myJSON}
outputs:
  test: ${myObject.result.test.password}
```

In this workaround we:

1. Base64 encode the JSON object we want to use in Pulumi YAML. This is
   necessary because Pulumi will attempt to JSON decode the value of config
   variables into objects on our behalf.

   ```sh
   pulumi config set --secret \
     myJSON \
     $(printf '{ "test": { "password": "secretpassword123" } }' | base64)
   ```

2. Use `fn::fromBase64` to decode that string into its original value.
3. Use `fn::std:jsondecode` to convert that string to an object.
4. Use `fn::secret` to ensure the value is marked as a secret. (Experimentally, this was necessary.)

The code change in the analyzer is necessary to allow indexing into the `Any` type on `${myObject.result}`.
AaronFriel added a commit that referenced this issue Apr 25, 2023
@AaronFriel
Enable workaround for using objects in config
395851b 
The type checker is modified to make property access on 'Any' return 'Any',
which enables workarounds for #434 using programs like below.

Unblocking this sans workaround will require further implementation of these issues to support more complex structured, hierarchical config:
- pulumi/pulumi#1052
- pulumi/pulumi#2307

The workaround program is:

```yaml
name: tmp.0T7TLEvBj8
runtime: yaml
description: A minimal Pulumi YAML program
variables:
  myObject:
    fn::secret:
      fn::std:jsondecode:
        input:
          fn::fromBase64: ${myJSON}
outputs:
  test: ${myObject.result.test.password}
```

In this workaround we:

1. Base64 encode the JSON object we want to use in Pulumi YAML. This is
   necessary because Pulumi will attempt to JSON decode the value of config
   variables into objects on our behalf.

   ```sh
   pulumi config set --secret \
     myJSON \
     $(printf '{ "test": { "password": "secretpassword123" } }' | base64)
   ```

2. Use `fn::fromBase64` to decode that string into its original value.
3. Use `fn::std:jsondecode` to convert that string to an object.
4. Use `fn::secret` to ensure the value is marked as a secret. (Experimentally, this was necessary.)

The code change in the analyzer is necessary to allow indexing into the `Any` type on `${myObject.result}`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/config kind/bug Some behavior is incorrect or out of spec
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@istvan-fodor @AaronFriel @Frassle @fuadsaud