Implement pipeform for Terraform Operations

[osterman]

Describe the Feature

Improve the console DX of terraform operations by presenting the user with a friendly UI to show terrafom progress.

https://github.com/magodo/pipeform

Expected Behavior

Running atmos terraform plan and atmos terraform apply will launch the pipeform UI. Note, this does not require pipeform to be installed, as it will be used as a native Go module.

Use Case

On the command line, the output from terraform is overwhelming and doesn't convey the overall progress of the operation. Terraform will feel much friendlier if we present the user with a progress indicator.

Describe Ideal Solution

The UI would be enabled by default, but can be disabled via the atmos.yaml configuration.

# atmos.yaml
terraform:
  ui:
    enable: [plan, apply]

pipeform.mp4

Requirements:

• If no TTY is attached, the UI should be automatically disabled
• If the CI=true environment variable is set, then UI should be automatically disabled
• If the --no-ui flag is explicitly passed, then the UI should be disabled.

Alternatives Considered

No response

Additional Context

The pipeform library is licensed MPL2 and is not currently implemented as a Go module. We've contacted the @magodo (author) and he said he would accept a PR converting it to a library. As part of this implementation, we will need to first open a PR for that work.

[suzuki-shunsuke]

pipeform is interesting, but honestly I'm not sure if we should really use this by default.
I tried pipeform a bit.
When we use pipeform, the plan result doesn't output in the terminal.

$ terraform plan -json | pipeform
# No output

I think this is inconvenient.
I'm not sure if we can convert the output of terraform plan -json to the conventional terraform output.

Furthermore, seems like we can't show the detail of changes.
I expected that we can choose a resource and show its detail, but seems like we can't.

By the way, the final view of terraform plan -json | pipeform looks strange.
I think we can contribute to pipeform.

pipeform shows nothing. This confused me.
I found I needed to move page by h and l key.

[osterman]

pipeform is interesting, but honestly I'm not sure if we should really use this by default. I tried pipeform a bit. When we use pipeform, the plan result doesn't output in the terminal.

$ terraform plan -json | pipeform
# No output

This sounds like a bug in pipeform. Per the docs, it says it's supported.

the following terraform commands are supported:

terraform refresh -json
terraform plan -json
terraform apply -auto-approve -json

---

Furthermore, seems like we can't show the detail of changes. I expected that we can choose a resource and show its detail, but seems like we can't.

This would definitely be a welcome enhancement. It would likely benefit your tooling for terraform as well. We can sponsor this development in pipeform, if you're interested.

---

By the way, the final view of terraform plan -json | pipeform looks strange. I think we can contribute to pipeform.

Yes, this is not good. It should probably work the way you described above, allowing individual resources to be expanded. They shouldn't "disappear".

---

Ideal World

• When using pipeform, raw terraform logs are written to disk, allowing further inspection upon failure
• The UI would enable inspection of any resource
• A flag like --ui=false would disable the UI on a per-run basis.
• When a TTY is not detected, or we're in CI (env), then UI is disabled

[suzuki-shunsuke]

This sounds like a bug in pipeform. Per the docs, it says it's supported.

Really? I can't find the docs.
In my envrionment, TUI is shown, but after TUI quits, no output is shown in the terminal.

[osterman]

@suzuki-shunsuke here is what I am referring to. Are we talking about the same thing?

[suzuki-shunsuke]

I think it means pipeform can provide TUI for these commands, but it doesn't mean pipeform can output Terraform's conventional outoputs to stdout.

[suzuki-shunsuke]

For pipeform to output the conventional Terraform command output, pipeform needs to convert the output of terraform plan -json to the conventional Terraform command output.
This means the output of terraform plan -json needs to include all information of the conventional Terraform command output.

I checked the output of terraform plan -json, but seems like it doesn't include the information of resource attributes.

For instance, terraform plan -json outputs the data like this:

{
  "@level": "info",
  "@message": "github_issue_label.fooo: Plan to create",
  "@module": "terraform.ui",
  "@timestamp": "2025-01-15T22:33:03.343780+09:00",
  "change": {
    "resource": {
      "addr": "github_issue_label.fooo",
      "module": "",
      "resource": "github_issue_label.fooo",
      "implied_provider": "github",
      "resource_type": "github_issue_label",
      "resource_name": "fooo",
      "resource_key": null
    },
    "action": "create"
  },
  "type": "planned_change"
}

We can see the resource github_issue_label.fooo is created, but we can't see the attributes of the resource.
We can't convert this output to the conventional output like this:

  # github_issue_label.fooo will be created
  + resource "github_issue_label" "fooo" {
      + color       = "FF0000"
      + description = "foo"
      + etag        = (known after apply)
      + id          = (known after apply)
      + name        = "fooo"
      + repository  = "hoge"
      + url         = (known after apply)
    }

[suzuki-shunsuke]

So unfortunately, to provide both TUI and the conventional output, we need to run terraform plan twice.

[suzuki-shunsuke]

to provide both TUI and the conventional output, we need to run terraform plan twice.

TUI has no meaning if you run terraform plan twice 😅

[osterman]

@suzuki-shunsuke if the objective is to also support the regular terraform output, I think that could still be accomplished.

  # github_issue_label.fooo will be created
  + resource "github_issue_label" "fooo" {
      + color       = "FF0000"
      + description = "foo"
      + etag        = (known after apply)
      + id          = (known after apply)
      + name        = "fooo"
      + repository  = "hoge"
      + url         = (known after apply)
    }

Isn't the workaround as simple as:

terraform plan -json -out planfile.out | pipeform
terraform show planfile.out

[suzuki-shunsuke]

Oh, I see. Sounds good.

[suzuki-shunsuke]

We need to improve pipeform.

1. Pipeform can't show the detail of resources from TUI
2. terraform plan -json | pipeform finishes at a last page which doesn't show any resources

We need to discuss these problems with pipeform's maintainers.

[osterman]

We need to discuss these problems with pipeform's maintainers.

I'll get the conversation started by opening an issue in the project and linking to this topic.

[osterman]

@suzuki-shunsuke the terraform plan handling was just resolved in:

• Display information in the "Plan" stage magodo/pipeform#10