-
-
Notifications
You must be signed in to change notification settings - Fork 108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix terraform clean
bugs
#870
base: main
Are you sure you want to change the base?
Conversation
📝 WalkthroughWalkthroughThe pull request introduces significant changes to the Changes
Assessment against linked issues
Possibly related PRs
Suggested reviewers
📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (5)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (1)
internal/exec/terraform.go (1)
Line range hint
78-98
: Overall changes effectively address all reported bugs.The modifications to stack processing and component validation logic successfully fix all three reported issues with the
atmos terraform clean
command. The changes are minimal, focused, and maintain the existing behavior for other commands.Consider adding integration tests to cover these specific scenarios to prevent regression:
atmos terraform clean --everything
atmos terraform clean -s <stack>
atmos terraform clean <component>
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
internal/exec/terraform.go
(2 hunks)
🧰 Additional context used
📓 Learnings (1)
internal/exec/terraform.go (1)
Learnt from: haitham911
PR: cloudposse/atmos#727
File: internal/exec/terraform_clean.go:407-416
Timestamp: 2024-11-24T19:13:10.287Z
Learning: In `internal/exec/terraform_clean.go`, when `getStackTerraformStateFolder` returns an error in the `handleCleanSubCommand` function, the error is logged, and the process continues without returning the error.
🔇 Additional comments (2)
internal/exec/terraform.go (2)
96-98
: LGTM! Component enabled check now correctly handles clean command.
The condition info.SubCommand != "clean"
ensures that disabled components can still be cleaned up, while maintaining the validation for other commands.
78-83
: LGTM! Stack processing logic aligns with command requirements.
The changes correctly handle all three use cases:
clean --everything
: No stack processing when component is emptyclean -s <stack>
: Stack check only when stack is providedclean <component>
: No stack requirement when only component is provided
Let's verify the command behavior:
✅ Verification successful
Stack processing logic changes are correctly implemented and verified!
The code changes align perfectly with the intended behavior:
- When no component is provided (
ComponentFromArg == ""
), stack processing is skipped, which correctly handlesclean --everything
- Stack validation (
shouldCheckStack
) only occurs when a stack is explicitly provided, fixing theclean -s <stack>
case - The implementation in
handleCleanSubCommand
confirms proper handling of component-only cleaning without requiring a stack
The changes are supported by:
- Clear error handling in
handleCleanSubCommand
- Proper integration with the command help system
- Consistent behavior with the documented CLI usage at
atmos.tools/cli/commands/terraform/clean
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Search for tests covering these scenarios
# Test: Look for clean command test cases
rg -A 5 "TestExecuteTerraform.*Clean"
# Test: Search for clean command implementation details
ast-grep --pattern 'func $_(t *testing.T) {
$$$
clean
$$$
}'
Length of output: 108
Script:
#!/bin/bash
# Let's try a broader search for test files and clean command implementation
# Search for test files related to terraform execution
fd "terraform.*test.*" --type f --exec rg -l "clean"
# Look for clean command implementation in terraform.go
rg -B 5 -A 10 "SubCommand == \"clean\"" internal/exec/terraform.go
# Search for any test functions that might test clean functionality
rg -B 2 -A 5 "func Test.*" --type go | rg -B 2 -A 5 -i "clean"
Length of output: 1832
Script:
#!/bin/bash
# Let's examine the handleCleanSubCommand function implementation
# Search for handleCleanSubCommand function
ast-grep --pattern 'func handleCleanSubCommand($$$) {
$$$
}'
# Look for any related error handling or validation logic
rg -B 2 -A 5 "handleCleanSubCommand" --type go
# Check for any clean-related constants or types that might affect the behavior
rg "clean" -i --type go internal/exec/
Length of output: 4429
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🔭 Outside diff range comments (2)
website/docs/cli/commands/terraform/usage.mdx (1)
Line range hint
62-66
: Ensure consistency with the clean command documentation.The description of the
--everything
flag's behavior should be consistent with the dedicated clean command documentation and clearly explain the confirmation prompt behavior.- Use the `--everything` flag is the default behavior to delete all the local Terraform state files and directories (including `terraform.tfstate.d`) for all components and stacks. - Use the `--force` flag to bypass the safety confirmation prompt and force the deletion (use with caution). - - :::warning - The `--everything` flag performs destructive operations that can lead to permanent state loss. Always ensure you have remote state configured in your components before proceeding. - ::: + The command now defaults to the behavior of the `--everything` flag, which means it will: + - Delete all local Terraform state files and directories (including `terraform.tfstate.d`) + - Apply to all components and stacks + - Show a confirmation prompt before proceeding + + Use the `--force` flag to bypass the safety confirmation prompt (use with extreme caution). + + :::danger Data Loss Risk + This command performs destructive operations that can lead to permanent state loss: + 1. Always ensure you have remote state configured in your components + 2. Verify your state backups are current + 3. Consider the impact on other team members + :::internal/exec/help.go (1)
Line range hint
82-87
: Fix typos and improve clarity in help text.The help text contains typos and could be clearer about the default behavior.
- "Use '--everything' flag —everything is the default behavior to delete the Terraform state files and and directories with confirm message.\n\n" + - "Use --force to forcefully delete Terraform state files and directories for the component.\n\n" + - "- If no component is specified, the command will apply to all components and stacks.\n" + - "- If no stack is specified, the command will apply to all stacks for the specified component.\n" + - "Use '--skip-lock-file' flag to skip deleting the '.terraform.lock.hcl' file.\n\n" + - "If no component or stack is specified, the clean operation will apply globally to all components.\n\n" + + "IMPORTANT: By default, this command will attempt to delete all Terraform state files and directories.\n" + + "A confirmation prompt will be shown before deletion.\n\n" + + "Flags:\n" + + " --force Bypass the confirmation prompt (use with extreme caution)\n" + + " --skip-lock-file Skip deleting the '.terraform.lock.hcl' file\n\n" + + "Scope:\n" + + " - No component specified: Applies to all components and stacks\n" + + " - No stack specified: Applies to all stacks for the specified component\n\n" +Issues fixed:
- Removed duplicate "and" in "state files and and directories"
- Fixed em dash usage
- Removed redundant information about global operation
- Improved formatting for better readability
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
internal/exec/help.go
(1 hunks)internal/exec/utils.go
(1 hunks)website/docs/cli/commands/terraform/terraform-clean.mdx
(1 hunks)website/docs/cli/commands/terraform/usage.mdx
(1 hunks)
🧰 Additional context used
📓 Learnings (3)
website/docs/cli/commands/terraform/terraform-clean.mdx (1)
Learnt from: osterman
PR: cloudposse/atmos#727
File: internal/exec/terraform.go:114-118
Timestamp: 2024-11-12T05:52:05.088Z
Learning: When `atmos terraform clean --everything` is used without specifying a component and without the `--force` flag, prompt the user for confirmation before deleting all components. Use the `--force` flag to skip the confirmation prompt.
website/docs/cli/commands/terraform/usage.mdx (1)
Learnt from: osterman
PR: cloudposse/atmos#727
File: internal/exec/terraform.go:114-118
Timestamp: 2024-11-12T05:52:05.088Z
Learning: When `atmos terraform clean --everything` is used without specifying a component and without the `--force` flag, prompt the user for confirmation before deleting all components. Use the `--force` flag to skip the confirmation prompt.
internal/exec/help.go (1)
Learnt from: osterman
PR: cloudposse/atmos#727
File: internal/exec/terraform.go:114-118
Timestamp: 2024-11-12T05:52:05.088Z
Learning: When `atmos terraform clean --everything` is used without specifying a component and without the `--force` flag, prompt the user for confirmation before deleting all components. Use the `--force` flag to skip the confirmation prompt.
🔇 Additional comments (1)
internal/exec/utils.go (1)
660-664
: LGTM! This change makes --everything
the default behavior for the clean command.
The modification correctly addresses the issue where atmos terraform clean --everything
was not working as expected. Now the --everything
flag is automatically appended for the clean command if not already present.
@coderabbitai can you add integration tests? |
Tip For best results, initiate chat on the files or code changes. @osterman: Here's a draft of minimal integration tests for the “terraform clean” command. The example below shows how you might structure a new test file named “terraform_clean_integration_test.go” in a dedicated integration tests folder (for instance, create a “test/integration” directory if it doesn't exist). This code invokes the “atmos terraform clean” command with different flag combinations, checking for exit codes and capturing stdout/stderr as needed. Below is an illustrative snippet in Go: package integration_test import ( func TestTerraformCleanEverything(t *testing.T) { func TestTerraformCleanSpecificStack(t *testing.T) { func TestTerraformCleanComponentNoStack(t *testing.T) { func TestTerraformCleanComponentAndStack(t *testing.T) { You can adjust the checks (e.g., the success string “deleted”) to something more reliable or parse logs or state files directly. If needed, you can also run prior setup steps in each test—such as creating dummy Terraform resources—to validate the cleanup more comprehensively. Hope this helps you get started! 🔗 Analysis chain🏁 Scripts executedThe following scripts were executed for the analysis: Script:
Length of output: 3303 Script:
Length of output: 2658 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (2)
tests/cli_terraform_test.go (2)
13-42
: Consider improving test setup maintainability.The test setup could be enhanced in two ways:
- Add documentation for the
PathManager
usage and its purpose- Consider making the working directory path configurable through environment variables or test parameters
+// TestCLITerraformClean validates the `atmos terraform clean` command by: +// 1. Setting up the test environment with the correct PATH +// 2. Running terraform apply commands to create state +// 3. Verifying the clean command removes all state files func TestCLITerraformClean(t *testing.T) { // ... existing code ... - workDir := "../examples/quick-start-simple" + workDir := os.Getenv("ATMOS_TEST_WORKDIR") + if workDir == "" { + workDir = "../examples/quick-start-simple" + }
53-58
: Remove unnecessary return after Fatalf.The
return
statement aftert.Fatalf()
is redundant asFatalf
already terminates the test.if err != nil { t.Log(stdout.String()) t.Fatalf("Failed to run terraform apply station -s dev: %v", stderr.String()) - return }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/cli_terraform_test.go
(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: [mock-macos] tests/fixtures/scenarios/complete
- GitHub Check: [mock-macos] examples/demo-vendoring
- GitHub Check: [mock-macos] examples/demo-component-versions
- GitHub Check: [mock-macos] examples/demo-atlantis
- GitHub Check: [mock-linux] examples/demo-component-versions
- GitHub Check: [k3s] demo-helmfile
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Summary
🔇 Additional comments (3)
tests/cli_terraform_test.go (3)
86-95
: Clean command execution looks good!The implementation has proper error handling and stdout/stderr capture.
96-110
: Clean verification logic looks good!The implementation follows best practices with proper error handling and comprehensive file checking.
13-112
: Add test cases for other clean command variations.The test only covers the
clean --force
scenario. Based on the PR objectives, we should add test cases for:
atmos terraform clean <component>
atmos terraform clean -s <stack>
atmos terraform clean <component> -s <stack>
This is particularly important as these scenarios were mentioned as having bugs in the PR objectives.
Would you like me to generate the additional test cases?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🧹 Nitpick comments (2)
tests/cli_terraform_test.go (2)
74-89
: Add timeout handling and improve logging consistency.The command execution should have a timeout to prevent hanging tests, and logging should be consistent across all command executions.
func runTerraformApply(t *testing.T, binaryPath, environment string) { + t.Logf("Running terraform apply for environment: %s", environment) cmd := exec.Command(binaryPath, "terraform", "apply", "station", "-s", environment) + // Add timeout + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute) + defer cancel() + cmd = exec.CommandContext(ctx, cmd.Path, cmd.Args[1:]...) envVars := os.Environ() envVars = append(envVars, "ATMOS_COMPONENTS_TERRAFORM_APPLY_AUTO_APPROVE=true") cmd.Env = envVars var stdout, stderr bytes.Buffer cmd.Stdout = &stdout cmd.Stderr = &stderr err := cmd.Run() - t.Log(stdout.String()) + t.Logf("Command output:\n%s", stdout.String()) if err != nil { + if ctx.Err() == context.DeadlineExceeded { + t.Fatalf("Command timed out after 5 minutes") + } t.Fatalf("Failed to run terraform apply station -s %s: %v", environment, stderr.String()) } }
117-131
: Improve error reporting in verification.The verification function could provide more descriptive error messages and aggregate multiple failures.
func verifyStateFilesDeleted(t *testing.T, stateFiles []string) { + var errors []string for _, file := range stateFiles { fileAbs, err := filepath.Abs(file) if err != nil { t.Fatalf("Failed to resolve absolute path for %q: %v", file, err) } _, err = os.Stat(fileAbs) if err == nil { - t.Errorf("Expected Terraform state file to be deleted: %q", fileAbs) + errors = append(errors, fmt.Sprintf("File still exists: %q", fileAbs)) } else if !errors.Is(err, os.ErrNotExist) { - t.Errorf("Unexpected error checking file %q: %v", fileAbs, err) + errors = append(errors, fmt.Sprintf("Error checking %q: %v", fileAbs, err)) } } + if len(errors) > 0 { + t.Errorf("Verification failed:\n%s", strings.Join(errors, "\n")) + } }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/cli_terraform_test.go
(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
internal/exec/help.go
Outdated
} else if componentType == "terraform" && command == "clean" { | ||
u.PrintMessage("\n'atmos terraform clean' command deletes the following folders and files from the component's directory:\n\n" + | ||
" - '.terraform' folder\n" + | ||
" - folder that the 'TF_DATA_DIR' ENV var points to\n" + | ||
" - '.terraform.lock.hcl' file\n" + | ||
" - generated varfile for the component in the stack\n" + | ||
" - generated planfile for the component in the stack\n" + | ||
" - generated 'backend.tf.json' file\n" + | ||
" - 'terraform.tfstate.d' folder (if '--everything' flag is used)\n\n" + | ||
"Usage: atmos terraform clean <component> -s <stack> <flags>\n\n" + | ||
"Use '--everything' flag to also delete the Terraform state files and and directories with confirm message.\n\n" + | ||
"Use --force to forcefully delete Terraform state files and directories for the component.\n\n" + | ||
"- If no component is specified, the command will apply to all components and stacks.\n" + | ||
"- If no stack is specified, the command will apply to all stacks for the specified component.\n" + | ||
"Use '--skip-lock-file' flag to skip deleting the '.terraform.lock.hcl' file.\n\n" + | ||
"If no component or stack is specified, the clean operation will apply globally to all components.\n\n" + | ||
"For more details refer to https://atmos.tools/cli/commands/terraform/clean\n") | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we're regressing here. This was heavily refactored in #914
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should go away from this file. Is there something we are missing that the code handle comes here?
what
Fix bugs related to:
references
Summary by CodeRabbit
New Features
--skip-init
flag for Terraform commands.--deploy-run-init
flag for Terraform deployment.--from-plan
and--planfile
flags for Terraform apply.Changes to Existing Features
atmos terraform clean
command to delete all Terraform state files by default.--everything
flag from clean command.Documentation
atmos terraform clean
command, including detailed deletion items and usage instructions.--everything
flag.Testing