Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provider Upgrade Testing #2719

Closed
6 of 12 tasks
t0yv0 opened this issue Aug 25, 2023 · 1 comment · Fixed by #2714
Closed
6 of 12 tasks

Provider Upgrade Testing #2719

t0yv0 opened this issue Aug 25, 2023 · 1 comment · Fixed by #2714
Assignees
@t0yv0
Labels
kind/engineering Work that is not visible to an external user resolution/fixed This issue was fixed
Milestone
0.93

Comments

@t0yv0
Copy link
Member

t0yv0 commented Aug 25, 2023 •
edited
Loading

Starting with #2714 we incubate provider upgrade testing.

  • TestProviderUpgradeQuick needs to work harder to use Check calls; current strategy fails with eks.Cluster
  • Switch validation test to be preview-only, currently performs redundant pulumi up operations
  • Split up resource example configurations into multiple stacks; recording is hard in presence of eks.Cluster
  • Add tests for covering covering the patch audit:
@t0yv0 t0yv0 self-assigned this Aug 25, 2023
@t0yv0 t0yv0 added this to the 0.93 milestone Aug 25, 2023
@mikhailshilkov mikhailshilkov added the kind/engineering Work that is not visible to an external user label Aug 28, 2023
@t0yv0 t0yv0 mentioned this issue Aug 28, 2023
Merged
t0yv0 added a commit that referenced this issue Sep 1, 2023
@t0yv0
Introduce provider upgrade tests (#2714)
8e8d685 
Partially fixes #2719

This PR adds coverage for testing provider upgrades from a baseline version (currently 5.42.0) to the in-source release candidate. The tests check that there are no upgrade or replace plans when upgrading the provider. This is verified on a few select resources.

Currently tested resources include:

aws:ec2/subnet:Subnet
aws:ec2/vpc:Vpc
aws:ec2:SecurityGroup
aws:eks:Cluster
aws:iam:Role
aws:iam:RolePolicyAttachment
aws:rds/instance:Instance
aws:route53:ResolverEndpoint
aws:s3/bucket:Bucket
The test relies on recording and replaying the behavior of the baseline version. There are two versions of the test, in-memory TestProviderUpgradeQuick version and full acceptance-style TestProviderUpgrade version. Full version is skipped for the moment with references to discovered issues.
@pulumi-bot pulumi-bot added the resolution/fixed This issue was fixed label Sep 1, 2023
@t0yv0
Copy link
Member Author

t0yv0 commented Sep 1, 2023

Unchecked items TBC in https://github.com/pulumi/home/issues/3061

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@t0yv0 t0yv0

Labels
kind/engineering Work that is not visible to an external user resolution/fixed This issue was fixed
Projects
None yet
Milestone
0.93
Development

Successfully merging a pull request may close this issue.

Introduce provider upgrade tests pulumi/pulumi-aws
3 participants
@t0yv0 @mikhailshilkov @pulumi-bot