Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce provider upgrade tests #2714

Merged
merged 51 commits into from
Sep 1, 2023
Merged

Introduce provider upgrade tests #2714

Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
bce00ed
Initial work
t0yv0 Aug 23, 2023
861f00f
Fix stack name discrepancies
t0yv0 Aug 23, 2023
a4554cd
Remove redundant
t0yv0 Aug 23, 2023
cd22d45
Sanity checks on ambient vs released provider
t0yv0 Aug 23, 2023
d694f77
Log baseline version
t0yv0 Aug 23, 2023
10c1b52
Add friendly Make targets
t0yv0 Aug 23, 2023
0e47acd
Fix quick test
t0yv0 Aug 23, 2023
9e0cc7a
Implement -short unit testing
t0yv0 Aug 23, 2023
0954a08
Column width
t0yv0 Aug 24, 2023
75f497e
Note limitations
t0yv0 Aug 24, 2023
132d28f
Make test names consistent
t0yv0 Aug 24, 2023
ad7045e
Record AWS instance data
t0yv0 Aug 24, 2023
9a1ff28
Merge branch 't0yv0/upgrade-tests' into t0yv0/upgrade-tests-v6
t0yv0 Aug 25, 2023
ada6650
go mod tidy
t0yv0 Aug 25, 2023
7b39f9f
Upgrade to V6
t0yv0 Aug 25, 2023
9173eee
Move under examples/ so we do not need to edit CI
t0yv0 Aug 25, 2023
5c7661b
Fix Go module name
t0yv0 Aug 25, 2023
47d921d
Headers and docs
t0yv0 Aug 25, 2023
b0328a9
Typo
t0yv0 Aug 25, 2023
d0db4ca
Fix makefile target
t0yv0 Aug 25, 2023
4d58b80
Drop ./tests for now
t0yv0 Aug 25, 2023
34f9bcf
Run as part of java test set instead of nodejs test set
t0yv0 Aug 25, 2023
f5b7176
Fix record target
t0yv0 Aug 25, 2023
62eef5c
Add timeout - these tests can be slow
t0yv0 Aug 25, 2023
188192b
Fix typo
t0yv0 Aug 25, 2023
f1bfcac
Define an EKS Cluster example
t0yv0 Aug 25, 2023
36a6f1a
Workaround refresh failure
t0yv0 Aug 25, 2023
3a32c6b
Record eks.Cluster logs
t0yv0 Aug 25, 2023
e3d0fc0
Fix make target
t0yv0 Aug 25, 2023
c373a8c
Ensure bucket names are unique per stack
t0yv0 Aug 25, 2023
fd518e9
Record baselines again
t0yv0 Aug 25, 2023
dc551fa
PR feedback - Truthy test
t0yv0 Aug 25, 2023
3bd18a3
PR feedback: notes around isDiffRecord
t0yv0 Aug 25, 2023
80bac60
Check for empty URN
t0yv0 Aug 25, 2023
4cacd60
Elaborate code based on PR feedback
t0yv0 Aug 25, 2023
bd4758b
Lint: unneeded return
t0yv0 Aug 25, 2023
16c4c80
PR nits
t0yv0 Aug 25, 2023
b46d88e
Typo
t0yv0 Aug 25, 2023
92706f2
Add tests for route53 resolverendpoint
t0yv0 Aug 25, 2023
d85e8e6
Enable quick replay of upgrade tests
t0yv0 Aug 25, 2023
ebe4463
Add a comment
t0yv0 Aug 28, 2023
a5646cd
Merged eks.Cluster tests
t0yv0 Aug 28, 2023
54c01ca
Merge route53ResolverEndpoint tests
t0yv0 Aug 28, 2023
e477bb3
Fix record target
t0yv0 Aug 28, 2023
7b9d238
Re-record resource upgrade baselines
t0yv0 Aug 28, 2023
f7d2295
Note why we are skipping tests
t0yv0 Aug 28, 2023
dc9a146
Merged master
t0yv0 Aug 31, 2023
f20f3ef
Rewrite preview-only tests for clarity
t0yv0 Aug 31, 2023
65d3cdc
Consolidate in one test
t0yv0 Aug 31, 2023
7792abe
Docs/makefile updates
t0yv0 Aug 31, 2023
1107e67
Make pre-recording faster
t0yv0 Aug 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,15 @@ test:
cd provider/shim && go test -v .
cd examples && go test -v -tags=all -parallel $(TESTPARALLELISM) -timeout 2h

# Tests that the work-in-progress provider causes no replace or update plans for stacks deployed
# using the last released (baseline) version.
test.upgrade::
t0yv0 marked this conversation as resolved.
Show resolved Hide resolved
cd examples && go test -v -tags=all -run TestProviderUpgrade -timeout 2h

# Runs integration tests on the baseline version and updates testdata to record expected behavior.
test.upgrade.record::
cd examples && PULUMI_ACCEPT=true go test -v -tags all -run TestProviderUpgradeRecord -timeout 2h
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Makefile feels like a non-ideal place to define a 2hr timeout. I would expect to define a timeout in the workflow for a CI job or just rely on the user for a timeout when running locally?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the state of the art in the Makefile already :) Just following existing practice.


tfgen: install_plugins patch_upstream
(cd provider && go build $(PULUMI_PROVIDER_BUILD_PARALLELISM) -o $(WORKING_DIR)/bin/$(TFGEN) -ldflags "-X $(PROJECT)/$(VERSION_PATH)=$(VERSION)" $(PROJECT)/$(PROVIDER_PATH)/cmd/$(TFGEN))
$(WORKING_DIR)/bin/$(TFGEN) schema --out provider/cmd/$(PROVIDER)
Expand Down
6 changes: 3 additions & 3 deletions examples/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@ require (
github.com/pulumi/pulumi-terraform-bridge/pf v0.16.0
github.com/pulumi/pulumi-terraform-bridge/testing v0.0.1
github.com/pulumi/pulumi/pkg/v3 v3.76.1
github.com/pulumi/pulumi/sdk/v3 v3.76.1
github.com/stretchr/testify v1.8.4
google.golang.org/protobuf v1.31.0
gopkg.in/yaml.v2 v2.4.0
)

// Replace to allow for correctly linking the aws provider.
Expand Down Expand Up @@ -244,7 +247,6 @@ require (
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/pulumi/pulumi-terraform-bridge/v3 v3.57.0 // indirect
github.com/pulumi/pulumi-terraform-bridge/x/muxer v0.0.7-0.20230801203955-5d215c892096 // indirect
github.com/pulumi/pulumi/sdk/v3 v3.76.1 // indirect
github.com/pulumi/terraform-diff-reader v0.0.2 // indirect
github.com/rivo/uniseg v0.4.4 // indirect
github.com/rogpeppe/go-internal v1.9.0 // indirect
Expand Down Expand Up @@ -291,10 +293,8 @@ require (
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/grpc v1.56.1 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/frand v1.4.2 // indirect
sourcegraph.com/sourcegraph/appdash v0.0.0-20211028080628-e2786a622600 // indirect
Expand Down
Loading
Loading