ignore_annotations change between 2.24.0 & 2.25.1

[quotidian-ennui]

Terraform version, Kubernetes provider version and Kubernetes version

Terraform version: 1.5.7
Kubernetes Provider version: 2.25.1
Kubernetes version: 1.28.3

Terraform configuration

provider "kubernetes" {
  ...
  ignore_annotations = [
    # Make sure we ignore the last-reloaded-from annotation
    "^reloader\\.stakater\\.com\\/last-reloaded-from$",
  ]
}

Question

#2345 tells me that ignore_annotations on the provider is not respected when gathering the annotations from templates and datasources.

This isn't going to play nicely with something like stakater/reloader since this will (depending on configuration) add an annotation to the deployment reloader.stakater.com/last-reloaded-from that marks the 'change' that triggered a redeploy.

If a config-map changes, and reloader does its thing; then we have the annotation applied. The next time terraform runs, it wants to remove the annotation (which causes a redeployment even though nothing concrete has changed).

What then is the correct way for us ignore annotations that might be added by server-side components that don't end in kubernetes.io because this possible in 2.24.0.

[arybolovlev]

Hi @quotidian-ennui,

Thank you for reporting this issue. I think we need to tune the logic of flatteners a bit more to cover the use case you just reported.

If my reproduction of this issue is valid, then 2.24.0 should work, but has a few side effects:

• It doesn't take into account ignore_annotations when it comes to flattening template metadata. In your case, it should work even without the ignore_annotations block.
• The provider doesn't detect changes that were made outside of Terraform in template metadata.

Could you please confirm the first statement?

In the meantime, I will work on this issue.

Thanks!

[quotidian-ennui]

@arybolovlev yes, 2.24.0 works fine for my use-case, and I've pinned it locally to that version.

(I haven't noticed side-effects on my side, but you're closer to the code than I am).

[gabegorelick]

This isn't going to play nicely with something like stakater/reloader since this will (depending on configuration) add an annotation to the deployment reloader.stakater.com/last-reloaded-from that marks the 'change' that triggered a redeploy.

Similar issue with kubectl rollout restart since it adds a kubectl.kubernetes.io/restartedAt annotation. I've had to resort to ignore_changes as a workaround since upgrading to 2.25.1.

[quotidian-ennui]

👍 for that top tip, I reported it since it's a breaking change in the sense that 2.24.0 is fine (with the ignore_annotations in the kubernetes provider config above), but 2.25.x isn't; so a semver upgrade to 3.0 would have caused less drama round my neck of the woods...

Even so adding this to every deployment managed by terraform doesn't fill me with joy...

  lifecycle {
    ignore_changes = [
      spec[0].template[0].metadata[0].annotations["reloader.stakater.com/last-reloaded-from"],
    ]
  }