Argo CD Synchronization Issues

[nasx]

There are a couple of issues with the deployment of CNV + Image Builder. Specifically, the config-cnv and rfe-image-builder-vm applications fail to synchronize.

1. HyperConverged resource in config-cnv application fails to synchronize with error: no matches for kind "HyperConverged" in version "hco.kubevirt.io/v1beta1". This appears to be because the HyperConverged resource defined in openshift/gitops/manifests/configs/cnv/base/hyperconverged.yaml is applied by Argo CD before the CNV operator can instantiate the CRD. Proposed solution: update sync-wave and/or use Argo CD health checks.

2. VirtualMachine resource in rfe-image-builder-vm application fails to synchronize with error: no matches for kind "VirtualMachine" in version "kubevirt.io/v1alpha3". This appears to be because the VirtualMachine resource defined in openshift/gitops/manifests/rfe/image-builder-vm/base/image-builder-vm.yaml is applied by Argo CD before the CNV operator can instantiate the CRD. Proposed solution: update sync-wave and/or use Argo CD health checks.

3. The Kubernetes Job image-builder-vm-ansible-job is failing here. The Ansible role openshift-image-builder-imi is called to dynamically build an in-memory inventory of the image builder host. For reasons unknown, Ansible is now trying to connect via the hostname (which the role defines as image-builder in the inventory) instead of the IP associated with the host in the inventory. Proposed solution:

   • Using the existing image-builder service (FQDN: image-builder.rfe.svc.cluster.local), create a ConfigMap with an Ansible inventory file containing a hard coded reference to the image-builder service.
   • Adjust ansible-playbook execution in the image-builder-vm-ansible-job Job to include a reference to the inventory mounted via volume/volumeMounts in the Job container spec.
   • Modify hosts parameter in playbooks to use the Kubernetes Service FQDN.

[sabre1041]

@nasx For #1 and #2, we can look to customize the retry policies. As long as SkipDryRunOnMissingResource is defined on the CR, it wont fail immediately. Otherwise, the sync will attempt until the retries have been exhausted. Regardless, if the operator does not instantiate the CRD's in 3 minutes, the sync will fail.

[pkliczewski]

Here is one more observed issue #70 related issue.

[sabre1041]

@nasx can this be closed?