From e8333fa79fc0bf4d72d7f8cdbdf6f929410c5395 Mon Sep 17 00:00:00 2001 From: akutz Date: Fri, 10 Jan 2025 15:41:29 -0600 Subject: [PATCH] Fast Deploy Direct & Linked (Experimental) This patch adds support for the Fast Deploy Direct and Linked features, i.e. the ability to cache images per-datastore and quickly provision a VM from these caches, either directly or as a linked clone. This is an experimental feature that must be enabled manually. There are many things about this feature that may change prior to it being ready for production. The patch notes below are broken down into several sections: * **Goals** -- What is currently supported * **Non-goals** -- What is not on the table right now * **Architecture** * **Activation** -- How to enable this experimental feature * **Placement** -- Request datastore recommendations * **Image cache** -- A general-purpose VM image cache * **Create VM** -- Create directly from cached disk The following goals are what is considered in-scope for this experimental feature at this time. Just because something is not listed, it does not mean it will not be added before the feature is made generally available: * Support all VM images that are OVFs * Support multiple zones * Support workload-domain isolation * Support all datastore types, including host-local and vSAN * Support for configuring a default fast-deploy mode * Support picking the fast-deploy mode per VM (direct, linked) * Support disabling fast-deploy per VM * Support VM encryption for VMs deployed with fast deploy direct * Support backup/restore for VMs deployed with fast deploy direct * Support site replication for VMs deployed with fast deploy direct * Support datastore maintenance/migration for VMs deployed with fast deploy direct The following is a list of non-goals that are not in scope at this time, although most of them should be revisited prior to this feature graduating to production: * Support VM images that are VM templates (VMTX) The architecture behind Fast Deploy makes it trivial to support deploying VM images that point to VM templates. While not in scope at this time, it is likely this becomes part of the feature prior to it graduating to production-ready. The architecture is broken down into the following sections: * **Activation** -- How to enable this experimental feature * **Placement** -- Request datastore recommendations * **Image cache** -- A general-purpose VM image cache * **Create VM** -- Create directly from cached disk Enabling the experimental Fast Deploy feature requires setting the environment variable `FSS_WCP_VMSERVICE_FAST_DEPLOY` to `true` in the VM Operator deployment. The environment variable `FAST_DEPLOY_MODE` may be set to one of the following values to configure the default mode for the fast-deploy feature: * `direct` -- VMs are deployed using cached disks * `linked` -- VMs are deployed as a linked clone * the value is empty -- `direct` mode is used * the value is anything else -- fast deploy is disabled It is possible to override the default mode per-VM by setting the annotation `vmoperator.vmware.com/fast-deploy`. The values of this annotation follow the same rules described above. Please note, setting the environment variable `FAST_DEPLOY_MODE` or the annotation `vmoperator.vmware.com/fast-deploy` has no effect if the feature is not enabled. Please refer to PR #823 for information on placement as the logic from that change has stayed the same in this one. The way the images/disks are cached has completely changed since PR * not visible to DevOps users * a namespace-scoped resource that only exists in the same namespace as the VM Operator controller pod * used to cache the OVF and an image's disks A `VirtualMachineImageCache` resource is created per unique library item resource. That means even if there are 20,000 VMI resources spread across a multitude of namespaces or at the cluster scope, if they all point to the same underlying library item, then for all those VMI resources there will be a single `VirtualMachineImageCache` resource in the VM Operator namespace. The `VirtualMachineImageCache` controller caches the OVF for the image in a `ConfigMap` resource in the VM Operator namespace. This completely obviates the need to maintain a bespoke, in-memory OVF cache. The `VirtualMachineImageCache` resource caches the image's disks on specified datastores by setting `spec.locations` with entries that map to unique datacenter/datastore IDs. The resource's status reveals the location(s) of the cached disk(s). For a more in-depth look on how the disks are actually cached, please refer to PR #823. If the `VirtualMachineImageCache` object is not ready with the cached OVF or disks, then the VM will be re-enqueued once the `VirtualMachineImageCache` _is_ ready. Please note, while placement is required to know where to cache the disks, additional placement calls are not issued if a VM is actively awaiting a `VirtualMachineImageCache` resource. Beyond that, the create VM workflow depends on the fast-deploy mode: 1. The cached disks are copied into the VM's folder. 2. The ConfigSpec is updated to reference the disks. a. Please note, if the VM is encrypted, the disks are not as part of the create call. This is because it is not possible to change the encryption state of disks when adding them to a VM. Thus the disks are encrypted after the VM is created, before it is powered on. 3. The `CreateVM_Task` VMODL1 API is used to create the VM. 1. The `VirtualDisk` devices in the ConfigSpec used to create the VM are updated with `VirtualDiskFlatVer2BackingInfo` backings that specify a parent backing which refers to the cached, base disk from above. The path to each of the VM's disks is constructed based on the index of the disk, ex.: `[] /-.vmdk`. 2. The `CreateVM_Task` VMODL1 API is used to create the VM. Because the the VM's disks have parent backings, this new VM is effectively a linked clone. --- .golangci.yml | 6 + api/v1alpha3/virtualmachineimage_types.go | 12 +- .../virtualmachineimagecache_types.go | 202 ++++ api/v1alpha3/zz_generated.deepcopy.go | 185 ++++ .../zz_virtualmachine_guestosid_generated.go | 2 +- ....vmware.com_virtualmachineimagecaches.yaml | 270 ++++++ config/rbac/role.yaml | 2 + .../clustercontentlibraryitem_controller.go | 324 +------ ...contentlibraryitem_controller_intg_test.go | 26 +- ...ontentlibraryitem_controller_suite_test.go | 2 +- ...contentlibraryitem_controller_unit_test.go | 329 ------- .../contentlibraryitem_controller.go | 292 +----- ...contentlibraryitem_controller_intg_test.go | 22 + ...ontentlibraryitem_controller_suite_test.go | 2 +- ...contentlibraryitem_controller_unit_test.go | 307 ------- .../utils/controller_builder.go | 468 ++++++++++ .../utils/controller_builder_test.go | 565 ++++++++++++ .../contentlibrary/utils/test_utils.go | 3 + .../contentlibrary/utils/utils_suite_test.go | 21 + .../contentlibrary/utils/utils_test.go | 219 +++-- controllers/controllers.go | 7 + .../virtualmachine_controller.go | 109 ++- ...almachineimagecache_controller_internal.go | 15 + .../virtualmachineimagecache_controller.go | 715 +++++++++++++++ ...machineimagecache_controller_suite_test.go | 24 + ...irtualmachineimagecache_controller_test.go | 862 ++++++++++++++++++ go.mod | 2 +- go.sum | 4 +- pkg/config/config.go | 25 +- pkg/config/default.go | 3 + pkg/config/env.go | 2 + pkg/config/env/env.go | 6 + pkg/config/env_test.go | 5 + pkg/constants/constants.go | 28 + .../clustercontentlibraryitem_context.go | 28 - pkg/context/contentlibraryitem_context.go | 29 - pkg/errors/errors_suite_test.go | 17 + pkg/errors/requeue_error.go | 42 + pkg/errors/requeue_error_test.go | 85 ++ pkg/errors/vmicache_not_ready_error.go | 46 + pkg/errors/vmicache_not_ready_error_test.go | 98 ++ .../content_library_provider.go | 41 + .../contentlibrary/content_library_test.go | 4 +- .../vsphere/placement/zone_placement.go | 105 +-- .../vsphere/placement/zone_placement_test.go | 19 +- .../vsphere/virtualmachine/publish_test.go | 2 +- pkg/providers/vsphere/vmlifecycle/create.go | 21 +- .../vmlifecycle/create_contentlibrary.go | 12 +- .../create_contentlibrary_linked_clone.go | 226 ----- .../vsphere/vmlifecycle/create_fastdeploy.go | 417 +++++++++ pkg/providers/vsphere/vmprovider.go | 133 ++- pkg/providers/vsphere/vmprovider_test.go | 425 +++++++-- pkg/providers/vsphere/vmprovider_vm.go | 259 +++++- pkg/providers/vsphere/vmprovider_vm_utils.go | 4 + pkg/providers/vsphere/vsphere_suite_test.go | 1 - pkg/util/devices.go | 34 + pkg/util/devices_test.go | 73 ++ pkg/util/hash.go | 25 + pkg/util/hash_test.go | 34 + pkg/util/vmopv1/image.go | 118 ++- pkg/util/vmopv1/image_test.go | 344 +++++++ pkg/util/vsphere/library/item_cache.go | 127 +-- pkg/util/vsphere/library/item_cache_test.go | 257 +----- pkg/util/vsphere/library/item_sync.go | 58 -- pkg/util/vsphere/library/item_sync_test.go | 171 ---- pkg/vmconfig/crypto/crypto_reconciler_pre.go | 100 +- test/builder/fake.go | 1 + .../testdata/images/ttylinux-pc_i486-16.1.mf | 4 +- .../testdata/images/ttylinux-pc_i486-16.1.ova | Bin 10604032 -> 10604032 bytes .../testdata/images/ttylinux-pc_i486-16.1.ovf | 35 +- test/builder/vcsim_test_context.go | 123 ++- 71 files changed, 6054 insertions(+), 2530 deletions(-) create mode 100644 api/v1alpha3/virtualmachineimagecache_types.go create mode 100644 config/crd/bases/vmoperator.vmware.com_virtualmachineimagecaches.yaml delete mode 100644 controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_unit_test.go delete mode 100644 controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_unit_test.go create mode 100644 controllers/contentlibrary/utils/controller_builder.go create mode 100644 controllers/contentlibrary/utils/controller_builder_test.go create mode 100644 controllers/contentlibrary/utils/utils_suite_test.go create mode 100644 controllers/virtualmachineimagecache/internal/virtualmachineimagecache_controller_internal.go create mode 100644 controllers/virtualmachineimagecache/virtualmachineimagecache_controller.go create mode 100644 controllers/virtualmachineimagecache/virtualmachineimagecache_controller_suite_test.go create mode 100644 controllers/virtualmachineimagecache/virtualmachineimagecache_controller_test.go delete mode 100644 pkg/context/clustercontentlibraryitem_context.go delete mode 100644 pkg/context/contentlibraryitem_context.go create mode 100644 pkg/errors/errors_suite_test.go create mode 100644 pkg/errors/requeue_error.go create mode 100644 pkg/errors/requeue_error_test.go create mode 100644 pkg/errors/vmicache_not_ready_error.go create mode 100644 pkg/errors/vmicache_not_ready_error_test.go delete mode 100644 pkg/providers/vsphere/vmlifecycle/create_contentlibrary_linked_clone.go create mode 100644 pkg/providers/vsphere/vmlifecycle/create_fastdeploy.go create mode 100644 pkg/util/hash.go create mode 100644 pkg/util/hash_test.go delete mode 100644 pkg/util/vsphere/library/item_sync.go delete mode 100644 pkg/util/vsphere/library/item_sync_test.go diff --git a/.golangci.yml b/.golangci.yml index f62c04fba..62273ebe6 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -82,6 +82,8 @@ linters-settings: pkg: github.com/vmware-tanzu/vm-operator/pkg/config - alias: pkgctx pkg: github.com/vmware-tanzu/vm-operator/pkg/context + - alias: pkgerr + pkg: github.com/vmware-tanzu/vm-operator/pkg/pkgerr - alias: ctxop pkg: github.com/vmware-tanzu/vm-operator/pkg/context/operation - alias: pkgmgr @@ -90,6 +92,10 @@ linters-settings: pkg: github.com/vmware-tanzu/vm-operator/pkg/util - alias: proberctx pkg: github.com/vmware-tanzu/vm-operator/pkg/prober/context + - alias: dsutil + pkg: github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/datastore + - alias: clsutil + pkg: github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/library depguard: rules: diff --git a/api/v1alpha3/virtualmachineimage_types.go b/api/v1alpha3/virtualmachineimage_types.go index 0daee779b..9a14484bf 100644 --- a/api/v1alpha3/virtualmachineimage_types.go +++ b/api/v1alpha3/virtualmachineimage_types.go @@ -252,6 +252,14 @@ type VirtualMachineImageStatus struct { Type string `json:"type,omitempty"` } +func (i VirtualMachineImageStatus) GetConditions() []metav1.Condition { + return i.Conditions +} + +func (i *VirtualMachineImageStatus) SetConditions(conditions []metav1.Condition) { + i.Conditions = conditions +} + // +kubebuilder:object:root=true // +kubebuilder:resource:scope=Namespaced,shortName=vmi;vmimage // +kubebuilder:storageversion @@ -273,7 +281,7 @@ type VirtualMachineImage struct { Status VirtualMachineImageStatus `json:"status,omitempty"` } -func (i *VirtualMachineImage) GetConditions() []metav1.Condition { +func (i VirtualMachineImage) GetConditions() []metav1.Condition { return i.Status.Conditions } @@ -311,7 +319,7 @@ type ClusterVirtualMachineImage struct { Status VirtualMachineImageStatus `json:"status,omitempty"` } -func (i *ClusterVirtualMachineImage) GetConditions() []metav1.Condition { +func (i ClusterVirtualMachineImage) GetConditions() []metav1.Condition { return i.Status.Conditions } diff --git a/api/v1alpha3/virtualmachineimagecache_types.go b/api/v1alpha3/virtualmachineimagecache_types.go new file mode 100644 index 000000000..95789390b --- /dev/null +++ b/api/v1alpha3/virtualmachineimagecache_types.go @@ -0,0 +1,202 @@ +// // © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package v1alpha3 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +const ( + // VirtualMachineImageCacheConditionProviderReady indicates the underlying + // provider is fully synced, including its disks being available locally on + // some datastore. + VirtualMachineImageCacheConditionProviderReady = "VirtualMachineImageCacheProviderReady" + + // VirtualMachineImageCacheConditionDisksReady indicates the disks are + // cached in a given location. + VirtualMachineImageCacheConditionDisksReady = "VirtualMachineImageCacheDisksReady" + + // VirtualMachineImageCacheConditionOVFReady indicates the OVF is cached. + VirtualMachineImageCacheConditionOVFReady = "VirtualMachineImageCacheOVFReady" +) + +type VirtualMachineImageCacheObjectRef struct { + // Kind is a string value representing the REST resource this object + // represents. + // Servers may infer this from the endpoint the client submits requests to. + // Cannot be updated. + // In CamelCase. + // More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + Kind string `json:"kind"` + + // +optional + + // Namespace refers to a namespace. + // This field will be empty if Kind refers to a cluster-scoped resource. + Namespace string `json:"namespace,omitempty"` + + // Name refers to a unique resource. + // More info: http://kubernetes.io/docs/user-guide/identifiers#names + Name string `json:"name"` +} + +type VirtualMachineImageCacheLocationSpec struct { + // DatacenterID describes the ID of the datacenter to which the image should + // be cached. + DatacenterID string `json:"datacenterID"` + + // DatastoreID describes the ID of the datastore to which the image should + // be cached. + DatastoreID string `json:"datastoreID"` +} + +// VirtualMachineImageCacheSpec defines the desired state of +// VirtualMachineImageCache. +type VirtualMachineImageCacheSpec struct { + // ProviderID describes the ID of the provider item to which the image + // corresponds. + // If the provider is Content Library, the ID refers to a Content Library + // item. + ProviderID string `json:"providerID"` + + // ProviderVersion describes the version of the provider item to which the + // image corresponds. + // The provider is Content Library, the version is the content version. + ProviderVersion string `json:"providerVersion"` + + // +optional + // +listType=map + // +listMapKey=datacenterID + // +listMapKey=datastoreID + + // Locations describes the locations where the image should be cached. + Locations []VirtualMachineImageCacheLocationSpec `json:"locations,omitempty"` +} + +func (s *VirtualMachineImageCacheSpec) SetLocation(dcID, dsID string) { + for i := range s.Locations { + l := s.Locations[i] + if l.DatacenterID == dcID && l.DatastoreID == dsID { + return + } + } + s.Locations = append(s.Locations, VirtualMachineImageCacheLocationSpec{ + DatacenterID: dcID, + DatastoreID: dsID, + }) +} + +type VirtualMachineImageCacheLocationStatus struct { + + // DatacenterID describes the ID of the datacenter to which the image should + // be cached. + DatacenterID string `json:"datacenterID"` + + // DatastoreID describes the ID of the datastore to which the image should + // be cached. + DatastoreID string `json:"datastoreID"` + + // +optional + + // Files describes the paths to the image's cached files on this datastore. + Files []string `json:"files,omitempty"` + + // +optional + + // Conditions describes any conditions associated with this cache location. + // + // Generally this should just include the ReadyType condition. + Conditions []metav1.Condition `json:"conditions,omitempty"` +} + +func (i VirtualMachineImageCacheLocationStatus) GetConditions() []metav1.Condition { + return i.Conditions +} + +func (i *VirtualMachineImageCacheLocationStatus) SetConditions(conditions []metav1.Condition) { + i.Conditions = conditions +} + +type VirtualMachineImageCacheOVFStatus struct { + + // +optional + + // ConfigMapName describes the name of the ConfigMap resource that contains + // the image's OVF envelope encoded as YAML. The data is located in the + // ConfigMap key "value". + ConfigMapName string `json:"configMapName,omitempty"` + + // +optional + + // ProviderVersion describes the observed provider version at which the OVF + // is cached. + // The provider is Content Library, the version is the content version. + ProviderVersion string `json:"providerVersion,omitempty"` +} + +// VirtualMachineImageCacheStatus defines the observed state of +// VirtualMachineImageCache. +type VirtualMachineImageCacheStatus struct { + + // +optional + // +listType=map + // +listMapKey=datacenterID + // +listMapKey=datastoreID + + // Locations describe the observed locations where the image is cached. + Locations []VirtualMachineImageCacheLocationStatus `json:"locations,omitempty"` + + // +optional + + // OVF describes the observed status of the cached OVF content. + OVF *VirtualMachineImageCacheOVFStatus `json:"ovf,omitempty"` + + // +optional + + // Conditions describes any conditions associated with this cached image. + // + // Generally this should just include the ReadyType condition, which will + // only be True if all of the cached locations also have True ReadyType + // condition. + Conditions []metav1.Condition `json:"conditions,omitempty"` +} + +func (i VirtualMachineImageCache) GetConditions() []metav1.Condition { + return i.Status.Conditions +} + +func (i *VirtualMachineImageCache) SetConditions(conditions []metav1.Condition) { + i.Status.Conditions = conditions +} + +// +kubebuilder:object:root=true +// +kubebuilder:resource:scope=Namespaced,shortName=vmic;vmicache;vmimagecache +// +kubebuilder:storageversion +// +kubebuilder:subresource:status + +// VirtualMachineImageCache is the schema for the +// virtualmachineimagecaches API. +type VirtualMachineImageCache struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec VirtualMachineImageCacheSpec `json:"spec,omitempty"` + Status VirtualMachineImageCacheStatus `json:"status,omitempty"` +} + +// +kubebuilder:object:root=true + +// VirtualMachineImageCacheList contains a list of VirtualMachineImageCache. +type VirtualMachineImageCacheList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []VirtualMachineImageCache `json:"items"` +} + +func init() { + objectTypes = append(objectTypes, + &VirtualMachineImageCache{}, + &VirtualMachineImageCacheList{}) +} diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index d76c538cb..38ffd8158 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -820,6 +820,191 @@ func (in *VirtualMachineImage) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCache) DeepCopyInto(out *VirtualMachineImageCache) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCache. +func (in *VirtualMachineImageCache) DeepCopy() *VirtualMachineImageCache { + if in == nil { + return nil + } + out := new(VirtualMachineImageCache) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *VirtualMachineImageCache) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCacheList) DeepCopyInto(out *VirtualMachineImageCacheList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]VirtualMachineImageCache, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCacheList. +func (in *VirtualMachineImageCacheList) DeepCopy() *VirtualMachineImageCacheList { + if in == nil { + return nil + } + out := new(VirtualMachineImageCacheList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *VirtualMachineImageCacheList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCacheLocationSpec) DeepCopyInto(out *VirtualMachineImageCacheLocationSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCacheLocationSpec. +func (in *VirtualMachineImageCacheLocationSpec) DeepCopy() *VirtualMachineImageCacheLocationSpec { + if in == nil { + return nil + } + out := new(VirtualMachineImageCacheLocationSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCacheLocationStatus) DeepCopyInto(out *VirtualMachineImageCacheLocationStatus) { + *out = *in + if in.Files != nil { + in, out := &in.Files, &out.Files + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCacheLocationStatus. +func (in *VirtualMachineImageCacheLocationStatus) DeepCopy() *VirtualMachineImageCacheLocationStatus { + if in == nil { + return nil + } + out := new(VirtualMachineImageCacheLocationStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCacheOVFStatus) DeepCopyInto(out *VirtualMachineImageCacheOVFStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCacheOVFStatus. +func (in *VirtualMachineImageCacheOVFStatus) DeepCopy() *VirtualMachineImageCacheOVFStatus { + if in == nil { + return nil + } + out := new(VirtualMachineImageCacheOVFStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCacheObjectRef) DeepCopyInto(out *VirtualMachineImageCacheObjectRef) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCacheObjectRef. +func (in *VirtualMachineImageCacheObjectRef) DeepCopy() *VirtualMachineImageCacheObjectRef { + if in == nil { + return nil + } + out := new(VirtualMachineImageCacheObjectRef) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCacheSpec) DeepCopyInto(out *VirtualMachineImageCacheSpec) { + *out = *in + if in.Locations != nil { + in, out := &in.Locations, &out.Locations + *out = make([]VirtualMachineImageCacheLocationSpec, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCacheSpec. +func (in *VirtualMachineImageCacheSpec) DeepCopy() *VirtualMachineImageCacheSpec { + if in == nil { + return nil + } + out := new(VirtualMachineImageCacheSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VirtualMachineImageCacheStatus) DeepCopyInto(out *VirtualMachineImageCacheStatus) { + *out = *in + if in.Locations != nil { + in, out := &in.Locations, &out.Locations + *out = make([]VirtualMachineImageCacheLocationStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.OVF != nil { + in, out := &in.OVF, &out.OVF + *out = new(VirtualMachineImageCacheOVFStatus) + **out = **in + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VirtualMachineImageCacheStatus. +func (in *VirtualMachineImageCacheStatus) DeepCopy() *VirtualMachineImageCacheStatus { + if in == nil { + return nil + } + out := new(VirtualMachineImageCacheStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *VirtualMachineImageDiskInfo) DeepCopyInto(out *VirtualMachineImageDiskInfo) { *out = *in diff --git a/api/v1alpha3/zz_virtualmachine_guestosid_generated.go b/api/v1alpha3/zz_virtualmachine_guestosid_generated.go index 7bbc88019..89d5b91e0 100644 --- a/api/v1alpha3/zz_virtualmachine_guestosid_generated.go +++ b/api/v1alpha3/zz_virtualmachine_guestosid_generated.go @@ -6,5 +6,5 @@ package v1alpha3 -// +kubebuilder:validation:Enum=dosGuest;win31Guest;win95Guest;win98Guest;winMeGuest;winNTGuest;win2000ProGuest;win2000ServGuest;win2000AdvServGuest;winXPHomeGuest;winXPProGuest;winXPPro64Guest;winNetWebGuest;winNetStandardGuest;winNetEnterpriseGuest;winNetDatacenterGuest;winNetBusinessGuest;winNetStandard64Guest;winNetEnterprise64Guest;winLonghornGuest;winLonghorn64Guest;winNetDatacenter64Guest;winVistaGuest;winVista64Guest;windows7Guest;windows7_64Guest;windows7Server64Guest;windows8Guest;windows8_64Guest;windows8Server64Guest;windows9Guest;windows9_64Guest;windows9Server64Guest;windows11_64Guest;windows12_64Guest;windowsHyperVGuest;windows2019srv_64Guest;windows2019srvNext_64Guest;windows2022srvNext_64Guest;freebsdGuest;freebsd64Guest;freebsd11Guest;freebsd11_64Guest;freebsd12Guest;freebsd12_64Guest;freebsd13Guest;freebsd13_64Guest;freebsd14Guest;freebsd14_64Guest;redhatGuest;rhel2Guest;rhel3Guest;rhel3_64Guest;rhel4Guest;rhel4_64Guest;rhel5Guest;rhel5_64Guest;rhel6Guest;rhel6_64Guest;rhel7Guest;rhel7_64Guest;rhel8_64Guest;rhel9_64Guest;centosGuest;centos64Guest;centos6Guest;centos6_64Guest;centos7Guest;centos7_64Guest;centos8_64Guest;centos9_64Guest;oracleLinuxGuest;oracleLinux64Guest;oracleLinux6Guest;oracleLinux6_64Guest;oracleLinux7Guest;oracleLinux7_64Guest;oracleLinux8_64Guest;oracleLinux9_64Guest;suseGuest;suse64Guest;slesGuest;sles64Guest;sles10Guest;sles10_64Guest;sles11Guest;sles11_64Guest;sles12Guest;sles12_64Guest;sles15_64Guest;sles16_64Guest;nld9Guest;oesGuest;sjdsGuest;mandrakeGuest;mandrivaGuest;mandriva64Guest;turboLinuxGuest;turboLinux64Guest;ubuntuGuest;ubuntu64Guest;debian4Guest;debian4_64Guest;debian5Guest;debian5_64Guest;debian6Guest;debian6_64Guest;debian7Guest;debian7_64Guest;debian8Guest;debian8_64Guest;debian9Guest;debian9_64Guest;debian10Guest;debian10_64Guest;debian11Guest;debian11_64Guest;debian12Guest;debian12_64Guest;asianux3Guest;asianux3_64Guest;asianux4Guest;asianux4_64Guest;asianux5_64Guest;asianux7_64Guest;asianux8_64Guest;asianux9_64Guest;opensuseGuest;opensuse64Guest;fedoraGuest;fedora64Guest;coreos64Guest;vmwarePhoton64Guest;other24xLinuxGuest;other26xLinuxGuest;otherLinuxGuest;other3xLinuxGuest;other4xLinuxGuest;other5xLinuxGuest;other6xLinuxGuest;genericLinuxGuest;other24xLinux64Guest;other26xLinux64Guest;other3xLinux64Guest;other4xLinux64Guest;other5xLinux64Guest;other6xLinux64Guest;otherLinux64Guest;solaris6Guest;solaris7Guest;solaris8Guest;solaris9Guest;solaris10Guest;solaris10_64Guest;solaris11_64Guest;os2Guest;eComStationGuest;eComStation2Guest;netware4Guest;netware5Guest;netware6Guest;openServer5Guest;openServer6Guest;unixWare7Guest;darwinGuest;darwin64Guest;darwin10Guest;darwin10_64Guest;darwin11Guest;darwin11_64Guest;darwin12_64Guest;darwin13_64Guest;darwin14_64Guest;darwin15_64Guest;darwin16_64Guest;darwin17_64Guest;darwin18_64Guest;darwin19_64Guest;darwin20_64Guest;darwin21_64Guest;darwin22_64Guest;darwin23_64Guest;vmkernelGuest;vmkernel5Guest;vmkernel6Guest;vmkernel65Guest;vmkernel7Guest;vmkernel8Guest;amazonlinux2_64Guest;amazonlinux3_64Guest;crxPod1Guest;crxSys1Guest;rockylinux_64Guest;almalinux_64Guest;otherGuest;otherGuest64 +// +kubebuilder:validation:Enum=dosGuest;win31Guest;win95Guest;win98Guest;winMeGuest;winNTGuest;win2000ProGuest;win2000ServGuest;win2000AdvServGuest;winXPHomeGuest;winXPProGuest;winXPPro64Guest;winNetWebGuest;winNetStandardGuest;winNetEnterpriseGuest;winNetDatacenterGuest;winNetBusinessGuest;winNetStandard64Guest;winNetEnterprise64Guest;winLonghornGuest;winLonghorn64Guest;winNetDatacenter64Guest;winVistaGuest;winVista64Guest;windows7Guest;windows7_64Guest;windows7Server64Guest;windows8Guest;windows8_64Guest;windows8Server64Guest;windows9Guest;windows9_64Guest;windows9Server64Guest;windows11_64Guest;windows12_64Guest;windowsHyperVGuest;windows2019srv_64Guest;windows2019srvNext_64Guest;windows2022srvNext_64Guest;freebsdGuest;freebsd64Guest;freebsd11Guest;freebsd11_64Guest;freebsd12Guest;freebsd12_64Guest;freebsd13Guest;freebsd13_64Guest;freebsd14Guest;freebsd14_64Guest;freebsd15Guest;freebsd15_64Guest;redhatGuest;rhel2Guest;rhel3Guest;rhel3_64Guest;rhel4Guest;rhel4_64Guest;rhel5Guest;rhel5_64Guest;rhel6Guest;rhel6_64Guest;rhel7Guest;rhel7_64Guest;rhel8_64Guest;rhel9_64Guest;rhel10_64Guest;centosGuest;centos64Guest;centos6Guest;centos6_64Guest;centos7Guest;centos7_64Guest;centos8_64Guest;centos9_64Guest;oracleLinuxGuest;oracleLinux64Guest;oracleLinux6Guest;oracleLinux6_64Guest;oracleLinux7Guest;oracleLinux7_64Guest;oracleLinux8_64Guest;oracleLinux9_64Guest;oracleLinux10_64Guest;suseGuest;suse64Guest;slesGuest;sles64Guest;sles10Guest;sles10_64Guest;sles11Guest;sles11_64Guest;sles12Guest;sles12_64Guest;sles15_64Guest;sles16_64Guest;nld9Guest;oesGuest;sjdsGuest;mandrakeGuest;mandrivaGuest;mandriva64Guest;turboLinuxGuest;turboLinux64Guest;ubuntuGuest;ubuntu64Guest;debian4Guest;debian4_64Guest;debian5Guest;debian5_64Guest;debian6Guest;debian6_64Guest;debian7Guest;debian7_64Guest;debian8Guest;debian8_64Guest;debian9Guest;debian9_64Guest;debian10Guest;debian10_64Guest;debian11Guest;debian11_64Guest;debian12Guest;debian12_64Guest;debian13Guest;debian13_64Guest;asianux3Guest;asianux3_64Guest;asianux4Guest;asianux4_64Guest;asianux5_64Guest;asianux7_64Guest;asianux8_64Guest;asianux9_64Guest;miraclelinux_64Guest;pardus_64Guest;opensuseGuest;opensuse64Guest;fedoraGuest;fedora64Guest;coreos64Guest;vmwarePhoton64Guest;other24xLinuxGuest;other26xLinuxGuest;otherLinuxGuest;other3xLinuxGuest;other4xLinuxGuest;other5xLinuxGuest;other6xLinuxGuest;other7xLinuxGuest;genericLinuxGuest;other24xLinux64Guest;other26xLinux64Guest;other3xLinux64Guest;other4xLinux64Guest;other5xLinux64Guest;other6xLinux64Guest;other7xLinux64Guest;otherLinux64Guest;solaris6Guest;solaris7Guest;solaris8Guest;solaris9Guest;solaris10Guest;solaris10_64Guest;solaris11_64Guest;fusionos_64Guest;prolinux_64Guest;kylinlinux_64Guest;os2Guest;eComStationGuest;eComStation2Guest;netware4Guest;netware5Guest;netware6Guest;openServer5Guest;openServer6Guest;unixWare7Guest;darwinGuest;darwin64Guest;darwin10Guest;darwin10_64Guest;darwin11Guest;darwin11_64Guest;darwin12_64Guest;darwin13_64Guest;darwin14_64Guest;darwin15_64Guest;darwin16_64Guest;darwin17_64Guest;darwin18_64Guest;darwin19_64Guest;darwin20_64Guest;darwin21_64Guest;darwin22_64Guest;darwin23_64Guest;vmkernelGuest;vmkernel5Guest;vmkernel6Guest;vmkernel65Guest;vmkernel7Guest;vmkernel8Guest;vmkernel9Guest;amazonlinux2_64Guest;amazonlinux3_64Guest;crxPod1Guest;crxSys1Guest;rockylinux_64Guest;almalinux_64Guest;otherGuest;otherGuest64 type VirtualMachineGuestOSIdentifier string diff --git a/config/crd/bases/vmoperator.vmware.com_virtualmachineimagecaches.yaml b/config/crd/bases/vmoperator.vmware.com_virtualmachineimagecaches.yaml new file mode 100644 index 000000000..d7b883b86 --- /dev/null +++ b/config/crd/bases/vmoperator.vmware.com_virtualmachineimagecaches.yaml @@ -0,0 +1,270 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: virtualmachineimagecaches.vmoperator.vmware.com +spec: + group: vmoperator.vmware.com + names: + kind: VirtualMachineImageCache + listKind: VirtualMachineImageCacheList + plural: virtualmachineimagecaches + shortNames: + - vmic + - vmicache + - vmimagecache + singular: virtualmachineimagecache + scope: Namespaced + versions: + - name: v1alpha3 + schema: + openAPIV3Schema: + description: |- + VirtualMachineImageCache is the schema for the + virtualmachineimagecaches API. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + VirtualMachineImageCacheSpec defines the desired state of + VirtualMachineImageCache. + properties: + locations: + description: Locations describes the locations where the image should + be cached. + items: + properties: + datacenterID: + description: |- + DatacenterID describes the ID of the datacenter to which the image should + be cached. + type: string + datastoreID: + description: |- + DatastoreID describes the ID of the datastore to which the image should + be cached. + type: string + required: + - datacenterID + - datastoreID + type: object + type: array + x-kubernetes-list-map-keys: + - datacenterID + - datastoreID + x-kubernetes-list-type: map + providerID: + description: |- + ProviderID describes the ID of the provider item to which the image + corresponds. + If the provider is Content Library, the ID refers to a Content Library + item. + type: string + providerVersion: + description: |- + ProviderVersion describes the version of the provider item to which the + image corresponds. + The provider is Content Library, the version is the content version. + type: string + required: + - providerID + - providerVersion + type: object + status: + description: |- + VirtualMachineImageCacheStatus defines the observed state of + VirtualMachineImageCache. + properties: + conditions: + description: |- + Conditions describes any conditions associated with this cached image. + + Generally this should just include the ReadyType condition, which will + only be True if all of the cached locations also have True ReadyType + condition. + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + locations: + description: Locations describe the observed locations where the image + is cached. + items: + properties: + conditions: + description: |- + Conditions describes any conditions associated with this cache location. + + Generally this should just include the ReadyType condition. + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + datacenterID: + description: |- + DatacenterID describes the ID of the datacenter to which the image should + be cached. + type: string + datastoreID: + description: |- + DatastoreID describes the ID of the datastore to which the image should + be cached. + type: string + files: + description: Files describes the paths to the image's cached + files on this datastore. + items: + type: string + type: array + required: + - datacenterID + - datastoreID + type: object + type: array + x-kubernetes-list-map-keys: + - datacenterID + - datastoreID + x-kubernetes-list-type: map + ovf: + description: OVF describes the observed status of the cached OVF content. + properties: + configMapName: + description: |- + ConfigMapName describes the name of the ConfigMap resource that contains + the image's OVF envelope encoded as YAML. The data is located in the + ConfigMap key "value". + type: string + providerVersion: + description: |- + ProviderVersion describes the observed provider version at which the OVF + is cached. + The provider is Content Library, the version is the content version. + type: string + type: object + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 28422953b..3704c232b 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -210,6 +210,7 @@ rules: resources: - clustervirtualmachineimages - virtualmachineclasses + - virtualmachineimagecaches - virtualmachineimages - virtualmachinepublishrequests - virtualmachines @@ -240,6 +241,7 @@ rules: - vmoperator.vmware.com resources: - virtualmachineclasses/status + - virtualmachineimagecaches/status - virtualmachinepublishrequests/status - virtualmachinereplicasets/status - virtualmachines/status diff --git a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller.go b/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller.go index 2613d9aca..43143d033 100644 --- a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller.go +++ b/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller.go @@ -5,338 +5,20 @@ package clustercontentlibraryitem import ( - "context" - "fmt" - "reflect" - "strings" - - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/controller" - "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/manager" - "github.com/go-logr/logr" - imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" - vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" - "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils" - "github.com/vmware-tanzu/vm-operator/pkg/conditions" - pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" - "github.com/vmware-tanzu/vm-operator/pkg/metrics" - "github.com/vmware-tanzu/vm-operator/pkg/providers" - "github.com/vmware-tanzu/vm-operator/pkg/record" - imgutil "github.com/vmware-tanzu/vm-operator/pkg/util/image" - "github.com/vmware-tanzu/vm-operator/pkg/util/ovfcache" ) -// AddToManager adds this package's controller to the provided manager. -func AddToManager(ctx *pkgctx.ControllerManagerContext, mgr manager.Manager) error { - var ( - cclItemType = &imgregv1a1.ClusterContentLibraryItem{} - cclItemTypeName = reflect.TypeOf(cclItemType).Elem().Name() - - controllerNameShort = fmt.Sprintf("%s-controller", strings.ToLower(cclItemTypeName)) - controllerNameLong = fmt.Sprintf("%s/%s/%s", ctx.Namespace, ctx.Name, controllerNameShort) - ) - - r := NewReconciler( - ctx, - mgr.GetClient(), - ctrl.Log.WithName("controllers").WithName(cclItemTypeName), - record.New(mgr.GetEventRecorderFor(controllerNameLong)), - ctx.VMProvider, - ) - - return ctrl.NewControllerManagedBy(mgr). - For(cclItemType). - // We do not set Owns(ClusterVirtualMachineImage) here as we call SetControllerReference() - // when creating such resources in the reconciling process below. - WithOptions(controller.Options{MaxConcurrentReconciles: ctx.MaxConcurrentReconciles}). - Complete(r) -} - -func NewReconciler( - ctx context.Context, - client client.Client, - logger logr.Logger, - recorder record.Recorder, - vmProvider providers.VirtualMachineProviderInterface) *Reconciler { - - return &Reconciler{ - Context: ctx, - Client: client, - Logger: logger, - Recorder: recorder, - VMProvider: vmProvider, - Metrics: metrics.NewContentLibraryItemMetrics(), - } -} - -// Reconciler reconciles an IaaS Image Registry Service's ClusterContentLibraryItem object -// by creating/updating the corresponding VM-Service's ClusterVirtualMachineImage resource. -type Reconciler struct { - client.Client - Context context.Context - Logger logr.Logger - Recorder record.Recorder - VMProvider providers.VirtualMachineProviderInterface - Metrics *metrics.ContentLibraryItemMetrics -} - // +kubebuilder:rbac:groups=imageregistry.vmware.com,resources=clustercontentlibraryitems,verbs=get;list;watch;update;patch;delete // +kubebuilder:rbac:groups=imageregistry.vmware.com,resources=clustercontentlibraryitems/status,verbs=get // +kubebuilder:rbac:groups=vmoperator.vmware.com,resources=clustervirtualmachineimages,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=vmoperator.vmware.com,resources=clustervirtualmachineimages/status,verbs=get;update;patch -func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (_ ctrl.Result, reterr error) { - ctx = pkgcfg.JoinContext(ctx, r.Context) - ctx = ovfcache.JoinContext(ctx, r.Context) - - logger := r.Logger.WithValues("cclItemName", req.Name) - logger.Info("Reconciling ClusterContentLibraryItem") - - cclItem := &imgregv1a1.ClusterContentLibraryItem{} - if err := r.Get(ctx, req.NamespacedName, cclItem); err != nil { - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - cvmiName, nameErr := utils.GetImageFieldNameFromItem(cclItem.Name) - if nameErr != nil { - logger.Error(nameErr, "Unsupported ClusterContentLibraryItem name, skip reconciling") - return ctrl.Result{}, nil - } - logger = logger.WithValues("cvmiName", cvmiName) - - cclItemCtx := &pkgctx.ClusterContentLibraryItemContext{ - Context: ctx, - Logger: logger, - CCLItem: cclItem, - ImageObjName: cvmiName, - } - - if !cclItem.DeletionTimestamp.IsZero() { - err := r.ReconcileDelete(cclItemCtx) - return ctrl.Result{}, err - } - - // Create or update the ClusterVirtualMachineImage resource accordingly. - err := r.ReconcileNormal(cclItemCtx) - return ctrl.Result{}, err -} - -// ReconcileDelete reconciles a deletion for a ClusterContentLibraryItem resource. -func (r *Reconciler) ReconcileDelete(ctx *pkgctx.ClusterContentLibraryItemContext) error { - if controllerutil.ContainsFinalizer(ctx.CCLItem, utils.CCLItemFinalizer) || - controllerutil.ContainsFinalizer(ctx.CCLItem, utils.DeprecatedCCLItemFinalizer) { - - r.Metrics.DeleteMetrics(ctx.Logger, ctx.ImageObjName, "") - - objPatch := client.MergeFromWithOptions( - ctx.CCLItem.DeepCopy(), - client.MergeFromWithOptimisticLock{}) - - controllerutil.RemoveFinalizer(ctx.CCLItem, utils.CCLItemFinalizer) - controllerutil.RemoveFinalizer(ctx.CCLItem, utils.DeprecatedCCLItemFinalizer) - - return r.Patch(ctx, ctx.CCLItem, objPatch) - } - - return nil -} - -// ReconcileNormal reconciles a ClusterContentLibraryItem resource by creating or -// updating the corresponding ClusterVirtualMachineImage resource. -func (r *Reconciler) ReconcileNormal(ctx *pkgctx.ClusterContentLibraryItemContext) error { - if !controllerutil.ContainsFinalizer(ctx.CCLItem, utils.CCLItemFinalizer) { - - // If the object has the deprecated finalizer, remove it. - if updated := controllerutil.RemoveFinalizer(ctx.CCLItem, utils.DeprecatedCCLItemFinalizer); updated { - ctx.Logger.V(5).Info("Removed deprecated finalizer", "finalizerName", utils.DeprecatedCCLItemFinalizer) - } - - objPatch := client.MergeFromWithOptions( - ctx.CCLItem.DeepCopy(), - client.MergeFromWithOptimisticLock{}) - - // The finalizer must be present before proceeding in order to ensure ReconcileDelete() will be called. - // Return immediately after here to update the object and then we'll proceed on the next reconciliation. - controllerutil.AddFinalizer(ctx.CCLItem, utils.CCLItemFinalizer) - - return r.Patch(ctx, ctx.CCLItem, objPatch) - } - - // Do not set additional fields here as they will be overwritten in CreateOrPatch below. - cvmi := &vmopv1.ClusterVirtualMachineImage{ - ObjectMeta: metav1.ObjectMeta{ - Name: ctx.ImageObjName, - }, - } - ctx.CVMI = cvmi - - var didSync bool - var syncErr error - var savedStatus *vmopv1.VirtualMachineImageStatus - - opRes, createOrPatchErr := controllerutil.CreateOrPatch(ctx, r.Client, cvmi, func() error { - defer func() { - savedStatus = cvmi.Status.DeepCopy() - }() - - if err := r.setUpCVMIFromCCLItem(ctx); err != nil { - ctx.Logger.Error(err, "Failed to set up ClusterVirtualMachineImage from ClusterContentLibraryItem") - return err - } - // Update image condition based on the security compliance of the provider item. - cclItemSecurityCompliance := ctx.CCLItem.Status.SecurityCompliance - if cclItemSecurityCompliance == nil || !*cclItemSecurityCompliance { - conditions.MarkFalse(cvmi, - vmopv1.ReadyConditionType, - vmopv1.VirtualMachineImageProviderSecurityNotCompliantReason, - "Provider item is not security compliant", - ) - // Since we want to persist a False condition if the CCL Item is - // not security compliant. - return nil - } - - // Check if the item is ready and skip the image content sync if not. - if !utils.IsItemReady(ctx.CCLItem.Status.Conditions) { - conditions.MarkFalse(cvmi, - vmopv1.ReadyConditionType, - vmopv1.VirtualMachineImageProviderNotReadyReason, - "Provider item is not in ready condition", - ) - ctx.Logger.Info("ClusterContentLibraryItem is not ready yet, skipping image content sync") - return nil - } - - syncErr = r.syncImageContent(ctx) - if syncErr == nil { - // In this block, we have confirmed that all the three sub-conditions constituting this - // Ready condition are true, hence mark it as true. - conditions.MarkTrue(cvmi, vmopv1.ReadyConditionType) - } - didSync = true - - // Do not return syncErr here as we still want to patch the updated fields we get above. - return nil - }) - - ctx.Logger = ctx.Logger.WithValues("operationResult", opRes) - - // Registry metrics based on the corresponding error captured. - defer func() { - r.Metrics.RegisterVMIResourceResolve(ctx.Logger, cvmi.Name, "", createOrPatchErr == nil) - r.Metrics.RegisterVMIContentSync(ctx.Logger, cvmi.Name, "", didSync && syncErr == nil) - }() - - if createOrPatchErr != nil { - ctx.Logger.Error(createOrPatchErr, "Failed to create or patch ClusterVirtualMachineImage resource") - return createOrPatchErr - } - - // CreateOrPatch/CreateOrUpdate doesn't patch sub-resource for creation. - if opRes == controllerutil.OperationResultCreated { - cvmi.Status = *savedStatus - if createOrPatchErr = r.Status().Update(ctx, cvmi); createOrPatchErr != nil { - ctx.Logger.Error(createOrPatchErr, "Failed to update ClusterVirtualMachineImage status") - return createOrPatchErr - } - } - - if syncErr != nil { - ctx.Logger.Error(syncErr, "Failed to sync ClusterVirtualMachineImage to the latest content version") - return syncErr - } - - ctx.Logger.Info("Successfully reconciled ClusterVirtualMachineImage", - "contentVersion", savedStatus.ProviderContentVersion) - return nil -} - -// setUpCVMIFromCCLItem sets up the ClusterVirtualMachineImage fields that -// are retrievable from the given ClusterContentLibraryItem resource. -func (r *Reconciler) setUpCVMIFromCCLItem(ctx *pkgctx.ClusterContentLibraryItemContext) error { - cclItem := ctx.CCLItem - cvmi := ctx.CVMI - - if err := controllerutil.SetControllerReference(cclItem, cvmi, r.Scheme()); err != nil { - return err - } - - // Setting the label Key Prefix on the basis of - // WCP_TKG_Multiple_CL FSS is enabled or not - - if cvmi.Labels == nil { - cvmi.Labels = make(map[string]string) - } - - labelKeyPrefix := utils.TKGServiceTypeLabelKeyPrefix - if pkgcfg.FromContext(ctx).Features.TKGMultipleCL { - labelKeyPrefix = utils.MultipleCLServiceTypeLabelKeyPrefix - - // Reconcile the labels between CCLItem and CVMI - // This should execute only for labels containing MultipleCLServiceTypeLabelKeyPrefix - for label := range cvmi.Labels { - if strings.HasPrefix(label, labelKeyPrefix) { - _, labelExists := cclItem.Labels[label] - if !labelExists { - delete(cvmi.Labels, label) - } - } - } - } - - // Only watch for service type labels from ClusterContentLibraryItem - for label := range cclItem.Labels { - if strings.HasPrefix(label, labelKeyPrefix) { - cvmi.Labels[label] = "" - } - } - - cvmi.Spec.ProviderRef = &common.LocalObjectRef{ - APIVersion: cclItem.APIVersion, - Kind: cclItem.Kind, - Name: cclItem.Name, - } - - cvmi.Status.Name = cclItem.Status.Name - cvmi.Status.ProviderItemID = string(cclItem.Spec.UUID) - cvmi.Status.Type = string(cclItem.Status.Type) - - return utils.AddContentLibraryRefToAnnotation(cvmi, ctx.CCLItem.Status.ContentLibraryRef) -} - -// syncImageContent syncs the ClusterVirtualMachineImage content from the provider. -// It skips syncing if the image content is already up-to-date. -func (r *Reconciler) syncImageContent(ctx *pkgctx.ClusterContentLibraryItemContext) error { - cclItem := ctx.CCLItem - cvmi := ctx.CVMI - latestVersion := cclItem.Status.ContentVersion - if cvmi.Status.ProviderContentVersion == latestVersion && len(cvmi.Status.Disks) != 0 { - return nil - } - - err := r.VMProvider.SyncVirtualMachineImage(ctx, cclItem, cvmi) - if err != nil { - conditions.MarkFalse(cvmi, - vmopv1.ReadyConditionType, - vmopv1.VirtualMachineImageNotSyncedReason, - "Failed to sync to the latest content version from provider") - } else { - cvmi.Status.ProviderContentVersion = latestVersion - } - - // Sync the image's type, OS information and capabilities to the resource's - // labels to make it easier for clients to search for images based on type, - // OS info or image capabilities. - imgutil.SyncStatusToLabels(cvmi, cvmi.Status) - - r.Recorder.EmitEvent(cvmi, "Update", err, false) - return err +// AddToManager adds this package's controller to the provided manager. +func AddToManager(ctx *pkgctx.ControllerManagerContext, mgr manager.Manager) error { + return utils.AddToManager(ctx, mgr, &imgregv1a1.ClusterContentLibraryItem{}) } diff --git a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_intg_test.go b/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_intg_test.go index ce07b6bb9..198de87f2 100644 --- a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_intg_test.go +++ b/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_intg_test.go @@ -184,7 +184,7 @@ func intgTestsReconcile() { image := &vmopv1.ClusterVirtualMachineImage{} Expect(ctx.Client.Get(ctx, cvmiKey, image)).To(Succeed()) - assertCVMImageFromCCLItem(image, cclItem) + assertVMImageFromCLItem(image, cclItem) By("ClusterContentLibraryItem has new content version", func() { Expect(ctx.Client.Get(ctx, cclItemKey, cclItem)).To(Succeed()) @@ -199,8 +199,30 @@ func intgTestsReconcile() { }).Should(Succeed()) cclItem.APIVersion, cclItem.Kind = gvk.ToAPIVersionAndKind() - assertCVMImageFromCCLItem(image, cclItem) + assertVMImageFromCLItem(image, cclItem) }) }) }) } + +func assertVMImageFromCLItem( + vmi *vmopv1.ClusterVirtualMachineImage, + clItem *imgregv1a1.ClusterContentLibraryItem) { + + Expect(metav1.IsControlledBy(vmi, clItem)).To(BeTrue()) + + By("Expected VMImage Spec", func() { + Expect(vmi.Spec.ProviderRef.Name).To(Equal(clItem.Name)) + Expect(vmi.Spec.ProviderRef.APIVersion).To(Equal(clItem.APIVersion)) + Expect(vmi.Spec.ProviderRef.Kind).To(Equal(clItem.Kind)) + }) + + By("Expected VMImage Status", func() { + Expect(vmi.Status.Name).To(Equal(clItem.Status.Name)) + Expect(vmi.Status.ProviderItemID).To(BeEquivalentTo(clItem.Spec.UUID)) + Expect(vmi.Status.ProviderContentVersion).To(Equal(clItem.Status.ContentVersion)) + Expect(vmi.Status.Type).To(BeEquivalentTo(clItem.Status.Type)) + + Expect(conditions.IsTrue(vmi, vmopv1.ReadyConditionType)).To(BeTrue()) + }) +} diff --git a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_suite_test.go b/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_suite_test.go index ce412a930..a64601757 100644 --- a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_suite_test.go +++ b/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_suite_test.go @@ -30,7 +30,7 @@ var suite = builder.NewTestSuiteForControllerWithContext( }) func TestClusterContentLibraryItem(t *testing.T) { - suite.Register(t, "ClusterContentLibraryItem controller suite", intgTests, unitTests) + suite.Register(t, "ClusterContentLibraryItem controller suite", intgTests, nil) } var _ = BeforeSuite(suite.BeforeSuite) diff --git a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_unit_test.go b/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_unit_test.go deleted file mode 100644 index f5147037a..000000000 --- a/controllers/contentlibrary/clustercontentlibraryitem/clustercontentlibraryitem_controller_unit_test.go +++ /dev/null @@ -1,329 +0,0 @@ -// © Broadcom. All Rights Reserved. -// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. -// SPDX-License-Identifier: Apache-2.0 - -package clustercontentlibraryitem_test - -import ( - "context" - "fmt" - - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" - - corev1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "sigs.k8s.io/controller-runtime/pkg/client" - - imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" - - vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" - "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" - "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/clustercontentlibraryitem" - "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils" - "github.com/vmware-tanzu/vm-operator/pkg/conditions" - "github.com/vmware-tanzu/vm-operator/pkg/constants/testlabels" - pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" - providerfake "github.com/vmware-tanzu/vm-operator/pkg/providers/fake" - "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" - "github.com/vmware-tanzu/vm-operator/test/builder" -) - -func unitTests() { - Describe( - "Reconcile", - Label( - testlabels.Controller, - testlabels.V1Alpha3, - ), - unitTestsReconcile, - ) -} - -func unitTestsReconcile() { - const firmwareValue = "my-firmware" - - var ( - ctx *builder.UnitTestContextForController - - reconciler *clustercontentlibraryitem.Reconciler - fakeVMProvider *providerfake.VMProvider - - cclItem *imgregv1a1.ClusterContentLibraryItem - cclItemCtx *pkgctx.ClusterContentLibraryItemContext - ) - - BeforeEach(func() { - ctx = suite.NewUnitTestContextForController() - - reconciler = clustercontentlibraryitem.NewReconciler( - ctx, - ctx.Client, - ctx.Logger, - ctx.Recorder, - ctx.VMProvider, - ) - - fakeVMProvider = ctx.VMProvider.(*providerfake.VMProvider) - fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, cvmiObj client.Object) error { - cvmi := cvmiObj.(*vmopv1.ClusterVirtualMachineImage) - // Use Firmware field to verify the provider function is called. - cvmi.Status.Firmware = firmwareValue - return nil - } - - cclItem = utils.DummyClusterContentLibraryItem(utils.ItemFieldNamePrefix + "-dummy") - // Add our finalizer so ReconcileNormal() does not return early. - cclItem.Finalizers = []string{utils.CCLItemFinalizer} - }) - - JustBeforeEach(func() { - Expect(ctx.Client.Create(ctx, cclItem)).To(Succeed()) - - imageName, err := utils.GetImageFieldNameFromItem(cclItem.Name) - Expect(err).ToNot(HaveOccurred()) - - cclItemCtx = &pkgctx.ClusterContentLibraryItemContext{ - Context: ctx, - Logger: ctx.Logger, - CCLItem: cclItem, - ImageObjName: imageName, - } - }) - - AfterEach(func() { - ctx.AfterEach() - ctx = nil - cclItem = nil - reconciler = nil - fakeVMProvider.Reset() - }) - - Context("ReconcileNormal", func() { - - When("ClusterContentLibraryItem doesn't have the VMOP finalizer", func() { - - BeforeEach(func() { - cclItem.Finalizers = nil - }) - - It("should add the finalizer", func() { - Expect(reconciler.ReconcileNormal(cclItemCtx)).To(Succeed()) - - Expect(cclItem.Finalizers).To(ContainElement(utils.CCLItemFinalizer)) - }) - }) - - When("ClusterContentLibraryItem is Not Ready", func() { - - BeforeEach(func() { - cclItem.Status.Conditions = []imgregv1a1.Condition{ - { - Type: imgregv1a1.ReadyCondition, - Status: corev1.ConditionFalse, - }, - } - }) - - It("should mark ClusterVirtualMachineImage condition as provider not ready", func() { - Expect(reconciler.ReconcileNormal(cclItemCtx)).To(Succeed()) - - cvmi := getClusterVMI(ctx, cclItemCtx.ImageObjName) - condition := conditions.Get(cvmi, vmopv1.ReadyConditionType) - Expect(condition).ToNot(BeNil()) - Expect(condition.Status).To(Equal(metav1.ConditionFalse)) - Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageProviderNotReadyReason)) - }) - }) - - When("ClusterContentLibraryItem is not security compliant", func() { - - BeforeEach(func() { - cclItem.Status.SecurityCompliance = ptr.To(false) - }) - - It("should mark ClusterVirtualMachineImage condition as provider security not compliant", func() { - Expect(reconciler.ReconcileNormal(cclItemCtx)).To(Succeed()) - - cvmi := getClusterVMI(ctx, cclItemCtx.ImageObjName) - condition := conditions.Get(cvmi, vmopv1.ReadyConditionType) - Expect(condition).ToNot(BeNil()) - Expect(condition.Status).To(Equal(metav1.ConditionFalse)) - Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageProviderSecurityNotCompliantReason)) - }) - - When("ClusterContentLibraryItem is Not Ready", func() { - - BeforeEach(func() { - cclItem.Status.Conditions = []imgregv1a1.Condition{ - { - Type: imgregv1a1.ReadyCondition, - Status: corev1.ConditionFalse, - }, - } - }) - - It("should mark the Ready condition as false", func() { - Expect(reconciler.ReconcileNormal(cclItemCtx)).To(Succeed()) - - cvmi := getClusterVMI(ctx, cclItemCtx.ImageObjName) - readyCondition := conditions.Get(cvmi, vmopv1.ReadyConditionType) - Expect(readyCondition).ToNot(BeNil()) - Expect(readyCondition.Status).To(Equal(metav1.ConditionFalse)) - Expect(readyCondition.Reason).To(Equal(vmopv1.VirtualMachineImageProviderSecurityNotCompliantReason)) - }) - - }) - }) - - When("SyncVirtualMachineImage returns an error", func() { - - BeforeEach(func() { - fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, _ client.Object) error { - return fmt.Errorf("sync-error") - } - }) - - It("should mark ClusterVirtualMachineImage Ready condition as failed", func() { - err := reconciler.ReconcileNormal(cclItemCtx) - Expect(err).To(MatchError("sync-error")) - - cvmi := getClusterVMI(ctx, cclItemCtx.ImageObjName) - condition := conditions.Get(cvmi, vmopv1.ReadyConditionType) - Expect(condition).ToNot(BeNil()) - Expect(condition.Status).To(Equal(metav1.ConditionFalse)) - Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageNotSyncedReason)) - }) - }) - - When("ClusterContentLibraryItem is ready and security complaint", func() { - - JustBeforeEach(func() { - // The DummyClusterContentLibraryItem() should meet these requirements. - var readyCond *imgregv1a1.Condition - for _, c := range cclItemCtx.CCLItem.Status.Conditions { - if c.Type == imgregv1a1.ReadyCondition { - c := c - readyCond = &c - break - } - } - Expect(readyCond).ToNot(BeNil()) - Expect(readyCond.Status).To(Equal(corev1.ConditionTrue)) - - Expect(cclItemCtx.CCLItem.Status.SecurityCompliance).To(Equal(ptr.To(true))) - }) - - When("ClusterVirtualMachineImage resource has not been created yet", func() { - - It("should create a new ClusterVirtualMachineImage syncing up with ClusterContentLibraryItem", func() { - Expect(reconciler.ReconcileNormal(cclItemCtx)).To(Succeed()) - - cvmi := getClusterVMI(ctx, cclItemCtx.ImageObjName) - assertCVMImageFromCCLItem(cvmi, cclItemCtx.CCLItem) - Expect(cvmi.Status.Firmware).To(Equal(firmwareValue)) - }) - }) - - When("ClusterVirtualMachineImage resource is exists but not up-to-date", func() { - - JustBeforeEach(func() { - cvmi := &vmopv1.ClusterVirtualMachineImage{ - ObjectMeta: metav1.ObjectMeta{ - Name: cclItemCtx.ImageObjName, - }, - Spec: vmopv1.VirtualMachineImageSpec{ - ProviderRef: &common.LocalObjectRef{ - Name: "bogus", - }, - }, - Status: vmopv1.VirtualMachineImageStatus{ - ProviderContentVersion: "stale", - Firmware: "should-be-updated", - }, - } - Expect(ctx.Client.Create(ctx, cvmi)).To(Succeed()) - }) - - It("should update the existing ClusterVirtualMachineImage with ClusterContentLibraryItem", func() { - cclItemCtx.CCLItem.Status.ContentVersion += "-updated" - Expect(reconciler.ReconcileNormal(cclItemCtx)).To(Succeed()) - - cvmi := getClusterVMI(ctx, cclItemCtx.ImageObjName) - assertCVMImageFromCCLItem(cvmi, cclItemCtx.CCLItem) - Expect(cvmi.Status.Firmware).To(Equal(firmwareValue)) - }) - }) - - When("ClusterVirtualMachineImage resource is created and already up-to-date", func() { - - JustBeforeEach(func() { - cvmi := &vmopv1.ClusterVirtualMachineImage{ - ObjectMeta: metav1.ObjectMeta{ - Name: cclItemCtx.ImageObjName, - }, - Status: vmopv1.VirtualMachineImageStatus{ - ProviderContentVersion: cclItemCtx.CCLItem.Status.ContentVersion, - Disks: make([]vmopv1.VirtualMachineImageDiskInfo, 1), - Firmware: "should-not-be-updated", - }, - } - Expect(ctx.Client.Create(ctx, cvmi)).To(Succeed()) - }) - - It("should skip updating the ClusterVirtualMachineImage with library item", func() { - fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, _ client.Object) error { - // Should not be called since the content versions match. - return fmt.Errorf("sync-error") - } - - Expect(reconciler.ReconcileNormal(cclItemCtx)).To(Succeed()) - - cvmi := getClusterVMI(ctx, cclItemCtx.ImageObjName) - Expect(cvmi.Status.Firmware).To(Equal("should-not-be-updated")) - }) - }) - }) - }) - - Context("ReconcileDelete", func() { - - It("should remove the finalizer from ClusterContentLibraryItem resource", func() { - Expect(cclItem.Finalizers).To(ContainElement(utils.CCLItemFinalizer)) - - Expect(reconciler.ReconcileDelete(cclItemCtx)).To(Succeed()) - Expect(cclItem.Finalizers).ToNot(ContainElement(utils.CCLItemFinalizer)) - }) - }) -} - -func getClusterVMI(ctx *builder.UnitTestContextForController, name string) *vmopv1.ClusterVirtualMachineImage { - cvmi := &vmopv1.ClusterVirtualMachineImage{} - Expect(ctx.Client.Get(ctx, client.ObjectKey{Name: name}, cvmi)).To(Succeed()) - return cvmi -} - -func assertCVMImageFromCCLItem( - cvmi *vmopv1.ClusterVirtualMachineImage, - cclItem *imgregv1a1.ClusterContentLibraryItem) { - - Expect(metav1.IsControlledBy(cvmi, cclItem)).To(BeTrue()) - for k := range utils.FilterServicesTypeLabels(cclItem.Labels) { - Expect(cvmi.Labels).To(HaveKey(k)) - } - - By("Expected ClusterVMImage Spec", func() { - Expect(cvmi.Spec.ProviderRef.Name).To(Equal(cclItem.Name)) - Expect(cvmi.Spec.ProviderRef.APIVersion).To(Equal(cclItem.APIVersion)) - Expect(cvmi.Spec.ProviderRef.Kind).To(Equal(cclItem.Kind)) - }) - - By("Expected ClusterVMImage Status", func() { - Expect(cvmi.Status.Name).To(Equal(cclItem.Status.Name)) - Expect(cvmi.Status.ProviderItemID).To(BeEquivalentTo(cclItem.Spec.UUID)) - Expect(cvmi.Status.ProviderContentVersion).To(Equal(cclItem.Status.ContentVersion)) - Expect(cvmi.Status.Type).To(BeEquivalentTo(cclItem.Status.Type)) - - Expect(conditions.IsTrue(cvmi, vmopv1.ReadyConditionType)).To(BeTrue()) - }) -} diff --git a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller.go b/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller.go index 78dbdae4b..0b0398e5b 100644 --- a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller.go +++ b/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller.go @@ -5,306 +5,20 @@ package contentlibraryitem import ( - "context" - "fmt" - "reflect" - "strings" - - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/controller" - "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/manager" - "github.com/go-logr/logr" - imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" - vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" - "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils" - "github.com/vmware-tanzu/vm-operator/pkg/conditions" - pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" - "github.com/vmware-tanzu/vm-operator/pkg/metrics" - "github.com/vmware-tanzu/vm-operator/pkg/providers" - "github.com/vmware-tanzu/vm-operator/pkg/record" - imgutil "github.com/vmware-tanzu/vm-operator/pkg/util/image" - "github.com/vmware-tanzu/vm-operator/pkg/util/ovfcache" ) -// AddToManager adds this package's controller to the provided manager. -func AddToManager(ctx *pkgctx.ControllerManagerContext, mgr manager.Manager) error { - var ( - clItemType = &imgregv1a1.ContentLibraryItem{} - clItemTypeName = reflect.TypeOf(clItemType).Elem().Name() - - controllerNameShort = fmt.Sprintf("%s-controller", strings.ToLower(clItemTypeName)) - controllerNameLong = fmt.Sprintf("%s/%s/%s", ctx.Namespace, ctx.Name, controllerNameShort) - ) - - r := NewReconciler( - ctx, - mgr.GetClient(), - ctrl.Log.WithName("controllers").WithName(clItemTypeName), - record.New(mgr.GetEventRecorderFor(controllerNameLong)), - ctx.VMProvider, - ) - - return ctrl.NewControllerManagedBy(mgr). - For(clItemType). - // We do not set Owns(VirtualMachineImage) here as we call SetControllerReference() - // when creating such resources in the reconciling process below. - WithOptions(controller.Options{MaxConcurrentReconciles: ctx.MaxConcurrentReconciles}). - Complete(r) -} - -func NewReconciler( - ctx context.Context, - client client.Client, - logger logr.Logger, - recorder record.Recorder, - vmProvider providers.VirtualMachineProviderInterface) *Reconciler { - - return &Reconciler{ - Context: ctx, - Client: client, - Logger: logger, - Recorder: recorder, - VMProvider: vmProvider, - Metrics: metrics.NewContentLibraryItemMetrics(), - } -} - -// Reconciler reconciles an IaaS Image Registry Service's ContentLibraryItem object -// by creating/updating the corresponding VM-Service's VirtualMachineImage resource. -type Reconciler struct { - client.Client - Context context.Context - Logger logr.Logger - Recorder record.Recorder - VMProvider providers.VirtualMachineProviderInterface - Metrics *metrics.ContentLibraryItemMetrics -} - // +kubebuilder:rbac:groups=imageregistry.vmware.com,resources=contentlibraryitems,verbs=get;list;watch;update;patch;delete // +kubebuilder:rbac:groups=imageregistry.vmware.com,resources=contentlibraryitems/status,verbs=get // +kubebuilder:rbac:groups=vmoperator.vmware.com,resources=virtualmachineimages,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=vmoperator.vmware.com,resources=virtualmachineimages/status,verbs=get;update;patch -func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (_ ctrl.Result, reterr error) { - ctx = pkgcfg.JoinContext(ctx, r.Context) - ctx = ovfcache.JoinContext(ctx, r.Context) - - logger := r.Logger.WithValues("clItemName", req.Name, "namespace", req.Namespace) - logger.Info("Reconciling ContentLibraryItem") - - clItem := &imgregv1a1.ContentLibraryItem{} - if err := r.Get(ctx, req.NamespacedName, clItem); err != nil { - return ctrl.Result{}, client.IgnoreNotFound(err) - } - - vmiName, nameErr := utils.GetImageFieldNameFromItem(clItem.Name) - if nameErr != nil { - logger.Error(nameErr, "Unsupported ContentLibraryItem name, skip reconciling") - return ctrl.Result{}, nil - } - logger = logger.WithValues("vmiName", vmiName) - - clItemCtx := &pkgctx.ContentLibraryItemContext{ - Context: ctx, - Logger: logger, - CLItem: clItem, - ImageObjName: vmiName, - } - - if !clItem.DeletionTimestamp.IsZero() { - err := r.ReconcileDelete(clItemCtx) - return ctrl.Result{}, err - } - - // Create or update the VirtualMachineImage resource accordingly. - err := r.ReconcileNormal(clItemCtx) - return ctrl.Result{}, err -} - -// ReconcileDelete reconciles a deletion for a ContentLibraryItem resource. -func (r *Reconciler) ReconcileDelete(ctx *pkgctx.ContentLibraryItemContext) error { - if controllerutil.ContainsFinalizer(ctx.CLItem, utils.CLItemFinalizer) || - controllerutil.ContainsFinalizer(ctx.CLItem, utils.DeprecatedCLItemFinalizer) { - - r.Metrics.DeleteMetrics(ctx.Logger, ctx.ImageObjName, ctx.CLItem.Namespace) - - objPatch := client.MergeFromWithOptions( - ctx.CLItem.DeepCopy(), - client.MergeFromWithOptimisticLock{}) - - controllerutil.RemoveFinalizer(ctx.CLItem, utils.CLItemFinalizer) - controllerutil.RemoveFinalizer(ctx.CLItem, utils.DeprecatedCLItemFinalizer) - - return r.Patch(ctx, ctx.CLItem, objPatch) - } - - return nil -} - -// ReconcileNormal reconciles a ContentLibraryItem resource by creating or -// updating the corresponding VirtualMachineImage resource. -func (r *Reconciler) ReconcileNormal(ctx *pkgctx.ContentLibraryItemContext) error { - if !controllerutil.ContainsFinalizer(ctx.CLItem, utils.CLItemFinalizer) { - - // If the object has the deprecated finalizer, remove it. - if updated := controllerutil.RemoveFinalizer(ctx.CLItem, utils.DeprecatedCLItemFinalizer); updated { - ctx.Logger.V(5).Info("Removed deprecated finalizer", "finalizerName", utils.DeprecatedCLItemFinalizer) - } - - objPatch := client.MergeFromWithOptions( - ctx.CLItem.DeepCopy(), - client.MergeFromWithOptimisticLock{}) - - // The finalizer must be present before proceeding in order to ensure ReconcileDelete() will be called. - // Return immediately after here to update the object and then we'll proceed on the next reconciliation. - controllerutil.AddFinalizer(ctx.CLItem, utils.CLItemFinalizer) - - return r.Patch(ctx, ctx.CLItem, objPatch) - } - - // Do not set additional fields here as they will be overwritten in CreateOrPatch below. - vmi := &vmopv1.VirtualMachineImage{ - ObjectMeta: metav1.ObjectMeta{ - Name: ctx.ImageObjName, - Namespace: ctx.CLItem.Namespace, - }, - } - ctx.VMI = vmi - - var didSync bool - var syncErr error - var savedStatus *vmopv1.VirtualMachineImageStatus - - opRes, createOrPatchErr := controllerutil.CreateOrPatch(ctx, r.Client, vmi, func() error { - defer func() { - savedStatus = vmi.Status.DeepCopy() - }() - - if err := r.setUpVMIFromCLItem(ctx); err != nil { - ctx.Logger.Error(err, "Failed to set up VirtualMachineImage from ContentLibraryItem") - return err - } - // Update image condition based on the security compliance of the provider item. - clItemSecurityCompliance := ctx.CLItem.Status.SecurityCompliance - if clItemSecurityCompliance == nil || !*clItemSecurityCompliance { - conditions.MarkFalse(vmi, - vmopv1.ReadyConditionType, - vmopv1.VirtualMachineImageProviderSecurityNotCompliantReason, - "Provider item is not security compliant", - ) - // Since we want to persist a False condition if the CCL Item is - // not security compliant. - return nil - } - - // Check if the item is ready and skip the image content sync if not. - if !utils.IsItemReady(ctx.CLItem.Status.Conditions) { - conditions.MarkFalse(vmi, - vmopv1.ReadyConditionType, - vmopv1.VirtualMachineImageProviderNotReadyReason, - "Provider item is not in ready condition", - ) - ctx.Logger.Info("ContentLibraryItem is not ready yet, skipping image content sync") - return nil - } - - syncErr = r.syncImageContent(ctx) - if syncErr == nil { - // In this block, we have confirmed that all the three sub-conditions constituting this - // Ready condition are true, hence mark it as true. - conditions.MarkTrue(vmi, vmopv1.ReadyConditionType) - } - didSync = true - - // Do not return syncErr here as we still want to patch the updated fields we get above. - return nil - }) - - ctx.Logger = ctx.Logger.WithValues("operationResult", opRes) - - // Registry metrics based on the corresponding error captured. - defer func() { - r.Metrics.RegisterVMIResourceResolve(ctx.Logger, vmi.Name, vmi.Namespace, createOrPatchErr == nil) - r.Metrics.RegisterVMIContentSync(ctx.Logger, vmi.Name, vmi.Namespace, didSync && syncErr == nil) - }() - - if createOrPatchErr != nil { - ctx.Logger.Error(createOrPatchErr, "failed to create or patch VirtualMachineImage resource") - return createOrPatchErr - } - - // CreateOrPatch/CreateOrUpdate doesn't patch sub-resource for creation. - if opRes == controllerutil.OperationResultCreated { - vmi.Status = *savedStatus - if createOrPatchErr = r.Status().Update(ctx, vmi); createOrPatchErr != nil { - ctx.Logger.Error(createOrPatchErr, "Failed to update VirtualMachineImage status") - return createOrPatchErr - } - } - - if syncErr != nil { - ctx.Logger.Error(syncErr, "Failed to sync VirtualMachineImage to the latest content version") - return syncErr - } - - ctx.Logger.Info("Successfully reconciled VirtualMachineImage", "contentVersion", savedStatus.ProviderContentVersion) - return nil -} - -// setUpVMIFromCLItem sets up the VirtualMachineImage fields that -// are retrievable from the given ContentLibraryItem resource. -func (r *Reconciler) setUpVMIFromCLItem(ctx *pkgctx.ContentLibraryItemContext) error { - clItem := ctx.CLItem - vmi := ctx.VMI - if err := controllerutil.SetControllerReference(clItem, vmi, r.Scheme()); err != nil { - return err - } - - vmi.Spec.ProviderRef = &common.LocalObjectRef{ - APIVersion: clItem.APIVersion, - Kind: clItem.Kind, - Name: clItem.Name, - } - vmi.Status.Name = clItem.Status.Name - vmi.Status.ProviderItemID = string(clItem.Spec.UUID) - vmi.Status.Type = string(clItem.Status.Type) - - return utils.AddContentLibraryRefToAnnotation(vmi, ctx.CLItem.Status.ContentLibraryRef) -} - -// syncImageContent syncs the VirtualMachineImage content from the provider. -// It skips syncing if the image content is already up-to-date. -func (r *Reconciler) syncImageContent(ctx *pkgctx.ContentLibraryItemContext) error { - clItem := ctx.CLItem - vmi := ctx.VMI - latestVersion := clItem.Status.ContentVersion - if vmi.Status.ProviderContentVersion == latestVersion && len(vmi.Status.Disks) != 0 { - return nil - } - - err := r.VMProvider.SyncVirtualMachineImage(ctx, clItem, vmi) - if err != nil { - conditions.MarkFalse(vmi, - vmopv1.ReadyConditionType, - vmopv1.VirtualMachineImageNotSyncedReason, - "Failed to sync to the latest content version from provider") - } else { - vmi.Status.ProviderContentVersion = latestVersion - } - - // Sync the image's type, OS information and capabilities to the resource's - // labels to make it easier for clients to search for images based on type, - // OS info or image capabilities. - imgutil.SyncStatusToLabels(vmi, vmi.Status) - - r.Recorder.EmitEvent(vmi, "Update", err, false) - return err +// AddToManager adds this package's controller to the provided manager. +func AddToManager(ctx *pkgctx.ControllerManagerContext, mgr manager.Manager) error { + return utils.AddToManager(ctx, mgr, &imgregv1a1.ContentLibraryItem{}) } diff --git a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_intg_test.go b/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_intg_test.go index 274f8800b..d1242fa19 100644 --- a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_intg_test.go +++ b/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_intg_test.go @@ -166,3 +166,25 @@ func intgTestsReconcile() { }) }) } + +func assertVMImageFromCLItem( + vmi *vmopv1.VirtualMachineImage, + clItem *imgregv1a1.ContentLibraryItem) { + + Expect(metav1.IsControlledBy(vmi, clItem)).To(BeTrue()) + + By("Expected VMImage Spec", func() { + Expect(vmi.Spec.ProviderRef.Name).To(Equal(clItem.Name)) + Expect(vmi.Spec.ProviderRef.APIVersion).To(Equal(clItem.APIVersion)) + Expect(vmi.Spec.ProviderRef.Kind).To(Equal(clItem.Kind)) + }) + + By("Expected VMImage Status", func() { + Expect(vmi.Status.Name).To(Equal(clItem.Status.Name)) + Expect(vmi.Status.ProviderItemID).To(BeEquivalentTo(clItem.Spec.UUID)) + Expect(vmi.Status.ProviderContentVersion).To(Equal(clItem.Status.ContentVersion)) + Expect(vmi.Status.Type).To(BeEquivalentTo(clItem.Status.Type)) + + Expect(conditions.IsTrue(vmi, vmopv1.ReadyConditionType)).To(BeTrue()) + }) +} diff --git a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_suite_test.go b/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_suite_test.go index a1eb76456..f84846bd0 100644 --- a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_suite_test.go +++ b/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_suite_test.go @@ -29,7 +29,7 @@ var suite = builder.NewTestSuiteForControllerWithContext( }) func TestContentLibraryItem(t *testing.T) { - suite.Register(t, "ContentLibraryItem controller suite", intgTests, unitTests) + suite.Register(t, "ContentLibraryItem controller suite", intgTests, nil) } var _ = BeforeSuite(suite.BeforeSuite) diff --git a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_unit_test.go b/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_unit_test.go deleted file mode 100644 index 77d6b40ea..000000000 --- a/controllers/contentlibrary/contentlibraryitem/contentlibraryitem_controller_unit_test.go +++ /dev/null @@ -1,307 +0,0 @@ -// © Broadcom. All Rights Reserved. -// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. -// SPDX-License-Identifier: Apache-2.0 - -package contentlibraryitem_test - -import ( - "context" - "fmt" - - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" - - corev1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "sigs.k8s.io/controller-runtime/pkg/client" - - imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" - - vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" - "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" - "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/contentlibraryitem" - "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils" - "github.com/vmware-tanzu/vm-operator/pkg/conditions" - "github.com/vmware-tanzu/vm-operator/pkg/constants/testlabels" - pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" - providerfake "github.com/vmware-tanzu/vm-operator/pkg/providers/fake" - "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" - "github.com/vmware-tanzu/vm-operator/test/builder" -) - -func unitTests() { - Describe( - "Reconcile", - Label( - testlabels.Controller, - testlabels.V1Alpha3, - ), - unitTestsReconcile, - ) -} - -func unitTestsReconcile() { - const firmwareValue = "my-firmware" - - var ( - ctx *builder.UnitTestContextForController - - reconciler *contentlibraryitem.Reconciler - fakeVMProvider *providerfake.VMProvider - - clItem *imgregv1a1.ContentLibraryItem - clItemCtx *pkgctx.ContentLibraryItemContext - ) - - BeforeEach(func() { - ctx = suite.NewUnitTestContextForController() - - reconciler = contentlibraryitem.NewReconciler( - ctx, - ctx.Client, - ctx.Logger, - ctx.Recorder, - ctx.VMProvider, - ) - - fakeVMProvider = ctx.VMProvider.(*providerfake.VMProvider) - - fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, vmiObj client.Object) error { - vmi := vmiObj.(*vmopv1.VirtualMachineImage) - - // Use Firmware field to verify the provider function is called. - vmi.Status.Firmware = firmwareValue - return nil - } - - clItem = utils.DummyContentLibraryItem(utils.ItemFieldNamePrefix+"-dummy", "dummy-ns") - clItem.Finalizers = []string{utils.CLItemFinalizer} - }) - - JustBeforeEach(func() { - Expect(ctx.Client.Create(ctx, clItem)).To(Succeed()) - - imageName, err := utils.GetImageFieldNameFromItem(clItem.Name) - Expect(err).ToNot(HaveOccurred()) - - clItemCtx = &pkgctx.ContentLibraryItemContext{ - Context: ctx, - Logger: ctx.Logger, - CLItem: clItem, - ImageObjName: imageName, - } - }) - - AfterEach(func() { - ctx.AfterEach() - ctx = nil - clItem = nil - reconciler = nil - fakeVMProvider.Reset() - }) - - Context("ReconcileNormal", func() { - - When("ContentLibraryItem doesn't have the VMOP finalizer", func() { - BeforeEach(func() { - clItem.Finalizers = nil - }) - - It("should add the finalizer", func() { - Expect(reconciler.ReconcileNormal(clItemCtx)).To(Succeed()) - - Expect(clItem.Finalizers).To(ContainElement(utils.CLItemFinalizer)) - }) - }) - - When("ContentLibraryItem is Not Ready", func() { - BeforeEach(func() { - clItem.Status.Conditions = []imgregv1a1.Condition{ - { - Type: imgregv1a1.ReadyCondition, - Status: corev1.ConditionFalse, - }, - } - }) - - It("should mark VirtualMachineImage condition as provider not ready", func() { - Expect(reconciler.ReconcileNormal(clItemCtx)).To(Succeed()) - - vmi := getVMI(ctx, clItemCtx) - condition := conditions.Get(vmi, vmopv1.ReadyConditionType) - Expect(condition).ToNot(BeNil()) - Expect(condition.Status).To(Equal(metav1.ConditionFalse)) - Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageProviderNotReadyReason)) - }) - }) - - When("ContentLibraryItem is not security compliant", func() { - - BeforeEach(func() { - clItem.Status.SecurityCompliance = ptr.To(false) - }) - - It("should mark VirtualMachineImage condition as provider security not compliant", func() { - Expect(reconciler.ReconcileNormal(clItemCtx)).To(Succeed()) - - vmi := getVMI(ctx, clItemCtx) - condition := conditions.Get(vmi, vmopv1.ReadyConditionType) - Expect(condition).ToNot(BeNil()) - Expect(condition.Status).To(Equal(metav1.ConditionFalse)) - Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageProviderSecurityNotCompliantReason)) - }) - }) - - When("SyncVirtualMachineImage returns an error", func() { - - BeforeEach(func() { - fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, _ client.Object) error { - return fmt.Errorf("sync-error") - } - }) - - It("should mark VirtualMachineImage condition synced failed", func() { - err := reconciler.ReconcileNormal(clItemCtx) - Expect(err).To(MatchError("sync-error")) - - vmi := getVMI(ctx, clItemCtx) - condition := conditions.Get(vmi, vmopv1.ReadyConditionType) - Expect(condition).ToNot(BeNil()) - Expect(condition.Status).To(Equal(metav1.ConditionFalse)) - Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageNotSyncedReason)) - }) - }) - - When("ContentLibraryItem is ready and security complaint", func() { - - JustBeforeEach(func() { - // The DummyContentLibraryItem() should meet these requirements. - var readyCond *imgregv1a1.Condition - for _, c := range clItemCtx.CLItem.Status.Conditions { - if c.Type == imgregv1a1.ReadyCondition { - c := c - readyCond = &c - break - } - } - Expect(readyCond).ToNot(BeNil()) - Expect(readyCond.Status).To(Equal(corev1.ConditionTrue)) - - Expect(clItemCtx.CLItem.Status.SecurityCompliance).To(Equal(ptr.To(true))) - }) - - When("VirtualMachineImage resource has not been created yet", func() { - - It("should create a new VirtualMachineImage syncing up with ContentLibraryItem", func() { - Expect(reconciler.ReconcileNormal(clItemCtx)).To(Succeed()) - - vmi := getVMI(ctx, clItemCtx) - assertVMImageFromCLItem(vmi, clItemCtx.CLItem) - Expect(vmi.Status.Firmware).To(Equal(firmwareValue)) - }) - }) - - When("VirtualMachineImage resource is exists but not up-to-date", func() { - - JustBeforeEach(func() { - vmi := &vmopv1.VirtualMachineImage{ - ObjectMeta: metav1.ObjectMeta{ - Name: clItemCtx.ImageObjName, - Namespace: clItem.Namespace, - }, - Spec: vmopv1.VirtualMachineImageSpec{ - ProviderRef: &common.LocalObjectRef{ - Name: "bogus", - }, - }, - Status: vmopv1.VirtualMachineImageStatus{ - ProviderContentVersion: "stale", - Firmware: "should-be-updated", - }, - } - Expect(ctx.Client.Create(ctx, vmi)).To(Succeed()) - }) - - It("should update the existing VirtualMachineImage with ContentLibraryItem", func() { - clItemCtx.CLItem.Status.ContentVersion += "-updated" - Expect(reconciler.ReconcileNormal(clItemCtx)).To(Succeed()) - - vmi := getVMI(ctx, clItemCtx) - assertVMImageFromCLItem(vmi, clItemCtx.CLItem) - Expect(vmi.Status.Firmware).To(Equal(firmwareValue)) - }) - }) - - When("VirtualMachineImage resource is created and already up-to-date", func() { - - JustBeforeEach(func() { - vmi := &vmopv1.VirtualMachineImage{ - ObjectMeta: metav1.ObjectMeta{ - Name: clItemCtx.ImageObjName, - Namespace: clItemCtx.CLItem.Namespace, - }, - Status: vmopv1.VirtualMachineImageStatus{ - ProviderContentVersion: clItemCtx.CLItem.Status.ContentVersion, - Firmware: "should-not-be-updated", - Disks: make([]vmopv1.VirtualMachineImageDiskInfo, 1), - }, - } - Expect(ctx.Client.Create(ctx, vmi)).To(Succeed()) - }) - - It("should skip updating the VirtualMachineImage with library item", func() { - fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, _ client.Object) error { - // Should not be called since the content versions match. - return fmt.Errorf("sync-error") - } - - Expect(reconciler.ReconcileNormal(clItemCtx)).To(Succeed()) - - vmi := getVMI(ctx, clItemCtx) - Expect(vmi.Status.Firmware).To(Equal("should-not-be-updated")) - }) - }) - }) - }) - - Context("ReconcileDelete", func() { - - It("should remove the finalizer from ContentLibraryItem resource", func() { - Expect(clItem.Finalizers).To(ContainElement(utils.CLItemFinalizer)) - - Expect(reconciler.ReconcileDelete(clItemCtx)).To(Succeed()) - Expect(clItem.Finalizers).ToNot(ContainElement(utils.CLItemFinalizer)) - }) - }) -} - -func getVMI( - ctx *builder.UnitTestContextForController, - clItemCtx *pkgctx.ContentLibraryItemContext) *vmopv1.VirtualMachineImage { - - vmi := &vmopv1.VirtualMachineImage{} - Expect(ctx.Client.Get(ctx, client.ObjectKey{Name: clItemCtx.ImageObjName, Namespace: clItemCtx.CLItem.Namespace}, vmi)).To(Succeed()) - return vmi -} - -func assertVMImageFromCLItem( - vmi *vmopv1.VirtualMachineImage, - clItem *imgregv1a1.ContentLibraryItem) { - - Expect(metav1.IsControlledBy(vmi, clItem)).To(BeTrue()) - - By("Expected VMImage Spec", func() { - Expect(vmi.Spec.ProviderRef.Name).To(Equal(clItem.Name)) - Expect(vmi.Spec.ProviderRef.APIVersion).To(Equal(clItem.APIVersion)) - Expect(vmi.Spec.ProviderRef.Kind).To(Equal(clItem.Kind)) - }) - - By("Expected VMImage Status", func() { - Expect(vmi.Status.Name).To(Equal(clItem.Status.Name)) - Expect(vmi.Status.ProviderItemID).To(BeEquivalentTo(clItem.Spec.UUID)) - Expect(vmi.Status.ProviderContentVersion).To(Equal(clItem.Status.ContentVersion)) - Expect(vmi.Status.Type).To(BeEquivalentTo(clItem.Status.Type)) - - Expect(conditions.IsTrue(vmi, vmopv1.ReadyConditionType)).To(BeTrue()) - }) -} diff --git a/controllers/contentlibrary/utils/controller_builder.go b/controllers/contentlibrary/utils/controller_builder.go new file mode 100644 index 000000000..c9c13a9d2 --- /dev/null +++ b/controllers/contentlibrary/utils/controller_builder.go @@ -0,0 +1,468 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package utils + +import ( + "context" + "fmt" + "reflect" + "strings" + + "github.com/go-logr/logr" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/manager" + + imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" + + vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" + "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" + pkgcnd "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" + pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" + "github.com/vmware-tanzu/vm-operator/pkg/metrics" + "github.com/vmware-tanzu/vm-operator/pkg/patch" + "github.com/vmware-tanzu/vm-operator/pkg/providers" + "github.com/vmware-tanzu/vm-operator/pkg/record" + imgutil "github.com/vmware-tanzu/vm-operator/pkg/util/image" + vmopv1util "github.com/vmware-tanzu/vm-operator/pkg/util/vmopv1" +) + +// SkipNameValidation is used for testing to allow multiple controllers with the +// same name since Controller-Runtime has a global singleton registry to +// prevent controllers with the same name, even if attached to different +// managers. +var SkipNameValidation *bool + +func AddToManager( + ctx *pkgctx.ControllerManagerContext, + mgr manager.Manager, + obj client.Object) error { + + var ( + controlledItemType = obj + controlledItemTypeName = reflect.TypeOf(controlledItemType).Elem().Name() + + controllerNameShort = fmt.Sprintf("%s-controller", strings.ToLower(controlledItemTypeName)) + controllerNameLong = fmt.Sprintf("%s/%s/%s", ctx.Namespace, ctx.Name, controllerNameShort) + ) + + r := NewReconciler( + ctx, + mgr.GetClient(), + ctrl.Log.WithName("controllers").WithName(controlledItemTypeName), + record.New(mgr.GetEventRecorderFor(controllerNameLong)), + ctx.VMProvider, + controlledItemTypeName, + ) + + builder := ctrl.NewControllerManagedBy(mgr). + For(controlledItemType). + WithOptions(controller.Options{ + MaxConcurrentReconciles: ctx.MaxConcurrentReconciles, + SkipNameValidation: SkipNameValidation, + }) + + if pkgcfg.FromContext(ctx).Features.FastDeploy { + builder = builder.Watches( + &vmopv1.VirtualMachineImageCache{}, + handler.EnqueueRequestsFromMapFunc( + vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + r.Logger.WithName("VirtualMachineImageCacheToItemMapper"), + r.Client, + imgregv1a1.GroupVersion, + controlledItemTypeName), + )) + } + + return builder.Complete(r) +} + +func NewReconciler( + ctx context.Context, + client client.Client, + logger logr.Logger, + recorder record.Recorder, + vmProvider providers.VirtualMachineProviderInterface, + kind string) *Reconciler { + + return &Reconciler{ + Context: ctx, + Client: client, + Logger: logger, + Recorder: recorder, + VMProvider: vmProvider, + Metrics: metrics.NewContentLibraryItemMetrics(), + Kind: kind, + } +} + +// Reconciler reconciles an IaaS Image Registry Service's ContentLibraryItem object +// by creating/updating the corresponding VM-Service's VirtualMachineImage resource. +type Reconciler struct { + client.Client + Context context.Context + Logger logr.Logger + Recorder record.Recorder + VMProvider providers.VirtualMachineProviderInterface + Metrics *metrics.ContentLibraryItemMetrics + Kind string +} + +func (r *Reconciler) Reconcile( + ctx context.Context, + req ctrl.Request) (_ ctrl.Result, reterr error) { + + ctx = pkgcfg.JoinContext(ctx, r.Context) + + logger := ctrl.Log.WithName(r.Kind).WithValues("name", req.String()) + + var ( + obj client.Object + spec *imgregv1a1.ContentLibraryItemSpec + status *imgregv1a1.ContentLibraryItemStatus + ) + + if req.Namespace != "" { + var o imgregv1a1.ContentLibraryItem + if err := r.Get(ctx, req.NamespacedName, &o); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + obj, spec, status = &o, &o.Spec, &o.Status + } else { + var o imgregv1a1.ClusterContentLibraryItem + if err := r.Get(ctx, req.NamespacedName, &o); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + obj, spec, status = &o, &o.Spec, &o.Status + } + + vmiName, nameErr := GetImageFieldNameFromItem(req.Name) + if nameErr != nil { + logger.Error(nameErr, "Unsupported library item name, skip reconciling") + return ctrl.Result{}, nil + } + logger = logger.WithValues("vmiName", vmiName) + + patchHelper, err := patch.NewHelper(obj, r.Client) + if err != nil { + return ctrl.Result{}, fmt.Errorf( + "failed to init patch helper for %s: %w", req.NamespacedName, err) + } + defer func() { + if err := patchHelper.Patch(ctx, obj); err != nil { + if reterr == nil { + reterr = err + } + logger.Error(err, "patch failed") + } + }() + + if !obj.GetDeletionTimestamp().IsZero() { + return ctrl.Result{}, r.ReconcileDelete(ctx, logger, obj, vmiName) + } + + // Create or update the VirtualMachineImage resource accordingly. + return ctrl.Result{}, r.ReconcileNormal( + ctx, + logger, + obj, + spec, + status, + vmiName) +} + +// ReconcileDelete reconciles a deletion for a ContentLibraryItem resource. +func (r *Reconciler) ReconcileDelete( + ctx context.Context, + logger logr.Logger, + obj client.Object, + vmiName string) error { + + if !controllerutil.ContainsFinalizer(obj, CLItemFinalizer) && + !controllerutil.ContainsFinalizer(obj, DeprecatedCLItemFinalizer) { + + return nil + } + + r.Metrics.DeleteMetrics(logger, vmiName, obj.GetNamespace()) + controllerutil.RemoveFinalizer(obj, CLItemFinalizer) + controllerutil.RemoveFinalizer(obj, DeprecatedCLItemFinalizer) + + return nil +} + +// ReconcileNormal reconciles a ContentLibraryItem resource by creating or +// updating the corresponding VirtualMachineImage resource. +func (r *Reconciler) ReconcileNormal( + ctx context.Context, + logger logr.Logger, + cliObj client.Object, + cliSpec *imgregv1a1.ContentLibraryItemSpec, + cliStatus *imgregv1a1.ContentLibraryItemStatus, + vmiName string) error { + + var ( + finalizer string + depFinalizer string + ) + + if cliObj.GetNamespace() != "" { + finalizer = CLItemFinalizer + depFinalizer = DeprecatedCLItemFinalizer + } else { + finalizer = CCLItemFinalizer + depFinalizer = DeprecatedCCLItemFinalizer + } + + if !controllerutil.ContainsFinalizer(cliObj, finalizer) { + + // If the object has the deprecated finalizer, remove it. + if controllerutil.RemoveFinalizer( + cliObj, + depFinalizer) { + + logger.V(5).Info( + "Removed deprecated finalizer", + "finalizerName", depFinalizer) + } + + // The finalizer must be present before proceeding in order to ensure + // ReconcileDelete() will be called. Return immediately after here to + // update the object and then we'll proceed on the next reconciliation. + controllerutil.AddFinalizer(cliObj, finalizer) + + return nil + } + + var ( + vmiObj client.Object + vmiSpec *vmopv1.VirtualMachineImageSpec + vmiStatus *vmopv1.VirtualMachineImageStatus + vmiKind string + ) + + if ns := cliObj.GetNamespace(); ns != "" { + o := vmopv1.VirtualMachineImage{ + ObjectMeta: metav1.ObjectMeta{ + Name: vmiName, + Namespace: ns, + }, + } + vmiKind = "VirtualMachineImage" + vmiObj, vmiSpec, vmiStatus = &o, &o.Spec, &o.Status + } else { + o := vmopv1.ClusterVirtualMachineImage{ + ObjectMeta: metav1.ObjectMeta{ + Name: vmiName, + }, + } + vmiKind = "ClusterVirtualMachineImage" + vmiObj, vmiSpec, vmiStatus = &o, &o.Spec, &o.Status + } + + logger = logger.WithValues("vmiKind", vmiKind) + + var ( + didSync bool + syncErr error + savedStatus *vmopv1.VirtualMachineImageStatus + ) + + opRes, copErr := controllerutil.CreateOrPatch( + ctx, + r.Client, + vmiObj, + func() error { + + defer func() { + savedStatus = vmiStatus.DeepCopy() + }() + + // Clear the status so it can be rebuilt. + *vmiStatus = vmopv1.VirtualMachineImageStatus{} + + if err := r.setUpVMIFromCLItem( + cliObj, + *cliSpec, + *cliStatus, + vmiObj, + vmiSpec, + vmiStatus); err != nil { + + logger.Error(err, "Failed to setup image from library item") + return err + } + + // Update image condition based on the security compliance of the + // underlying library item. + clItemSecurityCompliance := cliStatus.SecurityCompliance + if clItemSecurityCompliance == nil || !*clItemSecurityCompliance { + pkgcnd.MarkFalse( + vmiStatus, + vmopv1.ReadyConditionType, + vmopv1.VirtualMachineImageProviderSecurityNotCompliantReason, + "Provider item is not security compliant", + ) + // Since we want to persist a False condition if the CCL Item is + // not security compliant. + return nil + } + + // Check if the item is ready and skip the image content sync if + // not. + if !IsItemReady(cliStatus.Conditions) { + pkgcnd.MarkFalse( + vmiStatus, + vmopv1.ReadyConditionType, + vmopv1.VirtualMachineImageProviderNotReadyReason, + "Provider item is not in ready condition", + ) + logger.Info( + "Skipping image content sync", + "reason", "library item not ready") + return nil + } + + // If the sync is successful then the VMI resource is ready. + if syncErr = r.syncImageContent( + ctx, + cliObj, + *cliStatus, + vmiObj, + vmiStatus); syncErr == nil { + + pkgcnd.MarkTrue(vmiStatus, vmopv1.ReadyConditionType) + } + + didSync = true + + // Do not return syncErr here as we still want to patch the updated + // fields we get above. + return nil + }) + + logger = logger.WithValues("operationResult", opRes) + + // Registry metrics based on the corresponding error captured. + defer func() { + r.Metrics.RegisterVMIResourceResolve( + logger, + vmiObj.GetName(), + vmiObj.GetNamespace(), + copErr == nil) + r.Metrics.RegisterVMIContentSync(logger, + vmiObj.GetName(), + vmiObj.GetNamespace(), + didSync && syncErr == nil) + }() + + if copErr != nil { + logger.Error(copErr, "failed to create or patch image") + return copErr + } + + // CreateOrPatch/CreateOrUpdate doesn't patch sub-resource for creation. + if opRes == controllerutil.OperationResultCreated { + *vmiStatus = *savedStatus + if copErr = r.Status().Update(ctx, vmiObj); copErr != nil { + logger.Error(copErr, "Failed to update image status") + return copErr + } + } + + if syncErr != nil { + logger.Error( + syncErr, + "Failed to sync image to the latest content version") + return syncErr + } + + logger.Info( + "Successfully reconciled library item", + "contentVersion", savedStatus.ProviderContentVersion) + return nil +} + +// setUpVMIFromCLItem sets up the VirtualMachineImage fields that +// are retrievable from the given ContentLibraryItem resource. +func (r *Reconciler) setUpVMIFromCLItem( + cliObj client.Object, + cliSpec imgregv1a1.ContentLibraryItemSpec, + cliStatus imgregv1a1.ContentLibraryItemStatus, + vmiObj client.Object, + vmiSpec *vmopv1.VirtualMachineImageSpec, + vmiStatus *vmopv1.VirtualMachineImageStatus) error { + + if err := controllerutil.SetControllerReference( + cliObj, + vmiObj, + r.Scheme()); err != nil { + + return err + } + + cliGVK := cliObj.GetObjectKind().GroupVersionKind() + + vmiSpec.ProviderRef = &common.LocalObjectRef{ + APIVersion: cliGVK.GroupVersion().String(), + Kind: cliGVK.Kind, + Name: cliObj.GetName(), + } + + vmiStatus.Name = cliStatus.Name + vmiStatus.ProviderItemID = string(cliSpec.UUID) + vmiStatus.Type = string(cliStatus.Type) + + return AddContentLibraryRefToAnnotation( + vmiObj, cliStatus.ContentLibraryRef) +} + +// syncImageContent syncs the VirtualMachineImage content from the provider. +// It skips syncing if the image content is already up-to-date. +func (r *Reconciler) syncImageContent( + ctx context.Context, + cliObj client.Object, + cliStatus imgregv1a1.ContentLibraryItemStatus, + vmiObj client.Object, + vmiStatus *vmopv1.VirtualMachineImageStatus) error { + + latestVersion := cliStatus.ContentVersion + + err := r.VMProvider.SyncVirtualMachineImage(ctx, cliObj, vmiObj) + if err != nil { + if !pkgerr.WatchVMICacheIfNotReady(err, cliObj) { + pkgcnd.MarkFalse( + vmiStatus, + vmopv1.ReadyConditionType, + vmopv1.VirtualMachineImageNotSyncedReason, + "Failed to sync to the latest content version from provider") + } + } else { + vmiStatus.ProviderContentVersion = latestVersion + + if labels := cliObj.GetLabels(); labels != nil { + // Now that the VMI is synced, the CLI no longer needs to be + // reconciled when the VMI Cache object is updated. + delete(labels, pkgconst.VMICacheLabelKey) + cliObj.SetLabels(labels) + } + } + + // Sync the image's type, OS information and capabilities to the resource's + // labels to make it easier for clients to search for images based on type, + // OS info or image capabilities. + imgutil.SyncStatusToLabels(vmiObj, *vmiStatus) + + r.Recorder.EmitEvent(vmiObj, "Update", err, false) + return err +} diff --git a/controllers/contentlibrary/utils/controller_builder_test.go b/controllers/contentlibrary/utils/controller_builder_test.go new file mode 100644 index 000000000..8da53a662 --- /dev/null +++ b/controllers/contentlibrary/utils/controller_builder_test.go @@ -0,0 +1,565 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package utils_test + +import ( + "context" + "errors" + "fmt" + + "github.com/go-logr/logr" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + ctrlmgr "sigs.k8s.io/controller-runtime/pkg/manager" + + imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" + vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" + "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" + "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils" + pkgcnd "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" + "github.com/vmware-tanzu/vm-operator/pkg/constants/testlabels" + pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" + providerfake "github.com/vmware-tanzu/vm-operator/pkg/providers/fake" + pkgutil "github.com/vmware-tanzu/vm-operator/pkg/util" + "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" + "github.com/vmware-tanzu/vm-operator/test/builder" +) + +var _ = Describe("AddToManager", + Label( + testlabels.Controller, + testlabels.EnvTest, + testlabels.V1Alpha3, + ), + func() { + var ( + parentCtx context.Context + vcSimCtx *builder.IntegrationTestContextForVCSim + ) + + BeforeEach(func() { + parentCtx = pkgcfg.NewContext() + }) + + JustBeforeEach(func() { + vcSimCtx = builder.NewIntegrationTestContextForVCSim( + parentCtx, + builder.VCSimTestConfig{}, + func(ctx *pkgctx.ControllerManagerContext, mgr ctrlmgr.Manager) error { + return utils.AddToManager(ctx, mgr, &imgregv1a1.ContentLibraryItem{}) + }, + func(ctx *pkgctx.ControllerManagerContext, _ ctrlmgr.Manager) error { + return nil + }, + nil) + }) + + It("should not return an error", func() { + Expect(vcSimCtx).ToNot(BeNil()) + vcSimCtx.BeforeEach() + }) + + When("FSS FastDeploy is enabled", func() { + BeforeEach(func() { + pkgcfg.UpdateContext(parentCtx, func(config *pkgcfg.Config) { + config.Features.FastDeploy = true + }) + }) + It("should not return an error", func() { + Expect(vcSimCtx).ToNot(BeNil()) + vcSimCtx.BeforeEach() + }) + }) + }) + +var _ = Describe("Reconcile", + Label( + testlabels.Controller, + testlabels.V1Alpha3, + ), + func() { + const firmwareValue = "my-firmware" + + var ( + ctx *builder.UnitTestContextForController + + reconciler *utils.Reconciler + fakeVMProvider *providerfake.VMProvider + + cliObj client.Object + cliSpec *imgregv1a1.ContentLibraryItemSpec + cliStatus *imgregv1a1.ContentLibraryItemStatus + req ctrl.Request + + vmiName string + vmicName string + ) + + BeforeEach(func() { + ctx = builder.NewUnitTestContextForController(nil) + fakeVMProvider = ctx.VMProvider.(*providerfake.VMProvider) + }) + + JustBeforeEach(func() { + var err error + vmiName, err = utils.GetImageFieldNameFromItem(cliObj.GetName()) + Expect(err).ToNot(HaveOccurred()) + + Expect(ctx.Client.Create(ctx, cliObj)).To(Succeed()) + cliObj, cliSpec, cliStatus = getCLI(ctx, req.Namespace, req.Name) + }) + + AfterEach(func() { + ctx.AfterEach() + ctx = nil + cliObj = nil + reconciler = nil + fakeVMProvider.Reset() + }) + + Context("Namespace-scoped", func() { + + BeforeEach(func() { + reconciler = utils.NewReconciler( + ctx, + ctx.Client, + ctx.Logger, + ctx.Recorder, + ctx.VMProvider, + "ContentLibraryItem", + ) + + fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, vmiObj client.Object) error { + vmi := vmiObj.(*vmopv1.VirtualMachineImage) + + // Use Firmware field to verify the provider function is called. + vmi.Status.Firmware = firmwareValue + return nil + } + + o := utils.DummyContentLibraryItem( + utils.ItemFieldNamePrefix+"-dummy", "dummy-ns") + cliObj, cliSpec, cliStatus = o, &o.Spec, &o.Status + vmicName = pkgutil.VMIName(string(cliSpec.UUID)) + + // Add the finalizer so Reconcile does not return early. + cliObj.SetFinalizers([]string{utils.CLItemFinalizer}) + + req = ctrl.Request{} + req.Namespace = cliObj.GetNamespace() + req.Name = cliObj.GetName() + }) + + Context("ReconcileNormal", func() { + + When("Library item resource doesn't have the VMOP finalizer", func() { + BeforeEach(func() { + cliObj.SetFinalizers(nil) + }) + + It("should add the finalizer", func() { + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + cliObj, _, _ = getCLI(ctx, req.Namespace, req.Name) + + Expect(cliObj.GetFinalizers()).To(ContainElement(utils.CLItemFinalizer)) + }) + }) + + When("Library item resource is Not Ready", func() { + BeforeEach(func() { + cliStatus.Conditions = []imgregv1a1.Condition{ + { + Type: imgregv1a1.ReadyCondition, + Status: corev1.ConditionFalse, + }, + } + }) + + It("should mark image resource condition as provider not ready", func() { + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + + _, _, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + condition := pkgcnd.Get(vmiStatus, vmopv1.ReadyConditionType) + Expect(condition).ToNot(BeNil()) + Expect(condition.Status).To(Equal(metav1.ConditionFalse)) + Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageProviderNotReadyReason)) + }) + }) + + When("Library item resource is not security compliant", func() { + + BeforeEach(func() { + cliStatus.SecurityCompliance = ptr.To(false) + }) + + It("should mark image resource condition as provider security not compliant", func() { + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + + _, _, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + condition := pkgcnd.Get(vmiStatus, vmopv1.ReadyConditionType) + Expect(condition).ToNot(BeNil()) + Expect(condition.Status).To(Equal(metav1.ConditionFalse)) + Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageProviderSecurityNotCompliantReason)) + }) + }) + + When("SyncVirtualMachineImage returns an error", func() { + + BeforeEach(func() { + fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, _ client.Object) error { + return fmt.Errorf("sync-error") + } + }) + + It("should mark image resource condition synced failed", func() { + _, err := reconciler.Reconcile(ctx, req) + Expect(err).To(MatchError("sync-error")) + + _, _, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + condition := pkgcnd.Get(vmiStatus, vmopv1.ReadyConditionType) + Expect(condition).ToNot(BeNil()) + Expect(condition.Status).To(Equal(metav1.ConditionFalse)) + Expect(condition.Reason).To(Equal(vmopv1.VirtualMachineImageNotSyncedReason)) + }) + + When("error is ErrVMICacheNotReady", func() { + JustBeforeEach(func() { + fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, _ client.Object) error { + return fmt.Errorf("failed with %w", + pkgerr.VMICacheNotReadyError{Name: vmicName}) + } + }) + It("should place a label on the library item resource", func() { + _, err := reconciler.Reconcile(ctx, req) + + var e pkgerr.VMICacheNotReadyError + ExpectWithOffset(1, errors.As(err, &e)).To(BeTrue()) + ExpectWithOffset(1, e.Name).To(Equal(vmicName)) + + cliObj, _, _ = getCLI(ctx, req.Namespace, req.Name) + ExpectWithOffset(1, cliObj.GetLabels()).To(HaveKeyWithValue( + pkgconst.VMICacheLabelKey, vmicName)) + + _, _, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + condition := pkgcnd.Get(vmiStatus, vmopv1.ReadyConditionType) + Expect(condition).To(BeNil()) + }) + }) + }) + + When("Library item resource is ready and security complaint", func() { + + JustBeforeEach(func() { + // The dummy library item should meet these requirements. + var readyCond *imgregv1a1.Condition + for _, c := range cliStatus.Conditions { + if c.Type == imgregv1a1.ReadyCondition { + c := c + readyCond = &c + break + } + } + Expect(readyCond).ToNot(BeNil()) + Expect(readyCond.Status).To(Equal(corev1.ConditionTrue)) + + Expect(cliStatus.SecurityCompliance).To(Equal(ptr.To(true))) + }) + + When("Image resource has not been created yet", func() { + + It("should create a new image resource syncing up with the library item resource", func() { + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + cliObj, cliSpec, cliStatus = getCLI(ctx, req.Namespace, req.Name) + + vmiObj, vmiSpec, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + assertVMImageFromCLItem(cliObj, *cliSpec, *cliStatus, vmiObj, *vmiSpec, *vmiStatus) + Expect(vmiStatus.Firmware).To(Equal(firmwareValue)) + }) + }) + + When("Image resource is exists but not up-to-date", func() { + + JustBeforeEach(func() { + newVMI( + ctx, + req.Namespace, + vmiName, + vmopv1.VirtualMachineImageStatus{ + ProviderContentVersion: "stale", + Firmware: "should-be-updated", + }) + }) + + It("should update the existing image resource with the library item resource", func() { + cliStatus.ContentVersion += "-updated" + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + cliObj, cliSpec, cliStatus = getCLI(ctx, req.Namespace, req.Name) + + vmiObj, vmiSpec, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + assertVMImageFromCLItem(cliObj, *cliSpec, *cliStatus, vmiObj, *vmiSpec, *vmiStatus) + Expect(vmiStatus.Firmware).To(Equal(firmwareValue)) + }) + }) + + When("Image resource is created and already up-to-date", func() { + + JustBeforeEach(func() { + newVMI( + ctx, + req.Namespace, + vmiName, + vmopv1.VirtualMachineImageStatus{ + ProviderContentVersion: cliStatus.ContentVersion, + Firmware: "should-be-updated", + }) + }) + + It("should still update the image resource status from the library item resource", func() { + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + cliObj, cliSpec, cliStatus = getCLI(ctx, req.Namespace, req.Name) + + vmiObj, vmiSpec, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + assertVMImageFromCLItem(cliObj, *cliSpec, *cliStatus, vmiObj, *vmiSpec, *vmiStatus) + Expect(vmiStatus.Firmware).To(Equal(firmwareValue)) + }) + }) + }) + }) + + Context("ReconcileDelete", func() { + It("should remove the finalizer from the library item resource", func() { + Expect(cliObj.GetFinalizers()).To(ContainElement(utils.CLItemFinalizer)) + cliObj.SetDeletionTimestamp(ptr.To(metav1.Now())) + Expect(reconciler.ReconcileDelete(ctx, logr.Discard(), cliObj, vmiName)).To(Succeed()) + Expect(cliObj.GetFinalizers()).ToNot(ContainElement(utils.CLItemFinalizer)) + }) + }) + + }) + + Context("Cluster-scoped", func() { + + BeforeEach(func() { + reconciler = utils.NewReconciler( + ctx, + ctx.Client, + ctx.Logger, + ctx.Recorder, + ctx.VMProvider, + "ClusterContentLibraryItem", + ) + + fakeVMProvider.SyncVirtualMachineImageFn = func(_ context.Context, _, vmiObj client.Object) error { + vmi := vmiObj.(*vmopv1.ClusterVirtualMachineImage) + + // Use Firmware field to verify the provider function is called. + vmi.Status.Firmware = firmwareValue + return nil + } + + o := utils.DummyClusterContentLibraryItem( + utils.ItemFieldNamePrefix + "-dummy") + cliObj, cliSpec, cliStatus = o, &o.Spec, &o.Status + + // Add the finalizer so Reconcile does not return early. + cliObj.SetFinalizers([]string{utils.CLItemFinalizer}) + + req = ctrl.Request{} + req.Namespace = cliObj.GetNamespace() + req.Name = cliObj.GetName() + }) + + Context("ReconcileNormal", func() { + When("Library item resource is ready and security complaint", func() { + JustBeforeEach(func() { + // The dummy library item should meet these requirements. + var readyCond *imgregv1a1.Condition + for _, c := range cliStatus.Conditions { + if c.Type == imgregv1a1.ReadyCondition { + c := c + readyCond = &c + break + } + } + Expect(readyCond).ToNot(BeNil()) + Expect(readyCond.Status).To(Equal(corev1.ConditionTrue)) + Expect(cliStatus.SecurityCompliance).To(Equal(ptr.To(true))) + }) + + When("Image resource has not been created yet", func() { + + It("should create a new image resource syncing up with the library item resource", func() { + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + cliObj, cliSpec, cliStatus = getCLI(ctx, req.Namespace, req.Name) + + vmiObj, vmiSpec, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + assertVMImageFromCLItem(cliObj, *cliSpec, *cliStatus, vmiObj, *vmiSpec, *vmiStatus) + Expect(vmiStatus.Firmware).To(Equal(firmwareValue)) + }) + }) + + When("Image resource is exists but not up-to-date", func() { + JustBeforeEach(func() { + newVMI( + ctx, + req.Namespace, + vmiName, + vmopv1.VirtualMachineImageStatus{ + ProviderContentVersion: "stale", + Firmware: "should-be-updated", + }) + }) + It("should update the existing image resource with the library item resource", func() { + cliStatus.ContentVersion += "-updated" + _, err := reconciler.Reconcile(ctx, req) + Expect(err).ToNot(HaveOccurred()) + cliObj, cliSpec, cliStatus = getCLI(ctx, req.Namespace, req.Name) + + vmiObj, vmiSpec, vmiStatus := getVMI(ctx, req.Namespace, vmiName) + assertVMImageFromCLItem(cliObj, *cliSpec, *cliStatus, vmiObj, *vmiSpec, *vmiStatus) + Expect(vmiStatus.Firmware).To(Equal(firmwareValue)) + }) + }) + }) + }) + + Context("ReconcileDelete", func() { + It("should remove the finalizer from the library item resource", func() { + Expect(cliObj.GetFinalizers()).To(ContainElement(utils.CLItemFinalizer)) + cliObj.SetDeletionTimestamp(ptr.To(metav1.Now())) + Expect(reconciler.ReconcileDelete(ctx, logr.Discard(), cliObj, vmiName)).To(Succeed()) + Expect(cliObj.GetFinalizers()).ToNot(ContainElement(utils.CLItemFinalizer)) + }) + }) + }) + + }) + +func getCLI( + ctx *builder.UnitTestContextForController, + namespace, name string) (client.Object, *imgregv1a1.ContentLibraryItemSpec, *imgregv1a1.ContentLibraryItemStatus) { + + var ( + obj client.Object + spec *imgregv1a1.ContentLibraryItemSpec + status *imgregv1a1.ContentLibraryItemStatus + key = client.ObjectKey{ + Namespace: namespace, + Name: name, + } + ) + + if namespace != "" { + var o imgregv1a1.ContentLibraryItem + ExpectWithOffset(1, ctx.Client.Get(ctx, key, &o)).To(Succeed()) + obj, spec, status = &o, &o.Spec, &o.Status + } else { + var o imgregv1a1.ClusterContentLibraryItem + ExpectWithOffset(1, ctx.Client.Get(ctx, key, &o)).To(Succeed()) + obj, spec, status = &o, &o.Spec, &o.Status + } + + return obj, spec, status +} + +func getVMI( + ctx *builder.UnitTestContextForController, + namespace, name string) (client.Object, *vmopv1.VirtualMachineImageSpec, *vmopv1.VirtualMachineImageStatus) { + + var ( + obj client.Object + spec *vmopv1.VirtualMachineImageSpec + status *vmopv1.VirtualMachineImageStatus + key = client.ObjectKey{ + Namespace: namespace, + Name: name, + } + ) + + if namespace != "" { + var o vmopv1.VirtualMachineImage + ExpectWithOffset(1, ctx.Client.Get(ctx, key, &o)).To(Succeed()) + obj, spec, status = &o, &o.Spec, &o.Status + } else { + var o vmopv1.ClusterVirtualMachineImage + ExpectWithOffset(1, ctx.Client.Get(ctx, key, &o)).To(Succeed()) + obj, spec, status = &o, &o.Spec, &o.Status + } + + return obj, spec, status +} + +func newVMI( + ctx *builder.UnitTestContextForController, + namespace, name string, + status vmopv1.VirtualMachineImageStatus) { + + spec := vmopv1.VirtualMachineImageSpec{ + ProviderRef: &common.LocalObjectRef{ + Name: "bogus", + }, + } + + if namespace != "" { + o := vmopv1.VirtualMachineImage{ + ObjectMeta: metav1.ObjectMeta{ + Name: name, + Namespace: namespace, + }, + Spec: spec, + Status: status, + } + ExpectWithOffset(1, ctx.Client.Create(ctx, &o)).To(Succeed()) + } + + o := vmopv1.ClusterVirtualMachineImage{ + ObjectMeta: metav1.ObjectMeta{ + Name: name, + }, + Spec: spec, + Status: status, + } + ExpectWithOffset(1, ctx.Client.Create(ctx, &o)).To(Succeed()) +} + +func assertVMImageFromCLItem( + cliObj client.Object, + cliSpec imgregv1a1.ContentLibraryItemSpec, + cliStatus imgregv1a1.ContentLibraryItemStatus, + vmiObj client.Object, + vmiSpec vmopv1.VirtualMachineImageSpec, + vmiStatus vmopv1.VirtualMachineImageStatus) { + + Expect(metav1.IsControlledBy(vmiObj, cliObj)).To(BeTrue()) + + By("Expected VMImage Spec", func() { + Expect(vmiSpec.ProviderRef.Name).To(Equal(cliObj.GetName())) + cliGVK := cliObj.GetObjectKind().GroupVersionKind() + Expect(vmiSpec.ProviderRef.APIVersion).To(Equal(cliGVK.GroupVersion().String())) + Expect(vmiSpec.ProviderRef.Kind).To(Equal(cliGVK.Kind)) + }) + + By("Expected VMImage Status", func() { + Expect(vmiStatus.Name).To(Equal(cliStatus.Name)) + Expect(vmiStatus.ProviderItemID).To(BeEquivalentTo(cliSpec.UUID)) + Expect(vmiStatus.ProviderContentVersion).To(Equal(cliStatus.ContentVersion)) + Expect(vmiStatus.Type).To(BeEquivalentTo(cliStatus.Type)) + Expect(pkgcnd.IsTrue(vmiStatus, vmopv1.ReadyConditionType)).To(BeTrue()) + }) +} diff --git a/controllers/contentlibrary/utils/test_utils.go b/controllers/contentlibrary/utils/test_utils.go index e4f6432bf..fb7c05353 100644 --- a/controllers/contentlibrary/utils/test_utils.go +++ b/controllers/contentlibrary/utils/test_utils.go @@ -86,6 +86,9 @@ func DummyContentLibraryItem(name, namespace string) *imgregv1a1.ContentLibraryI }, } + clItem.SetName(clItem.Name) + clItem.SetNamespace(clItem.Namespace) + return clItem } diff --git a/controllers/contentlibrary/utils/utils_suite_test.go b/controllers/contentlibrary/utils/utils_suite_test.go new file mode 100644 index 000000000..17092b668 --- /dev/null +++ b/controllers/contentlibrary/utils/utils_suite_test.go @@ -0,0 +1,21 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package utils_test + +import ( + "testing" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + + "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils" + "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" +) + +func TestContentLibraryItemControllerUtils(t *testing.T) { + utils.SkipNameValidation = ptr.To(true) + RegisterFailHandler(Fail) + RunSpecs(t, "ContentLibraryItem Controller Utils Test Suite") +} diff --git a/controllers/contentlibrary/utils/utils_test.go b/controllers/contentlibrary/utils/utils_test.go index 5e12d3306..a4ec54a3b 100644 --- a/controllers/contentlibrary/utils/utils_test.go +++ b/controllers/contentlibrary/utils/utils_test.go @@ -6,156 +6,141 @@ package utils_test import ( "encoding/json" - "testing" + . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" "github.com/onsi/gomega/gstruct" - corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" - vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" "github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils" ) const fake = "fake" -func TestDummyClusterContentLibraryItem(t *testing.T) { - g := NewWithT(t) - obj := utils.DummyClusterContentLibraryItem(fake) - g.Expect(obj).ToNot(BeNil()) - g.Expect(obj.Name).To(Equal(fake)) - g.Expect(obj.Namespace).To(BeEmpty()) -} - -func TestDummyContentLibraryItem(t *testing.T) { - g := NewWithT(t) - obj := utils.DummyContentLibraryItem(fake, fake+fake) - g.Expect(obj).ToNot(BeNil()) - g.Expect(obj.Name).To(Equal(fake)) - g.Expect(obj.Namespace).To(Equal(fake + fake)) -} - -func TestFilterServicesTypeLabels(t *testing.T) { - filtered := utils.FilterServicesTypeLabels(map[string]string{ - "hello": "world", - "type.services.vmware.com/": "bar"}) - g := NewWithT(t) - g.Expect(filtered).To(HaveLen(1)) - g.Expect(filtered).To(HaveKeyWithValue("type.services.vmware.com/", "")) -} +var _ = Describe("DummyClusterContentLibraryItem", func() { + It("should succeed", func() { + obj := utils.DummyClusterContentLibraryItem(fake) + Expect(obj).ToNot(BeNil()) + Expect(obj.Name).To(Equal(fake)) + Expect(obj.Namespace).To(BeEmpty()) + }) +}) + +var _ = Describe("DummyContentLibraryItem", func() { + It("should succeed", func() { + obj := utils.DummyContentLibraryItem(fake, fake+fake) + Expect(obj).ToNot(BeNil()) + Expect(obj.Name).To(Equal(fake)) + Expect(obj.Namespace).To(Equal(fake + fake)) + }) +}) + +var _ = Describe("FilterServicesTypeLabels", func() { + It("should succeed", func() { + filtered := utils.FilterServicesTypeLabels(map[string]string{ + "hello": "world", + "type.services.vmware.com/": "bar"}) + Expect(filtered).To(HaveLen(1)) + Expect(filtered).To(HaveKeyWithValue("type.services.vmware.com/", "")) + }) +}) + +var _ = Describe("IsItemReady", func() { + It("should succeed", func() { + Expect(utils.IsItemReady(nil)).To(BeFalse()) + Expect(utils.IsItemReady(imgregv1a1.Conditions{})).To(BeFalse()) + Expect(utils.IsItemReady(imgregv1a1.Conditions{ + { + Type: imgregv1a1.ConditionType(fake), + }, + })).To(BeFalse()) + Expect(utils.IsItemReady(imgregv1a1.Conditions{ + { + Type: imgregv1a1.ReadyCondition, + }, + })).To(BeFalse()) + Expect(utils.IsItemReady(imgregv1a1.Conditions{ + { + Type: imgregv1a1.ReadyCondition, + Status: corev1.ConditionFalse, + }, + })).To(BeFalse()) + Expect(utils.IsItemReady(imgregv1a1.Conditions{ + { + Type: imgregv1a1.ReadyCondition, + Status: corev1.ConditionTrue, + }, + })).To(BeTrue()) + }) +}) + +var _ = DescribeTable("GetImageFieldNameFromItem", + func(in, expOut, expErr string) { + out, err := utils.GetImageFieldNameFromItem(in) + if expErr != "" { + Expect(err).To(MatchError(expErr)) + Expect(out).To(BeEmpty()) + } else { + Expect(out).To(Equal(expOut)) + } + }, + Entry("missing prefix", "123", "", "item name does not start with \"clitem\""), + Entry("missing identifier", "clitem", "", "item name does not have an identifier after clitem-"), + Entry("valid", "clitem-123", "vmi-123", ""), +) -func TestIsItemReady(t *testing.T) { - g := NewWithT(t) - g.Expect(utils.IsItemReady(nil)).To(BeFalse()) - g.Expect(utils.IsItemReady(imgregv1a1.Conditions{})).To(BeFalse()) - g.Expect(utils.IsItemReady(imgregv1a1.Conditions{ - { - Type: imgregv1a1.ConditionType(fake), - }, - })).To(BeFalse()) - g.Expect(utils.IsItemReady(imgregv1a1.Conditions{ - { - Type: imgregv1a1.ReadyCondition, - }, - })).To(BeFalse()) - g.Expect(utils.IsItemReady(imgregv1a1.Conditions{ - { - Type: imgregv1a1.ReadyCondition, - Status: corev1.ConditionFalse, - }, - })).To(BeFalse()) - g.Expect(utils.IsItemReady(imgregv1a1.Conditions{ - { - Type: imgregv1a1.ReadyCondition, - Status: corev1.ConditionTrue, - }, - })).To(BeTrue()) -} +var _ = Describe("AddContentLibRefToAnnotation", func() { -func TestGetImageFieldNameFromItem(t *testing.T) { - testCases := []struct { - name string - in string - expectedErr string - expectedOut string - }{ - { - name: "missing prefix", - in: "123", - expectedErr: "item name does not start with \"clitem\"", - }, - { - name: "missing identifier", - in: "clitem", - expectedErr: "item name does not have an identifier after clitem-", - }, - { - name: "valid", - in: "clitem-123", - expectedOut: "vmi-123", - }, - } - - for i := range testCases { - tc := testCases[i] - t.Run(tc.name, func(t *testing.T) { - g := NewWithT(t) - out, err := utils.GetImageFieldNameFromItem(tc.in) - if tc.expectedErr != "" { - g.Expect(err).To(MatchError(tc.expectedErr)) - g.Expect(out).To(BeEmpty()) - } else { - g.Expect(out).To(Equal(tc.expectedOut)) - } - }) - } -} + var obj client.Object -func Test_AddContentLibRefToAnnotation(t *testing.T) { - obj := &vmopv1.ClusterVirtualMachineImage{ - ObjectMeta: metav1.ObjectMeta{}, - } + BeforeEach(func() { + obj = &vmopv1.ClusterVirtualMachineImage{ + ObjectMeta: metav1.ObjectMeta{}, + } + }) - t.Run("when the CL ref is missing", func(t *testing.T) { - g := NewWithT(t) - g.Expect(utils.AddContentLibraryRefToAnnotation(obj, nil)).To(Succeed()) + When("the CL ref is missing", func() { + Expect(utils.AddContentLibraryRefToAnnotation(obj, nil)).To(Succeed()) }) - t.Run("with a valid CL ref", func(t *testing.T) { - g := NewWithT(t) + When("there is a valid CL ref", func() { ref := &imgregv1a1.NameAndKindRef{ Kind: "FooKind", Name: "foo", } - t.Run("annotation gets correctly set", func(t *testing.T) { - g.Expect(utils.AddContentLibraryRefToAnnotation(obj, ref)).To(Succeed()) - g.Expect(obj.Annotations).To(HaveLen(1)) - assertAnnotation(g, obj) + When("annotation gets correctly set", func() { + It("should have expected result", func() { + Expect(utils.AddContentLibraryRefToAnnotation(obj, ref)).To(Succeed()) + Expect(obj.GetAnnotations()).To(HaveLen(1)) + assertAnnotation(obj) + }) }) - t.Run("the new annotation does not override existing annotations", func(t *testing.T) { - obj.Annotations = map[string]string{"bar": "baz"} - g.Expect(utils.AddContentLibraryRefToAnnotation(obj, ref)).To(Succeed()) - g.Expect(len(obj.Annotations)).To(BeNumerically(">=", 1)) - assertAnnotation(g, obj) + When("the new annotation does not override existing annotations", func() { + It("should have expected result", func() { + obj.SetAnnotations(map[string]string{"bar": "baz"}) + Expect(utils.AddContentLibraryRefToAnnotation(obj, ref)).To(Succeed()) + Expect(len(obj.GetAnnotations())).To(BeNumerically(">=", 1)) + assertAnnotation(obj) + }) }) }) -} +}) -func assertAnnotation(g *WithT, obj client.Object) { - g.Expect(obj.GetAnnotations()).To(HaveKey(vmopv1.VMIContentLibRefAnnotation)) +func assertAnnotation(obj client.Object) { + ExpectWithOffset(1, obj.GetAnnotations()).To(HaveKey(vmopv1.VMIContentLibRefAnnotation)) val := obj.GetAnnotations()[vmopv1.VMIContentLibRefAnnotation] coreRef := corev1.TypedLocalObjectReference{} - g.Expect(json.Unmarshal([]byte(val), &coreRef)).To(Succeed()) + ExpectWithOffset(1, json.Unmarshal([]byte(val), &coreRef)).To(Succeed()) - g.Expect(coreRef.APIGroup).To(gstruct.PointTo(Equal(imgregv1a1.GroupVersion.Group))) - g.Expect(coreRef.Kind).To(Equal("FooKind")) - g.Expect(coreRef.Name).To(Equal("foo")) + ExpectWithOffset(1, coreRef.APIGroup).To(gstruct.PointTo(Equal(imgregv1a1.GroupVersion.Group))) + ExpectWithOffset(1, coreRef.Kind).To(Equal("FooKind")) + ExpectWithOffset(1, coreRef.Name).To(Equal("foo")) } diff --git a/controllers/controllers.go b/controllers/controllers.go index b3a8f27bf..7bb504e4b 100644 --- a/controllers/controllers.go +++ b/controllers/controllers.go @@ -15,6 +15,7 @@ import ( spq "github.com/vmware-tanzu/vm-operator/controllers/storagepolicyquota" "github.com/vmware-tanzu/vm-operator/controllers/virtualmachine" "github.com/vmware-tanzu/vm-operator/controllers/virtualmachineclass" + "github.com/vmware-tanzu/vm-operator/controllers/virtualmachineimagecache" "github.com/vmware-tanzu/vm-operator/controllers/virtualmachinepublishrequest" "github.com/vmware-tanzu/vm-operator/controllers/virtualmachinereplicaset" "github.com/vmware-tanzu/vm-operator/controllers/virtualmachineservice" @@ -68,5 +69,11 @@ func AddToManager(ctx *pkgctx.ControllerManagerContext, mgr manager.Manager) err } } + if pkgcfg.FromContext(ctx).Features.FastDeploy { + if err := virtualmachineimagecache.AddToManager(ctx, mgr); err != nil { + return fmt.Errorf("failed to initialize VMI controllers: %w", err) + } + } + return nil } diff --git a/controllers/virtualmachine/virtualmachine/virtualmachine_controller.go b/controllers/virtualmachine/virtualmachine/virtualmachine_controller.go index f22e8df01..8d0febf49 100644 --- a/controllers/virtualmachine/virtualmachine/virtualmachine_controller.go +++ b/controllers/virtualmachine/virtualmachine/virtualmachine_controller.go @@ -9,7 +9,6 @@ import ( "errors" "fmt" "reflect" - "strconv" "strings" "time" @@ -31,8 +30,10 @@ import ( byokv1 "github.com/vmware-tanzu/vm-operator/external/byok/api/v1alpha1" "github.com/vmware-tanzu/vm-operator/pkg/conditions" pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" ctxop "github.com/vmware-tanzu/vm-operator/pkg/context/operation" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" "github.com/vmware-tanzu/vm-operator/pkg/metrics" "github.com/vmware-tanzu/vm-operator/pkg/patch" "github.com/vmware-tanzu/vm-operator/pkg/prober" @@ -136,6 +137,21 @@ func AddToManager(ctx *pkgctx.ControllerManagerContext, mgr manager.Manager) err } } + if pkgcfg.FromContext(ctx).Features.FastDeploy { + builder = builder.Watches( + &vmopv1.VirtualMachineImageCache{}, + handler.EnqueueRequestsFromMapFunc( + vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + r.Logger.WithName("VirtualMachineImageCacheToItemMapper"), + r.Client, + vmopv1.GroupVersion, + controlledTypeName, + ), + ), + ) + } + return builder.Complete(r) } @@ -269,10 +285,14 @@ type Reconciler struct { // +kubebuilder:rbac:groups="",resources=resourcequotas;namespaces,verbs=get;list;watch // +kubebuilder:rbac:groups=encryption.vmware.com,resources=encryptionclasses,verbs=get;list;watch +// Reconcile the object. func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (_ ctrl.Result, reterr error) { ctx = pkgcfg.JoinContext(ctx, r.Context) - if pkgcfg.FromContext(ctx).Features.UnifiedStorageQuota || pkgcfg.FromContext(ctx).Features.WorkloadDomainIsolation { + if pkgcfg.FromContext(ctx).Features.UnifiedStorageQuota || + pkgcfg.FromContext(ctx).Features.WorkloadDomainIsolation || + pkgcfg.FromContext(ctx).Features.FastDeploy { + ctx = cource.JoinContext(ctx, r.Context) } @@ -293,17 +313,27 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (_ ctrl.Re logger := ctrl.Log.WithName("VirtualMachine").WithValues("name", vm.NamespacedName()) if pkgcfg.FromContext(ctx).Features.FastDeploy { - // Allow the use of an annotation to control whether fast-deploy is used - // per-VM to deploy the VM. - if val := vm.Annotations["vmoperator.vmware.com/fast-deploy"]; val != "" { - if ok, _ := strconv.ParseBool(val); !ok { - // Create a copy of the config so the feature-state for - // FastDeploy can also be influenced by a VM annotation. - cfg := pkgcfg.FromContext(ctx) - cfg.Features.FastDeploy = false - ctx = pkgcfg.WithContext(ctx, cfg) - logger.Info("Disabled fast-deploy for this VM") - } + + // Check to see if the VM's annotations specify a fast-deploy mode. + mode := vm.Annotations[pkgconst.FastDeployAnnotationKey] + + if mode == "" { + // If the VM's annotation does not specify a fast-deploy mode then + // get the mode from the global config. + mode = pkgcfg.FromContext(ctx).FastDeployMode + } + + switch strings.ToLower(mode) { + case pkgconst.FastDeployModeDirect, pkgconst.FastDeployModeLinked: + logger.V(4).Info("Using fast-deploy for this VM", "mode", mode) + // No-op + default: + // Create a copy of the config where the Fast Deploy feature is + // disabled for this call-stack. + cfg := pkgcfg.FromContext(ctx) + cfg.Features.FastDeploy = false + ctx = pkgcfg.WithContext(ctx, cfg) + logger.Info("Disabled fast-deploy for this VM") } } @@ -334,11 +364,13 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (_ ctrl.Re }() if !vm.DeletionTimestamp.IsZero() { - err = r.ReconcileDelete(vmCtx) - return ctrl.Result{}, err + return pkgerr.ResultFromError(r.ReconcileDelete(vmCtx)) } - if err = r.ReconcileNormal(vmCtx); err != nil && !ignoredCreateErr(err) { + if err := r.ReconcileNormal(vmCtx); err != nil && !ignoredCreateErr(err) { + if result, err := pkgerr.ResultFromError(err); err == nil { + return result, nil + } vmCtx.Logger.Error(err, "Failed to reconcile VirtualMachine") return ctrl.Result{}, err } @@ -434,7 +466,7 @@ func (r *Reconciler) ReconcileDelete(ctx *pkgctx.VirtualMachineContext) (reterr } // ReconcileNormal processes a level trigger for this VM: create if it doesn't exist otherwise update the existing VM. -func (r *Reconciler) ReconcileNormal(ctx *pkgctx.VirtualMachineContext) (reterr error) { +func (r *Reconciler) ReconcileNormal(ctx *pkgctx.VirtualMachineContext) (reterr error) { //nolint:gocyclo // Return early if the VM reconciliation is paused. if _, exists := ctx.VM.Annotations[vmopv1.PauseAnnotation]; exists { ctx.Logger.Info("Skipping reconciliation since VirtualMachine contains the pause annotation") @@ -481,6 +513,13 @@ func (r *Reconciler) ReconcileNormal(ctx *pkgctx.VirtualMachineContext) (reterr // Upgrade schema fields where needed upgradeSchema(ctx) + if pkgcfg.FromContext(ctx).Features.FastDeploy { + // Do not proceed unless the VMI cache this VM needs is ready. + if !r.isVMICacheReady(ctx) { + return nil + } + } + var ( err error chanErr <-chan error @@ -582,3 +621,39 @@ func ignoredCreateErr(err error) bool { } return false } + +func (r *Reconciler) isVMICacheReady(vmCtx *pkgctx.VirtualMachineContext) bool { + vmicName := vmCtx.VM.Labels[pkgconst.VMICacheLabelKey] + if vmicName == "" { + // The object is not waiting on a VMI cache object and can proceed. + return true + } + + var ( + vmic vmopv1.VirtualMachineImageCache + vmicKey = client.ObjectKey{ + Namespace: pkgcfg.FromContext(vmCtx).PodNamespace, + Name: vmicName, + } + ) + + if err := r.Client.Get(vmCtx, vmicKey, &vmic); err != nil { + vmCtx.Logger.Error( + err, + "Skipping due to failure getting vmicache object") + return false + } + + for _, t := range []string{ + vmopv1.VirtualMachineImageCacheConditionOVFReady, + vmopv1.VirtualMachineImageCacheConditionDisksReady, + } { + if !conditions.IsTrue(vmic, t) { + vmCtx.Logger.V(4).Info("Skipping due to missing true condition", + "conditionType", t) + return false + } + } + + return true +} diff --git a/controllers/virtualmachineimagecache/internal/virtualmachineimagecache_controller_internal.go b/controllers/virtualmachineimagecache/internal/virtualmachineimagecache_controller_internal.go new file mode 100644 index 000000000..a1d7d80b0 --- /dev/null +++ b/controllers/virtualmachineimagecache/internal/virtualmachineimagecache_controller_internal.go @@ -0,0 +1,15 @@ +// // © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package internal + +type contextKey uint8 + +const ( + // NewCacheStorageURIsClientContextKey is used for testing. + NewCacheStorageURIsClientContextKey contextKey = iota + + // NewContentLibraryProviderContextKey is used for testing. + NewContentLibraryProviderContextKey +) diff --git a/controllers/virtualmachineimagecache/virtualmachineimagecache_controller.go b/controllers/virtualmachineimagecache/virtualmachineimagecache_controller.go new file mode 100644 index 000000000..577cb6a57 --- /dev/null +++ b/controllers/virtualmachineimagecache/virtualmachineimagecache_controller.go @@ -0,0 +1,715 @@ +// // © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package virtualmachineimagecache + +import ( + "context" + "errors" + "fmt" + "reflect" + "regexp" + "strings" + + "github.com/go-logr/logr" + "github.com/vmware/govmomi/fault" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/property" + "github.com/vmware/govmomi/vapi/rest" + "github.com/vmware/govmomi/vim25" + "github.com/vmware/govmomi/vim25/mo" + vimtypes "github.com/vmware/govmomi/vim25/types" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/manager" + "sigs.k8s.io/controller-runtime/pkg/source" + "sigs.k8s.io/yaml" + + vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" + "github.com/vmware-tanzu/vm-operator/controllers/virtualmachineimagecache/internal" + pkgcond "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" + "github.com/vmware-tanzu/vm-operator/pkg/patch" + "github.com/vmware-tanzu/vm-operator/pkg/providers" + clprov "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/contentlibrary" + "github.com/vmware-tanzu/vm-operator/pkg/record" + pkgutil "github.com/vmware-tanzu/vm-operator/pkg/util" + "github.com/vmware-tanzu/vm-operator/pkg/util/kube/cource" + "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/client" + clsutil "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/library" +) + +// SkipNameValidation is used for testing to allow multiple controllers with the +// same name since Controller-Runtime has a global singleton registry to +// prevent controllers with the same name, even if attached to different +// managers. +var SkipNameValidation *bool + +// AddToManager adds this package's controller to the provided manager. +func AddToManager(ctx *pkgctx.ControllerManagerContext, mgr manager.Manager) error { + var ( + controlledType = &vmopv1.VirtualMachineImageCache{} + controlledTypeName = reflect.TypeOf(controlledType).Elem().Name() + + controllerNameShort = fmt.Sprintf("%s-controller", strings.ToLower(controlledTypeName)) + controllerNameLong = fmt.Sprintf("%s/%s/%s", ctx.Namespace, ctx.Name, controllerNameShort) + ) + + r := &reconciler{ + Context: ctx, + Client: mgr.GetClient(), + Logger: ctx.Logger.WithName("controllers").WithName(controlledTypeName), + Recorder: record.New(mgr.GetEventRecorderFor(controllerNameLong)), + VMProvider: ctx.VMProvider, + + newCLSProvdrFn: newContentLibraryProviderOrDefault(ctx), + newSRIClientFn: newCacheStorageURIsClientOrDefault(ctx), + } + + return ctrl.NewControllerManagedBy(mgr). + For(controlledType). + WithOptions(controller.Options{ + SkipNameValidation: SkipNameValidation, + }). + WatchesRawSource(source.Channel( + cource.FromContextWithBuffer(ctx, "VirtualMachineImageCache", 100), + &handler.EnqueueRequestForObject{})). + Complete(r) +} + +// reconciler reconciles a VirtualMachineImageCache object. +type reconciler struct { + ctrlclient.Client + Context context.Context + Logger logr.Logger + Recorder record.Recorder + VMProvider providers.VirtualMachineProviderInterface + + newCLSProvdrFn newContentLibraryProviderFn + newSRIClientFn newCacheStorageURIsClientFn +} + +// +kubebuilder:rbac:groups=vmoperator.vmware.com,resources=virtualmachineimagecaches,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=vmoperator.vmware.com,resources=virtualmachineimagecaches/status,verbs=get;update;patch + +func (r *reconciler) Reconcile(ctx context.Context, req ctrl.Request) (_ ctrl.Result, reterr error) { + ctx = pkgcfg.JoinContext(ctx, r.Context) + + var obj vmopv1.VirtualMachineImageCache + if err := r.Get(ctx, req.NamespacedName, &obj); err != nil { + return ctrl.Result{}, ctrlclient.IgnoreNotFound(err) + } + + logger := r.Logger.WithValues("name", req.NamespacedName) + ctx = logr.NewContext(ctx, logger) + + patchHelper, err := patch.NewHelper(&obj, r.Client) + if err != nil { + return ctrl.Result{}, fmt.Errorf( + "failed to init patch helper for %s: %w", req.NamespacedName, err) + } + defer func() { + if err := patchHelper.Patch(ctx, &obj); err != nil { + if reterr == nil { + reterr = err + } + logger.Error(err, "patch failed") + } + }() + + if !obj.DeletionTimestamp.IsZero() { + // Noop. + return ctrl.Result{}, nil + } + + return pkgerr.ResultFromError(r.ReconcileNormal(ctx, &obj)) +} + +const conditionReasonFailed = "Failed" + +var endsWithVMDK = regexp.MustCompile(`(?i)^.+\.vmdk$`) + +func (r *reconciler) ReconcileNormal( + ctx context.Context, + obj *vmopv1.VirtualMachineImageCache) (retErr error) { + + // Reset the version status so it is constructed from scratch each time. + obj.Status = vmopv1.VirtualMachineImageCacheStatus{} + + // If the reconcile failed with an error, then make sure it is reflected in + // the object's Ready condition. + defer func() { + if retErr != nil { + pkgcond.MarkFalse( + obj, + vmopv1.ReadyConditionType, + conditionReasonFailed, + retErr.Error()) + } + }() + + // Verify the item's ID. + if obj.Spec.ProviderID == "" { + return errors.New("spec.providerID is empty") + } + + // Verify the item's version. + if obj.Spec.ProviderVersion == "" { + return errors.New("spec.providerVersion is empty") + } + + logger := r.Logger.WithValues( + "providerID", obj.Spec.ProviderID, + "providerVersion", obj.Spec.ProviderVersion) + ctx = logr.NewContext(ctx, logger) + + // Get a vSphere client. + c, err := r.VMProvider.VSphereClient(ctx) + if err != nil { + return fmt.Errorf("failed to get vSphere client: %w", err) + } + + // Get the content library provider. + clProv := r.newCLSProvdrFn(ctx, c.RestClient()) + + // Reconcile the OVF envelope. + if err := reconcileOVF(ctx, r.Client, clProv, obj); err != nil { + pkgcond.MarkFalse( + obj, + vmopv1.VirtualMachineImageCacheConditionOVFReady, + conditionReasonFailed, + err.Error()) + } else { + pkgcond.MarkTrue( + obj, + vmopv1.VirtualMachineImageCacheConditionOVFReady) + } + + if len(obj.Spec.Locations) > 0 { + // Reconcile the underlying library item. + if err := reconcileLibraryItem(ctx, clProv, obj); err != nil { + pkgcond.MarkFalse( + obj, + vmopv1.VirtualMachineImageCacheConditionProviderReady, + conditionReasonFailed, + err.Error()) + } else { + pkgcond.MarkTrue( + obj, + vmopv1.VirtualMachineImageCacheConditionProviderReady) + } + + // Reconcile the disks. + if err := r.reconcileDisks(ctx, c, clProv, obj); err != nil { + pkgcond.MarkFalse( + obj, + vmopv1.VirtualMachineImageCacheConditionDisksReady, + conditionReasonFailed, + err.Error()) + } else { + // Aggregate each location's Ready condition into the top-level + // VirtualMachineImageCacheConditionDisksReady condition. + getters := make([]pkgcond.Getter, len(obj.Status.Locations)) + for i := range obj.Status.Locations { + getters[i] = obj.Status.Locations[i] + } + pkgcond.SetAggregate( + obj, + vmopv1.VirtualMachineImageCacheConditionDisksReady, + getters, + pkgcond.WithStepCounter()) + } + } + + // Create the object's Ready condition based on its other conditions. + pkgcond.SetSummary(obj, pkgcond.WithStepCounter()) + + return nil +} + +func (r *reconciler) reconcileDisks( + ctx context.Context, + vcClient *client.Client, + clProv clprov.Provider, + obj *vmopv1.VirtualMachineImageCache) error { + + var ( + srcDatacenter = vcClient.Datacenter() + vimClient = vcClient.VimClient() + ) + + // Get the library item's storage paths. + srcDiskURIs, err := getSourceDiskPaths( + ctx, + clProv, + srcDatacenter, + obj.Spec.ProviderID) + if err != nil { + return err + } + + // Get the datacenters used by the item. + dstDatacenters, err := getDatacenters(ctx, vimClient, obj) + if err != nil { + return err + } + + // Get the datastores used by the item. + dstDatastores, err := getDatastores(ctx, vimClient, obj) + if err != nil { + return err + } + + // Get the top-level cache directories for each datastore. + dstTopLevelDirs, err := r.getTopLevelCacheDirs( + ctx, + vimClient, + dstDatacenters, + dstDatastores) + if err != nil { + return err + } + + // Reconcile the locations. + r.reconcileLocations( + ctx, + vimClient, + dstDatacenters, + srcDatacenter, + dstDatastores, + obj, + dstTopLevelDirs, + srcDiskURIs) + + return nil +} + +func (r *reconciler) reconcileLocations( + ctx context.Context, + vimClient *vim25.Client, + dstDatacenters map[string]*object.Datacenter, + srcDatacenter *object.Datacenter, + dstDatastores map[string]datastore, + obj *vmopv1.VirtualMachineImageCache, + dstTopLevelDirs map[string]string, + srcDiskURIs []string) { + + obj.Status.Locations = make( + []vmopv1.VirtualMachineImageCacheLocationStatus, + len(obj.Spec.Locations)) + + for i := range obj.Spec.Locations { + + var ( + spec = obj.Spec.Locations[i] + status = &obj.Status.Locations[i] + conditions = pkgcond.Conditions(status.Conditions) + ) + + status.DatacenterID = spec.DatacenterID + status.DatastoreID = spec.DatastoreID + + // Get the preferred disk format for the datastore. + dstDiskFormat := pkgutil.GetPreferredDiskFormat( + dstDatastores[spec.DatastoreID].mo.Info. + GetDatastoreInfo().SupportedVDiskFormats...) + + files, err := r.cacheDisks( + ctx, + vimClient, + dstDatacenters[spec.DatacenterID], + srcDatacenter, + dstTopLevelDirs[spec.DatastoreID], + obj.Spec.ProviderID, + obj.Spec.ProviderVersion, + dstDiskFormat, + srcDiskURIs) + if err != nil { + conditions = conditions.MarkFalse( + vmopv1.ReadyConditionType, + conditionReasonFailed, + err.Error()) + } else { + status.Files = files + conditions = conditions.MarkTrue(vmopv1.ReadyConditionType) + } + + status.Conditions = conditions + } +} + +func (r *reconciler) cacheDisks( + ctx context.Context, + vimClient *vim25.Client, + dstDatacenter, srcDatacenter *object.Datacenter, + tldPath, itemID, itemVersion string, + dstDiskFormat vimtypes.DatastoreSectorFormat, + srcDiskURIs []string) ([]string, error) { + + itemCacheDir := clsutil.GetCacheDirForLibraryItem( + tldPath, + itemID, + itemVersion) + + sriClient := r.newSRIClientFn(vimClient) + + logger := logr.FromContextOrDiscard(ctx) + logger.Info("Caching disks", + "dstDatacenter", dstDatacenter.Reference().Value, + "srcDatacenter", srcDatacenter.Reference().Value, + "itemCacheDir", itemCacheDir, + "dstDiskFormat", dstDiskFormat, + "srcDiskPaths", srcDiskURIs) + + cachedPaths, err := clsutil.CacheStorageURIs( + ctx, + sriClient, + dstDatacenter, + srcDatacenter, + itemCacheDir, + dstDiskFormat, + srcDiskURIs...) + if err != nil { + return nil, fmt.Errorf("failed to cache storage items: %w", err) + } + + return cachedPaths, nil +} + +const ( + ovfConfigMapValueKey = "value" + ovfConfigMapContentVersionKey = "contentVersion" +) + +func reconcileOVF( + ctx context.Context, + k8sClient ctrlclient.Client, + clProv clprov.Provider, + obj *vmopv1.VirtualMachineImageCache) error { + + // Ensure the OVF ConfigMap is up-to-date. Please note, this may be a no-op. + configMap := corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: obj.Namespace, + Name: obj.Name, + }, + } + if _, err := controllerutil.CreateOrPatch( + ctx, + k8sClient, + &configMap, + func() error { + + if configMap.Data[ovfConfigMapValueKey] != "" && + configMap.Data[ovfConfigMapContentVersionKey] == obj.Spec.ProviderVersion { + // Do nothing if the ConfigMap has the marshaled OVF and it is + // the latest content version. + return nil + } + + if configMap.Data == nil { + configMap.Data = map[string]string{} + } + + logger := logr.FromContextOrDiscard(ctx) + logger.Info("Fetching OVF") + + // Get the OVF. + ovfEnv, err := clProv.RetrieveOvfEnvelopeByLibraryItemID( + ctx, obj.Spec.ProviderID) + if err != nil { + return fmt.Errorf("failed to retrieve ovf envelope: %w", err) + } + + // Marshal the OVF envelope to YAML. + data, err := yaml.Marshal(ovfEnv) + if err != nil { + return fmt.Errorf("failed to marshal ovf envelope to YAML: %w", err) + } + + configMap.Data[ovfConfigMapContentVersionKey] = obj.Spec.ProviderVersion + configMap.Data[ovfConfigMapValueKey] = string(data) + + return nil + }); err != nil { + + return fmt.Errorf("failed to create or patch ovf configmap: %w", err) + } + + if obj.Status.OVF == nil { + obj.Status.OVF = &vmopv1.VirtualMachineImageCacheOVFStatus{} + } + + obj.Status.OVF.ProviderVersion = obj.Spec.ProviderVersion + obj.Status.OVF.ConfigMapName = configMap.Name + + return nil +} + +func reconcileLibraryItem( + ctx context.Context, + p clprov.Provider, + obj *vmopv1.VirtualMachineImageCache) error { + + logger := logr.FromContextOrDiscard(ctx) + + // Get the content library item to be cached. + item, err := p.GetLibraryItemID(ctx, obj.Spec.ProviderID) + if err != nil { + return fmt.Errorf("failed to get library item: %w", err) + } + + // If the item is not cached locally, then issue a sync so content library + // fetches the item's disks. + // + // Please note, the m.SyncLibraryItem method is reentrant on the remote + // side. That is to say, if the call gets interrupted and we sync the item + // again while an existing sync is occurring, the client will block until + // the original sync is complete. + if !item.Cached { + logger.Info("Syncing library item") + if err := p.SyncLibraryItem(ctx, item, true); err != nil { + return fmt.Errorf("failed to sync library item: %w", err) + } + } + + return nil +} + +type datastore struct { + datacenterID string + mo mo.Datastore + obj *object.Datastore +} + +func getSourceDiskPaths( + ctx context.Context, + p clprov.Provider, + datacenter *object.Datacenter, + itemID string) ([]string, error) { + + // Get the storage URIs for the library item's files. + itemStor, err := p.ListLibraryItemStorage(ctx, itemID) + if err != nil { + return nil, fmt.Errorf("failed to list library item storage: %w", err) + } + + // Resolve the item's storage URIs into datastore paths, ex. + // [my-datastore] path/to/file.ext + if err := p.ResolveLibraryItemStorage( + ctx, + datacenter, + itemStor); err != nil { + + return nil, fmt.Errorf("failed to resolve library item storage: %w", err) + } + + // Get the storage URIs for just the files that are disks. + var srcDiskURIs []string + for i := range itemStor { + is := itemStor[i] + for j := range is.StorageURIs { + s := is.StorageURIs[j] + if endsWithVMDK.MatchString(s) { + srcDiskURIs = append(srcDiskURIs, s) + } + } + } + + return srcDiskURIs, nil +} + +func getDatacenters( + ctx context.Context, + vimClient *vim25.Client, + obj *vmopv1.VirtualMachineImageCache) (map[string]*object.Datacenter, error) { + + var ( + refList []vimtypes.ManagedObjectReference + objMap = map[string]*object.Datacenter{} + ) + + // Get a set of unique datacenters used by the item's storage. + for i := range obj.Spec.Locations { + l := obj.Spec.Locations[i] + if _, ok := objMap[l.DatacenterID]; !ok { + ref := vimtypes.ManagedObjectReference{ + Type: "Datacenter", + Value: l.DatacenterID, + } + objMap[l.DatacenterID] = object.NewDatacenter(vimClient, ref) + refList = append(refList, ref) + } + } + + var ( + moList []mo.Datacenter + pc = property.DefaultCollector(vimClient) + ) + + // Populate the properties of the unique datacenters. + if err := pc.Retrieve( + ctx, + refList, + []string{"name"}, + &moList); err != nil { + + if fault.Is(err, &vimtypes.ManagedObjectNotFound{}) { + return nil, errors.New("invalid datacenter ID") + } + + return nil, fmt.Errorf("failed to get datacenter properties: %w", err) + } + + return objMap, nil +} + +func getDatastores( + ctx context.Context, + vimClient *vim25.Client, + obj *vmopv1.VirtualMachineImageCache) (map[string]datastore, error) { + + var ( + refList []vimtypes.ManagedObjectReference + objMap = map[string]datastore{} + ) + + // Get a set of unique datastores used by the item's storage. + for i := range obj.Spec.Locations { + l := obj.Spec.Locations[i] + if _, ok := objMap[l.DatastoreID]; !ok { + ref := vimtypes.ManagedObjectReference{ + Type: "Datastore", + Value: l.DatastoreID, + } + objMap[l.DatastoreID] = datastore{ + datacenterID: l.DatacenterID, + mo: mo.Datastore{ + ManagedEntity: mo.ManagedEntity{ + ExtensibleManagedObject: mo.ExtensibleManagedObject{ + Self: ref, + }, + }, + }, + obj: object.NewDatastore(vimClient, ref), + } + refList = append(refList, ref) + } + } + + var ( + moList []mo.Datastore + pc = property.DefaultCollector(vimClient) + ) + + // Populate the properties of the unique datastores. + if err := pc.Retrieve( + ctx, + refList, + []string{"name", "info"}, + &moList); err != nil { + + if fault.Is(err, &vimtypes.ManagedObjectNotFound{}) { + return nil, errors.New("invalid datastore ID") + } + + return nil, fmt.Errorf("failed to get datastore properties: %w", err) + } + + for i := range moList { + v := moList[i].Reference().Value + o := objMap[v] + o.mo = moList[i] + objMap[v] = o + } + + return objMap, nil +} + +func (r *reconciler) getTopLevelCacheDirs( + ctx context.Context, + vimClient *vim25.Client, + dstDatacenters map[string]*object.Datacenter, + dstDatastores map[string]datastore) (map[string]string, error) { + + client := r.newSRIClientFn(vimClient) + + // Iterate over the unique datastores and ensure there is a top-level + // cache directory present on each one. + tldMap := map[string]string{} + for k, ds := range dstDatastores { + p := fmt.Sprintf("[%s] %s", ds.mo.Name, clsutil.TopLevelCacheDirName) + if err := client.MakeDirectory( + ctx, + p, + dstDatacenters[ds.datacenterID], + true); err != nil { + return nil, fmt.Errorf( + "failed to create top-level directory %q: %w", + p, err) + } + tldMap[k] = p + } + return tldMap, nil +} + +type newContentLibraryProviderFn func(context.Context, *rest.Client) clprov.Provider +type newCacheStorageURIsClientFn func(*vim25.Client) clsutil.CacheStorageURIsClient + +func newContentLibraryProviderOrDefault( + ctx context.Context) newContentLibraryProviderFn { + + out := clprov.NewProvider + obj := ctx.Value(internal.NewContentLibraryProviderContextKey) + if fn, ok := obj.(func(context.Context, *rest.Client) clprov.Provider); ok { + out = func(ctx context.Context, c *rest.Client) clprov.Provider { + if p := fn(ctx, c); p != nil { + return p + } + return clprov.NewProvider(ctx, c) + } + } + return out +} + +func newCacheStorageURIsClientOrDefault( + ctx context.Context) newCacheStorageURIsClientFn { + + out := newCacheStorageURIsClient + obj := ctx.Value(internal.NewCacheStorageURIsClientContextKey) + if fn, ok := obj.(func(*vim25.Client) clsutil.CacheStorageURIsClient); ok { + out = func(c *vim25.Client) clsutil.CacheStorageURIsClient { + if p := fn(c); p != nil { + return p + } + return newCacheStorageURIsClient(c) + } + } + return out +} + +func newCacheStorageURIsClient(c *vim25.Client) clsutil.CacheStorageURIsClient { + return &cacheStorageURIsClient{ + FileManager: object.NewFileManager(c), + VirtualDiskManager: object.NewVirtualDiskManager(c), + } +} + +type cacheStorageURIsClient struct { + *object.FileManager + *object.VirtualDiskManager +} + +func (c *cacheStorageURIsClient) WaitForTask( + ctx context.Context, task *object.Task) error { + + return task.Wait(ctx) +} diff --git a/controllers/virtualmachineimagecache/virtualmachineimagecache_controller_suite_test.go b/controllers/virtualmachineimagecache/virtualmachineimagecache_controller_suite_test.go new file mode 100644 index 000000000..25a2d742e --- /dev/null +++ b/controllers/virtualmachineimagecache/virtualmachineimagecache_controller_suite_test.go @@ -0,0 +1,24 @@ +// Copyright (c) 2024 Broadcom. All Rights Reserved. +// Broadcom Confidential. The term "Broadcom" refers to Broadcom Inc. +// and/or its subsidiaries. + +package virtualmachineimagecache_test + +import ( + "testing" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + + "github.com/vmware-tanzu/vm-operator/controllers/virtualmachineimagecache" + "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" +) + +func TestVirtualMachineImageCacheController(t *testing.T) { + RegisterFailHandler(Fail) + RunSpecs(t, "VirtualMachineImageCache Controller Test Suite") +} + +var _ = BeforeSuite(func() { + virtualmachineimagecache.SkipNameValidation = ptr.To(true) +}) diff --git a/controllers/virtualmachineimagecache/virtualmachineimagecache_controller_test.go b/controllers/virtualmachineimagecache/virtualmachineimagecache_controller_test.go new file mode 100644 index 000000000..5b288d918 --- /dev/null +++ b/controllers/virtualmachineimagecache/virtualmachineimagecache_controller_test.go @@ -0,0 +1,862 @@ +// // © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package virtualmachineimagecache_test + +import ( + "context" + "errors" + "fmt" + "path" + "time" + + "github.com/go-logr/logr" + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/ovf" + "github.com/vmware/govmomi/vapi/library" + "github.com/vmware/govmomi/vapi/rest" + "github.com/vmware/govmomi/vim25" + vimtypes "github.com/vmware/govmomi/vim25/types" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/klog/v2/textlogger" + ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" + ctrlmgr "sigs.k8s.io/controller-runtime/pkg/manager" + + vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" + "github.com/vmware-tanzu/vm-operator/controllers/virtualmachineimagecache" + "github.com/vmware-tanzu/vm-operator/controllers/virtualmachineimagecache/internal" + pkgcond "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + "github.com/vmware-tanzu/vm-operator/pkg/constants/testlabels" + pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" + providerfake "github.com/vmware-tanzu/vm-operator/pkg/providers/fake" + clprov "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/contentlibrary" + "github.com/vmware-tanzu/vm-operator/pkg/util/kube/cource" + vsclient "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/client" + clsutil "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/library" + "github.com/vmware-tanzu/vm-operator/test/builder" +) + +var _ = Describe( + "Reconcile", + Label( + testlabels.Controller, + testlabels.EnvTest, + testlabels.V1Alpha3, + ), + func() { + + const ( + fakeString = "fake" + + cndOVFReady = vmopv1.VirtualMachineImageCacheConditionOVFReady + cndDskReady = vmopv1.VirtualMachineImageCacheConditionDisksReady + cndPrvReady = vmopv1.VirtualMachineImageCacheConditionProviderReady + cndRdyReady = vmopv1.ReadyConditionType + ) + + getVMICacheObj := func( + namespace, + providerID, + providerVersion string, + locations ...vmopv1.VirtualMachineImageCacheLocationSpec) vmopv1.VirtualMachineImageCache { + + return vmopv1.VirtualMachineImageCache{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: namespace, + GenerateName: "vmi-", + }, + Spec: vmopv1.VirtualMachineImageCacheSpec{ + ProviderID: providerID, + ProviderVersion: providerVersion, + Locations: locations, + }, + } + } + + var ( + ctx context.Context + vcSimCtx *builder.IntegrationTestContextForVCSim + provider *providerfake.VMProvider + initEnvFn builder.InitVCSimEnvFn + nsInfo builder.WorkloadNamespaceInfo + + itemID string + itemVersion string + + faker fakeClient + ) + + assertCondNil := func(g Gomega, o pkgcond.Getter, t string) { + g.ExpectWithOffset(1, pkgcond.Get(o, t)).To(BeNil()) + } + + assertCondTrue := func(g Gomega, o pkgcond.Getter, t string) { + c := pkgcond.Get(o, t) + g.ExpectWithOffset(1, c).ToNot(BeNil()) + g.ExpectWithOffset(1, c.Message).To(BeEmpty()) + g.ExpectWithOffset(1, c.Reason).To(Equal(string(metav1.ConditionTrue))) + g.ExpectWithOffset(1, c.Status).To(Equal(metav1.ConditionTrue)) + } + + assertCondFalse := func(g Gomega, o pkgcond.Getter, t, reason, message string) { + c := pkgcond.Get(o, t) + g.ExpectWithOffset(1, c).ToNot(BeNil()) + g.ExpectWithOffset(1, c.Message).To(HavePrefix(message)) + g.ExpectWithOffset(1, c.Reason).To(Equal(reason)) + g.ExpectWithOffset(1, c.Status).To(Equal(metav1.ConditionFalse)) + } + + assertOVFConfigMap := func(g Gomega, key ctrlclient.ObjectKey) { + var obj corev1.ConfigMap + g.ExpectWithOffset(1, vcSimCtx.Client.Get(ctx, key, &obj)).To(Succeed()) + g.ExpectWithOffset(1, obj.Data["value"]).To(MatchYAML(ovfEnvelopeYAML)) + } + + assertLocation := func( + g Gomega, + obj vmopv1.VirtualMachineImageCache, + locationIndex int) { + + var ( + spec = obj.Spec.Locations[locationIndex] + status = obj.Status.Locations[locationIndex] + ) + + g.ExpectWithOffset(1, status.DatacenterID).To(Equal(spec.DatacenterID)) + g.ExpectWithOffset(1, status.DatastoreID).To(Equal(spec.DatastoreID)) + + ds := object.NewDatastore( + vcSimCtx.VCClient.Client, + vimtypes.ManagedObjectReference{ + Type: "Datastore", + Value: spec.DatastoreID, + }) + dsName, err := ds.ObjectName(ctx) + g.ExpectWithOffset(1, err).ToNot(HaveOccurred()) + + topLevelCacheDir := fmt.Sprintf( + "[%s] %s", + dsName, + clsutil.TopLevelCacheDirName) + + itemCacheDir := clsutil.GetCacheDirForLibraryItem( + topLevelCacheDir, + obj.Spec.ProviderID, + obj.Spec.ProviderVersion) + + vmdkFileName := clsutil.GetCachedFileNameForVMDK( + "ttylinux-pc_i486-16.1-disk1.vmdk") + ".vmdk" + vmdkFilePath := path.Join(itemCacheDir, vmdkFileName) + + g.ExpectWithOffset(1, status.Files).To(HaveLen(1)) + g.ExpectWithOffset(1, status.Files[0]).To(Equal(vmdkFilePath)) + } + + Context("Ordered", Ordered, func() { + + BeforeAll(func() { + _ = internal.NewContentLibraryProviderContextKey + + ctx = context.Background() + ctx = logr.NewContext( + ctx, + textlogger.NewLogger(textlogger.NewConfig( + textlogger.Verbosity(5), + textlogger.Output(GinkgoWriter), + ))) + ctx = pkgcfg.WithContext(ctx, pkgcfg.Default()) + ctx = cource.WithContext(ctx) + + ctx = context.WithValue( + ctx, + internal.NewContentLibraryProviderContextKey, + faker.newContentLibraryProviderFn) + ctx = context.WithValue( + ctx, + internal.NewCacheStorageURIsClientContextKey, + faker.newCacheStorageURIsClientFn) + + provider = providerfake.NewVMProvider() + + vcSimCtx = builder.NewIntegrationTestContextForVCSim( + ctx, + builder.VCSimTestConfig{ + WithContentLibrary: true, + }, + virtualmachineimagecache.AddToManager, + func(ctx *pkgctx.ControllerManagerContext, _ ctrlmgr.Manager) error { + ctx.VMProvider = provider + return nil + }, + initEnvFn) + Expect(vcSimCtx).ToNot(BeNil()) + + vcSimCtx.BeforeEach() + ctx = vcSimCtx + + // Get the library item ID and content version. + itemID = vcSimCtx.ContentLibraryItemID + libMgr := library.NewManager(vcSimCtx.RestClient) + item, err := libMgr.GetLibraryItem(ctx, itemID) + Expect(err).ToNot(HaveOccurred()) + itemVersion = item.ContentVersion + + // Create one namespace for all the ordered tests. + nsInfo = vcSimCtx.CreateWorkloadNamespace() + }) + + BeforeEach(func() { + provider.VSphereClientFn = func(ctx context.Context) (*vsclient.Client, error) { + return vsclient.NewClient(ctx, vcSimCtx.VCClientConfig) + } + }) + + AfterEach(func() { + faker.reset() + }) + + AfterAll(func() { + vcSimCtx.AfterEach() + }) + + tableFn := func( + fn func() vmopv1.VirtualMachineImageCache, + expOVFRdy bool, expOVFMsg string, + expPrvRdy bool, expPrvMsg string, + expDskRdy bool, expDskMsg, expLocMsg string, + expRdyRdy bool, expRdyMsg string, + ) { + obj := fn() + Expect(vcSimCtx.Client.Create(ctx, &obj)).To(Succeed()) + key := ctrlclient.ObjectKey{ + Namespace: obj.Namespace, + Name: obj.Name, + } + + type expCondResult struct { + ready bool + msg string + } + + conditionTypesAndExpVals := map[string]expCondResult{ + cndOVFReady: { + ready: expOVFRdy, + msg: expOVFMsg, + }, + cndPrvReady: { + ready: expPrvRdy, + msg: expPrvMsg, + }, + cndDskReady: { + ready: expDskRdy, + msg: expDskMsg, + }, + cndRdyReady: { + ready: expRdyRdy, + msg: expRdyMsg, + }, + } + + Eventually(func(g Gomega) { + var obj vmopv1.VirtualMachineImageCache + g.Expect(vcSimCtx.Client.Get(ctx, key, &obj)).To(Succeed()) + + for t, v := range conditionTypesAndExpVals { + + switch { + case v.ready: + + // Verify the specified condition is true. + assertCondTrue(g, obj, t) + + switch t { + case cndOVFReady: + + // Verify the OVF is cached and ready. + assertOVFConfigMap(g, key) + + case cndDskReady: + + // Verify the disks are cached and ready. + g.Expect(obj.Status.Locations).To(HaveLen(1)) + assertCondTrue(g, obj.Status.Locations[0], cndRdyReady) + assertLocation(g, obj, 0) + + } + + case v.msg == "", + len(obj.Spec.Locations) == 0 && + (t == cndDskReady || t == cndPrvReady): + + // There will be no condition when the expected + // message is empty OR spec.locations is empty and + // the tested condition is DisksReady. + assertCondNil(g, obj, t) + + case v.msg != "": + + // Expect a failure condition. + assertCondFalse(g, obj, t, "Failed", v.msg) + + if t == cndDskReady && expLocMsg != "" { + assertCondFalse( + g, + obj.Status.Locations[0], + cndRdyReady, + "Failed", + expLocMsg) + } + } + } + + }, 5*time.Second, 1*time.Second).Should(Succeed()) + } + + DescribeTable("Failures", + tableFn, + + Entry( + "spec.providerID is empty", + func() vmopv1.VirtualMachineImageCache { + return getVMICacheObj( + nsInfo.Namespace, + "", + fakeString) + }, + false, "", // OVFReady + false, "", // ProviderReady + false, "", "", // DisksReady + false, "spec.providerID is empty", // Ready + ), + + Entry( + "spec.providerVersion is empty", + func() vmopv1.VirtualMachineImageCache { + return getVMICacheObj( + nsInfo.Namespace, + fakeString, + "") + }, + false, "", // OVFReady + false, "", // ProviderReady + false, "", "", // DisksReady + false, "spec.providerVersion is empty", // Ready + ), + + Entry( + "failure to get vSphere client", + func() vmopv1.VirtualMachineImageCache { + provider.VSphereClientFn = func(ctx context.Context) (*vsclient.Client, error) { + return nil, errors.New("fubar") + } + return getVMICacheObj( + nsInfo.Namespace, + fakeString, + fakeString) + }, + false, "", // OVFReady + false, "", // ProviderReady + false, "", "", // DisksReady + false, "failed to get vSphere client: fubar", // Ready + ), + + Entry( + "library item does not exist", + func() vmopv1.VirtualMachineImageCache { + return getVMICacheObj( + nsInfo.Namespace, + fakeString, + fakeString) + }, + false, "failed to create or patch ovf configmap: failed to retrieve ovf envelope:", // OVFReady + false, "", // ProviderReady + false, "", "", // DisksReady + false, "0 of 1 completed", // Ready + ), + + Entry( + "library item exists and location has invalid datacenter ID", + func() vmopv1.VirtualMachineImageCache { + return getVMICacheObj( + nsInfo.Namespace, + itemID, + itemVersion, + vmopv1.VirtualMachineImageCacheLocationSpec{ + DatacenterID: fakeString, + DatastoreID: vcSimCtx.Datastore.Reference().Value, + }) + }, + true, "", // OVFReady + true, "", // ProviderReady + false, "invalid datacenter ID", "", // DisksReady + false, "2 of 3 completed", // Ready + ), + + Entry( + "library item exists and location has invalid datastore ID", + func() vmopv1.VirtualMachineImageCache { + return getVMICacheObj( + nsInfo.Namespace, + itemID, + itemVersion, + vmopv1.VirtualMachineImageCacheLocationSpec{ + DatacenterID: vcSimCtx.Datacenter.Reference().Value, + DatastoreID: "invalid", + }) + }, + true, "", // OVFReady + true, "", // ProviderReady + false, "invalid datastore ID", "", // DisksReady + false, "2 of 3 completed", // Ready + ), + + Entry( + "cannot cache storage uris", + func() vmopv1.VirtualMachineImageCache { + + faker.fakeSRIClient = true + faker.queryVirtualDiskUuidFn = func( + context.Context, + string, + *object.Datacenter) (string, error) { + + return "", errors.New("query disk error") + } + + return getVMICacheObj( + nsInfo.Namespace, + itemID, + itemVersion, + vmopv1.VirtualMachineImageCacheLocationSpec{ + DatacenterID: vcSimCtx.Datacenter.Reference().Value, + DatastoreID: vcSimCtx.Datastore.Reference().Value, + }) + }, + true, "", // OVFReady + true, "", // ProviderReady + false, "0 of 1 completed", "failed to cache storage items: failed to query disk uuid: query disk error", // DisksReady + false, "2 of 3 completed", // Ready + ), + ) + + DescribeTable("Successes", + tableFn, + + Entry( + "no cache locations", + func() vmopv1.VirtualMachineImageCache { + return getVMICacheObj(nsInfo.Namespace, itemID, itemVersion) + }, + true, "", // OVFReady + false, "", // ProviderReady + false, "", "", // DisksReady + true, "", // Ready + ), + + Entry( + "a single cache location", + func() vmopv1.VirtualMachineImageCache { + return getVMICacheObj( + nsInfo.Namespace, + itemID, + itemVersion, + vmopv1.VirtualMachineImageCacheLocationSpec{ + DatacenterID: vcSimCtx.Datacenter.Reference().Value, + DatastoreID: vcSimCtx.Datastore.Reference().Value, + }) + }, + true, "", // OVFReady + true, "", // ProviderReady + true, "", "", // DisksReady + true, "", // Ready + ), + ) + }) + + }) + +type fakeClient struct { + fakeCLSProvdr bool + fakeSRIClient bool + + queryVirtualDiskUuidFn func( //nolint:revive,stylecheck + ctx context.Context, + name string, + datacenter *object.Datacenter) (string, error) + + copyVirtualDiskFn func( + ctx context.Context, + srcName string, srcDatacenter *object.Datacenter, + dstName string, dstDatacenter *object.Datacenter, + dstSpec vimtypes.BaseVirtualDiskSpec, force bool) (*object.Task, error) + + makeDirectoryFn func( + ctx context.Context, + name string, + datacenter *object.Datacenter, + createParentDirectories bool) error + + waitForTaskFn func( + ctx context.Context, task *object.Task) error + + getLibraryItemsFn func( + ctx context.Context, + libraryID string) ([]library.Item, error) + + getLibraryItemFn func( + ctx context.Context, + libraryID, + itemName string, + notFoundReturnErr bool) (*library.Item, error) + + getLibraryItemIDFn func( + ctx context.Context, + itemID string) (*library.Item, error) + + listLibraryItemsFn func( + ctx context.Context, + libraryID string) ([]string, error) + + updateLibraryItemFn func( + ctx context.Context, + itemID, + newName string, + newDescription *string) error + + retrieveOvfEnvelopeFromLibraryItemFn func( + ctx context.Context, + item *library.Item) (*ovf.Envelope, error) + + retrieveOvfEnvelopeByLibraryItemIDFn func( + ctx context.Context, + itemID string) (*ovf.Envelope, error) + + syncLibraryItemFn func( + ctx context.Context, + item *library.Item, + force bool) error + + listLibraryItemStorageFn func( + ctx context.Context, + itemID string) ([]library.Storage, error) + + resolveLibraryItemStorageFn func( + ctx context.Context, + datacenter *object.Datacenter, + storage []library.Storage) error + + createLibraryItemFn func( + ctx context.Context, + libraryItem library.Item, + path string) error +} + +func (m *fakeClient) reset() { + m.fakeCLSProvdr = false + m.fakeSRIClient = false + + m.queryVirtualDiskUuidFn = nil + m.copyVirtualDiskFn = nil + m.makeDirectoryFn = nil + m.waitForTaskFn = nil + m.getLibraryItemsFn = nil + m.getLibraryItemFn = nil + m.getLibraryItemIDFn = nil + m.listLibraryItemsFn = nil + m.updateLibraryItemFn = nil + m.retrieveOvfEnvelopeFromLibraryItemFn = nil + m.retrieveOvfEnvelopeByLibraryItemIDFn = nil + m.syncLibraryItemFn = nil + m.listLibraryItemStorageFn = nil + m.resolveLibraryItemStorageFn = nil + m.createLibraryItemFn = nil +} + +func (m *fakeClient) newContentLibraryProviderFn( + context.Context, + *rest.Client) clprov.Provider { + + if m.fakeCLSProvdr { + return m + } + return nil +} + +func (m *fakeClient) newCacheStorageURIsClientFn( + c *vim25.Client) clsutil.CacheStorageURIsClient { + + if m.fakeSRIClient { + return m + } + return nil +} + +func (m *fakeClient) QueryVirtualDiskUuid( //nolint:revive,stylecheck + ctx context.Context, + name string, + datacenter *object.Datacenter) (string, error) { + + if fn := m.queryVirtualDiskUuidFn; fn != nil { + return fn(ctx, name, datacenter) + } + return "", nil +} + +func (m *fakeClient) CopyVirtualDisk( + ctx context.Context, + srcName string, srcDatacenter *object.Datacenter, + dstName string, dstDatacenter *object.Datacenter, + dstSpec vimtypes.BaseVirtualDiskSpec, force bool) (*object.Task, error) { + + if fn := m.copyVirtualDiskFn; fn != nil { + return fn(ctx, srcName, srcDatacenter, dstName, dstDatacenter, dstSpec, force) + } + return nil, nil +} + +func (m *fakeClient) MakeDirectory( + ctx context.Context, + name string, + datacenter *object.Datacenter, + createParentDirectories bool) error { + + if fn := m.makeDirectoryFn; fn != nil { + return fn(ctx, name, datacenter, createParentDirectories) + } + return nil +} + +func (m *fakeClient) WaitForTask( + ctx context.Context, task *object.Task) error { + + if fn := m.waitForTaskFn; fn != nil { + return fn(ctx, task) + } + return nil +} + +func (m *fakeClient) GetLibraryItems( + ctx context.Context, + libraryID string) ([]library.Item, error) { + + if fn := m.getLibraryItemsFn; fn != nil { + return fn(ctx, libraryID) + } + return nil, nil +} + +func (m *fakeClient) GetLibraryItem( + ctx context.Context, + libraryID, + itemName string, + notFoundReturnErr bool) (*library.Item, error) { + + if fn := m.getLibraryItemFn; fn != nil { + return fn(ctx, libraryID, itemName, notFoundReturnErr) + } + return nil, nil +} + +func (m *fakeClient) GetLibraryItemID( + ctx context.Context, + itemID string) (*library.Item, error) { + + if fn := m.getLibraryItemIDFn; fn != nil { + return fn(ctx, itemID) + } + return nil, nil +} + +func (m *fakeClient) ListLibraryItems( + ctx context.Context, + libraryID string) ([]string, error) { + + if fn := m.listLibraryItemsFn; fn != nil { + return fn(ctx, libraryID) + } + return nil, nil +} + +func (m *fakeClient) UpdateLibraryItem( + ctx context.Context, + itemID, + newName string, + newDescription *string) error { + + if fn := m.updateLibraryItemFn; fn != nil { + return fn(ctx, itemID, newName, newDescription) + } + return nil +} + +func (m *fakeClient) RetrieveOvfEnvelopeFromLibraryItem( + ctx context.Context, + item *library.Item) (*ovf.Envelope, error) { + + if fn := m.retrieveOvfEnvelopeFromLibraryItemFn; fn != nil { + return fn(ctx, item) + } + return nil, nil +} + +func (m *fakeClient) RetrieveOvfEnvelopeByLibraryItemID( + ctx context.Context, + itemID string) (*ovf.Envelope, error) { + + if fn := m.retrieveOvfEnvelopeByLibraryItemIDFn; fn != nil { + return fn(ctx, itemID) + } + return nil, nil +} + +func (m *fakeClient) SyncLibraryItem( + ctx context.Context, + item *library.Item, + force bool) error { + + if fn := m.syncLibraryItemFn; fn != nil { + return fn(ctx, item, force) + } + return nil +} + +func (m *fakeClient) ListLibraryItemStorage( + ctx context.Context, + itemID string) ([]library.Storage, error) { + + if fn := m.listLibraryItemStorageFn; fn != nil { + return fn(ctx, itemID) + } + return nil, nil +} + +func (m *fakeClient) ResolveLibraryItemStorage( + ctx context.Context, + datacenter *object.Datacenter, + storage []library.Storage) error { + + if fn := m.resolveLibraryItemStorageFn; fn != nil { + return fn(ctx, datacenter, storage) + } + return nil +} + +func (m *fakeClient) CreateLibraryItem( + ctx context.Context, + item library.Item, + path string) error { + + if fn := m.createLibraryItemFn; fn != nil { + return fn(ctx, item, path) + } + return nil +} + +const ovfEnvelopeYAML = ` +diskSection: + disk: + - capacity: "30" + capacityAllocationUnits: byte * 2^20 + diskId: vmdisk1 + fileRef: file1 + format: http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized + populatedSize: 18743296 + info: Virtual disk information +networkSection: + info: The list of logical networks + network: + - description: The nat network + name: nat +references: +- href: ttylinux-pc_i486-16.1-disk1.vmdk + id: file1 + size: 10595840 +virtualSystem: + id: vm + info: A virtual machine + name: ttylinux-pc_i486-16.1 + operatingSystemSection: + id: 36 + info: The kind of installed guest operating system + osType: otherLinuxGuest + virtualHardwareSection: + - config: + - key: firmware + required: false + value: efi + - key: powerOpInfo.powerOffType + required: false + value: soft + - key: powerOpInfo.resetType + required: false + value: soft + - key: powerOpInfo.suspendType + required: false + value: soft + - key: tools.syncTimeWithHost + required: false + value: "true" + - key: tools.toolsUpgradePolicy + required: false + value: upgradeAtPowerCycle + id: null + info: Virtual hardware requirements + item: + - allocationUnits: hertz * 10^6 + description: Number of Virtual CPUs + elementName: 1 virtual CPU(s) + instanceID: "1" + resourceType: 3 + virtualQuantity: 1 + - allocationUnits: byte * 2^20 + description: Memory Size + elementName: 32MB of memory + instanceID: "2" + resourceType: 4 + virtualQuantity: 32 + - address: "0" + description: IDE Controller + elementName: ideController0 + instanceID: "3" + resourceType: 5 + - addressOnParent: "0" + elementName: disk0 + hostResource: + - ovf:/disk/vmdisk1 + instanceID: "4" + parent: "3" + resourceType: 17 + - addressOnParent: "1" + automaticAllocation: true + config: + - key: wakeOnLanEnabled + required: false + value: "false" + connection: + - nat + description: E1000 ethernet adapter on "nat" + elementName: ethernet0 + instanceID: "5" + resourceSubType: E1000 + resourceType: 10 + - automaticAllocation: false + elementName: video + instanceID: "6" + required: false + resourceType: 24 + - automaticAllocation: false + elementName: vmci + instanceID: "7" + required: false + resourceSubType: vmware.vmci + resourceType: 1 + system: + elementName: Virtual Hardware Family + instanceID: "0" + virtualSystemIdentifier: ttylinux-pc_i486-16.1 + virtualSystemType: vmx-09` diff --git a/go.mod b/go.mod index ece694f04..e383ad68b 100644 --- a/go.mod +++ b/go.mod @@ -34,7 +34,7 @@ require ( github.com/vmware-tanzu/vm-operator/external/tanzu-topology v0.0.0-00010101000000-000000000000 github.com/vmware-tanzu/vm-operator/pkg/backup/api v0.0.0-00010101000000-000000000000 github.com/vmware-tanzu/vm-operator/pkg/constants/testlabels v0.0.0-00010101000000-000000000000 - github.com/vmware/govmomi v0.47.0-alpha.0.0.20241219162111-46d5d8739f1e + github.com/vmware/govmomi v0.47.0-alpha.0.0.20250102163115-80de29b4051e golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc // * https://github.com/vmware-tanzu/vm-operator/security/dependabot/24 golang.org/x/text v0.21.0 diff --git a/go.sum b/go.sum index 3d98863af..516c375f0 100644 --- a/go.sum +++ b/go.sum @@ -116,8 +116,8 @@ github.com/vmware-tanzu/net-operator-api v0.0.0-20240523152550-862e2c4eb0e0 h1:y github.com/vmware-tanzu/net-operator-api v0.0.0-20240523152550-862e2c4eb0e0/go.mod h1:w6QJGm3crIA16ZIz1FVQXD2NVeJhOgGXxW05RbVTSTo= github.com/vmware-tanzu/nsx-operator/pkg/apis v0.0.0-20241112044858-9da8637c1b0d h1:z9lrzKVtNlujduv9BilzPxuge/LE2F0N1ms3TP4JZvw= github.com/vmware-tanzu/nsx-operator/pkg/apis v0.0.0-20241112044858-9da8637c1b0d/go.mod h1:Q4JzNkNMvjo7pXtlB5/R3oME4Nhah7fAObWgghVmtxk= -github.com/vmware/govmomi v0.47.0-alpha.0.0.20241219162111-46d5d8739f1e h1:xWk68LmieEMkyArGnvKLSn6APFUVbm/I5swqHX+3WGs= -github.com/vmware/govmomi v0.47.0-alpha.0.0.20241219162111-46d5d8739f1e/go.mod h1:bYwUHpGpisE4AOlDl5eph90T+cjJMIcKx/kaa5v5rQM= +github.com/vmware/govmomi v0.47.0-alpha.0.0.20250102163115-80de29b4051e h1:o49EvG0ToH1Oq6lg4fCOjFYoNi4K3K+5d9/yNmBstuY= +github.com/vmware/govmomi v0.47.0-alpha.0.0.20250102163115-80de29b4051e/go.mod h1:bYwUHpGpisE4AOlDl5eph90T+cjJMIcKx/kaa5v5rQM= github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= diff --git a/pkg/config/config.go b/pkg/config/config.go index 3590a7c7d..8a535a993 100644 --- a/pkg/config/config.go +++ b/pkg/config/config.go @@ -68,6 +68,11 @@ type Config struct { // Defaults to 10 seconds. PoweredOnVMHasIPRequeueDelay time.Duration + // SyncImageRequeueDelay is the requeue delay that is used to requeue an + // image that wants to be synced but is not yet ready. + // Defaults to 10 seconds. + SyncImageRequeueDelay time.Duration + NetworkProviderType NetworkProviderType VSphereNetworking bool LoadBalancerProvider string @@ -118,6 +123,23 @@ type Config struct { // // Defaults to false. AsyncCreateDisabled bool + + // FastDeployMode determines the default mode for Fast Deploy + // feature. + // + // Please note, this flag has no impact if the Fast Deploy feature is not + // enabled. + // + // The valid values are "direct" and "linked." If the FSS is enabled and: + // + // - the value is "direct," then the VM is deployed using cached disks. + // - the value is "linked," then the VM is deployed as a linked clone. + // - the value is empty, then "direct" mode is used. + // - the value is anything else, then fast deploy is not used to deploy + // VMs. + // + // Defaults to "direct." + FastDeployMode string } // GetMaxDeployThreadsOnProvider returns MaxDeployThreadsOnProvider if it is >0 @@ -148,8 +170,7 @@ type FeatureStates struct { VMIncrementalRestore bool // FSS_WCP_VMSERVICE_INCREMENTAL_RESTORE BringYourOwnEncryptionKey bool // FSS_WCP_VMSERVICE_BYOK SVAsyncUpgrade bool // FSS_WCP_SUPERVISOR_ASYNC_UPGRADE - // TODO(akutz) This FSS is placeholder. - FastDeploy bool // FSS_WCP_VMSERVICE_FAST_DEPLOY + FastDeploy bool // FSS_WCP_VMSERVICE_FAST_DEPLOY } type InstanceStorage struct { diff --git a/pkg/config/default.go b/pkg/config/default.go index d09dedca3..62e4a1c04 100644 --- a/pkg/config/default.go +++ b/pkg/config/default.go @@ -8,6 +8,7 @@ import ( "time" "github.com/vmware-tanzu/vm-operator/pkg" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" ) const defaultPrefix = "vmoperator-" @@ -40,8 +41,10 @@ func Default() Config { MaxConcurrentReconciles: 1, AsyncSignalDisabled: false, AsyncCreateDisabled: false, + FastDeployMode: pkgconst.FastDeployModeDirect, CreateVMRequeueDelay: 10 * time.Second, PoweredOnVMHasIPRequeueDelay: 10 * time.Second, + SyncImageRequeueDelay: 10 * time.Second, NetworkProviderType: NetworkProviderTypeNamed, PodName: defaultPrefix + "controller-manager", PodNamespace: defaultPrefix + "system", diff --git a/pkg/config/env.go b/pkg/config/env.go index 2ee88b852..26a91f6f7 100644 --- a/pkg/config/env.go +++ b/pkg/config/env.go @@ -23,6 +23,7 @@ func FromEnv() Config { setInt(env.MaxCreateVMsOnProvider, &config.MaxCreateVMsOnProvider) setDuration(env.CreateVMRequeueDelay, &config.CreateVMRequeueDelay) setDuration(env.PoweredOnVMHasIPRequeueDelay, &config.PoweredOnVMHasIPRequeueDelay) + setDuration(env.SyncImageRequeueDelay, &config.SyncImageRequeueDelay) setNetworkProviderType(env.NetworkProviderType, &config.NetworkProviderType) setString(env.LoadBalancerProvider, &config.LoadBalancerProvider) setBool(env.VSphereNetworking, &config.VSphereNetworking) @@ -30,6 +31,7 @@ func FromEnv() Config { setBool(env.LogSensitiveData, &config.LogSensitiveData) setBool(env.AsyncSignalDisabled, &config.AsyncSignalDisabled) setBool(env.AsyncCreateDisabled, &config.AsyncCreateDisabled) + setString(env.FastDeployMode, &config.FastDeployMode) setDuration(env.InstanceStoragePVPlacementFailedTTL, &config.InstanceStorage.PVPlacementFailedTTL) setFloat64(env.InstanceStorageJitterMaxFactor, &config.InstanceStorage.JitterMaxFactor) diff --git a/pkg/config/env/env.go b/pkg/config/env/env.go index b23c4dd70..a4b217910 100644 --- a/pkg/config/env/env.go +++ b/pkg/config/env/env.go @@ -18,6 +18,7 @@ const ( MaxCreateVMsOnProvider CreateVMRequeueDelay PoweredOnVMHasIPRequeueDelay + SyncImageRequeueDelay PrivilegedUsers NetworkProviderType LoadBalancerProvider @@ -27,6 +28,7 @@ const ( LogSensitiveData AsyncSignalDisabled AsyncCreateDisabled + FastDeployMode InstanceStoragePVPlacementFailedTTL InstanceStorageJitterMaxFactor InstanceStorageSeedRequeueDuration @@ -94,6 +96,8 @@ func (n VarName) String() string { return "CREATE_VM_REQUEUE_DELAY" case PoweredOnVMHasIPRequeueDelay: return "POWERED_ON_VM_HAS_IP_REQUEUE_DELAY" + case SyncImageRequeueDelay: + return "SYNC_IMAGE_REQUEUE_DELAY" case PrivilegedUsers: return "PRIVILEGED_USERS" case NetworkProviderType: @@ -112,6 +116,8 @@ func (n VarName) String() string { return "ASYNC_SIGNAL_DISABLED" case AsyncCreateDisabled: return "ASYNC_CREATE_DISABLED" + case FastDeployMode: + return "FAST_DEPLOY_MODE" case InstanceStoragePVPlacementFailedTTL: return "INSTANCE_STORAGE_PV_PLACEMENT_FAILED_TTL" case InstanceStorageJitterMaxFactor: diff --git a/pkg/config/env_test.go b/pkg/config/env_test.go index b03b6c2ab..860f5c762 100644 --- a/pkg/config/env_test.go +++ b/pkg/config/env_test.go @@ -13,6 +13,7 @@ import ( pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" "github.com/vmware-tanzu/vm-operator/pkg/config/env" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" ) var _ = Describe( @@ -79,6 +80,7 @@ var _ = Describe( Expect(os.Setenv("MAX_CONCURRENT_RECONCILES", "114")).To(Succeed()) Expect(os.Setenv("ASYNC_SIGNAL_DISABLED", "true")).To(Succeed()) Expect(os.Setenv("ASYNC_CREATE_DISABLED", "true")).To(Succeed()) + Expect(os.Setenv("FAST_DEPLOY_MODE", pkgconst.FastDeployModeLinked)).To(Succeed()) Expect(os.Setenv("LEADER_ELECTION_ID", "115")).To(Succeed()) Expect(os.Setenv("POD_NAME", "116")).To(Succeed()) Expect(os.Setenv("POD_NAMESPACE", "117")).To(Succeed()) @@ -104,6 +106,7 @@ var _ = Describe( Expect(os.Setenv("FSS_WCP_VMSERVICE_FAST_DEPLOY", "true")).To(Succeed()) Expect(os.Setenv("CREATE_VM_REQUEUE_DELAY", "125h")).To(Succeed()) Expect(os.Setenv("POWERED_ON_VM_HAS_IP_REQUEUE_DELAY", "126h")).To(Succeed()) + Expect(os.Setenv("SYNC_IMAGE_REQUEUE_DELAY", "127h")).To(Succeed()) }) It("Should return a default config overridden by the environment", func() { Expect(config).To(BeComparableTo(pkgcfg.Config{ @@ -128,6 +131,7 @@ var _ = Describe( MaxConcurrentReconciles: 114, AsyncSignalDisabled: true, AsyncCreateDisabled: true, + FastDeployMode: pkgconst.FastDeployModeLinked, LeaderElectionID: "115", PodName: "116", PodNamespace: "117", @@ -155,6 +159,7 @@ var _ = Describe( }, CreateVMRequeueDelay: 125 * time.Hour, PoweredOnVMHasIPRequeueDelay: 126 * time.Hour, + SyncImageRequeueDelay: 127 * time.Hour, })) }) }) diff --git a/pkg/constants/constants.go b/pkg/constants/constants.go index 193e6cfaf..325209c9f 100644 --- a/pkg/constants/constants.go +++ b/pkg/constants/constants.go @@ -38,4 +38,32 @@ const ( // MinSupportedHWVersionForPCIPassthruDevices is the supported virtual // hardware version for NVidia PCI devices. MinSupportedHWVersionForPCIPassthruDevices = vimtypes.VMX17 + + // VMICacheLabelKey is applied to resources that need to be reconciled when + // the VirtualMachineImageCache resource specified by the label's value is + // updated. + VMICacheLabelKey = "vmoperator.vmware.com/vmi-cache" + + // FastDeployAnnotationKey is applied to VirtualMachine resources that want + // to control the mode of FastDeploy used to create the underlying VM. + // Please note, this annotation only has any effect if the FastDeploy FSS is + // enabled. + // The valid values for this annotation are "direct" and "linked." If the + // FSS is enabled and: + // + // - the value is "direct," then the VM is deployed from cached disks. + // - the value is "linked," then the VM is deployed as a linked clone. + // - the value is empty or the annotation is not present, then the mode + // is derived from the environment variable FAST_DEPLOY_MODE. + // - the value is anything else, then fast deploy is not used to deploy + // the VM. + FastDeployAnnotationKey = "vmoperator.vmware.com/fast-deploy" + + // FastDeployModeDirect is a fast deploy mode. See FastDeployAnnotationKey + // for more information. + FastDeployModeDirect = "direct" + + // FastDeployModeLinked is a fast deploy mode. See FastDeployAnnotationKey + // for more information. + FastDeployModeLinked = "linked" ) diff --git a/pkg/context/clustercontentlibraryitem_context.go b/pkg/context/clustercontentlibraryitem_context.go deleted file mode 100644 index c11f4ce90..000000000 --- a/pkg/context/clustercontentlibraryitem_context.go +++ /dev/null @@ -1,28 +0,0 @@ -// © Broadcom. All Rights Reserved. -// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. -// SPDX-License-Identifier: Apache-2.0 - -package context - -import ( - "context" - "fmt" - - "github.com/go-logr/logr" - - imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" - vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" -) - -// ClusterContentLibraryItemContext is the context used for ClusterContentLibraryItem controller. -type ClusterContentLibraryItemContext struct { - context.Context - Logger logr.Logger - CCLItem *imgregv1a1.ClusterContentLibraryItem - CVMI *vmopv1.ClusterVirtualMachineImage - ImageObjName string -} - -func (c *ClusterContentLibraryItemContext) String() string { - return fmt.Sprintf("%s %s", c.CCLItem.GroupVersionKind(), c.CCLItem.Name) -} diff --git a/pkg/context/contentlibraryitem_context.go b/pkg/context/contentlibraryitem_context.go deleted file mode 100644 index 692727197..000000000 --- a/pkg/context/contentlibraryitem_context.go +++ /dev/null @@ -1,29 +0,0 @@ -// © Broadcom. All Rights Reserved. -// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. -// SPDX-License-Identifier: Apache-2.0 - -package context - -import ( - "context" - "fmt" - - "github.com/go-logr/logr" - - imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" - - vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" -) - -// ContentLibraryItemContext is the context used for ContentLibraryItem controller. -type ContentLibraryItemContext struct { - context.Context - Logger logr.Logger - CLItem *imgregv1a1.ContentLibraryItem - VMI *vmopv1.VirtualMachineImage - ImageObjName string -} - -func (c *ContentLibraryItemContext) String() string { - return fmt.Sprintf("%s %s/%s", c.CLItem.GroupVersionKind(), c.CLItem.Namespace, c.CLItem.Name) -} diff --git a/pkg/errors/errors_suite_test.go b/pkg/errors/errors_suite_test.go new file mode 100644 index 000000000..a1848f278 --- /dev/null +++ b/pkg/errors/errors_suite_test.go @@ -0,0 +1,17 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package errors_test + +import ( + "testing" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" +) + +func TestErrors(t *testing.T) { + RegisterFailHandler(Fail) + RunSpecs(t, "Errors Suite") +} diff --git a/pkg/errors/requeue_error.go b/pkg/errors/requeue_error.go new file mode 100644 index 000000000..1e5834626 --- /dev/null +++ b/pkg/errors/requeue_error.go @@ -0,0 +1,42 @@ +// // © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package errors + +import ( + "errors" + "fmt" + "time" + + ctrl "sigs.k8s.io/controller-runtime" +) + +// RequeueError may be returned from any part of a reconcile call stack and the +// controller should requeue the request. If After > 0 then the request is +// requeued with the provided value, otherwise the request is requeued +// immediately. +type RequeueError struct { + After time.Duration +} + +func (e RequeueError) Error() string { + if e.After == 0 { + return "requeue immediately" + } + return fmt.Sprintf("requeue after %s", e.After) +} + +// ResultFromError returns a ReconcileResult based on the provided error. If +// the error contains an embedded RequeueError, then it is used to influence +// the result. +func ResultFromError(err error) (ctrl.Result, error) { + var dst RequeueError + if err != nil && errors.As(err, &dst) { + if dst.After == 0 { + return ctrl.Result{Requeue: true}, nil + } + return ctrl.Result{RequeueAfter: dst.After}, nil + } + return ctrl.Result{}, err +} diff --git a/pkg/errors/requeue_error_test.go b/pkg/errors/requeue_error_test.go new file mode 100644 index 000000000..cb775b48d --- /dev/null +++ b/pkg/errors/requeue_error_test.go @@ -0,0 +1,85 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package errors_test + +import ( + "errors" + "fmt" + "time" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + + ctrl "sigs.k8s.io/controller-runtime" + + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" +) + +var _ = Describe("RequeueError", func() { + + DescribeTable("Error", + func(e error, expErr string) { + Expect(e).To(MatchError(expErr)) + }, + + Entry( + "after is 0", + pkgerr.RequeueError{}, + "requeue immediately", + ), + + Entry( + "after is 1s", + pkgerr.RequeueError{After: time.Second * 1}, + "requeue after 1s", + ), + ) + + DescribeTable("ResultFromError", + func(e error, expResult ctrl.Result, expErr string) { + res, resErr := pkgerr.ResultFromError(e) + Expect(res).To(Equal(expResult)) + if expErr == "" { + Expect(resErr).ToNot(HaveOccurred()) + } else { + Expect(resErr).To(MatchError(expErr)) + } + }, + + Entry( + "err is not RequeueError", + errors.New("hi"), + ctrl.Result{}, + "hi", + ), + + Entry( + "err is RequeueError", + pkgerr.RequeueError{}, + ctrl.Result{Requeue: true}, + "", + ), + + Entry( + "err is wrapped RequeueError", + fmt.Errorf("hi: %w", pkgerr.RequeueError{After: time.Second * 1}), + ctrl.Result{RequeueAfter: time.Second * 1}, + "", + ), + + Entry( + "err is RequeueError wrapped with multiple errors", + fmt.Errorf( + "hi: %w", + fmt.Errorf("there: %w, %w", + errors.New("hello"), + pkgerr.RequeueError{After: time.Minute * 1}, + ), + ), + ctrl.Result{RequeueAfter: time.Minute * 1}, + "", + ), + ) +}) diff --git a/pkg/errors/vmicache_not_ready_error.go b/pkg/errors/vmicache_not_ready_error.go new file mode 100644 index 000000000..0f21ca678 --- /dev/null +++ b/pkg/errors/vmicache_not_ready_error.go @@ -0,0 +1,46 @@ +// // © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package errors + +import ( + "errors" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" +) + +// VMICacheNotReadyError is returned from a method that cannot proceed until a +// VirtualMachineImageCache object is not ready. +type VMICacheNotReadyError struct { + // Name of the VirtualMachineImageCache object. + Name string +} + +func (e VMICacheNotReadyError) Error() string { + return "cache not ready" +} + +// WatchVMICacheIfNotReady adds a label to the provided object that allows it +// to be reconciled when a VMI Cache object is updated. This occurs when the +// provided error is or contains an ErrVMICacheNotReady error. +// This function returns true if a label was added to the object, otherwise +// false is returned. +func WatchVMICacheIfNotReady(err error, obj metav1.Object) bool { + if err == nil || obj == nil { + return false + } + var e VMICacheNotReadyError + if !errors.As(err, &e) { + return false + } + labels := obj.GetLabels() + if labels == nil { + labels = map[string]string{} + } + labels[pkgconst.VMICacheLabelKey] = e.Name + obj.SetLabels(labels) + return true +} diff --git a/pkg/errors/vmicache_not_ready_error_test.go b/pkg/errors/vmicache_not_ready_error_test.go new file mode 100644 index 000000000..74bd4f06b --- /dev/null +++ b/pkg/errors/vmicache_not_ready_error_test.go @@ -0,0 +1,98 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package errors_test + +import ( + "errors" + "fmt" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" +) + +var _ = Describe("VMICacheNotReadyError", func() { + + Context("Error", func() { + It("should return \"cache not ready\"", func() { + Expect(pkgerr.VMICacheNotReadyError{}.Error()).To(Equal("cache not ready")) + }) + }) + + DescribeTable("WatchVMICacheIfNotReady", + func(e error, obj metav1.Object, expResult bool, expName string) { + ok := pkgerr.WatchVMICacheIfNotReady(e, obj) + Expect(ok).To(Equal(expResult)) + if obj != nil { + labels := obj.GetLabels() + if ok { + Expect(labels).To(HaveKeyWithValue( + pkgconst.VMICacheLabelKey, expName)) + } else { + Expect(labels).ToNot(HaveKey(pkgconst.VMICacheLabelKey)) + } + } + }, + + Entry( + "error is nil", + (error)(nil), + &corev1.ConfigMap{}, + false, + "", + ), + + Entry( + "object is nil", + errors.New("hi"), + (metav1.Object)(nil), + false, + "", + ), + + Entry( + "error is not VMICacheNotReadyError", + errors.New("hi"), + &corev1.ConfigMap{}, + false, + "", + ), + + Entry( + "error is VMICacheNotReadyError", + pkgerr.VMICacheNotReadyError{Name: "vmi-123"}, + &corev1.ConfigMap{}, + true, + "vmi-123", + ), + + Entry( + "err is wrapped VMICacheNotReadyError", + fmt.Errorf("hi: %w", pkgerr.VMICacheNotReadyError{Name: "vmi-123"}), + &corev1.ConfigMap{}, + true, + "vmi-123", + ), + + Entry( + "err is VMICacheNotReadyError wrapped with multiple errors", + fmt.Errorf( + "hi: %w", + fmt.Errorf("there: %w, %w", + errors.New("hello"), + pkgerr.VMICacheNotReadyError{Name: "vmi-123"}, + ), + ), + &corev1.ConfigMap{}, + true, + "vmi-123", + ), + ) +}) diff --git a/pkg/providers/vsphere/contentlibrary/content_library_provider.go b/pkg/providers/vsphere/contentlibrary/content_library_provider.go index 8f74f43c9..2dd88a777 100644 --- a/pkg/providers/vsphere/contentlibrary/content_library_provider.go +++ b/pkg/providers/vsphere/contentlibrary/content_library_provider.go @@ -17,8 +17,10 @@ import ( "k8s.io/apimachinery/pkg/util/wait" "github.com/go-logr/logr" + "github.com/vmware/govmomi/object" "github.com/vmware/govmomi/ovf" "github.com/vmware/govmomi/vapi/library" + "github.com/vmware/govmomi/vapi/library/finder" "github.com/vmware/govmomi/vapi/rest" "github.com/vmware/govmomi/vim25/soap" @@ -35,6 +37,9 @@ type Provider interface { UpdateLibraryItem(ctx context.Context, itemID, newName string, newDescription *string) error RetrieveOvfEnvelopeFromLibraryItem(ctx context.Context, item *library.Item) (*ovf.Envelope, error) RetrieveOvfEnvelopeByLibraryItemID(ctx context.Context, itemID string) (*ovf.Envelope, error) + SyncLibraryItem(ctx context.Context, item *library.Item, force bool) error + ListLibraryItemStorage(ctx context.Context, itemID string) ([]library.Storage, error) + ResolveLibraryItemStorage(ctx context.Context, datacenter *object.Datacenter, storage []library.Storage) error // TODO: Testing only. Remove these from this file. CreateLibraryItem(ctx context.Context, libraryItem library.Item, path string) error @@ -245,6 +250,16 @@ func (cs *provider) UpdateLibraryItem(ctx context.Context, itemID, newName strin return cs.libMgr.UpdateLibraryItem(ctx, item) } +// SyncLibraryItem issues a sync call against a subscribed library item, +// fetching its latest OVF and disks. +func (cs *provider) SyncLibraryItem( + ctx context.Context, + item *library.Item, + force bool) error { + + return cs.libMgr.SyncLibraryItem(ctx, item, force) +} + // Only used in testing. func (cs *provider) CreateLibraryItem(ctx context.Context, libraryItem library.Item, path string) error { log.Info("Creating Library Item", "item", libraryItem, "path", path) @@ -380,3 +395,29 @@ func (cs *provider) generateDownloadURLForLibraryItem( return url.Parse(fileURL) } + +func (cs *provider) ListLibraryItemStorage( + ctx context.Context, + itemID string) ([]library.Storage, error) { + + return cs.libMgr.ListLibraryItemStorage(ctx, itemID) +} + +func (cs *provider) ResolveLibraryItemStorage( + ctx context.Context, + datacenter *object.Datacenter, + storage []library.Storage) error { + + if err := finder.NewPathFinder( + cs.libMgr, + datacenter.Client()).ResolveLibraryItemStorage( + ctx, + datacenter, + nil, + storage); err != nil { + + return fmt.Errorf("failed to resolve library item storage URIs: %w", err) + } + + return nil +} diff --git a/pkg/providers/vsphere/contentlibrary/content_library_test.go b/pkg/providers/vsphere/contentlibrary/content_library_test.go index b8fcf311d..b03e1a6d4 100644 --- a/pkg/providers/vsphere/contentlibrary/content_library_test.go +++ b/pkg/providers/vsphere/contentlibrary/content_library_test.go @@ -119,13 +119,13 @@ func clTests() { libItem := library.Item{ Name: libItemName, Type: "ovf", - LibraryID: ctx.ContentLibraryID, + LibraryID: ctx.LocalContentLibraryID, } err = clProvider.CreateLibraryItem(ctx, libItem, ovfPath) Expect(err).NotTo(HaveOccurred()) - libItem2, err := clProvider.GetLibraryItem(ctx, ctx.ContentLibraryID, libItemName, true) + libItem2, err := clProvider.GetLibraryItem(ctx, ctx.LocalContentLibraryID, libItemName, true) Expect(err).ToNot(HaveOccurred()) Expect(libItem2).ToNot(BeNil()) Expect(libItem2.Name).To(Equal(libItem.Name)) diff --git a/pkg/providers/vsphere/placement/zone_placement.go b/pkg/providers/vsphere/placement/zone_placement.go index 791db4680..219fc9c1d 100644 --- a/pkg/providers/vsphere/placement/zone_placement.go +++ b/pkg/providers/vsphere/placement/zone_placement.go @@ -14,6 +14,7 @@ import ( "github.com/vmware/govmomi/object" "github.com/vmware/govmomi/property" "github.com/vmware/govmomi/vim25" + "github.com/vmware/govmomi/vim25/mo" vimtypes "github.com/vmware/govmomi/vim25/types" "golang.org/x/exp/maps" "k8s.io/apimachinery/pkg/util/sets" @@ -53,10 +54,10 @@ type Result struct { } type DatastoreResult struct { - Name string - MoRef vimtypes.ManagedObjectReference - URL string - TopLevelDirectoryCreateSupported bool + Name string + MoRef vimtypes.ManagedObjectReference + URL string + DiskFormats []string // ForDisk is false if the recommendation is for the VM's home directory and // true if for a disk. DiskKey is only valid if ForDisk is true. @@ -446,7 +447,7 @@ func Placement( if pkgcfg.FromContext(vmCtx).Features.FastDeploy { // Get the name and type of the datastores. - if err := getDatastoreNameAndType(vmCtx, vcClient, &rec); err != nil { + if err := getDatastoreProperties(vmCtx, vcClient, &rec); err != nil { return nil, err } } @@ -465,83 +466,49 @@ func Placement( return &result, nil } -func getDatastoreNameAndType( +func getDatastoreProperties( vmCtx pkgctx.VirtualMachineContext, vcClient *vim25.Client, rec *Recommendation) error { - var objSet []vimtypes.ObjectSpec - for i := range rec.Datastores { - d := rec.Datastores[i] - if d.Name == "" { - objSet = append(objSet, vimtypes.ObjectSpec{ - Obj: d.MoRef, - }) - } + if len(rec.Datastores) == 0 { + return nil } - if len(objSet) == 0 { - return nil + dsRefs := make([]vimtypes.ManagedObjectReference, len(rec.Datastores)) + for i := range rec.Datastores { + dsRefs[i] = rec.Datastores[i].MoRef } + var moDSs []mo.Datastore pc := property.DefaultCollector(vcClient) - res, err := pc.RetrieveProperties(vmCtx, vimtypes.RetrieveProperties{ - SpecSet: []vimtypes.PropertyFilterSpec{ - { - PropSet: []vimtypes.PropertySpec{ - { - Type: "Datastore", - PathSet: []string{ - "capability.topLevelDirectoryCreateSupported", - "info.url", - "name", - }, - }, - }, - ObjectSet: objSet, - }, + if err := pc.Retrieve( + vmCtx, + dsRefs, + []string{ + // TODO(akutz) There is a bug in GoVmomi that is preventing getting + // the property info.supportedVDiskFormats via the + // explicit path. Doug Mac is working on a fix, but + // until then, just get all of info. + // "info.supportedVDiskFormats", + // "info.url", + "info", + "name", }, - }) - if err != nil { - return fmt.Errorf("failed to get datastore names: %w", err) + &moDSs); err != nil { + + return fmt.Errorf("failed to get datastore properties: %w", err) } - for i := range res.Returnval { - r := res.Returnval[i] + for i := range moDSs { + moDS := moDSs[i] for j := range rec.Datastores { - if r.Obj == rec.Datastores[j].MoRef { - for k := range r.PropSet { - p := r.PropSet[k] - switch p.Name { - case "capability.topLevelDirectoryCreateSupported": - switch tVal := p.Val.(type) { - case bool: - rec.Datastores[j].TopLevelDirectoryCreateSupported = tVal - default: - return fmt.Errorf( - "datastore %[1]s is not bool: %[2]T, %+[2]v", - p.Name, p.Val) - } - case "info.url": - switch tVal := p.Val.(type) { - case string: - rec.Datastores[j].URL = tVal - default: - return fmt.Errorf( - "datastore %[1]s is not string: %[2]T, %+[2]v", - p.Name, p.Val) - } - case "name": - switch tVal := p.Val.(type) { - case string: - rec.Datastores[j].Name = tVal - default: - return fmt.Errorf( - "datastore %[1]s is not string: %[2]T, %+[2]v", - p.Name, p.Val) - } - } - } + ds := &rec.Datastores[j] + if moDS.Reference() == rec.Datastores[j].MoRef { + ds.Name = moDS.Name + dsInfo := moDS.Info.GetDatastoreInfo() + ds.DiskFormats = dsInfo.SupportedVDiskFormats + ds.URL = dsInfo.Url } } } diff --git a/pkg/providers/vsphere/placement/zone_placement_test.go b/pkg/providers/vsphere/placement/zone_placement_test.go index c50170c81..8c7d0957d 100644 --- a/pkg/providers/vsphere/placement/zone_placement_test.go +++ b/pkg/providers/vsphere/placement/zone_placement_test.go @@ -10,6 +10,7 @@ import ( . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" + "github.com/vmware/govmomi/simulator" vimtypes "github.com/vmware/govmomi/vim25/types" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/sets" @@ -124,6 +125,18 @@ func vcSimPlacement() { testConfig, initObjects...) + for _, dsEnt := range simulator.Map.All("Datastore") { + simulator.Map.WithLock( + simulator.SpoofContext(), + dsEnt.Reference(), + func() { + ds := simulator.Map.Get(dsEnt.Reference()).(*simulator.Datastore) + ds.Info.GetDatastoreInfo().SupportedVDiskFormats = []string{ + "native_512", "native_4k", + } + }) + } + nsInfo = ctx.CreateWorkloadNamespace() vm.Namespace = nsInfo.Namespace @@ -573,13 +586,13 @@ func vcSimPlacement() { Expect(result.Datastores[0].Name).ToNot(BeEmpty()) Expect(result.Datastores[0].MoRef).ToNot(BeZero()) Expect(result.Datastores[0].URL).ToNot(BeZero()) - Expect(result.Datastores[0].TopLevelDirectoryCreateSupported).To(BeTrue()) + Expect(result.Datastores[0].DiskFormats).ToNot(BeEmpty()) Expect(result.Datastores[1].ForDisk).To(BeTrue()) Expect(result.Datastores[1].DiskKey).ToNot(BeZero()) Expect(result.Datastores[1].Name).ToNot(BeEmpty()) Expect(result.Datastores[1].MoRef).ToNot(BeZero()) Expect(result.Datastores[1].URL).ToNot(BeZero()) - Expect(result.Datastores[1].TopLevelDirectoryCreateSupported).To(BeTrue()) + Expect(result.Datastores[1].DiskFormats).ToNot(BeEmpty()) }) Context("Only one zone exists", func() { @@ -601,7 +614,7 @@ func vcSimPlacement() { Expect(result.Datastores[0].Name).ToNot(BeEmpty()) Expect(result.Datastores[0].MoRef).ToNot(BeZero()) Expect(result.Datastores[0].URL).ToNot(BeZero()) - Expect(result.Datastores[0].TopLevelDirectoryCreateSupported).To(BeTrue()) + Expect(result.Datastores[0].DiskFormats).ToNot(BeEmpty()) }) }) }) diff --git a/pkg/providers/vsphere/virtualmachine/publish_test.go b/pkg/providers/vsphere/virtualmachine/publish_test.go index 25a1372bd..11910a43c 100644 --- a/pkg/providers/vsphere/virtualmachine/publish_test.go +++ b/pkg/providers/vsphere/virtualmachine/publish_test.go @@ -40,7 +40,7 @@ func publishTests() { vm = builder.DummyVirtualMachine() vm.Status.UniqueID = vcVM.Reference().Value - cl = builder.DummyContentLibrary("dummy-cl", "dummy-ns", ctx.ContentLibraryID) + cl = builder.DummyContentLibrary("dummy-cl", "dummy-ns", ctx.LocalContentLibraryID) vmPub = builder.DummyVirtualMachinePublishRequest("dummy-vmpub", "dummy-ns", vcVM.Name(), "dummy-item-name", "dummy-cl") vmPub.Status.SourceRef = &vmPub.Spec.Source diff --git a/pkg/providers/vsphere/vmlifecycle/create.go b/pkg/providers/vsphere/vmlifecycle/create.go index 0428e018e..ca3a487ee 100644 --- a/pkg/providers/vsphere/vmlifecycle/create.go +++ b/pkg/providers/vsphere/vmlifecycle/create.go @@ -6,7 +6,6 @@ package vmlifecycle import ( "github.com/vmware/govmomi/find" - "github.com/vmware/govmomi/object" "github.com/vmware/govmomi/vapi/rest" "github.com/vmware/govmomi/vim25" vimtypes "github.com/vmware/govmomi/vim25/types" @@ -22,20 +21,22 @@ type CreateArgs struct { ConfigSpec vimtypes.VirtualMachineConfigSpec StorageProvisioning string + DatacenterMoID string FolderMoID string ResourcePoolMoID string HostMoID string StorageProfileID string DatastoreMoID string // gce2e only: used only if StorageProfileID is unset Datastores []DatastoreRef + DiskPaths []string ZoneName string } type DatastoreRef struct { - Name string - MoRef vimtypes.ManagedObjectReference - URL string - TopLevelDirectoryCreateSupported bool + Name string + MoRef vimtypes.ManagedObjectReference + URL string + DiskFormats []string // ForDisk is false if the recommendation is for the VM's home directory and // true if for a disk. DiskKey is only valid if ForDisk is true. @@ -49,18 +50,10 @@ func CreateVirtualMachine( restClient *rest.Client, vimClient *vim25.Client, finder *find.Finder, - datacenter *object.Datacenter, createArgs *CreateArgs) (*vimtypes.ManagedObjectReference, error) { if createArgs.UseContentLibrary { - return deployFromContentLibrary( - vmCtx, - k8sClient, - restClient, - vimClient, - datacenter, - createArgs) + return deployFromContentLibrary(vmCtx, restClient, vimClient, createArgs) } - return cloneVMFromInventory(vmCtx, finder, createArgs) } diff --git a/pkg/providers/vsphere/vmlifecycle/create_contentlibrary.go b/pkg/providers/vsphere/vmlifecycle/create_contentlibrary.go index 8c6df7555..bf46c18dc 100644 --- a/pkg/providers/vsphere/vmlifecycle/create_contentlibrary.go +++ b/pkg/providers/vsphere/vmlifecycle/create_contentlibrary.go @@ -14,7 +14,6 @@ import ( "github.com/vmware/govmomi/vapi/vcenter" "github.com/vmware/govmomi/vim25" vimtypes "github.com/vmware/govmomi/vim25/types" - ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" @@ -117,10 +116,8 @@ func deployVMTX( func deployFromContentLibrary( vmCtx pkgctx.VirtualMachineContext, - k8sClient ctrlclient.Client, restClient *rest.Client, vimClient *vim25.Client, - datacenter *object.Datacenter, createArgs *CreateArgs) (*vimtypes.ManagedObjectReference, error) { // This call is needed to get the item type. We could avoid going to CL here, and @@ -134,14 +131,7 @@ func deployFromContentLibrary( switch item.Type { case library.ItemTypeOVF: if pkgcfg.FromContext(vmCtx).Features.FastDeploy { - return linkedCloneOVF( - vmCtx, - k8sClient, - vimClient, - restClient, - datacenter, - item, - createArgs) + return fastDeploy(vmCtx, vimClient, createArgs) } return deployOVF(vmCtx, restClient, item, createArgs) case library.ItemTypeVMTX: diff --git a/pkg/providers/vsphere/vmlifecycle/create_contentlibrary_linked_clone.go b/pkg/providers/vsphere/vmlifecycle/create_contentlibrary_linked_clone.go deleted file mode 100644 index 085258eb9..000000000 --- a/pkg/providers/vsphere/vmlifecycle/create_contentlibrary_linked_clone.go +++ /dev/null @@ -1,226 +0,0 @@ -// © Broadcom. All Rights Reserved. -// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. -// SPDX-License-Identifier: Apache-2.0 - -package vmlifecycle - -import ( - "context" - "errors" - "fmt" - "path" - - "github.com/vmware/govmomi/object" - "github.com/vmware/govmomi/vapi/library" - "github.com/vmware/govmomi/vapi/rest" - "github.com/vmware/govmomi/vim25" - vimtypes "github.com/vmware/govmomi/vim25/types" - ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" - - pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" - "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" - vmopv1util "github.com/vmware-tanzu/vm-operator/pkg/util/vmopv1" - clsutil "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/library" -) - -func linkedCloneOVF( - vmCtx pkgctx.VirtualMachineContext, - k8sClient ctrlclient.Client, - vimClient *vim25.Client, - restClient *rest.Client, - datacenter *object.Datacenter, - item *library.Item, - createArgs *CreateArgs) (*vimtypes.ManagedObjectReference, error) { - - logger := vmCtx.Logger.WithName("linkedCloneOVF") - - if len(createArgs.Datastores) == 0 { - return nil, errors.New("no compatible datastores") - } - - // Get the information required to do the linked clone. - imgInfo, err := getImageLinkedCloneInfo( - vmCtx, - k8sClient, - restClient, - item) - if err != nil { - return nil, err - } - - topLevelCacheDir, err := clsutil.GetTopLevelCacheDir( - vmCtx, - object.NewDatastoreNamespaceManager(vimClient), - datacenter, - object.NewDatastore(vimClient, createArgs.Datastores[0].MoRef), - createArgs.Datastores[0].Name, - createArgs.Datastores[0].URL, - createArgs.Datastores[0].TopLevelDirectoryCreateSupported) - if err != nil { - return nil, fmt.Errorf("failed to create top-level cache dir: %w", err) - } - logger.Info("Got top-level cache dir", "topLevelCacheDir", topLevelCacheDir) - - dstDir := clsutil.GetCacheDirForLibraryItem( - topLevelCacheDir, - imgInfo.ItemID, - imgInfo.ItemContentVersion) - logger.Info("Got item cache dir", "dstDir", dstDir) - - dstURIs, err := clsutil.CacheStorageURIs( - vmCtx, - newCacheStorageURIsClient(vimClient), - datacenter, - datacenter, - dstDir, - imgInfo.DiskURIs...) - if err != nil { - return nil, fmt.Errorf("failed to cache library item disks: %w", err) - } - logger.Info("Got parent disks", "dstURIs", dstURIs) - - vmDir := path.Dir(createArgs.ConfigSpec.Files.VmPathName) - logger.Info("Got vm dir", "vmDir", vmDir) - - // Update the ConfigSpec with the disk chains. - var disks []*vimtypes.VirtualDisk - for i := range createArgs.ConfigSpec.DeviceChange { - dc := createArgs.ConfigSpec.DeviceChange[i].GetVirtualDeviceConfigSpec() - if d, ok := dc.Device.(*vimtypes.VirtualDisk); ok { - disks = append(disks, d) - - // The profile is no longer needed since we have placement. - dc.Profile = nil - } - } - logger.Info("Got disks", "disks", disks) - - if a, b := len(dstURIs), len(disks); a != b { - return nil, fmt.Errorf( - "invalid disk count: len(uris)=%d, len(disks)=%d", a, b) - } - - for i := range disks { - d := disks[i] - if bfb, ok := d.Backing.(vimtypes.BaseVirtualDeviceFileBackingInfo); ok { - fb := bfb.GetVirtualDeviceFileBackingInfo() - fb.Datastore = &createArgs.Datastores[0].MoRef - fb.FileName = fmt.Sprintf("%s/%s-%d.vmdk", vmDir, vmCtx.VM.Name, i) - } - if fb, ok := d.Backing.(*vimtypes.VirtualDiskFlatVer2BackingInfo); ok { - fb.Parent = &vimtypes.VirtualDiskFlatVer2BackingInfo{ - VirtualDeviceFileBackingInfo: vimtypes.VirtualDeviceFileBackingInfo{ - Datastore: &createArgs.Datastores[0].MoRef, - FileName: dstURIs[i], - }, - DiskMode: string(vimtypes.VirtualDiskModePersistent), - ThinProvisioned: ptr.To(true), - } - } - } - - // The profile is no longer needed since we have placement. - createArgs.ConfigSpec.VmProfile = nil - - vmCtx.Logger.Info( - "Deploying OVF Library Item as linked clone", - "itemID", item.ID, - "itemName", item.Name, - "configSpec", createArgs.ConfigSpec) - - folder := object.NewFolder( - vimClient, - vimtypes.ManagedObjectReference{ - Type: "Folder", - Value: createArgs.FolderMoID, - }) - pool := object.NewResourcePool( - vimClient, - vimtypes.ManagedObjectReference{ - Type: "ResourcePool", - Value: createArgs.ResourcePoolMoID, - }) - - createTask, err := folder.CreateVM( - vmCtx, - createArgs.ConfigSpec, - pool, - nil) - if err != nil { - return nil, fmt.Errorf("failed to call create task: %w", err) - } - - createTaskInfo, err := createTask.WaitForResult(vmCtx) - if err != nil { - return nil, fmt.Errorf("failed to wait for create task: %w", err) - } - - vmRef, ok := createTaskInfo.Result.(vimtypes.ManagedObjectReference) - if !ok { - return nil, fmt.Errorf( - "failed to assert create task result is ref: %[1]T %+[1]v", - createTaskInfo.Result) - } - - return &vmRef, nil -} - -func getImageLinkedCloneInfo( - vmCtx pkgctx.VirtualMachineContext, - k8sClient ctrlclient.Client, - restClient *rest.Client, - item *library.Item) (vmopv1util.ImageDiskInfo, error) { - - imgInfo, err := vmopv1util.GetImageDiskInfo( - vmCtx, - k8sClient, - *vmCtx.VM.Spec.Image, - vmCtx.VM.Namespace) - - if err == nil { - return imgInfo, nil - } - - if !errors.Is(err, vmopv1util.ErrImageNotSynced) { - return vmopv1util.ImageDiskInfo{}, - fmt.Errorf("failed to get image info before syncing: %w", err) - } - - if err := clsutil.SyncLibraryItem( - vmCtx, - library.NewManager(restClient), - item.ID); err != nil { - - return vmopv1util.ImageDiskInfo{}, fmt.Errorf("failed to sync: %w", err) - } - - if imgInfo, err = vmopv1util.GetImageDiskInfo( - vmCtx, - k8sClient, - *vmCtx.VM.Spec.Image, - vmCtx.VM.Namespace); err != nil { - - return vmopv1util.ImageDiskInfo{}, - fmt.Errorf("failed to get image info after syncing: %w", err) - } - - return imgInfo, nil -} - -func newCacheStorageURIsClient(c *vim25.Client) clsutil.CacheStorageURIsClient { - return &cacheStorageURIsClient{ - FileManager: object.NewFileManager(c), - VirtualDiskManager: object.NewVirtualDiskManager(c), - } -} - -type cacheStorageURIsClient struct { - *object.FileManager - *object.VirtualDiskManager -} - -func (c *cacheStorageURIsClient) WaitForTask( - ctx context.Context, task *object.Task) error { - - return task.Wait(ctx) -} diff --git a/pkg/providers/vsphere/vmlifecycle/create_fastdeploy.go b/pkg/providers/vsphere/vmlifecycle/create_fastdeploy.go new file mode 100644 index 000000000..d7f4c36c9 --- /dev/null +++ b/pkg/providers/vsphere/vmlifecycle/create_fastdeploy.go @@ -0,0 +1,417 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package vmlifecycle + +import ( + "context" + "errors" + "fmt" + "path" + "strings" + "sync" + + "github.com/go-logr/logr" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/vim25" + vimtypes "github.com/vmware/govmomi/vim25/types" + + pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" + pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" + pkgutil "github.com/vmware-tanzu/vm-operator/pkg/util" + "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" +) + +func fastDeploy( + vmCtx pkgctx.VirtualMachineContext, + vimClient *vim25.Client, + createArgs *CreateArgs) (vmRef *vimtypes.ManagedObjectReference, retErr error) { + + logger := vmCtx.Logger.WithName("fastDeploy") + + if len(createArgs.Datastores) == 0 { + return nil, errors.New("no compatible datastores") + } + + vmDir := path.Dir(createArgs.ConfigSpec.Files.VmPathName) + logger.Info("Got vm dir", "vmDir", vmDir) + + srcDiskPaths := createArgs.DiskPaths + logger.Info("Got source disk paths", "srcDiskPaths", srcDiskPaths) + + dstDiskFormat := pkgutil.GetPreferredDiskFormat( + createArgs.Datastores[0].DiskFormats...) + logger.Info("Got destination disk format", "dstDiskFormat", dstDiskFormat) + + dstDiskPaths := make([]string, len(srcDiskPaths)) + for i := 0; i < len(dstDiskPaths); i++ { + dstDiskPaths[i] = fmt.Sprintf("%s/disk-%d.vmdk", vmDir, i) + } + logger.Info("Got destination disk paths", "dstDiskPaths", dstDiskPaths) + + // Collect the disks and remove the storage profile from them. + var ( + disks []*vimtypes.VirtualDisk + diskSpecs []*vimtypes.VirtualDeviceConfigSpec + ) + for i := range createArgs.ConfigSpec.DeviceChange { + dc := createArgs.ConfigSpec.DeviceChange[i].GetVirtualDeviceConfigSpec() + if d, ok := dc.Device.(*vimtypes.VirtualDisk); ok { + d.VirtualDiskFormat = string(dstDiskFormat) + disks = append(disks, d) + diskSpecs = append(diskSpecs, dc) + } + } + + if a, b := len(srcDiskPaths), len(disks); a != b { + logger.Info("Got disks", "disks", disks) + return nil, fmt.Errorf( + "invalid disk count: len(srcDiskPaths)=%d, len(disks)=%d", a, b) + } + + // Update the disks with their expected file names. + for i := range disks { + d := disks[i] + if bfb, ok := d.Backing.(vimtypes.BaseVirtualDeviceFileBackingInfo); ok { + fb := bfb.GetVirtualDeviceFileBackingInfo() + fb.Datastore = &createArgs.Datastores[0].MoRef + fb.FileName = dstDiskPaths[i] + } + } + logger.Info("Got disks", "disks", disks) + + datacenter := object.NewDatacenter( + vimClient, + vimtypes.ManagedObjectReference{ + Type: string(vimtypes.ManagedObjectTypesDatacenter), + Value: createArgs.DatacenterMoID, + }) + logger.Info("Got datacenter", "datacenter", datacenter.Reference()) + + // Create the directory where the VM will be created. + fm := object.NewFileManager(vimClient) + if err := fm.MakeDirectory(vmCtx, vmDir, datacenter, true); err != nil { + return nil, fmt.Errorf("failed to create vm dir %q: %w", vmDir, err) + } + + // If any error occurs after this point, the newly created VM directory and + // its contents need to be cleaned up. + defer func() { + if retErr == nil { + // Make sure the VM is no longer reconciled when the VMI Cache + // resource is updated. + delete(vmCtx.VM.Labels, pkgconst.VMICacheLabelKey) + + // Do not delete the VM directory if this function was successful. + return + } + + // Use a new context to ensure cleanup happens even if the context + // is cancelled. + ctx := context.Background() + + // Delete the VM directory and its contents. + t, err := fm.DeleteDatastoreFile(ctx, vmDir, datacenter) + if err != nil { + err = fmt.Errorf( + "failed to call delete api for vm dir %q: %w", vmDir, err) + if retErr == nil { + retErr = err + } else { + retErr = fmt.Errorf("%w,%w", err, retErr) + } + return + } + + // Wait for the delete call to return. + if err := t.Wait(ctx); err != nil { + err = fmt.Errorf("failed to delete vm dir %q: %w", vmDir, err) + if retErr == nil { + retErr = err + } else { + retErr = fmt.Errorf("%w,%w", err, retErr) + } + } + }() + + folder := object.NewFolder(vimClient, vimtypes.ManagedObjectReference{ + Type: "Folder", + Value: createArgs.FolderMoID, + }) + logger.Info("Got folder", "folder", folder.Reference()) + + pool := object.NewResourcePool(vimClient, vimtypes.ManagedObjectReference{ + Type: "ResourcePool", + Value: createArgs.ResourcePoolMoID, + }) + logger.Info("Got pool", "pool", pool.Reference()) + + // Determine the type of fast deploy operation. + fastDeployMode := vmCtx.VM.Annotations[pkgconst.FastDeployAnnotationKey] + if fastDeployMode == "" { + fastDeployMode = pkgcfg.FromContext(vmCtx).FastDeployMode + } + logger.Info( + "Deploying OVF Library Item with Fast Deploy", + "mode", fastDeployMode) + + if strings.EqualFold(fastDeployMode, "linked") { + return fastDeployLinked( + vmCtx, + folder, + pool, + createArgs.ConfigSpec, + createArgs.Datastores[0].MoRef, + disks, + diskSpecs, + srcDiskPaths) + } + + return fastDeployDirect( + vmCtx, + datacenter, + folder, + pool, + createArgs.ConfigSpec, + diskSpecs, + dstDiskFormat, + dstDiskPaths, + srcDiskPaths) +} + +func fastDeployLinked( + ctx context.Context, + folder *object.Folder, + pool *object.ResourcePool, + configSpec vimtypes.VirtualMachineConfigSpec, + datastoreRef vimtypes.ManagedObjectReference, + disks []*vimtypes.VirtualDisk, + diskSpecs []*vimtypes.VirtualDeviceConfigSpec, + srcDiskPaths []string) (*vimtypes.ManagedObjectReference, error) { + + logger := logr.FromContextOrDiscard(ctx).WithName("fastDeployLinked") + + // Linked clones do not fully support encryption, so remove the profile + // and any possible crypto information. + configSpec.VmProfile = nil + for i := range diskSpecs { + ds := diskSpecs[i] + ds.Profile = nil + if ds.Backing != nil { + ds.Backing.Crypto = nil + } + } + + for i := range disks { + fileBackingInfo := vimtypes.VirtualDeviceFileBackingInfo{ + Datastore: &datastoreRef, + FileName: srcDiskPaths[i], + } + switch tBack := disks[i].Backing.(type) { + case *vimtypes.VirtualDiskFlatVer2BackingInfo: + // Point the disk to its parent. + tBack.Parent = &vimtypes.VirtualDiskFlatVer2BackingInfo{ + VirtualDeviceFileBackingInfo: fileBackingInfo, + DiskMode: string(vimtypes.VirtualDiskModePersistent), + ThinProvisioned: ptr.To(true), + } + case *vimtypes.VirtualDiskSeSparseBackingInfo: + // Point the disk to its parent. + tBack.Parent = &vimtypes.VirtualDiskSeSparseBackingInfo{ + VirtualDeviceFileBackingInfo: fileBackingInfo, + DiskMode: string(vimtypes.VirtualDiskModePersistent), + } + case *vimtypes.VirtualDiskSparseVer2BackingInfo: + // Point the disk to its parent. + tBack.Parent = &vimtypes.VirtualDiskSparseVer2BackingInfo{ + VirtualDeviceFileBackingInfo: fileBackingInfo, + DiskMode: string(vimtypes.VirtualDiskModePersistent), + } + } + } + + return fastDeployCreateVM(ctx, logger, folder, pool, configSpec) +} + +func fastDeployDirect( + ctx context.Context, + datacenter *object.Datacenter, + folder *object.Folder, + pool *object.ResourcePool, + configSpec vimtypes.VirtualMachineConfigSpec, + diskSpecs []*vimtypes.VirtualDeviceConfigSpec, + diskFormat vimtypes.DatastoreSectorFormat, + dstDiskPaths, + srcDiskPaths []string) (*vimtypes.ManagedObjectReference, error) { + + logger := logr.FromContextOrDiscard(ctx).WithName("fastDeployDirect") + + // Copy each disk into the VM directory. + if err := fastDeployDirectCopyDisks( + ctx, + logger, + datacenter, + configSpec, + srcDiskPaths, + dstDiskPaths, + diskFormat); err != nil { + + return nil, err + } + + _, isVMEncrypted := configSpec.Crypto.(*vimtypes.CryptoSpecEncrypt) + + for i := range diskSpecs { + ds := diskSpecs[i] + + // Set the file operation to an empty string since the disk already + // exists. + ds.FileOperation = "" + + if isVMEncrypted { + // If the VM is to be encrypted, then the disks need to be updated + // so they are not marked as encrypted upon VM creation. This is + // because it is not possible to change the encryption state of VM + // disks when they are being attached. Instead the disks must be + // encrypted after they are attached to the VM. + ds.Profile = nil + if ds.Backing != nil { + ds.Backing.Crypto = nil + } + } + } + + return fastDeployCreateVM(ctx, logger, folder, pool, configSpec) +} + +func fastDeployCreateVM( + ctx context.Context, + logger logr.Logger, + folder *object.Folder, + pool *object.ResourcePool, + configSpec vimtypes.VirtualMachineConfigSpec) (*vimtypes.ManagedObjectReference, error) { + + logger.Info("Creating VM", "configSpec", vimtypes.ToString(configSpec)) + + createTask, err := folder.CreateVM( + ctx, + configSpec, + pool, + nil) + if err != nil { + return nil, fmt.Errorf("failed to call create task: %w", err) + } + + createTaskInfo, err := createTask.WaitForResult(ctx) + if err != nil { + return nil, fmt.Errorf("failed to wait for create task: %w", err) + } + + vmRefVal, ok := createTaskInfo.Result.(vimtypes.ManagedObjectReference) + if !ok { + return nil, fmt.Errorf( + "failed to assert create task result is ref: %[1]T %+[1]v", + createTaskInfo.Result) + } + + return &vmRefVal, nil +} + +func fastDeployDirectCopyDisks( + ctx context.Context, + logger logr.Logger, + datacenter *object.Datacenter, + configSpec vimtypes.VirtualMachineConfigSpec, + srcDiskPaths, + dstDiskPaths []string, + diskFormat vimtypes.DatastoreSectorFormat) error { + + var ( + wg sync.WaitGroup + copyDiskTasks = make([]*object.Task, len(srcDiskPaths)) + copyDiskErrs = make(chan error, len(srcDiskPaths)) + copyDiskSpec = vimtypes.FileBackedVirtualDiskSpec{ + VirtualDiskSpec: vimtypes.VirtualDiskSpec{ + AdapterType: string(vimtypes.VirtualDiskAdapterTypeLsiLogic), + DiskType: string(vimtypes.VirtualDiskTypeThin), + }, + // TODO(akutz) Remove the below commented out code. + // The disk is encrypted (or not) using the same settings as the VM. + //Crypto: configSpec.Crypto, + SectorFormat: string(diskFormat), + Profile: configSpec.VmProfile, + } + diskManager = object.NewVirtualDiskManager(datacenter.Client()) + ) + + for i := range srcDiskPaths { + s := srcDiskPaths[i] + d := dstDiskPaths[i] + + logger.Info( + "Copying disk", + "dstDiskPath", d, + "srcDiskPath", s, + "copyDiskSpec", copyDiskSpec) + + t, err := diskManager.CopyVirtualDisk( + ctx, + s, + datacenter, + d, + datacenter, + ©DiskSpec, + false) + if err != nil { + logger.Error(err, "failed to copy disk, cancelling other tasks") + + // Cancel any other outstanding disk copies. + for _, t := range copyDiskTasks { + if t != nil { + // Cancel the task using a background context to ensure it + // goes through. + _ = t.Cancel(context.Background()) + } + } + + logger.Info("waiting on other copy tasks to complete") + // Wait on any other outstanding disk copies to complete before + // returning to ensure the parent folder can be cleaned up. + wg.Wait() + logger.Info("waited on other copy tasks to complete") + + return fmt.Errorf("failed to call copy disk %q to %q: %w", s, d, err) + } + + wg.Add(1) + copyDiskTasks[i] = t + + go func() { + defer wg.Done() + if err := t.Wait(context.Background()); err != nil { + copyDiskErrs <- fmt.Errorf( + "failed to copy disk %q to %q: %w", s, d, err) + } + }() + } + + // Wait on all the disk copies to complete before proceeding. + go func() { + wg.Wait() + close(copyDiskErrs) + }() + var copyDiskErr error + for err := range copyDiskErrs { + if err != nil { + if copyDiskErr == nil { + copyDiskErr = err + } else { + copyDiskErr = fmt.Errorf("%w,%w", copyDiskErr, err) + } + } + } + + return copyDiskErr +} diff --git a/pkg/providers/vsphere/vmprovider.go b/pkg/providers/vsphere/vmprovider.go index 0d7d5966b..b56c8b553 100644 --- a/pkg/providers/vsphere/vmprovider.go +++ b/pkg/providers/vsphere/vmprovider.go @@ -13,20 +13,27 @@ import ( "sync" "sync/atomic" + "github.com/go-logr/logr" "github.com/vmware/govmomi/object" "github.com/vmware/govmomi/ovf" "github.com/vmware/govmomi/task" "github.com/vmware/govmomi/vapi/library" vimtypes "github.com/vmware/govmomi/vim25/types" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" apierrorsutil "k8s.io/apimachinery/pkg/util/errors" ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/yaml" imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" + pkgcnd "github.com/vmware-tanzu/vm-operator/pkg/conditions" pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" "github.com/vmware-tanzu/vm-operator/pkg/providers" vcclient "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/client" vcconfig "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/config" @@ -35,6 +42,7 @@ import ( "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/vcenter" "github.com/vmware-tanzu/vm-operator/pkg/record" "github.com/vmware-tanzu/vm-operator/pkg/topology" + "github.com/vmware-tanzu/vm-operator/pkg/util" "github.com/vmware-tanzu/vm-operator/pkg/util/ovfcache" vsclient "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/client" ) @@ -146,34 +154,139 @@ func (vs *vSphereVMProvider) clearAndLogoutVcClient(ctx context.Context) { } } -// SyncVirtualMachineImage syncs the vmi object with the OVF Envelope retrieved from the cli object. -func (vs *vSphereVMProvider) SyncVirtualMachineImage(ctx context.Context, cli, vmi ctrlclient.Object) error { - var itemID, contentVersion string - var itemType imgregv1a1.ContentLibraryItemType +// SyncVirtualMachineImage syncs the vmi object with the OVF Envelope retrieved +// from the content library item object. +func (vs *vSphereVMProvider) SyncVirtualMachineImage( + ctx context.Context, + cli, + vmi ctrlclient.Object) error { + + var ( + itemID string + itemVersion string + itemType imgregv1a1.ContentLibraryItemType + ) switch cli := cli.(type) { case *imgregv1a1.ContentLibraryItem: itemID = string(cli.Spec.UUID) - contentVersion = cli.Status.ContentVersion + itemVersion = cli.Status.ContentVersion itemType = cli.Status.Type case *imgregv1a1.ClusterContentLibraryItem: itemID = string(cli.Spec.UUID) - contentVersion = cli.Status.ContentVersion + itemVersion = cli.Status.ContentVersion itemType = cli.Status.Type default: return fmt.Errorf("unexpected content library item K8s object type %T", cli) } - logger := log.V(4).WithValues("vmiName", vmi.GetName(), "cliName", cli.GetName()) + logger := log.V(4).WithValues( + "vmiName", vmi.GetName(), + "cliName", cli.GetName()) // Exit early if the library item type is not an OVF. if itemType != imgregv1a1.ContentLibraryItemTypeOvf { - logger.Info("Skip syncing VMI content as the library item is not OVF", - "libraryItemType", itemType) + logger.Info( + "Skip syncing VMI content as the library item is not OVF", + "itemType", itemType) return nil } - ovfEnvelope, err := ovfcache.GetOVFEnvelope(ctx, itemID, contentVersion) + if pkgcfg.FromContext(ctx).Features.FastDeploy { + return vs.syncVirtualMachineImageFastDeploy(ctx, vmi, logger, itemID, itemVersion) + } + return vs.syncVirtualMachineImage(ctx, vmi, itemID, itemVersion) +} + +func (vs *vSphereVMProvider) syncVirtualMachineImageFastDeploy( + ctx context.Context, + vmi ctrlclient.Object, + logger logr.Logger, + itemID, + itemVersion string) error { + + // Create or patch the VMICache object. + vmiCache := vmopv1.VirtualMachineImageCache{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: pkgcfg.FromContext(ctx).PodNamespace, + Name: util.VMIName(itemID), + }, + } + if _, err := controllerutil.CreateOrPatch( + ctx, + vs.k8sClient, + &vmiCache, + func() error { + vmiCache.Spec.ProviderID = itemID + vmiCache.Spec.ProviderVersion = itemVersion + return nil + }); err != nil { + return fmt.Errorf( + "failed to createOrPatch image cache resource: %w", err) + } + + // Check if the OVF data is ready. + c := pkgcnd.Get(vmiCache, vmopv1.VirtualMachineImageCacheConditionOVFReady) + switch { + case c != nil && c.Status == metav1.ConditionFalse: + + return fmt.Errorf( + "failed to get ovf: %s: %w", + c.Message, + pkgerr.VMICacheNotReadyError{Name: vmiCache.Name}) + + case c == nil, + c.Status != metav1.ConditionTrue, + vmiCache.Status.OVF == nil, + vmiCache.Status.OVF.ProviderVersion != itemVersion: + + logger.V(4).Info( + "Skip sync VMI", + "vmiCache.ovfCond", c, + "vmiCache.status.ovf", vmiCache.Status.OVF, + "expectedContentVersion", itemVersion) + + return pkgerr.VMICacheNotReadyError{Name: vmiCache.Name} + } + + // Get the OVF data. + var ( + ovfConfigMap corev1.ConfigMap + ovfConfigMapKey = ctrlclient.ObjectKey{ + Namespace: vmiCache.Namespace, + Name: vmiCache.Status.OVF.ConfigMapName, + } + ) + if err := vs.k8sClient.Get( + ctx, + ovfConfigMapKey, + &ovfConfigMap); err != nil { + + return fmt.Errorf( + "failed to get ovf configmap: %w, %w", + err, + pkgerr.VMICacheNotReadyError{Name: vmiCache.Name}) + } + + var ovfEnvelope ovf.Envelope + if err := yaml.Unmarshal( + []byte(ovfConfigMap.Data["value"]), &ovfEnvelope); err != nil { + + return fmt.Errorf( + "failed to unmarshal ovf yaml into envelope: %w", err) + } + + contentlibrary.UpdateVmiWithOvfEnvelope(vmi, ovfEnvelope) + return nil +} + +func (vs *vSphereVMProvider) syncVirtualMachineImage( + ctx context.Context, + vmi ctrlclient.Object, + itemID, + itemVersion string) error { + + ovfEnvelope, err := ovfcache.GetOVFEnvelope(ctx, itemID, itemVersion) if err != nil { return fmt.Errorf("failed to get OVF envelope for library item %q: %w", itemID, err) } diff --git a/pkg/providers/vsphere/vmprovider_test.go b/pkg/providers/vsphere/vmprovider_test.go index 9bfa446cf..e4bbc5516 100644 --- a/pkg/providers/vsphere/vmprovider_test.go +++ b/pkg/providers/vsphere/vmprovider_test.go @@ -5,19 +5,25 @@ package vsphere_test import ( + "errors" "fmt" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" + ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/vmware/govmomi/vapi/library" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" + pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" "github.com/vmware-tanzu/vm-operator/pkg/providers" vsphere "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere" + "github.com/vmware-tanzu/vm-operator/pkg/util" "github.com/vmware-tanzu/vm-operator/test/builder" ) @@ -51,7 +57,7 @@ func cpuFreqTests() { }) } -func syncVirtualMachineImageTests() { +var _ = Describe("SyncVirtualMachineImage", func() { var ( ctx *builder.TestContextForVCSim testConfig builder.VCSimTestConfig @@ -77,7 +83,7 @@ func syncVirtualMachineImageTests() { }) When("content library item is not an OVF type", func() { - It("should exit early without updating VM Image status", func() { + It("should return early without updating VM Image status", func() { isoItem := &imgregv1a1.ContentLibraryItem{ Status: imgregv1a1.ContentLibraryItemStatus{ Type: imgregv1a1.ContentLibraryItemTypeIso, @@ -89,73 +95,362 @@ func syncVirtualMachineImageTests() { }) }) - When("content library item is an OVF type but it fails to get the OVF envelope", func() { - It("should return an error", func() { - ovfItem := &imgregv1a1.ContentLibraryItem{ - Spec: imgregv1a1.ContentLibraryItemSpec{ - // Use an invalid item ID to fail to get the OVF envelope. - UUID: "invalid-library-ID", - }, - Status: imgregv1a1.ContentLibraryItemStatus{ - Type: imgregv1a1.ContentLibraryItemTypeOvf, - }, - } - err := vmProvider.SyncVirtualMachineImage(ctx, ovfItem, &vmopv1.VirtualMachineImage{}) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring("failed to get OVF envelope for library item \"invalid-library-ID\"")) - }) - }) + When("content library item is an OVF type", func() { + // TODO(akutz) Promote this block when the FSS WCP_VMService_FastDeploy is + // removed. + When("FSS WCP_VMService_FastDeploy is enabled", func() { - When("content library item is an OVF type but the OVF envelope is nil", func() { - It("should return an error", func() { - libraryItem := library.Item{ - Name: "test-image-ovf-empty", - // Use a non-OVF type here to cause the retrieved OVF envelope to be nil. - Type: "empty", - LibraryID: ctx.ContentLibraryID, - } - itemID := builder.CreateContentLibraryItem( - ctx, - library.NewManager(ctx.RestClient), - libraryItem, - "", + var ( + err error + cli imgregv1a1.ContentLibraryItem + vmi vmopv1.VirtualMachineImage + vmic vmopv1.VirtualMachineImageCache + vmicm corev1.ConfigMap + createVMIC bool ) - Expect(itemID).NotTo(BeEmpty()) - ovfItem := &imgregv1a1.ContentLibraryItem{ - Spec: imgregv1a1.ContentLibraryItemSpec{ - UUID: types.UID(itemID), - }, - Status: imgregv1a1.ContentLibraryItemStatus{ - Type: imgregv1a1.ContentLibraryItemTypeOvf, - }, + BeforeEach(func() { + pkgcfg.UpdateContext(ctx, func(config *pkgcfg.Config) { + config.Features.FastDeploy = true + }) + + cli = imgregv1a1.ContentLibraryItem{ + Spec: imgregv1a1.ContentLibraryItemSpec{ + UUID: types.UID(ctx.ContentLibraryItemID), + }, + Status: imgregv1a1.ContentLibraryItemStatus{ + ContentVersion: "v1", + Type: imgregv1a1.ContentLibraryItemTypeOvf, + }, + } + + vmi = vmopv1.VirtualMachineImage{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "my-namespace", + Name: "my-vmi", + }, + } + + createVMIC = true + vmicName := util.VMIName(ctx.ContentLibraryItemID) + vmic = vmopv1.VirtualMachineImageCache{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: pkgcfg.FromContext(ctx).PodNamespace, + Name: vmicName, + }, + Status: vmopv1.VirtualMachineImageCacheStatus{ + OVF: &vmopv1.VirtualMachineImageCacheOVFStatus{ + ConfigMapName: vmicName, + ProviderVersion: "v1", + }, + Conditions: []metav1.Condition{ + { + Type: vmopv1.VirtualMachineImageCacheConditionOVFReady, + Status: metav1.ConditionTrue, + }, + }, + }, + } + + vmicm = corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: vmic.Namespace, + Name: vmic.Name, + }, + Data: map[string]string{ + "value": ovfEnvelopeYAML, + }, + } + Expect(ctx.Client.Create(ctx, &vmicm)).To(Succeed()) + }) + + JustBeforeEach(func() { + if createVMIC { + status := vmic.Status.DeepCopy() + Expect(ctx.Client.Create(ctx, &vmic)).To(Succeed()) + vmic.Status = *status + Expect(ctx.Client.Status().Update(ctx, &vmic)).To(Succeed()) + } + err = vmProvider.SyncVirtualMachineImage(ctx, &cli, &vmi) + }) + + When("it fails to createOrPatch the VMICache resource", func() { + // TODO(akutz) Add interceptors to the vcSim test context so + // this can be tested. + XIt("should return an error", func() { + + }) + }) + + assertVMICExists := func(namespace, name string) { + var ( + obj vmopv1.VirtualMachineImageCache + key = ctrlclient.ObjectKey{ + Namespace: namespace, + Name: name, + } + ) + ExpectWithOffset(1, ctx.Client.Get(ctx, key, &obj)).To(Succeed()) } - err := vmProvider.SyncVirtualMachineImage(ctx, ovfItem, &vmopv1.VirtualMachineImage{}) - Expect(err).To(HaveOccurred()) - Expect(err.Error()).To(ContainSubstring(fmt.Sprintf("OVF envelope is nil for library item %q", itemID))) - }) - }) - When("content library item is an OVF type with valid OVF envelope", func() { - It("should return success and update VM Image status accordingly", func() { - ovfItem := &imgregv1a1.ContentLibraryItem{ - Spec: imgregv1a1.ContentLibraryItemSpec{ - UUID: types.UID(ctx.ContentLibraryItemID), - }, - Status: imgregv1a1.ContentLibraryItemStatus{ - Type: imgregv1a1.ContentLibraryItemTypeOvf, - }, + assertVMICNotReady := func(err error, name string) { + var e pkgerr.VMICacheNotReadyError + ExpectWithOffset(1, errors.As(err, &e)).To(BeTrue()) + ExpectWithOffset(1, e.Name).To(Equal(name)) } - var vmi vmopv1.VirtualMachineImage - Expect(vmProvider.SyncVirtualMachineImage(ctx, ovfItem, &vmi)).To(Succeed()) - Expect(vmi.Status.Firmware).To(Equal("efi")) - Expect(vmi.Status.HardwareVersion).NotTo(BeNil()) - Expect(*vmi.Status.HardwareVersion).To(Equal(int32(9))) - Expect(vmi.Status.OSInfo.ID).To(Equal("36")) - Expect(vmi.Status.OSInfo.Type).To(Equal("otherLinuxGuest")) - Expect(vmi.Status.Disks).To(HaveLen(1)) - Expect(vmi.Status.Disks[0].Capacity.String()).To(Equal("30Mi")) - Expect(vmi.Status.Disks[0].Size.String()).To(Equal("18304Ki")) + + When("OVF condition is False", func() { + BeforeEach(func() { + vmic.Status.Conditions[0].Status = metav1.ConditionFalse + vmic.Status.Conditions[0].Message = "fubar" + }) + It("should return an error", func() { + Expect(err).To(MatchError("failed to get ovf: fubar: cache not ready")) + }) + }) + + When("OVF is not ready", func() { + When("condition is missing", func() { + BeforeEach(func() { + createVMIC = false + }) + It("should return ErrVMICacheNotReady", func() { + assertVMICExists(vmic.Namespace, vmic.Name) + assertVMICNotReady(err, vmic.Name) + }) + }) + When("condition is unknown", func() { + BeforeEach(func() { + vmic.Status.Conditions[0].Status = metav1.ConditionUnknown + }) + It("should return ErrVMICacheNotReady", func() { + assertVMICNotReady(err, vmic.Name) + }) + }) + When("status.ovf is nil", func() { + BeforeEach(func() { + vmic.Status.OVF = nil + }) + It("should return ErrVMICacheNotReady", func() { + assertVMICNotReady(err, vmic.Name) + }) + }) + When("status.ovf.providerVersion does not match expected version", func() { + BeforeEach(func() { + vmic.Status.OVF.ProviderVersion = "" + }) + It("should return ErrVMICacheNotReady", func() { + assertVMICNotReady(err, vmic.Name) + }) + }) + When("configmap is missing", func() { + BeforeEach(func() { + Expect(ctx.Client.Delete(ctx, &vmicm)).To(Succeed()) + }) + It("should return ErrVMICacheNotReady", func() { + assertVMICNotReady(err, vmic.Name) + }) + }) + }) + + When("OVF is ready", func() { + When("marshaled ovf data is invalid", func() { + BeforeEach(func() { + vmicm.Data["value"] = "invalid" + Expect(ctx.Client.Update(ctx, &vmicm)).To(Succeed()) + }) + It("should return an error", func() { + Expect(err).To(MatchError("failed to unmarshal ovf yaml into envelope: " + + "error unmarshaling JSON: while decoding JSON: " + + "json: cannot unmarshal string into Go value of type ovf.Envelope")) + }) + }) + When("marshaled ovf data is valid", func() { + It("should return success and update VM Image status accordingly", func() { + Expect(err).ToNot(HaveOccurred()) + Expect(vmi.Status.Firmware).To(Equal("efi")) + Expect(vmi.Status.HardwareVersion).NotTo(BeNil()) + Expect(*vmi.Status.HardwareVersion).To(Equal(int32(9))) + Expect(vmi.Status.OSInfo.ID).To(Equal("36")) + Expect(vmi.Status.OSInfo.Type).To(Equal("otherLinuxGuest")) + Expect(vmi.Status.Disks).To(HaveLen(1)) + Expect(vmi.Status.Disks[0].Capacity.String()).To(Equal("30Mi")) + Expect(vmi.Status.Disks[0].Size.String()).To(Equal("18304Ki")) + }) + }) + + }) + }) + + // TODO(akutz) Remove this block when the FSS WCP_VMService_FastDeploy is + // removed. + When("FSS WCP_VMService_FastDeploy is disabled", func() { + + BeforeEach(func() { + pkgcfg.UpdateContext(ctx, func(config *pkgcfg.Config) { + config.Features.FastDeploy = false + }) + }) + + When("it fails to get the OVF envelope", func() { + It("should return an error", func() { + cli := &imgregv1a1.ContentLibraryItem{ + Spec: imgregv1a1.ContentLibraryItemSpec{ + // Use an invalid item ID to fail to get the OVF envelope. + UUID: "invalid-library-ID", + }, + Status: imgregv1a1.ContentLibraryItemStatus{ + Type: imgregv1a1.ContentLibraryItemTypeOvf, + }, + } + err := vmProvider.SyncVirtualMachineImage(ctx, cli, &vmopv1.VirtualMachineImage{}) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("failed to get OVF envelope for library item \"invalid-library-ID\"")) + }) + }) + + When("OVF envelope is nil", func() { + It("should return an error", func() { + ovfItem := &imgregv1a1.ContentLibraryItem{ + Spec: imgregv1a1.ContentLibraryItemSpec{ + UUID: types.UID(ctx.ContentLibraryIsoItemID), + }, + Status: imgregv1a1.ContentLibraryItemStatus{ + Type: imgregv1a1.ContentLibraryItemTypeOvf, + }, + } + err := vmProvider.SyncVirtualMachineImage(ctx, ovfItem, &vmopv1.VirtualMachineImage{}) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring(fmt.Sprintf("OVF envelope is nil for library item %q", ctx.ContentLibraryIsoItemID))) + }) + }) + + When("there is a valid OVF envelope", func() { + It("should return success and update VM Image status accordingly", func() { + cli := &imgregv1a1.ContentLibraryItem{ + Spec: imgregv1a1.ContentLibraryItemSpec{ + UUID: types.UID(ctx.ContentLibraryItemID), + }, + Status: imgregv1a1.ContentLibraryItemStatus{ + Type: imgregv1a1.ContentLibraryItemTypeOvf, + }, + } + var vmi vmopv1.VirtualMachineImage + Expect(vmProvider.SyncVirtualMachineImage(ctx, cli, &vmi)).To(Succeed()) + Expect(vmi.Status.Firmware).To(Equal("efi")) + Expect(vmi.Status.HardwareVersion).NotTo(BeNil()) + Expect(*vmi.Status.HardwareVersion).To(Equal(int32(9))) + Expect(vmi.Status.OSInfo.ID).To(Equal("36")) + Expect(vmi.Status.OSInfo.Type).To(Equal("otherLinuxGuest")) + Expect(vmi.Status.Disks).To(HaveLen(1)) + Expect(vmi.Status.Disks[0].Capacity.String()).To(Equal("30Mi")) + Expect(vmi.Status.Disks[0].Size.String()).To(Equal("18304Ki")) + }) + }) }) }) -} +}) + +const ovfEnvelopeYAML = ` +diskSection: + disk: + - capacity: "30" + capacityAllocationUnits: byte * 2^20 + diskId: vmdisk1 + fileRef: file1 + format: http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized + populatedSize: 18743296 + info: Virtual disk information +networkSection: + info: The list of logical networks + network: + - description: The nat network + name: nat +references: +- href: ttylinux-pc_i486-16.1-disk1.vmdk + id: file1 + size: 10595840 +virtualSystem: + id: vm + info: A virtual machine + name: ttylinux-pc_i486-16.1 + operatingSystemSection: + id: 36 + info: The kind of installed guest operating system + osType: otherLinuxGuest + virtualHardwareSection: + - config: + - key: firmware + required: false + value: efi + - key: powerOpInfo.powerOffType + required: false + value: soft + - key: powerOpInfo.resetType + required: false + value: soft + - key: powerOpInfo.suspendType + required: false + value: soft + - key: tools.syncTimeWithHost + required: false + value: "true" + - key: tools.toolsUpgradePolicy + required: false + value: upgradeAtPowerCycle + id: null + info: Virtual hardware requirements + item: + - allocationUnits: hertz * 10^6 + description: Number of Virtual CPUs + elementName: 1 virtual CPU(s) + instanceID: "1" + resourceType: 3 + virtualQuantity: 1 + - allocationUnits: byte * 2^20 + description: Memory Size + elementName: 32MB of memory + instanceID: "2" + resourceType: 4 + virtualQuantity: 32 + - address: "0" + description: IDE Controller + elementName: ideController0 + instanceID: "3" + resourceType: 5 + - addressOnParent: "0" + elementName: disk0 + hostResource: + - ovf:/disk/vmdisk1 + instanceID: "4" + parent: "3" + resourceType: 17 + - addressOnParent: "1" + automaticAllocation: true + config: + - key: wakeOnLanEnabled + required: false + value: "false" + connection: + - nat + description: E1000 ethernet adapter on "nat" + elementName: ethernet0 + instanceID: "5" + resourceSubType: E1000 + resourceType: 10 + - automaticAllocation: false + elementName: video + instanceID: "6" + required: false + resourceType: 24 + - automaticAllocation: false + elementName: vmci + instanceID: "7" + required: false + resourceSubType: vmware.vmci + resourceType: 1 + system: + elementName: Virtual Hardware Family + instanceID: "0" + virtualSystemIdentifier: ttylinux-pc_i486-16.1 + virtualSystemType: vmx-09` diff --git a/pkg/providers/vsphere/vmprovider_vm.go b/pkg/providers/vsphere/vmprovider_vm.go index c46e0c80c..d07b33712 100644 --- a/pkg/providers/vsphere/vmprovider_vm.go +++ b/pkg/providers/vsphere/vmprovider_vm.go @@ -16,25 +16,31 @@ import ( "time" "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/ovf" "github.com/vmware/govmomi/pbm" "github.com/vmware/govmomi/pbm/types" "github.com/vmware/govmomi/property" "github.com/vmware/govmomi/vim25/mo" vimtypes "github.com/vmware/govmomi/vim25/types" + corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" apierrorsutil "k8s.io/apimachinery/pkg/util/errors" ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/yaml" imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" - "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgcnd "github.com/vmware-tanzu/vm-operator/pkg/conditions" pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" pkgctx "github.com/vmware-tanzu/vm-operator/pkg/context" ctxop "github.com/vmware-tanzu/vm-operator/pkg/context/operation" + pkgerr "github.com/vmware-tanzu/vm-operator/pkg/errors" "github.com/vmware-tanzu/vm-operator/pkg/providers" vcclient "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/client" "github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/clustermodules" @@ -49,7 +55,7 @@ import ( "github.com/vmware-tanzu/vm-operator/pkg/topology" pkgutil "github.com/vmware-tanzu/vm-operator/pkg/util" kubeutil "github.com/vmware-tanzu/vm-operator/pkg/util/kube" - "github.com/vmware-tanzu/vm-operator/pkg/util/ovfcache" + "github.com/vmware-tanzu/vm-operator/pkg/util/kube/cource" vmopv1util "github.com/vmware-tanzu/vm-operator/pkg/util/vmopv1" "github.com/vmware-tanzu/vm-operator/pkg/vmconfig" ) @@ -545,6 +551,9 @@ func (vs *vSphereVMProvider) getCreateArgs( } if pkgcfg.FromContext(vmCtx).Features.FastDeploy { + if err := vs.vmCreateGetSourceDiskPaths(vmCtx, vcClient, createArgs); err != nil { + return nil, err + } if err := vs.vmCreatePathNameFromDatastoreRecommendation(vmCtx, createArgs); err != nil { return nil, err } @@ -576,19 +585,18 @@ func (vs *vSphereVMProvider) createVirtualMachine( vcClient.RestClient(), vcClient.VimClient(), vcClient.Finder(), - vcClient.Datacenter(), &args.CreateArgs) if err != nil { ctx.Logger.Error(err, "CreateVirtualMachine failed") - conditions.MarkFalse( + pkgcnd.MarkFalse( ctx.VM, vmopv1.VirtualMachineConditionCreated, "Error", err.Error()) if pkgcfg.FromContext(ctx).Features.FastDeploy { - conditions.MarkFalse( + pkgcnd.MarkFalse( ctx.VM, vmopv1.VirtualMachineConditionPlacementReady, "Error", @@ -599,7 +607,7 @@ func (vs *vSphereVMProvider) createVirtualMachine( } ctx.VM.Status.UniqueID = moRef.Reference().Value - conditions.MarkTrue(ctx.VM, vmopv1.VirtualMachineConditionCreated) + pkgcnd.MarkTrue(ctx.VM, vmopv1.VirtualMachineConditionCreated) if pkgcfg.FromContext(ctx).Features.FastDeploy { if zoneName := args.ZoneName; zoneName != "" { @@ -631,7 +639,6 @@ func (vs *vSphereVMProvider) createVirtualMachineAsync( vcClient.RestClient(), vcClient.VimClient(), vcClient.Finder(), - vcClient.Datacenter(), &args.CreateArgs) if vimErr != nil { @@ -646,14 +653,14 @@ func (vs *vSphereVMProvider) createVirtualMachineAsync( func() error { if vimErr != nil { - conditions.MarkFalse( + pkgcnd.MarkFalse( ctx.VM, vmopv1.VirtualMachineConditionCreated, "Error", vimErr.Error()) if pkgcfg.FromContext(ctx).Features.FastDeploy { - conditions.MarkFalse( + pkgcnd.MarkFalse( ctx.VM, vmopv1.VirtualMachineConditionPlacementReady, "Error", @@ -673,7 +680,7 @@ func (vs *vSphereVMProvider) createVirtualMachineAsync( } ctx.VM.Status.UniqueID = moRef.Reference().Value - conditions.MarkTrue(ctx.VM, vmopv1.VirtualMachineConditionCreated) + pkgcnd.MarkTrue(ctx.VM, vmopv1.VirtualMachineConditionCreated) return nil }, @@ -812,7 +819,7 @@ func (vs *vSphereVMProvider) vmCreateDoPlacement( defer func() { if retErr != nil { - conditions.MarkFalse( + pkgcnd.MarkFalse( vmCtx.VM, vmopv1.VirtualMachineConditionPlacementReady, "NotReady", @@ -860,6 +867,7 @@ func (vs *vSphereVMProvider) vmCreateDoPlacement( } if pkgcfg.FromContext(vmCtx).Features.FastDeploy { + createArgs.DatacenterMoID = vcClient.Datacenter().Reference().Value createArgs.Datastores = make([]vmlifecycle.DatastoreRef, len(result.Datastores)) for i := range result.Datastores { createArgs.Datastores[i].DiskKey = result.Datastores[i].DiskKey @@ -867,7 +875,7 @@ func (vs *vSphereVMProvider) vmCreateDoPlacement( createArgs.Datastores[i].MoRef = result.Datastores[i].MoRef createArgs.Datastores[i].Name = result.Datastores[i].Name createArgs.Datastores[i].URL = result.Datastores[i].URL - createArgs.Datastores[i].TopLevelDirectoryCreateSupported = result.Datastores[i].TopLevelDirectoryCreateSupported + createArgs.Datastores[i].DiskFormats = result.Datastores[i].DiskFormats } } @@ -903,7 +911,7 @@ func (vs *vSphereVMProvider) vmCreateDoPlacement( } } - conditions.MarkTrue(vmCtx.VM, vmopv1.VirtualMachineConditionPlacementReady) + pkgcnd.MarkTrue(vmCtx.VM, vmopv1.VirtualMachineConditionPlacementReady) return nil } @@ -974,6 +982,130 @@ func (vs *vSphereVMProvider) vmCreateGetFolderAndRPMoIDs( return nil } +// vmCreateGetSourceDiskPaths gets paths to the source disk(s) used to create +// the VM. +func (vs *vSphereVMProvider) vmCreateGetSourceDiskPaths( + vmCtx pkgctx.VirtualMachineContext, + vcClient *vcclient.Client, + createArgs *VMCreateArgs) error { + + if len(createArgs.Datastores) == 0 { + return errors.New("no compatible datastores") + } + + var ( + datacenterID = vs.vcClient.Datacenter().Reference().Value + datastoreID = createArgs.Datastores[0].MoRef.Value + itemID = createArgs.ImageStatus.ProviderItemID + itemVersion = createArgs.ImageStatus.ProviderContentVersion + ) + + // Create/patch/get the VirtualMachineImageCache resource. + obj := vmopv1.VirtualMachineImageCache{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: pkgcfg.FromContext(vmCtx).PodNamespace, + Name: pkgutil.VMIName(itemID), + }, + } + if _, err := controllerutil.CreateOrPatch( + vmCtx, + vs.k8sClient, + &obj, + func() error { + obj.Spec.ProviderID = itemID + obj.Spec.ProviderVersion = itemVersion + obj.Spec.SetLocation(datacenterID, datastoreID) + return nil + }); err != nil { + return fmt.Errorf( + "failed to createOrPatch image cache resource: %w", err) + } + + // Check if the disks are cached. + for i := range obj.Status.Locations { + l := obj.Status.Locations[i] + if l.DatacenterID == datacenterID && l.DatastoreID == datastoreID { + if c := pkgcnd.Get(l, vmopv1.ReadyConditionType); c != nil { + switch c.Status { + case metav1.ConditionTrue: + + // Verify the disks are still cached. If the disks are + // found to no longer exist, a reconcile request is enqueued + // for the VMI cache object. + if vs.vmCreateGetSourceDiskPathsVerify( + vmCtx, + vcClient, + obj, + l.Files) { + + // The location has the cached disks. + createArgs.DiskPaths = l.Files + vmCtx.Logger.Info("got source disks", "disks", l.Files) + + // Make sure the VM is no longer reconciled when the VMI + // Cache resource is updated. + delete(vmCtx.VM.Labels, pkgconst.VMICacheLabelKey) + + return nil + } + + case metav1.ConditionFalse: + // The disks could not be cached at that location. + return fmt.Errorf("failed to cache disks: %s", c.Message) + } + } + } + } + + // If the disks are not yet cached, add a label to the VM that will cause + // it to be reconciled when the VMICache resource is updated. + // + // This label is removed once the disks are available. + if vmCtx.VM.Labels == nil { + vmCtx.VM.Labels = map[string]string{} + } + vmCtx.VM.Labels[pkgconst.VMICacheLabelKey] = obj.Name + + return errors.New("disks not yet available") +} + +// vmCreateGetSourceDiskPathsVerify verifies the provided disks are still +// available. If not, a reconcile request is enqueued for the VMI cache object. +func (vs *vSphereVMProvider) vmCreateGetSourceDiskPathsVerify( + vmCtx pkgctx.VirtualMachineContext, + vcClient *vcclient.Client, + obj vmopv1.VirtualMachineImageCache, + srcDiskPaths []string) bool { + + vdm := object.NewVirtualDiskManager(vcClient.VimClient()) + + for i := range srcDiskPaths { + s := srcDiskPaths[i] + if _, err := vdm.QueryVirtualDiskUuid( + vmCtx, + s, + vcClient.Datacenter()); err != nil { + + vmCtx.Logger.Error(err, "disk is invalid", "diskPath", s) + + chanSource := cource.FromContextWithBuffer( + vmCtx, "VirtualMachineImageCache", 100) + chanSource <- event.GenericEvent{ + Object: &vmopv1.VirtualMachineImageCache{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: obj.Namespace, + Name: obj.Name, + }, + }, + } + + return false + } + } + + return true +} + func (vs *vSphereVMProvider) vmCreateFixupConfigSpec( vmCtx pkgctx.VirtualMachineContext, vcClient *vcclient.Client, @@ -1161,7 +1293,7 @@ func (vs *vSphereVMProvider) vmCreateGetVirtualMachineImage( default: if !SkipVMImageCLProviderCheck { err := fmt.Errorf("unsupported image provider kind: %s", providerRef.Kind) - conditions.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionImageReady, "NotSupported", err.Error()) + pkgcnd.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionImageReady, "NotSupported", err.Error()) return err } // Testing only: we'll clone the source VM found in the Inventory. @@ -1233,20 +1365,20 @@ func (vs *vSphereVMProvider) vmCreateGetStoragePrereqs( // This will be true in WCP. if cfg.StorageClassRequired { err := fmt.Errorf("StorageClass is required but not specified") - conditions.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, "StorageClassRequired", err.Error()) + pkgcnd.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, "StorageClassRequired", err.Error()) return err } // Testing only for standalone gce2e. if cfg.Datastore == "" { err := fmt.Errorf("no Datastore provided in configuration") - conditions.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, "DatastoreNotFound", err.Error()) + pkgcnd.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, "DatastoreNotFound", err.Error()) return err } datastore, err := vcClient.Finder().Datastore(vmCtx, cfg.Datastore) if err != nil { - conditions.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, "DatastoreNotFound", err.Error()) + pkgcnd.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, "DatastoreNotFound", err.Error()) return fmt.Errorf("failed to find Datastore %s: %w", cfg.Datastore, err) } @@ -1256,7 +1388,7 @@ func (vs *vSphereVMProvider) vmCreateGetStoragePrereqs( vmStorage, err := storage.GetVMStorageData(vmCtx, vs.k8sClient) if err != nil { reason, msg := errToConditionReasonAndMessage(err) - conditions.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, reason, msg) + pkgcnd.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, reason, msg) return err } @@ -1264,14 +1396,14 @@ func (vs *vSphereVMProvider) vmCreateGetStoragePrereqs( provisioningType, err := virtualmachine.GetDefaultDiskProvisioningType(vmCtx, vcClient, vmStorageProfileID) if err != nil { reason, msg := errToConditionReasonAndMessage(err) - conditions.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, reason, msg) + pkgcnd.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady, reason, msg) return err } createArgs.Storage = vmStorage createArgs.StorageProvisioning = provisioningType createArgs.StorageProfileID = vmStorageProfileID - conditions.MarkTrue(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady) + pkgcnd.MarkTrue(vmCtx.VM, vmopv1.VirtualMachineConditionStorageReady) return nil } @@ -1283,7 +1415,7 @@ func (vs *vSphereVMProvider) vmCreateDoNetworking( networkSpec := vmCtx.VM.Spec.Network if networkSpec == nil || networkSpec.Disabled { - conditions.Delete(vmCtx.VM, vmopv1.VirtualMachineConditionNetworkReady) + pkgcnd.Delete(vmCtx.VM, vmopv1.VirtualMachineConditionNetworkReady) return nil } @@ -1295,12 +1427,12 @@ func (vs *vSphereVMProvider) vmCreateDoNetworking( nil, // Don't know the CCR yet (needed to resolve backings for NSX-T) networkSpec) if err != nil { - conditions.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionNetworkReady, "NotReady", err.Error()) + pkgcnd.MarkFalse(vmCtx.VM, vmopv1.VirtualMachineConditionNetworkReady, "NotReady", err.Error()) return err } createArgs.NetworkResults = results - conditions.MarkTrue(vmCtx.VM, vmopv1.VirtualMachineConditionNetworkReady) + pkgcnd.MarkTrue(vmCtx.VM, vmopv1.VirtualMachineConditionNetworkReady) return nil } @@ -1341,6 +1473,7 @@ func (vs *vSphereVMProvider) vmCreateGenConfigSpec( if pkgcfg.FromContext(vmCtx).Features.FastDeploy { if err := vs.vmCreateGenConfigSpecImage(vmCtx, createArgs); err != nil { + _ = pkgerr.WatchVMICacheIfNotReady(err, vmCtx.VM) return err } createArgs.ConfigSpec.VmProfile = []vimtypes.BaseVirtualMachineProfileSpec{ @@ -1383,28 +1516,90 @@ func (vs *vSphereVMProvider) vmCreateGenConfigSpec( func (vs *vSphereVMProvider) vmCreateGenConfigSpecImage( vmCtx pkgctx.VirtualMachineContext, - createArgs *VMCreateArgs) error { + createArgs *VMCreateArgs) (retErr error) { if createArgs.ImageStatus.Type != "OVF" { return nil } - if createArgs.ImageStatus.ProviderItemID == "" { + var ( + itemID = createArgs.ImageStatus.ProviderItemID + itemVersion = createArgs.ImageStatus.ProviderContentVersion + ) + + if itemID == "" { return errors.New("empty image provider item id") } - if createArgs.ImageStatus.ProviderContentVersion == "" { + if itemVersion == "" { return errors.New("empty image provider content version") } - ovf, err := ovfcache.GetOVFEnvelope( + var ( + vmiCache vmopv1.VirtualMachineImageCache + vmiCacheKey = ctrlclient.ObjectKey{ + Namespace: pkgcfg.FromContext(vmCtx).PodNamespace, + Name: pkgutil.VMIName(itemID), + } + ) + + // Get the VirtualMachineImageCache resource. + if err := vs.k8sClient.Get( vmCtx, - createArgs.ImageStatus.ProviderItemID, - createArgs.ImageStatus.ProviderContentVersion) - if err != nil { - return fmt.Errorf("failed to get ovf from cache: %w", err) + vmiCacheKey, + &vmiCache); err != nil { + + return fmt.Errorf( + "failed to get vmi cache object: %w, %w", + err, + pkgerr.VMICacheNotReadyError{Name: vmiCacheKey.Name}) + } + + // Check if the OVF data is ready. + c := pkgcnd.Get(vmiCache, vmopv1.VirtualMachineImageCacheConditionOVFReady) + switch { + case c != nil && c.Status == metav1.ConditionFalse: + + return fmt.Errorf( + "failed to get ovf: %s: %w", + c.Message, + pkgerr.VMICacheNotReadyError{Name: vmiCacheKey.Name}) + + case c == nil, + c.Status != metav1.ConditionTrue, + vmiCache.Status.OVF == nil, + vmiCache.Status.OVF.ProviderVersion != itemVersion: + + return pkgerr.VMICacheNotReadyError{Name: vmiCacheKey.Name} + } + + // Get the OVF data. + var ovfConfigMap corev1.ConfigMap + if err := vs.k8sClient.Get( + vmCtx, + ctrlclient.ObjectKey{ + Namespace: vmiCache.Namespace, + Name: vmiCache.Status.OVF.ConfigMapName, + }, + &ovfConfigMap); err != nil { + + return fmt.Errorf( + "failed to get ovf configmap: %w, %w", + err, + pkgerr.VMICacheNotReadyError{Name: vmiCacheKey.Name}) + } + + // Remove the label from the VM that causes it to get reconciled when the + // VMI Cache object is updated. + delete(vmCtx.VM.Labels, pkgconst.VMICacheLabelKey) + + var ovfEnvelope ovf.Envelope + if err := yaml.Unmarshal( + []byte(ovfConfigMap.Data["value"]), &ovfEnvelope); err != nil { + + return fmt.Errorf("failed to unmarshal ovf yaml into envelope: %w", err) } - ovfConfigSpec, err := ovf.ToConfigSpec() + ovfConfigSpec, err := ovfEnvelope.ToConfigSpec() if err != nil { return fmt.Errorf("failed to transform ovf to config spec: %w", err) } diff --git a/pkg/providers/vsphere/vmprovider_vm_utils.go b/pkg/providers/vsphere/vmprovider_vm_utils.go index 89bcaa58f..50d782f92 100644 --- a/pkg/providers/vsphere/vmprovider_vm_utils.go +++ b/pkg/providers/vsphere/vmprovider_vm_utils.go @@ -243,11 +243,15 @@ func GetVirtualMachineImageSpecAndStatus( } vmiNotReadyMessage := "VirtualMachineImage is not ready" + + // Mirror the image's ReadyConditionType into the VM's + // VirtualMachineConditionImageReady. conditions.SetMirror( vmCtx.VM, vmopv1.VirtualMachineConditionImageReady, obj.(conditions.Getter), conditions.WithFallbackValue(false, "NotReady", vmiNotReadyMessage)) + if conditions.IsFalse(vmCtx.VM, vmopv1.VirtualMachineConditionImageReady) { return nil, vmopv1.VirtualMachineImageSpec{}, diff --git a/pkg/providers/vsphere/vsphere_suite_test.go b/pkg/providers/vsphere/vsphere_suite_test.go index 009ae6490..b0f72d76b 100644 --- a/pkg/providers/vsphere/vsphere_suite_test.go +++ b/pkg/providers/vsphere/vsphere_suite_test.go @@ -25,7 +25,6 @@ var suite = builder.NewTestSuite() func vcSimTests() { Describe("CPUFreq", cpuFreqTests) - Describe("SyncVirtualMachineImage", syncVirtualMachineImageTests) Describe("ResourcePolicyTests", resourcePolicyTests) Describe("VirtualMachine", vmTests) Describe("VirtualMachineE2E", vmE2ETests) diff --git a/pkg/util/devices.go b/pkg/util/devices.go index f88755dd0..40f4ce443 100644 --- a/pkg/util/devices.go +++ b/pkg/util/devices.go @@ -202,3 +202,37 @@ func isNonRDMDisk(dev vimtypes.BaseVirtualDevice) bool { return false } + +// GetPreferredDiskFormat gets the preferred disk format. This function returns +// 4k if available, native 512 if available, an empty value if no formats are +// provided, else the first available format is returned. +func GetPreferredDiskFormat[T string | vimtypes.DatastoreSectorFormat]( + diskFormats ...T) vimtypes.DatastoreSectorFormat { + + if len(diskFormats) == 0 { + return "" + } + + var ( + supports4k bool + supportsNative512 bool + ) + + for i := range diskFormats { + switch diskFormats[i] { + case T(vimtypes.DatastoreSectorFormatNative_4k): + supports4k = true + case T(vimtypes.DatastoreSectorFormatNative_512): + supportsNative512 = true + } + } + + if supports4k { + return vimtypes.DatastoreSectorFormatNative_4k + } + if supportsNative512 { + return vimtypes.DatastoreSectorFormatNative_512 + } + + return vimtypes.DatastoreSectorFormat(diskFormats[0]) +} diff --git a/pkg/util/devices_test.go b/pkg/util/devices_test.go index 7df2ecbd2..84db2c00f 100644 --- a/pkg/util/devices_test.go +++ b/pkg/util/devices_test.go @@ -297,3 +297,76 @@ var _ = Describe("SelectDevicesByTypes", func() { }) }) }) + +var _ = Describe("GetPreferredDiskFormat", func() { + + DescribeTable("[]string", + func(in []string, exp vimtypes.DatastoreSectorFormat) { + Expect(util.GetPreferredDiskFormat(in...)).To(Equal(exp)) + }, + Entry( + "no available formats", + []string{}, + vimtypes.DatastoreSectorFormat(""), + ), + Entry( + "4kn is available", + []string{ + string(vimtypes.DatastoreSectorFormatEmulated_512), + string(vimtypes.DatastoreSectorFormatNative_512), + string(vimtypes.DatastoreSectorFormatNative_4k), + }, + vimtypes.DatastoreSectorFormatNative_4k, + ), + Entry( + "native 512 is available", + []string{ + string(vimtypes.DatastoreSectorFormatEmulated_512), + string(vimtypes.DatastoreSectorFormatNative_512), + }, + vimtypes.DatastoreSectorFormatNative_512, + ), + Entry( + "neither 4kn nor 512 are available", + []string{ + string(vimtypes.DatastoreSectorFormatEmulated_512), + }, + vimtypes.DatastoreSectorFormatEmulated_512, + ), + ) + + DescribeTable("[]vimtypes.DatastoreSectorFormat", + func(in []vimtypes.DatastoreSectorFormat, exp vimtypes.DatastoreSectorFormat) { + Expect(util.GetPreferredDiskFormat(in...)).To(Equal(exp)) + }, + Entry( + "no available formats", + []vimtypes.DatastoreSectorFormat{}, + vimtypes.DatastoreSectorFormat(""), + ), + Entry( + "4kn is available", + []vimtypes.DatastoreSectorFormat{ + vimtypes.DatastoreSectorFormatEmulated_512, + vimtypes.DatastoreSectorFormatNative_512, + vimtypes.DatastoreSectorFormatNative_4k, + }, + vimtypes.DatastoreSectorFormatNative_4k, + ), + Entry( + "native 512 is available", + []vimtypes.DatastoreSectorFormat{ + vimtypes.DatastoreSectorFormatEmulated_512, + vimtypes.DatastoreSectorFormatNative_512, + }, + vimtypes.DatastoreSectorFormatNative_512, + ), + Entry( + "neither 4kn nor 512 are available", + []vimtypes.DatastoreSectorFormat{ + vimtypes.DatastoreSectorFormatEmulated_512, + }, + vimtypes.DatastoreSectorFormatEmulated_512, + ), + ) +}) diff --git a/pkg/util/hash.go b/pkg/util/hash.go new file mode 100644 index 000000000..87312a93e --- /dev/null +++ b/pkg/util/hash.go @@ -0,0 +1,25 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package util + +import ( + "crypto/sha1" //nolint:gosec // used for creating safe names + "encoding/hex" + "io" + "strings" +) + +// SHA1Sum17 returns the first 17 characters of the base64-encoded, SHA1 +// sum created from the provided string. +func SHA1Sum17(s string) string { + h := sha1.New() //nolint:gosec // used for creating safe names + _, _ = io.WriteString(h, s) + return hex.EncodeToString(h.Sum(nil))[:17] +} + +// VMIName returns the VMI name for a given library item ID. +func VMIName(itemID string) string { + return "vmi-" + SHA1Sum17(strings.ReplaceAll(itemID, "-", "")) +} diff --git a/pkg/util/hash_test.go b/pkg/util/hash_test.go new file mode 100644 index 000000000..5626f9604 --- /dev/null +++ b/pkg/util/hash_test.go @@ -0,0 +1,34 @@ +// © Broadcom. All Rights Reserved. +// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. +// SPDX-License-Identifier: Apache-2.0 + +package util_test + +import ( + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + + "github.com/vmware-tanzu/vm-operator/pkg/util" +) + +var _ = DescribeTable("SHA1Sum17", + func(in, out string) { + Expect(util.SHA1Sum17(in)).To(Equal(out)) + }, + Entry("empty string", "", "da39a3ee5e6b4b0d3"), + Entry("a", "a", "86f7e437faa5a7fce"), + Entry("b", "b", "e9d71f5ee7c92d6dc"), + Entry("c", "c", "84a516841ba77a5b4"), + Entry("a/b/c", "a/b/c", "ee2be6da91ae87947"), +) + +var _ = DescribeTable("VMIName", + func(in, out string) { + Expect(util.VMIName(in)).To(Equal(out)) + }, + Entry("empty string", "", "vmi-da39a3ee5e6b4b0d3"), + Entry("a", "a", "vmi-86f7e437faa5a7fce"), + Entry("-b", "-b", "vmi-e9d71f5ee7c92d6dc"), + Entry("c-", "c-", "vmi-84a516841ba77a5b4"), + Entry("a/--b/-c", "a/--b-/c", "vmi-ee2be6da91ae87947"), +) diff --git a/pkg/util/vmopv1/image.go b/pkg/util/vmopv1/image.go index c54addae0..73453ef10 100644 --- a/pkg/util/vmopv1/image.go +++ b/pkg/util/vmopv1/image.go @@ -11,11 +11,18 @@ import ( "path" "strings" + "github.com/go-logr/logr" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/reconcile" imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" - "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgcnd "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgconst "github.com/vmware-tanzu/vm-operator/pkg/constants" pkgutil "github.com/vmware-tanzu/vm-operator/pkg/util" ) @@ -132,10 +139,10 @@ func GetImage( } func IsImageReady(img vmopv1.VirtualMachineImage) error { - if !conditions.IsTrue(&img, vmopv1.ReadyConditionType) { + if !pkgcnd.IsTrue(&img, vmopv1.ReadyConditionType) { return fmt.Errorf( "image condition is not ready: %v", - conditions.Get(&img, vmopv1.ReadyConditionType)) + pkgcnd.Get(&img, vmopv1.ReadyConditionType)) } if img.Spec.ProviderRef == nil || img.Spec.ProviderRef.Name == "" { return errors.New("image provider ref is empty") @@ -231,3 +238,108 @@ func GetStorageURIsForLibraryItemDisks( } return storageURIs, nil } + +// VirtualMachineImageCacheToItemMapper returns a mapper function used to +// enqueue reconcile requests for resources in response to an event on +// a VirtualMachineImageCache resource. +func VirtualMachineImageCacheToItemMapper( + ctx context.Context, + logger logr.Logger, + k8sClient ctrlclient.Client, + groupVersion schema.GroupVersion, + kind string) handler.MapFunc { + + if ctx == nil { + panic("context is nil") + } + if k8sClient == nil { + panic("k8sClient is nil") + } + if groupVersion.Empty() { + panic("groupVersion is empty") + } + if kind == "" { + panic("kind is empty") + } + + gvkString := groupVersion.WithKind(kind).String() + listGVK := groupVersion.WithKind(kind + "List") + + // For a given VirtualMachineImageCache, return reconcile requests for + // resources that have the label pkgconst.VMICacheLabelKey with a value set + // to the name of a VMI cache resource. + return func(ctx context.Context, o ctrlclient.Object) []reconcile.Request { + if ctx == nil { + panic("context is nil") + } + if o == nil { + panic("object is nil") + } + obj, ok := o.(*vmopv1.VirtualMachineImageCache) + if !ok { + panic(fmt.Sprintf("object is %T", o)) + } + + // Do not reconcile anything if the referred object is being deleted. + if obj.DeletionTimestamp != nil && !obj.DeletionTimestamp.IsZero() { + return nil + } + + // Do not reconcile anything unless the OVF for this VMI Cache resource + // is ready. + if !pkgcnd.IsTrue(obj, vmopv1.VirtualMachineImageCacheConditionOVFReady) { + return nil + } + + logger := logger.WithValues( + "name", o.GetName(), + "namespace", o.GetNamespace()) + logger.V(4).Info( + "Reconciling all resources referencing an VirtualMachineImageCache", + "resourceGVK", gvkString) + + list := unstructured.UnstructuredList{ + Object: map[string]interface{}{}, + } + list.SetGroupVersionKind(listGVK) + + // Find all the referrers. + if err := k8sClient.List( + ctx, + &list, + ctrlclient.MatchingLabels(map[string]string{ + pkgconst.VMICacheLabelKey: obj.Name, + })); err != nil { + + if !apierrors.IsNotFound(err) { + logger.Error( + err, + "Failed to list resources due to VirtualMachineImageCache watch", + "resourceGVK", gvkString) + } + return nil + } + + // Populate reconcile requests for referrers. + var requests []reconcile.Request + for i := range list.Items { + requests = append( + requests, + reconcile.Request{ + NamespacedName: ctrlclient.ObjectKey{ + Namespace: list.Items[i].GetNamespace(), + Name: list.Items[i].GetName(), + }, + }) + } + + if len(requests) > 0 { + logger.V(4).Info( + "Reconciling resources due to VirtualMachineImageCache watch", + "resourceGVK", gvkString, + "requests", requests) + } + + return requests + } +} diff --git a/pkg/util/vmopv1/image_test.go b/pkg/util/vmopv1/image_test.go index 62f4a5aa7..25481ed71 100644 --- a/pkg/util/vmopv1/image_test.go +++ b/pkg/util/vmopv1/image_test.go @@ -6,19 +6,28 @@ package vmopv1_test import ( "context" + "errors" "fmt" + "github.com/go-logr/logr" . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" + corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/resource" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" ctrlclient "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/interceptor" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/reconcile" imgregv1a1 "github.com/vmware-tanzu/image-registry-operator-api/api/v1alpha1" vmopv1 "github.com/vmware-tanzu/vm-operator/api/v1alpha3" vmopv1common "github.com/vmware-tanzu/vm-operator/api/v1alpha3/common" vmopv1util "github.com/vmware-tanzu/vm-operator/pkg/util/vmopv1" + "github.com/vmware-tanzu/vm-operator/test/builder" ) var _ = DescribeTable("IsImageReady", @@ -305,3 +314,338 @@ var _ = XDescribe("GetImage", func() { // TODO(akutz) Implement test }) + +var _ = Describe("VirtualMachineImageCacheToItemMapper", func() { + const ( + vmiCacheName = "my-vmi" + namespaceName = "fake" + ) + + var ( + ctx context.Context + logger logr.Logger + k8sClient ctrlclient.Client + groupVersion schema.GroupVersion + kind string + withObjs []ctrlclient.Object + withFuncs interceptor.Funcs + obj *vmopv1.VirtualMachineImageCache + mapFn handler.MapFunc + mapFnCtx context.Context + mapFnObj ctrlclient.Object + reqs []reconcile.Request + ) + BeforeEach(func() { + reqs = nil + withObjs = nil + withFuncs = interceptor.Funcs{} + + ctx = context.Background() + logger = logr.Discard() + mapFnCtx = ctx + groupVersion = schema.GroupVersion{ + Group: "", + Version: "v1", + } + kind = "ConfigMap" + + obj = &vmopv1.VirtualMachineImageCache{ + ObjectMeta: metav1.ObjectMeta{ + Name: vmiCacheName, + Namespace: namespaceName, + }, + Status: vmopv1.VirtualMachineImageCacheStatus{ + Conditions: []metav1.Condition{ + { + Type: vmopv1.VirtualMachineImageCacheConditionOVFReady, + Status: metav1.ConditionTrue, + }, + }, + }, + } + mapFnObj = obj + }) + JustBeforeEach(func() { + withObjs = append(withObjs, obj) + k8sClient = builder.NewFakeClientWithInterceptors(withFuncs, withObjs...) + Expect(k8sClient).ToNot(BeNil()) + }) + When("panic is expected", func() { + When("ctx is nil", func() { + JustBeforeEach(func() { + ctx = nil + }) + It("should panic", func() { + Expect(func() { + _ = vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + logger, + k8sClient, + groupVersion, + kind) + }).To(PanicWith("context is nil")) + }) + }) + When("k8sClient is nil", func() { + JustBeforeEach(func() { + k8sClient = nil + }) + It("should panic", func() { + Expect(func() { + _ = vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + logger, + k8sClient, + groupVersion, + kind) + }).To(PanicWith("k8sClient is nil")) + }) + }) + When("groupVersion is empty", func() { + JustBeforeEach(func() { + groupVersion = schema.GroupVersion{} + }) + It("should panic", func() { + Expect(func() { + _ = vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + logger, + k8sClient, + groupVersion, + kind) + }).To(PanicWith("groupVersion is empty")) + }) + }) + When("kind is empty", func() { + JustBeforeEach(func() { + kind = "" + }) + It("should panic", func() { + Expect(func() { + _ = vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + logger, + k8sClient, + groupVersion, + kind) + }).To(PanicWith("kind is empty")) + }) + }) + Context("mapFn", func() { + JustBeforeEach(func() { + mapFn = vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + logger, + k8sClient, + groupVersion, + kind) + Expect(mapFn).ToNot(BeNil()) + }) + When("ctx is nil", func() { + BeforeEach(func() { + mapFnCtx = nil + }) + It("should panic", func() { + Expect(func() { + _ = mapFn(mapFnCtx, mapFnObj) + }).To(PanicWith("context is nil")) + }) + }) + When("object is nil", func() { + BeforeEach(func() { + mapFnObj = nil + }) + It("should panic", func() { + Expect(func() { + _ = mapFn(mapFnCtx, mapFnObj) + }).To(PanicWith("object is nil")) + }) + }) + When("object is invalid", func() { + BeforeEach(func() { + mapFnObj = &vmopv1.VirtualMachine{} + }) + It("should panic", func() { + Expect(func() { + _ = mapFn(mapFnCtx, mapFnObj) + }).To(PanicWith(fmt.Sprintf("object is %T", mapFnObj))) + }) + }) + }) + }) + When("panic is not expected", func() { + JustBeforeEach(func() { + mapFn = vmopv1util.VirtualMachineImageCacheToItemMapper( + ctx, + logger, + k8sClient, + groupVersion, + kind) + Expect(mapFn).ToNot(BeNil()) + reqs = mapFn(mapFnCtx, mapFnObj) + }) + When("there is an error listing resources", func() { + BeforeEach(func() { + withFuncs.List = func( + ctx context.Context, + client ctrlclient.WithWatch, + list ctrlclient.ObjectList, + opts ...ctrlclient.ListOption) error { + + if _, ok := list.(*corev1.ConfigMapList); ok { + return errors.New("fake") + } + return client.List(ctx, list, opts...) + } + }) + Specify("no reconcile requests should be returned", func() { + Expect(reqs).To(BeEmpty()) + }) + }) + When("there are no matching resources", func() { + BeforeEach(func() { + withObjs = append(withObjs, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "configmap-1", + }, + }, + ) + }) + Specify("no reconcile requests should be returned", func() { + Expect(reqs).To(BeEmpty()) + }) + }) + When("there is a single matching resource but ovf is not ready", func() { + BeforeEach(func() { + obj.Status.Conditions = nil + + withObjs = append(withObjs, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "configmap-1", + Labels: map[string]string{ + "vmoperator.vmware.com/vmi-cache": vmiCacheName, + }, + }, + }, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-2", + Name: "configmap-2", + }, + }, + ) + }) + Specify("no reconcile requests should be returned", func() { + Expect(reqs).To(BeEmpty()) + }) + }) + When("there is a single matching resource", func() { + BeforeEach(func() { + withObjs = append(withObjs, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "configmap-1", + Labels: map[string]string{ + "vmoperator.vmware.com/vmi-cache": vmiCacheName, + }, + }, + }, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-2", + Name: "configmap-2", + }, + }, + ) + }) + Specify("one reconcile request should be returned", func() { + Expect(reqs).To(ConsistOf( + reconcile.Request{ + NamespacedName: types.NamespacedName{ + Namespace: "ns-1", + Name: "configmap-1", + }, + }, + )) + }) + }) + When("there are multiple matching resources across multiple namespaces", func() { + BeforeEach(func() { + withObjs = append(withObjs, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "configmap-1", + }, + }, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-2", + Name: "configmap-2", + Labels: map[string]string{ + "vmoperator.vmware.com/vmi-cache": vmiCacheName, + }, + }, + }, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-3", + Name: "configmap-3", + }, + }, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-4", + Name: "configmap-4", + Labels: map[string]string{ + "vmoperator.vmware.com/vmi-cache": vmiCacheName, + }, + }, + }, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-5", + Name: "configmap-5", + Labels: map[string]string{ + "vmoperator.vmware.com/vmi-cache": vmiCacheName, + }, + }, + }, + &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-6", + Name: "configmap-6", + }, + }, + ) + }) + Specify("an equal number of requests should be returned", func() { + Expect(reqs).To(ConsistOf( + reconcile.Request{ + NamespacedName: types.NamespacedName{ + Namespace: "ns-2", + Name: "configmap-2", + }, + }, + reconcile.Request{ + NamespacedName: types.NamespacedName{ + Namespace: "ns-4", + Name: "configmap-4", + }, + }, + reconcile.Request{ + NamespacedName: types.NamespacedName{ + Namespace: "ns-5", + Name: "configmap-5", + }, + }, + )) + }) + }) + }) +}) diff --git a/pkg/util/vsphere/library/item_cache.go b/pkg/util/vsphere/library/item_cache.go index 18bea8c67..03957dbef 100644 --- a/pkg/util/vsphere/library/item_cache.go +++ b/pkg/util/vsphere/library/item_cache.go @@ -6,13 +6,10 @@ package library import ( "context" - "crypto/sha1" //nolint:gosec // used for creating directory name "fmt" - "io" "path" "strings" - "github.com/go-logr/logr" "github.com/vmware/govmomi/fault" "github.com/vmware/govmomi/object" vimtypes "github.com/vmware/govmomi/vim25/types" @@ -50,6 +47,7 @@ func CacheStorageURIs( client CacheStorageURIsClient, dstDatacenter, srcDatacenter *object.Datacenter, dstDir string, + dstDisksFormat vimtypes.DatastoreSectorFormat, srcDiskURIs ...string) ([]string, error) { if pkgutil.IsNil(ctx) { @@ -76,6 +74,7 @@ func CacheStorageURIs( client, dstDir, srcDiskURIs[i], + dstDisksFormat, dstDatacenter, srcDatacenter) if err != nil { @@ -91,6 +90,7 @@ func copyDisk( ctx context.Context, client CacheStorageURIsClient, dstDir, srcFilePath string, + dstDisksFormat vimtypes.DatastoreSectorFormat, dstDatacenter, srcDatacenter *object.Datacenter) (string, error) { var ( @@ -112,7 +112,7 @@ func copyDisk( return "", fmt.Errorf("failed to query disk uuid: %w", queryDiskErr) } - // Create the VM folder. + // Ensure the directory where the disks will be cached exists. if err := client.MakeDirectory( ctx, dstDir, @@ -134,6 +134,7 @@ func copyDisk( AdapterType: string(vimtypes.VirtualDiskAdapterTypeLsiLogic), DiskType: string(vimtypes.VirtualDiskTypeThin), }, + SectorFormat: string(dstDisksFormat), }, false) if err != nil { @@ -146,112 +147,9 @@ func copyDisk( return dstFilePath, nil } -const topLevelCacheDirName = ".contentlib-cache" - -// GetTopLevelCacheDirClient implements the client methods used by the -// GetTopLevelCacheDir method. -type GetTopLevelCacheDirClient interface { - CreateDirectory( - ctx context.Context, - datastore *object.Datastore, - displayName, policy string) (string, error) - - ConvertNamespacePathToUuidPath( - ctx context.Context, - datacenter *object.Datacenter, - datastoreURL string) (string, error) -} - -// GetTopLevelCacheDir returns the top-level cache directory at the root of the -// datastore. -// If the datastore uses vSAN, this function also ensures the top-level -// directory exists. -func GetTopLevelCacheDir( - ctx context.Context, - client GetTopLevelCacheDirClient, - dstDatacenter *object.Datacenter, - dstDatastore *object.Datastore, - dstDatastoreName, dstDatastoreURL string, - topLevelDirectoryCreateSupported bool) (string, error) { - - if pkgutil.IsNil(ctx) { - panic("context is nil") - } - if pkgutil.IsNil(client) { - panic("client is nil") - } - if dstDatacenter == nil { - panic("dstDatacenter is nil") - } - if dstDatastore == nil { - panic("dstDatastore is nil") - } - if dstDatastoreName == "" { - panic("dstDatastoreName is empty") - } - if dstDatastoreURL == "" { - panic("dstDatastoreURL is empty") - } - - logger := logr.FromContextOrDiscard(ctx).WithName("GetTopLevelCacheDir") - - logger.V(4).Info( - "Args", - "dstDatastoreName", dstDatastoreName, - "dstDatastoreURL", dstDatastoreURL, - "topLevelDirectoryCreateSupported", topLevelDirectoryCreateSupported) - - if topLevelDirectoryCreateSupported { - return fmt.Sprintf( - "[%s] %s", dstDatastoreName, topLevelCacheDirName), nil - } - - // TODO(akutz) Figure out a way to test if the directory already exists - // instead of trying to just create it again and using the - // FileAlreadyExists error as signal. - - dstDatastorePath, _ := strings.CutPrefix(dstDatastoreURL, "ds://") - topLevelCacheDirPath := path.Join(dstDatastorePath, topLevelCacheDirName) - - logger.V(4).Info( - "CreateDirectory", - "dstDatastorePath", dstDatastorePath, - "topLevelCacheDirPath", topLevelCacheDirPath) - - uuidTopLevelCacheDirPath, err := client.CreateDirectory( - ctx, - dstDatastore, - topLevelCacheDirPath, - "") - if err != nil { - if !fault.Is(err, &vimtypes.FileAlreadyExists{}) { - return "", fmt.Errorf("failed to create directory: %w", err) - } - - logger.V(4).Info( - "ConvertNamespacePathToUuidPath", - "dstDatacenter", dstDatacenter.Reference().Value, - "topLevelCacheDirPath", topLevelCacheDirPath) - - uuidTopLevelCacheDirPath, err = client.ConvertNamespacePathToUuidPath( - ctx, - dstDatacenter, - topLevelCacheDirPath) - if err != nil { - return "", fmt.Errorf( - "failed to convert namespace path=%q: %w", - topLevelCacheDirPath, err) - } - } - - logger.V(4).Info( - "Got absolute top level cache dir path", - "uuidTopLevelCacheDirPath", uuidTopLevelCacheDirPath) - - topLevelCacheDirName := path.Base(uuidTopLevelCacheDirPath) - - return fmt.Sprintf("[%s] %s", dstDatastoreName, topLevelCacheDirName), nil -} +// TopLevelCacheDirName is the name of the top-level cache directory created on +// each datastore. +const TopLevelCacheDirName = ".contentlib-cache" // GetCacheDirForLibraryItem returns the cache directory for a library item // beneath a top-level cache directory. @@ -267,6 +165,11 @@ func GetCacheDirForLibraryItem( if contentVersion == "" { panic("contentVersion is empty") } + + // Encode the contentVersion to ensure it is safe to use as part of the + // directory name. + contentVersion = pkgutil.SHA1Sum17(contentVersion) + return path.Join(topLevelCacheDir, itemUUID, contentVersion) } @@ -279,7 +182,5 @@ func GetCachedFileNameForVMDK(vmdkFileName string) string { if ext := path.Ext(vmdkFileName); ext != "" { vmdkFileName, _ = strings.CutSuffix(vmdkFileName, ext) } - h := sha1.New() //nolint:gosec // used for creating directory name - _, _ = io.WriteString(h, vmdkFileName) - return fmt.Sprintf("%x", h.Sum(nil))[0:17] + return pkgutil.SHA1Sum17(vmdkFileName) } diff --git a/pkg/util/vsphere/library/item_cache_test.go b/pkg/util/vsphere/library/item_cache_test.go index 0e3ecc19d..bd8e5e353 100644 --- a/pkg/util/vsphere/library/item_cache_test.go +++ b/pkg/util/vsphere/library/item_cache_test.go @@ -6,7 +6,6 @@ package library_test import ( "context" - "fmt" "sync/atomic" . "github.com/onsi/ginkgo/v2" @@ -108,7 +107,8 @@ var _ = Describe("CacheStorageURIs", func() { client, dstDatacenter, srcDatacenter, - dstDir) + dstDir, + vimtypes.DatastoreSectorFormatNative_512) } Expect(f).To(PanicWith(expPanic)) @@ -209,6 +209,7 @@ var _ = Describe("CacheStorageURIs", func() { dstDatacenter, srcDatacenter, dstDir, + vimtypes.DatastoreSectorFormatNative_512, srcDiskURIs...) }) @@ -318,254 +319,6 @@ var _ = Describe("CacheStorageURIs", func() { }) }) -type fakeGetTopLevelCacheDirClient struct { - createErr error - createResult string - createCalls int32 - - convertErr error - convertResult string - convertCalls int32 -} - -func (m *fakeGetTopLevelCacheDirClient) CreateDirectory( - ctx context.Context, - datastore *object.Datastore, - displayName, policy string) (string, error) { - - _ = atomic.AddInt32(&m.createCalls, 1) - return m.createResult, m.createErr -} - -func (m *fakeGetTopLevelCacheDirClient) ConvertNamespacePathToUuidPath( //nolint:revive,stylecheck - ctx context.Context, - datacenter *object.Datacenter, - datastoreURL string) (string, error) { - - _ = atomic.AddInt32(&m.convertCalls, 1) - return m.convertResult, m.convertErr -} - -var _ = Describe("GetTopLevelCacheDir", func() { - - var _ = DescribeTable("it should panic", - func( - ctx context.Context, - client clsutil.GetTopLevelCacheDirClient, - dstDatacenter *object.Datacenter, - dstDatastore *object.Datastore, - dstDatastoreName, - dstDatastoreURL string, - topLevelDirectoryCreateSupported bool, - expPanic string) { - - if ctx == nilContext { - ctx = nil - } - - f := func() { - _, _ = clsutil.GetTopLevelCacheDir( - ctx, - client, - dstDatacenter, - dstDatastore, - dstDatastoreName, - dstDatastoreURL, - topLevelDirectoryCreateSupported) - } - - Expect(f).To(PanicWith(expPanic)) - }, - - Entry( - "nil ctx", - nilContext, - &fakeGetTopLevelCacheDirClient{}, - &object.Datacenter{}, - &object.Datastore{}, - "my-datastore", - "ds://my-datastore", - false, - "context is nil", - ), - Entry( - "nil client", - context.Background(), - nil, - &object.Datacenter{}, - &object.Datastore{}, - "my-datastore", - "ds://my-datastore", - false, - "client is nil", - ), - Entry( - "nil dstDatacenter", - context.Background(), - &fakeGetTopLevelCacheDirClient{}, - nil, - &object.Datastore{}, - "my-datastore", - "ds://my-datastore", - false, - "dstDatacenter is nil", - ), - Entry( - "nil dstDatastore", - context.Background(), - &fakeGetTopLevelCacheDirClient{}, - &object.Datacenter{}, - nil, - "my-datastore", - "ds://my-datastore", - false, - "dstDatastore is nil", - ), - Entry( - "empty dstDatastoreName", - context.Background(), - &fakeGetTopLevelCacheDirClient{}, - &object.Datacenter{}, - &object.Datastore{}, - "", - "ds://my-datastore", - false, - "dstDatastoreName is empty", - ), - Entry( - "empty dstDatastoreURL", - context.Background(), - &fakeGetTopLevelCacheDirClient{}, - &object.Datacenter{}, - &object.Datastore{}, - "my-datastore", - "", - false, - "dstDatastoreURL is empty", - ), - ) - - var _ = When("it should not panic", func() { - - var ( - ctx context.Context - client *fakeGetTopLevelCacheDirClient - dstDatacenter *object.Datacenter - dstDatastore *object.Datastore - dstDatastoreName string - dstDatastoreURL string - topLevelDirectoryCreateSupported bool - - err error - out string - ) - - BeforeEach(func() { - ctx = context.Background() - client = &fakeGetTopLevelCacheDirClient{} - dstDatacenter = object.NewDatacenter( - nil, vimtypes.ManagedObjectReference{ - Type: "Datacenter", - Value: "datacenter-1", - }) - dstDatastore = object.NewDatastore( - nil, vimtypes.ManagedObjectReference{ - Type: "Datastore", - Value: "datastore-1", - }) - dstDatastoreName = "my-datastore" - dstDatastoreURL = "ds://my-datastore" - topLevelDirectoryCreateSupported = true - }) - - JustBeforeEach(func() { - out, err = clsutil.GetTopLevelCacheDir( - ctx, - client, - dstDatacenter, - dstDatastore, - dstDatastoreName, - dstDatastoreURL, - topLevelDirectoryCreateSupported) - }) - - When("datastore supports top-level directories", func() { - It("should return the expected path", func() { - Expect(err).ToNot(HaveOccurred()) - Expect(out).To(Equal("[my-datastore] .contentlib-cache")) - }) - }) - - When("datastore does not support top-level directories", func() { - BeforeEach(func() { - topLevelDirectoryCreateSupported = false - client.createResult = "ds://vmfs/volumes/123/abc" - }) - It("should return the expected path", func() { - Expect(client.createCalls).To(Equal(int32(1))) - Expect(client.convertCalls).To(BeZero()) - - Expect(err).ToNot(HaveOccurred()) - Expect(out).To(Equal("[my-datastore] abc")) - }) - - When("create directory returns an error", func() { - When("error is not FileAlreadyExists", func() { - BeforeEach(func() { - client.createErr = fmt.Errorf( - "nested %w", - soap.WrapVimFault(&vimtypes.RuntimeFault{})) - }) - - It("should return the error", func() { - Expect(client.createCalls).To(Equal(int32(1))) - Expect(client.convertCalls).To(BeZero()) - - Expect(err).To(HaveOccurred()) - Expect(err).To(MatchError(soap.WrapVimFault( - &vimtypes.RuntimeFault{}))) - }) - }) - When("error is FileAlreadyExists", func() { - BeforeEach(func() { - client.createErr = fmt.Errorf( - "nested %w", - soap.WrapVimFault(&vimtypes.FileAlreadyExists{})) - - client.convertResult = "abc" - }) - - It("should return the expected path", func() { - Expect(client.createCalls).To(Equal(int32(1))) - Expect(client.convertCalls).To(Equal(int32(1))) - - Expect(err).ToNot(HaveOccurred()) - Expect(out).To(Equal("[my-datastore] abc")) - }) - - When("convert path returns an error", func() { - BeforeEach(func() { - client.convertErr = fmt.Errorf( - "nested %w", - soap.WrapVimFault(&vimtypes.RuntimeFault{})) - }) - - It("should return the error", func() { - Expect(client.createCalls).To(Equal(int32(1))) - Expect(client.convertCalls).To(Equal(int32(1))) - - Expect(err).To(HaveOccurred()) - Expect(err).To(MatchError(soap.WrapVimFault( - &vimtypes.RuntimeFault{}))) - }) - }) - }) - }) - - }) - }) -}) - var _ = DescribeTable("GetCacheDirForLibraryItem", func(topLevelCacheDir, itemUUID, contentVersion, expOut, expPanic string) { var out string @@ -603,13 +356,13 @@ var _ = DescribeTable("GetCacheDirForLibraryItem", Entry( "absolute topLevelCacheDir", "/a", "b", "c", - "/a/b/c", + "/a/b/84a516841ba77a5b4", "", ), Entry( "relative topLevelCacheDir", "a", "b", "c", - "a/b/c", + "a/b/84a516841ba77a5b4", "", ), ) diff --git a/pkg/util/vsphere/library/item_sync.go b/pkg/util/vsphere/library/item_sync.go deleted file mode 100644 index 7b77b4a93..000000000 --- a/pkg/util/vsphere/library/item_sync.go +++ /dev/null @@ -1,58 +0,0 @@ -// © Broadcom. All Rights Reserved. -// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. -// SPDX-License-Identifier: Apache-2.0 - -package library - -import ( - "context" - "fmt" - - "github.com/go-logr/logr" - "github.com/vmware/govmomi/vapi/library" - - pkgutil "github.com/vmware-tanzu/vm-operator/pkg/util" -) - -// SyncLibraryItemClient implements the client methods used by the -// SyncLibraryItem method. -type SyncLibraryItemClient interface { - GetLibraryItem(ctx context.Context, id string) (*library.Item, error) - SyncLibraryItem(ctx context.Context, item *library.Item, force bool) error -} - -// SyncLibraryItem issues a sync call to the provided library item. -func SyncLibraryItem( - ctx context.Context, - client SyncLibraryItemClient, - itemID string) error { - - if pkgutil.IsNil(ctx) { - panic("context is nil") - } - if pkgutil.IsNil(client) { - panic("client is nil") - } - if itemID == "" { - panic("itemID is empty") - } - - logger := logr.FromContextOrDiscard(ctx) - - // A file from a library item that belongs to a subscribed library may not - // be fully available. Sync the file to ensure it is present. - logger.Info("Syncing content library item", "libraryItemID", itemID) - libItem, err := client.GetLibraryItem(ctx, itemID) - if err != nil { - return fmt.Errorf( - "error getting library item %s: %w", itemID, err) - } - if libItem.Type == "LOCAL" { - return nil - } - if err := client.SyncLibraryItem(ctx, libItem, true); err != nil { - return fmt.Errorf( - "error syncing library item %s: %w", itemID, err) - } - return nil -} diff --git a/pkg/util/vsphere/library/item_sync_test.go b/pkg/util/vsphere/library/item_sync_test.go deleted file mode 100644 index 0a07aff2d..000000000 --- a/pkg/util/vsphere/library/item_sync_test.go +++ /dev/null @@ -1,171 +0,0 @@ -// © Broadcom. All Rights Reserved. -// The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. -// SPDX-License-Identifier: Apache-2.0 - -package library_test - -import ( - "context" - "errors" - "fmt" - "sync/atomic" - - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" - - "github.com/go-logr/logr" - "github.com/vmware/govmomi/vapi/library" - - clsutil "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/library" -) - -type fakeSyncLibraryItemClient struct { - getLibItemItem *library.Item - getLibItemErr error - getLibItemCalls int32 - syncLibItemErr error - syncLibItemCalls int32 -} - -func (m *fakeSyncLibraryItemClient) GetLibraryItem( - ctx context.Context, - id string) (*library.Item, error) { - - _ = atomic.AddInt32(&m.getLibItemCalls, 1) - return m.getLibItemItem, m.getLibItemErr -} - -func (m *fakeSyncLibraryItemClient) SyncLibraryItem( - ctx context.Context, - item *library.Item, force bool) error { - - _ = atomic.AddInt32(&m.syncLibItemCalls, 1) - return m.syncLibItemErr -} - -var _ = Describe("SyncLibraryItem", func() { - - var ( - ctx context.Context - client *fakeSyncLibraryItemClient - itemID string - ) - - BeforeEach(func() { - ctx = context.Background() - client = &fakeSyncLibraryItemClient{} - itemID = "fake" - }) - - When("it should panic", func() { - When("context is nil", func() { - BeforeEach(func() { - ctx = nil - }) - It("should panic", func() { - Expect(func() { - _ = clsutil.SyncLibraryItem(ctx, client, itemID) - }).To(PanicWith("context is nil")) - }) - }) - When("client is nil", func() { - BeforeEach(func() { - client = nil - }) - It("should panic", func() { - client = nil - Expect(func() { - _ = clsutil.SyncLibraryItem(ctx, client, itemID) - }).To(PanicWith("client is nil")) - }) - }) - When("library item ID is empty", func() { - BeforeEach(func() { - itemID = "" - }) - It("should panic", func() { - Expect(func() { - _ = clsutil.SyncLibraryItem(ctx, client, itemID) - }).To(PanicWith("itemID is empty")) - }) - }) - }) - - When("it should not panic", func() { - - var ( - syncErr error - ) - - BeforeEach(func() { - ctx = logr.NewContext(context.Background(), GinkgoLogr) - }) - - JustBeforeEach(func() { - syncErr = clsutil.SyncLibraryItem(ctx, client, itemID) - }) - - When("item does not exist", func() { - BeforeEach(func() { - client.getLibItemErr = errors.New("404 Not Found") - }) - It("should 404", func() { - Expect(syncErr).To(HaveOccurred()) - Expect(syncErr).To(MatchError(fmt.Errorf( - "error getting library item %s: %w", itemID, - client.getLibItemErr))) - Expect(atomic.LoadInt32(&client.getLibItemCalls)).To(Equal(int32(1))) - - }) - }) - - When("item does exist", func() { - BeforeEach(func() { - client.getLibItemItem = &library.Item{ - ID: itemID, - Type: "LOCAL", - } - }) - When("the item is local", func() { - BeforeEach(func() { - client.getLibItemItem.Type = "LOCAL" - }) - It("should succeed without calling sync", func() { - Expect(syncErr).ToNot(HaveOccurred()) - Expect(atomic.LoadInt32(&client.getLibItemCalls)).To(Equal(int32(1))) - Expect(atomic.LoadInt32(&client.syncLibItemCalls)).To(Equal(int32(0))) - }) - }) - When("the item is subscribed", func() { - BeforeEach(func() { - client.getLibItemItem.Type = "SUBSCRIBED" - }) - When("there is an issue with syncing", func() { - BeforeEach(func() { - client.syncLibItemErr = errors.New("timed out") - }) - It("should return an error", func() { - Expect(syncErr).To(HaveOccurred()) - Expect(syncErr).To(MatchError(fmt.Errorf( - "error syncing library item %s: %w", itemID, - client.syncLibItemErr))) - Expect(atomic.LoadInt32(&client.getLibItemCalls)).To(Equal(int32(1))) - Expect(atomic.LoadInt32(&client.syncLibItemCalls)).To(Equal(int32(1))) - }) - }) - When("there is no issue with syncing", func() { - BeforeEach(func() { - client.getLibItemItem = &library.Item{ - ID: itemID, - } - }) - It("should succeed", func() { - Expect(syncErr).ToNot(HaveOccurred()) - Expect(atomic.LoadInt32(&client.getLibItemCalls)).To(Equal(int32(1))) - Expect(atomic.LoadInt32(&client.syncLibItemCalls)).To(Equal(int32(1))) - }) - }) - }) - }) - }) -}) diff --git a/pkg/vmconfig/crypto/crypto_reconciler_pre.go b/pkg/vmconfig/crypto/crypto_reconciler_pre.go index a23731a3a..586ccc040 100644 --- a/pkg/vmconfig/crypto/crypto_reconciler_pre.go +++ b/pkg/vmconfig/crypto/crypto_reconciler_pre.go @@ -18,6 +18,7 @@ import ( byokv1 "github.com/vmware-tanzu/vm-operator/external/byok/api/v1alpha1" "github.com/vmware-tanzu/vm-operator/pkg/conditions" + pkgcfg "github.com/vmware-tanzu/vm-operator/pkg/config" kubeutil "github.com/vmware-tanzu/vm-operator/pkg/util/kube" "github.com/vmware-tanzu/vm-operator/pkg/util/paused" "github.com/vmware-tanzu/vm-operator/pkg/vmconfig/crypto/internal" @@ -43,6 +44,7 @@ type reconcileArgs struct { curKey cryptoKey newKey cryptoKey isEncStorClass bool + profileID string hasVTPM bool addVTPM bool remVTPM bool @@ -130,7 +132,7 @@ func (r reconciler) Reconcile( args.curKey = getCurCryptoKey(moVM) // Check whether or not the StorageClass supports encryption. - args.isEncStorClass, _, err = kubeutil.IsEncryptedStorageClass( + args.isEncStorClass, args.profileID, err = kubeutil.IsEncryptedStorageClass( ctx, k8sClient, vm.Spec.StorageClass) @@ -511,7 +513,7 @@ func doUpdateEncrypted( args reconcileArgs) (string, Reason, []string, error) { op := "updating encrypted" - r, m, err := validateUpdateEncrypted(ctx, args) + r, m, err := onUpdateEncrypted(ctx, args) return op, r, m, err } @@ -691,6 +693,100 @@ func updateDiskBackingForRecrypt( return true } +func onUpdateEncrypted( + ctx context.Context, + args reconcileArgs) (Reason, []string, error) { + + logger := logr.FromContextOrDiscard(ctx) + + reason, msgs, err := validateUpdateEncrypted(ctx, args) + if reason > 0 || len(msgs) > 0 || err != nil { + return reason, msgs, err + } + + if pkgcfg.FromContext(ctx).Features.FastDeploy { + var encryptedDisks []string + if args.isEncStorClass { + encryptedDisks = onEncryptDisks(args) + } + if len(encryptedDisks) > 0 { + logger.Info( + "Update encrypted VM", + "currentKeyID", args.curKey.id, + "currentProviderID", args.curKey.provider, + "encryptedDisks", encryptedDisks) + } + } + + return 0, nil, nil +} + +func onEncryptDisks(args reconcileArgs) []string { + var fileNames []string + for _, baseDev := range args.moVM.Config.Hardware.Device { + if disk, ok := baseDev.(*vimtypes.VirtualDisk); ok { + if disk.VDiskId == nil { // Skip FCDs + + switch tBack := disk.Backing.(type) { + case *vimtypes.VirtualDiskFlatVer2BackingInfo: + if tBack.Parent == nil && tBack.KeyId == nil { + if updateDiskBackingForEncrypt(args, disk) { + fileNames = append(fileNames, tBack.FileName) + } + } + case *vimtypes.VirtualDiskSeSparseBackingInfo: + if tBack.Parent == nil && tBack.KeyId == nil { + if updateDiskBackingForEncrypt(args, disk) { + fileNames = append(fileNames, tBack.FileName) + } + } + case *vimtypes.VirtualDiskSparseVer2BackingInfo: + if tBack.Parent == nil && tBack.KeyId == nil { + if updateDiskBackingForEncrypt(args, disk) { + fileNames = append(fileNames, tBack.FileName) + } + } + } + } + } + } + return fileNames +} + +func updateDiskBackingForEncrypt( + args reconcileArgs, + disk *vimtypes.VirtualDisk) bool { + + devSpec := getOrCreateDeviceChangeForDisk(args, disk) + if devSpec == nil { + return false + } + + if devSpec.Backing == nil { + devSpec.Backing = &vimtypes.VirtualDeviceConfigSpecBackingSpec{} + } + + // Update the device change's profile to use the encryption storage profile. + devSpec.Profile = []vimtypes.BaseVirtualMachineProfileSpec{ + &vimtypes.VirtualMachineDefinedProfileSpec{ + ProfileId: args.profileID, + }, + } + + // Set the device change's crypto spec to be the same as the one the VM is + // currently using. + devSpec.Backing.Crypto = &vimtypes.CryptoSpecEncrypt{ + CryptoKeyId: vimtypes.CryptoKeyId{ + KeyId: args.curKey.id, + ProviderId: &vimtypes.KeyProviderId{ + Id: args.curKey.provider, + }, + }, + } + + return true +} + // getOrCreateDeviceChangeForDisk returns the device change for the specified // disk. If there is an existing Edit change, that is returned. If there is an // existing Add/Remove change, nil is returned. Otherwise a new Edit change is diff --git a/test/builder/fake.go b/test/builder/fake.go index d6654821f..55fa92caf 100644 --- a/test/builder/fake.go +++ b/test/builder/fake.go @@ -60,6 +60,7 @@ func KnownObjectTypes() []client.Object { &vmopv1.VirtualMachinePublishRequest{}, &vmopv1.ClusterVirtualMachineImage{}, &vmopv1.VirtualMachineImage{}, + &vmopv1.VirtualMachineImageCache{}, &vmopv1.VirtualMachineWebConsoleRequest{}, &vmopv1a1.WebConsoleRequest{}, &cnsv1alpha1.CnsNodeVmAttachment{}, diff --git a/test/builder/testdata/images/ttylinux-pc_i486-16.1.mf b/test/builder/testdata/images/ttylinux-pc_i486-16.1.mf index fd935c37f..bcfe83915 100644 --- a/test/builder/testdata/images/ttylinux-pc_i486-16.1.mf +++ b/test/builder/testdata/images/ttylinux-pc_i486-16.1.mf @@ -1,2 +1,2 @@ -SHA1(ttylinux-pc_i486-16.1-disk1.vmdk)= ed64564a37366bfe1c93af80e2ead0cbd398c3d3 - \ No newline at end of file +SHA1 (ttylinux-pc_i486-16.1.ovf) = 87271da9a5ae979f29e0ce7be824df428c178914 +SHA1 (ttylinux-pc_i486-16.1-disk1.vmdk) = ed64564a37366bfe1c93af80e2ead0cbd398c3d3 diff --git a/test/builder/testdata/images/ttylinux-pc_i486-16.1.ova b/test/builder/testdata/images/ttylinux-pc_i486-16.1.ova index 5d878c74fb714b6138e957175e999605df92a1f9..30f078bd7b7042f60897bd0ba4931c90128161b9 100644 GIT binary patch delta 938 zcmbV|Npn;M6h@&TqA>~~h#?&1B?=_s@TTt9jRs^!WKvXAP<$la7$FWJgCj(SgN!EK zDyS&1v0S*qg@4b7EL`b=TXpNJ@7BFvox0Q4&$Ue#J#(Z|Nz-6X7=~FT`$xn0t}jMW zREg3cO0%RACWTLfFe;amcuo+uW>*Y%OdKE2$BNth=*0Nd&OdI^Rty@A+1}*~i|wNB z1{PYZ|9-o)v9r4|GB!GukJbBv$sw zjFaqNtT?%Ym8AEG)$E>M3i@Z3B?G}=P|e~j8qRBZIj`5US|hI2!=ZY1xL%FZ;YJ!) zhoY=ni_-Re+c!i(f3qhj4Fs)K-E00@9JJ?in?3&xUJjEuyYrJMPUCWx#%Z}yfLF34 zy|;g!Lgq?I=E;0%llx?W+%FGEyF4fz@{l|%3uTdX%3|q~CGv2q zd0bXVAidHjPso$9QdY@o>6Za{O4i8J@{Bwy&q*ke#F9uVWvNIeRjJ8Zd0t+SL0Kp3 zMEAp+($gDJ_CEv;S@`GHJYx1M~BtOe_`9*$} L8*=kEZr%PHSLX}5 delta 859 zcmb7==~B-D6oxA$dy-OlP|2F**MG~tWG74ZohaEVTI@^0lO?JC$P)6!1>_3cLAT-# z{66@i8HRV}%sKO(ne#q#+6Ou_+QaIpPXvirEF}n49t6Mj&?%3HUU?)IiC4rTv3Mff zhgK|_5+r+bYqQ!~T5Fn9ic(sd8tR%F>w1d|GJlQz-%C@6htevuzjfy{rzHn-tNQZ( z8WjYQXmn&$ED;aU{w1m}KRsEUUy^LiFN+lYx00>7UH!wQsXzJ({|p%qqLGU5O(P)_ zk!bwiAUlv$Nt02ME*UafGG&a6m2om&CdfpYBv~?9vSo_o$W+OdJeel>QXqvgUBa6) zWTwoLA}N*SV=$VIs%m!&}(rAe;HRcV$MX_Yp)CfDVL+>~2#TkgnRxhMCf zT^>k>Jd{WBSd>nAB2VR+JeMwcA>Hy)dZbtSBq^`twY-sjc`NVay?l@X`6!>{vkZRa G%g|5Ufc9|! diff --git a/test/builder/testdata/images/ttylinux-pc_i486-16.1.ovf b/test/builder/testdata/images/ttylinux-pc_i486-16.1.ovf index ccf48c497..3c6ca9adf 100644 --- a/test/builder/testdata/images/ttylinux-pc_i486-16.1.ovf +++ b/test/builder/testdata/images/ttylinux-pc_i486-16.1.ovf @@ -1,12 +1,22 @@ - - + + - + Virtual disk information - + The list of logical networks @@ -68,7 +78,7 @@ 5 E1000 10 - + false @@ -83,12 +93,13 @@ vmware.vmci 1 - - - - - - + + + + + + - + \ No newline at end of file diff --git a/test/builder/vcsim_test_context.go b/test/builder/vcsim_test_context.go index 48580166d..cad0547f6 100644 --- a/test/builder/vcsim_test_context.go +++ b/test/builder/vcsim_test_context.go @@ -66,6 +66,7 @@ import ( pkgmgr "github.com/vmware-tanzu/vm-operator/pkg/manager" "github.com/vmware-tanzu/vm-operator/pkg/record" "github.com/vmware-tanzu/vm-operator/pkg/util/ovfcache" + "github.com/vmware-tanzu/vm-operator/pkg/util/ptr" pkgclient "github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/client" "github.com/vmware-tanzu/vm-operator/test/testutil" ) @@ -157,6 +158,7 @@ type TestContextForVCSim struct { withWorkloadIsolation bool // When WithContentLibrary is true: + LocalContentLibraryID string ContentLibraryImageName string ContentLibraryID string ContentLibraryItemID string @@ -620,8 +622,9 @@ func (c *TestContextForVCSim) setupContentLibrary(config VCSimTestConfig) { libMgr := library.NewManager(c.RestClient) - libSpec := library.Library{ - Name: "vmop-content-library", + // Create a local library. + localLibSpec := library.Library{ + Name: "vmop-local", Type: "LOCAL", Storage: []library.StorageBacking{ { @@ -629,64 +632,116 @@ func (c *TestContextForVCSim) setupContentLibrary(config VCSimTestConfig) { Type: "DATASTORE", }, }, - // Making it published to be able to verify SyncLibraryItem() API. + // Publish the library. Publication: &library.Publication{ - Published: vimtypes.NewBool(true), + Published: ptr.To(true), }, } - - clID, err := libMgr.CreateLibrary(c, libSpec) + localLibID, err := libMgr.CreateLibrary(c, localLibSpec) Expect(err).ToNot(HaveOccurred()) - Expect(clID).ToNot(BeEmpty()) - c.ContentLibraryID = clID + Expect(localLibID).ToNot(BeEmpty()) + localLib, err := libMgr.GetLibraryByID(c, localLibID) + Expect(err).ToNot(HaveOccurred()) + Expect(localLib).ToNot(BeNil()) - // OVA - libraryItem := library.Item{ + c.LocalContentLibraryID = localLibID + + // Create an OVA in the local library. + localLibItemOVA := library.Item{ Name: "test-image-ovf", Type: library.ItemTypeOVF, - LibraryID: clID, + LibraryID: localLibID, } - c.ContentLibraryImageName = libraryItem.Name - - itemID := CreateContentLibraryItem( + localLibItemOVAID := CreateContentLibraryItem( c, libMgr, - libraryItem, + localLibItemOVA, path.Join( testutil.GetRootDirOrDie(), "test", "builder", "testdata", - "images", "ttylinux-pc_i486-16.1.ovf"), + "images", "ttylinux-pc_i486-16.1.ova"), ) - c.ContentLibraryItemID = itemID + Expect(localLibItemOVAID).ToNot(BeEmpty()) - // The image isn't quite as prod but sufficient for what we need here ATM. - clusterVMImage := DummyClusterVirtualMachineImage(c.ContentLibraryImageName) - clusterVMImage.Spec.ProviderRef = &common.LocalObjectRef{ - Kind: "ClusterContentLibraryItem", + { + li, err := libMgr.GetLibraryItem(c, localLibItemOVAID) + Expect(err).ToNot(HaveOccurred()) + Expect(li.Cached).To(BeTrue()) + Expect(li.ContentVersion).ToNot(BeEmpty()) } - Expect(c.Client.Create(c, clusterVMImage)).To(Succeed()) - clusterVMImage.Status.ProviderItemID = itemID - conditions.MarkTrue(clusterVMImage, vmopv1.ReadyConditionType) - Expect(c.Client.Status().Update(c, clusterVMImage)).To(Succeed()) - // ISO - libraryItem = library.Item{ + // Create an ISO in the local library. + localLibItemISO := library.Item{ Name: "test-image-iso", Type: library.ItemTypeISO, - LibraryID: clID, + LibraryID: localLibID, } - c.ContentLibraryIsoImageName = libraryItem.Name - - itemID = CreateContentLibraryItem( + localLibItemISOID := CreateContentLibraryItem( c, libMgr, - libraryItem, + localLibItemISO, path.Join( testutil.GetRootDirOrDie(), "test", "builder", "testdata", "images", "ttylinux-pc_i486-16.1.iso"), ) - c.ContentLibraryIsoItemID = itemID + Expect(localLibItemISOID).ToNot(BeEmpty()) + + // Create a subscribed library. + subLibSpec := library.Library{ + Name: "vmop-subscribed", + Type: "SUBSCRIBED", + Storage: []library.StorageBacking{ + { + DatastoreID: c.Datastore.Reference().Value, + Type: "DATASTORE", + }, + }, + Subscription: &library.Subscription{ + SubscriptionURL: localLib.Publication.PublishURL, + OnDemand: ptr.To(true), + }, + } + subLibID, err := libMgr.CreateLibrary(c, subLibSpec) + Expect(err).ToNot(HaveOccurred()) + Expect(subLibID).ToNot(BeEmpty()) + + // Get the library item IDs for the OVA & ISO from the subscribed library. + var ( + subLibItemOVAID string + subLibItemISOID string + ) + subLibItems, err := libMgr.GetLibraryItems(c, subLibID) + Expect(err).ToNot(HaveOccurred()) + for i := range subLibItems { + sli := subLibItems[i] + switch sli.Name { + case localLibItemOVA.Name: + subLibItemOVAID = sli.ID + case localLibItemISO.Name: + subLibItemISOID = sli.ID + } + } + Expect(subLibItemOVAID).ToNot(BeEmpty()) + Expect(subLibItemISOID).ToNot(BeEmpty()) + + c.ContentLibraryID = subLibID + + c.ContentLibraryImageName = localLibItemOVA.Name + c.ContentLibraryItemID = subLibItemOVAID + + c.ContentLibraryIsoImageName = localLibItemISO.Name + c.ContentLibraryIsoItemID = subLibItemISOID + + // The image isn't quite as prod but sufficient for what we need here ATM. + clusterVMImage := DummyClusterVirtualMachineImage(c.ContentLibraryImageName) + clusterVMImage.Spec.ProviderRef = &common.LocalObjectRef{ + Kind: "ClusterContentLibraryItem", + } + Expect(c.Client.Create(c, clusterVMImage)).To(Succeed()) + clusterVMImage.Status.ProviderItemID = subLibItemOVAID + conditions.MarkTrue(clusterVMImage, vmopv1.ReadyConditionType) + Expect(c.Client.Status().Update(c, clusterVMImage)).To(Succeed()) // The image isn't quite as prod but sufficient for what we need here ATM. clusterVMImage = DummyClusterVirtualMachineImage(c.ContentLibraryIsoImageName) @@ -694,7 +749,7 @@ func (c *TestContextForVCSim) setupContentLibrary(config VCSimTestConfig) { Kind: "ClusterContentLibraryItem", } Expect(c.Client.Create(c, clusterVMImage)).To(Succeed()) - clusterVMImage.Status.ProviderItemID = itemID + clusterVMImage.Status.ProviderItemID = subLibItemISOID conditions.MarkTrue(clusterVMImage, vmopv1.ReadyConditionType) Expect(c.Client.Status().Update(c, clusterVMImage)).To(Succeed()) }