Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

boundary-init job hangs #23

Open
lucardcoder opened this issue Nov 17, 2023 · 6 comments
Open

boundary-init job hangs #23

lucardcoder opened this issue Nov 17, 2023 · 6 comments

Comments

@lucardcoder
Copy link

lucardcoder commented Nov 17, 2023

I am trying the helm chart you provided in EKS and it hangs at the boundary-init job. any idea why would that would be?
job Logs:
Error creating: pods “boundary-init-” is forbidden: error looking up service account boundary/boundary: serviceaccount “boundary” not found.
EKS Version: 1.26

@jbouse
Copy link
Member

jbouse commented Nov 17, 2023

Hmm... let me take a look into this and see what's going on. I haven't looked at this chart in a while now, and it was before the EKS upgrade so might be something that needs to be updated.

@arogic
Copy link

arogic commented Nov 30, 2023

Hi, I am having the same issue. It seems that the ServiceAccount doesn't get created. Regards

@lucardcoder
Copy link
Author

there is a fix for that. you need to add below annotations to serviceaccount.yaml in templates:

annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation
helm.sh/hook-weight: "-10"

@arogic
Copy link

arogic commented Nov 30, 2023

Thanks @lucardcoder. I deployed the Service Account using the following Manifest, before deploying boundary with the helm chart to get around the issue.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: boundary
  namespace: boundary
  labels:
    app.kubernetes.io/managed-by: Helm
  annotations:
    meta.helm.sh/release-name: boundary
    meta.helm.sh/release-namespace: boundary
    helm.sh/hook: pre-install,pre-upgrade
    helm.sh/hook-delete-policy: before-hook-creation
    helm.sh/hook-weight: "-10"
kubectl create namespace boundary
kubectl apply -f service_account.yaml

However, I am getting the following error on the pods:

Back-off restarting failed container boundary-controller-scale-down in pod boundary-init-dr6zn_boundary(421cb507-6ec5-4351-a9bd-b1bfee9de02c)

Any ideas?

@jbouse
Copy link
Member

jbouse commented Nov 30, 2023

I already have the helm.sh/hook* annotations in the develop branch I just don't think it's been released in a version yet as the main branch still doesn't have it. As the init process is attempting to scale-down and scale-up the controller while it runs without the SA in place that will not have the proper RBAC access and I can see that causing errors.

@arogic
Copy link

arogic commented Dec 1, 2023

The Back-off issue comes from the initContainers in the https://github.com/ugns/boundary-chart/blob/v0.1.0/templates/database-init-job.yaml#L48, because the scale-down container is specified there instead of the init container as you have in the develop branch https://github.com/ugns/boundary-chart/blob/develop/templates/database-init-job.yaml#L49. Another issue I ran into is that there is no postgresql defined in the chart, so you need to deploy it before it seems. I haven't fully tested it yet, but it would be a great addition to the helm chart, if the postgresql is added there as an optional argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants