Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] using the env injector with a key vault with private endpoint #782

Open
jason-berk-k1x opened this issue Nov 5, 2024 · 2 comments
Labels
question Further information is requested

Comments

@jason-berk-k1x
Copy link

Note: Make sure to check out known issues (https://github.com/sparebankenvest/azure-key-vault-to-kubernetes#known-issues) before submitting

Your question
how exactly is authentication/authorization happening in the EnvInjector?

i am using akv2k8s with an AKS cluster configured with Microsoft Entra ID authentication with Azure RBAC. My key vault is limited public access but has Allow trusted Microsoft services to bypass this firewall enabled. The vault also has a private endpoint configured. My initial thought was that aks would use the private endpoint to connect to the vault and pull secrets, but that does not appear to be true. Seems like some OICD(?) token exchange is happening and that if we were to disable the trusted services, akv2k8s would no longer be able to connect to the vault through the firewall.

Just to be clear, I got it all working....I see my secrets injected into my pods using the env injector and the identity assigned to the agentpool. I'm just trying to understand if akv2k8s is somehow actually using the privatelink I setup or not.

where in the logs would I see the actual URL used to get secrets from the vault?

To Reproduce
If question relates to a certain behavior, describe steps to reproduce:

Logs
If applicable, add logs to help add context to your question.

Additional context
Add any other context about the question here.

@jason-berk-k1x jason-berk-k1x added the question Further information is requested label Nov 5, 2024
@jason-berk-k1x
Copy link
Author

update:

I deleted the private endpoint and redeployed my test app:

{
	  "error": {
	    "code": "Forbidden",
	    "message": "Client address is not authorized and caller is not a trusted service.\r\nClient address: X.X.X.X\r\nCaller:.....stuff.....",
	    "innererror": {
	      "code": "ForbiddenByFirewall"
	    }
	  }
	}

recreated the PE and the boot looping app came up....

so my guess is, somewhere under the covers akv2k8s is making a request to the DNS of the vault which is being resolved by the private DNS zone to the private IP of the vault on my vnet.

@andreasthuen
Copy link
Contributor

The app uses the DNS setup of the cluster it is running in. I do not know how you have set up DNS resolution for private endpoints in your clusters, but the way we have set this up is by forwarding all requests related to azure services from our internal DNS servers to our private azure DNS zone, which returns the private IP of the vault.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants