Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(helm chart): add kubelet path as arg to csi-node #596

Merged
merged 1 commit into from
Jan 10, 2025

Conversation

Abhinandan-Purkait
Copy link
Member

  • Adds the kubelet path as an arg to csi-node, to use the kubelet path as prefix in exhaustive mount path search during cleanup.

@Abhinandan-Purkait
Copy link
Member Author

bors try

bors-openebs-mayastor bot pushed a commit that referenced this pull request Jan 9, 2025
@bors-openebs-mayastor
Copy link
Contributor

try

Build failed:

@niladrih
Copy link
Member

niladrih commented Jan 9, 2025

error: unexpected argument '--kubelet-path' found

Usage: csi-node --node-name <NAME> --csi-socket <PATH> --rest-endpoint <rest-endpoint> --enable-rest --enable-registration --grpc-ip <GRPC_IP> --grpc-port <GRPC_PORT> --nvme-io-timeout <nvme-io-timeout> --nvme-core-io-timeout <TIMEOUT> --nvme-ctrl-loss-tmo <NUMBER> --nvme-nr-io-queues <NUMBER> --nvme-connect-fallback <nvme-connect-fallback>

For more information, try '--help'.

The test requires the develop image (on docker hub) from control-plane to work with the helm chart.

One of the two csi-nodes is crashing. The other one seems to be doing fine:

CSI Driver (csi-node) revision 35aa8366271f (v2.7.2+0)
["csi-node", "--csi-socket=/csi/csi.sock", "--node-name=kind-worker2", "--rest-endpoint=http://mayastor-api-rest:8081", "--enable-rest", "--enable-registration", "--grpc-endpoint=172.18.0.3:10199", "--nvme-io-timeout=110s10s", "--nvme-core-io-timeout=110s10s", "--nvme-ctrl-loss-tmo=1980", "--nvme-nr-io-queues=2", "--node-selector=openebs.io/csi-node=mayastor", "--fmt-style=pretty", "--ansi-colors=true"]
  2025-01-09T21:13:42.794587Z  INFO csi_node::client: API client is initialized with endpoint http://mayastor-api-rest:8081, request timeout = 5s
    at control-plane/csi-driver/src/bin/node/client.rs:127

  2025-01-09T21:13:42.794711Z  INFO csi_node::main_: CSI plugin bound to /csi/csi.sock
    at control-plane/csi-driver/src/bin/node/main_.rs:320

  2025-01-09T21:13:42.794768Z  INFO csi_node::client: VolumesClient API is initialized with endpoint http://mayastor-api-rest:8081, request timeout = 10s
    at control-plane/csi-driver/src/bin/node/client.rs:187

  2025-01-09T21:13:42.794787Z  INFO csi_node::node: Node topology segments: {"openebs.io/csi-node": "mayastor", "openebs.io/nodename": "kind-worker2"}
    at control-plane/csi-driver/src/bin/node/node.rs:61

  2025-01-09T21:13:42.794842Z  INFO csi_node::nodeplugin_grpc: node plugin gRPC server configured at address 172.18.0.3:10199
    at control-plane/csi-driver/src/bin/node/nodeplugin_grpc.rs:153

  2025-01-09T21:13:42.795873Z ERROR csi_node::registration: Failed to register app node: ServerCommunication("error in request: error trying to connect: tcp connect error: Connection refused (os error 111)")
    at control-plane/csi-driver/src/bin/node/registration.rs:46

  2025-01-09T21:14:42.802980Z  INFO csi_node::registration: Successfully re-registered the app node
    at control-plane/csi-driver/src/bin/node/registration.rs:39
NAME                                            READY   STATUS             RESTARTS        AGE
mayastor-agent-core-6f8499f6f6-jxtlj            2/2     Running            0               6m17s
mayastor-agent-ha-node-q9bg9                    1/1     Running            0               5m56s
mayastor-agent-ha-node-xt6b9                    1/1     Running            0               6m17s
mayastor-api-rest-575b4dd9c-7kv59               1/1     Running            0               6m17s
mayastor-csi-controller-6766dbc7ff-8jbn2        6/6     Running            0               6m17s
mayastor-csi-node-j7r4g                         0/2     CrashLoopBackOff   11 (21s ago)    6m17s
mayastor-csi-node-xqkvc                         2/2     Running            0               8m6s
mayastor-etcd-0                                 1/1     Running            0               6m16s
mayastor-io-engine-7z7jp                        2/2     Running            0               8m6s
mayastor-io-engine-s9mlr                        2/2     Running            0               8m6s
mayastor-localpv-provisioner-68c747444f-f8sf5   1/1     Running            0               8m6s
mayastor-loki-0                                 1/1     Running            0               8m6s
mayastor-obs-callhome-65454497c6-l2l5r          1/1     Running            0               6m17s
mayastor-operator-diskpool-678d6d479b-4fxm7     1/1     Running            0               6m17s
mayastor-promtail-kntsv                         1/1     Running            0               8m6s
mayastor-promtail-t45sz                         1/1     Running            5 (5m57s ago)   8m6s
mayastor-upgrade-v2-8-0-8b9wn                   1/1     Running            0               6m27s

This has happened because the kubernetes daemonset tries to upgrade the daemonset pods one-by-one. The first one never succeeds, and the hence the second one is never upgraded and continues to live in generation 1. Generation 2 is failing.

@niladrih
Copy link
Member

niladrih commented Jan 9, 2025

openebs/mayastor-control-plane#913 is a prerequisite for this PR.

@Abhinandan-Purkait
Copy link
Member Author

bors try

bors-openebs-mayastor bot pushed a commit that referenced this pull request Jan 10, 2025
@bors-openebs-mayastor
Copy link
Contributor

try

Build succeeded:

@Abhinandan-Purkait
Copy link
Member Author

bors merge

@bors-openebs-mayastor
Copy link
Contributor

👎 Rejected by too few approved reviews

@Abhinandan-Purkait
Copy link
Member Author

oops! Thanks @niladrih

bors merge

@bors-openebs-mayastor
Copy link
Contributor

Build succeeded:

@bors-openebs-mayastor bors-openebs-mayastor bot merged commit dde1fd2 into develop Jan 10, 2025
5 checks passed
@bors-openebs-mayastor bors-openebs-mayastor bot deleted the find_mnt_refactor branch January 10, 2025 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants