hello_world can not run

[czq693497091]

I have installed the k8s (v1.18.2) in the local cluster and used helm(v2.17.0) to install adaptdl, adaptdl-sched successfully:

root@k8s-master:/home/czq/Pollux/adaptdl_v2/examples/mnist# kubectl get pod -A | grep adaptdl
adaptdl adaptdl-registry-697884b65-wf4w6 1/1 Running 0 17h
adaptdl jazzed-koala-adaptdl-sched-85d75fdb5d-9lvzq 3/3 Running 6 17h
adaptdl jazzed-koala-validator-98f8fcf7c-jj959 1/1 Running 0 17h
adaptdl peeking-ostrich-adaptdl-sched-667c78f9fb-fr2zj 3/3 Running 4 17h

and I write the hello_world protect the same as the introduction with the following structure:
└── hello_world
├── adaptdljob.yaml
├── Dockerfile
└── hello_world.py

I execute the "adaptdl submit hello_world" and get the following information:

/usr/lib/python3/dist-packages/requests/init.py:80: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Using AdaptDL insecure registry.
Sending build context to Docker daemon 4.096kB
Step 1/4 : FROM python:3.7-slim
---> d3c9ad326043
Step 2/4 : RUN python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple adaptdl
---> Using cache
---> 05dae174d67e
Step 3/4 : COPY hello_world.py /root/hello_world.py
---> Using cache
---> 10d12170490d
Step 4/4 : ENV PYTHONUNBUFFERED=true
---> Using cache
---> bc04efd29920
Successfully built bc04efd29920
Successfully tagged localhost:59283/adaptdl-submit:latest
Using default tag: latest
The push refers to repository [localhost:59283/adaptdl-submit]
2cab9519a560: Layer already exists
16f13637494a: Layer already exists
25ad0307b4c1: Layer already exists
874b45955cb1: Layer already exists
85c923303735: Layer already exists
d0fa20bfdce7: Layer already exists
2edcec3590a4: Layer already exists
latest: digest: sha256:7346ece45037f13481a30a50907418bbd460035f488a1aab3cfb0f8ebdf35644 size: 1790
W0126 21:25:38.652722 75926 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
Unsupported storageclass from available storageclasses []

and I execute "adaptdl ls" but cannot get the information about this demo:
root@k8s-master:/home/czq/Pollux/adaptdl_v2/examples/HelloWorld# adaptdl ls
/usr/lib/python3/dist-packages/requests/init.py:80: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
No adaptdljobs
Name Status Start(UTC) Runtime Rplc Rtrt

I wonder how to cope with this problem and the job can correctly execute.

[aurickq]

Unsupported storageclass from available storageclasses []

It looks like your K8s might not have any storageclasses installed. AdaptDL requires a shared filesystem which can be used to store checkpoints and other information when a job is restarted. Once you have a storageclass for a shared filesystem installed, you can pass it into the submit command with --checkpoint-storage-class=....

[SHu0421]

Hello @aurickq ,
I also can't run hello_world, but I met a different problem:

The push refers to repository [docker.io/cindybrain/adaptdl-submit]
42247853ddec: Layer already exists 
7bb07b4b650b: Layer already exists 
6f3a145bdf9a: Layer already exists 
07752303aace: Layer already exists 
a598855c21e0: Layer already exists 
2c7950a1245f: Layer already exists 
9c1b6dd6c1e6: Layer already exists 
latest: digest: sha256:cc13db8e078711414917d022979ca54f73e48167d32f33a59cd3eb38830df392 size: 1790
W0502 17:37:55.091458  148014 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
Failure (InternalError): Internal error occurred: failed calling webhook "adaptdl-validator.adaptdl.svc.cluster.local": Post https://adaptdl-validator.adaptdl.svc:443/validate?timeout=10s: dial tcp 10.152.183.186:443: connect: connection refused

Because I am not familiar with ValidatingWebhookConfiguration, I don't know how to slove it.

[aurickq]

@SHu0421 This error could be caused by a variety of reasons. You can start by checking kubectl -n <adaptdl namespace> get all (replacing <adaptdl namespace> with the namespace in which you installed the adaptdl scheduler).

[gudiandian]

Hello @aurickq , I also can't run hello_world, but I met a different problem:

The push refers to repository [docker.io/cindybrain/adaptdl-submit]
42247853ddec: Layer already exists 
7bb07b4b650b: Layer already exists 
6f3a145bdf9a: Layer already exists 
07752303aace: Layer already exists 
a598855c21e0: Layer already exists 
2c7950a1245f: Layer already exists 
9c1b6dd6c1e6: Layer already exists 
latest: digest: sha256:cc13db8e078711414917d022979ca54f73e48167d32f33a59cd3eb38830df392 size: 1790
W0502 17:37:55.091458  148014 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
Failure (InternalError): Internal error occurred: failed calling webhook "adaptdl-validator.adaptdl.svc.cluster.local": Post https://adaptdl-validator.adaptdl.svc:443/validate?timeout=10s: dial tcp 10.152.183.186:443: connect: connection refused

Because I am not familiar with ValidatingWebhookConfiguration, I don't know how to slove it.

Hi, have you solved the problem?

[aurickq]

@gudiandian it sounds like it's related to the problem you are having in #124

[SHu0421]

Hello @aurickq , I also can't run hello_world, but I met a different problem:

The push refers to repository [docker.io/cindybrain/adaptdl-submit]
42247853ddec: Layer already exists 
7bb07b4b650b: Layer already exists 
6f3a145bdf9a: Layer already exists 
07752303aace: Layer already exists 
a598855c21e0: Layer already exists 
2c7950a1245f: Layer already exists 
9c1b6dd6c1e6: Layer already exists 
latest: digest: sha256:cc13db8e078711414917d022979ca54f73e48167d32f33a59cd3eb38830df392 size: 1790
W0502 17:37:55.091458  148014 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
Failure (InternalError): Internal error occurred: failed calling webhook "adaptdl-validator.adaptdl.svc.cluster.local": Post https://adaptdl-validator.adaptdl.svc:443/validate?timeout=10s: dial tcp 10.152.183.186:443: connect: connection refused

Because I am not familiar with ValidatingWebhookConfiguration, I don't know how to slove it.

Hi, have you solved the problem?

I changed microk8s to standard k8s instance (with three nodes), and I didn't met the problem again. By the way, I used the insecure registry rather than external registry.

[gudiandian]

Hello @aurickq , I also can't run hello_world, but I met a different problem:

The push refers to repository [docker.io/cindybrain/adaptdl-submit]
42247853ddec: Layer already exists 
7bb07b4b650b: Layer already exists 
6f3a145bdf9a: Layer already exists 
07752303aace: Layer already exists 
a598855c21e0: Layer already exists 
2c7950a1245f: Layer already exists 
9c1b6dd6c1e6: Layer already exists 
latest: digest: sha256:cc13db8e078711414917d022979ca54f73e48167d32f33a59cd3eb38830df392 size: 1790
W0502 17:37:55.091458  148014 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
Failure (InternalError): Internal error occurred: failed calling webhook "adaptdl-validator.adaptdl.svc.cluster.local": Post https://adaptdl-validator.adaptdl.svc:443/validate?timeout=10s: dial tcp 10.152.183.186:443: connect: connection refused

Because I am not familiar with ValidatingWebhookConfiguration, I don't know how to slove it.

Hi, have you solved the problem?

I changed microk8s to standard k8s instance (with three nodes), and I didn't met the problem again. By the way, I used the insecure registry rather than external registry.

Unfortunately, I am using standard k8s already. Thank you for your reply.