Cannot build VMs from Windows 10

[terzieff]

Hey,

after following the documentation (creating VM and adding valid SSH keys) and trying to use docker_build() (for a Shiny app, but that's not important, I guess), I was getting this:
Error in cli_tools() : ssh, scp not found on your computer Install the missing tool(s) and try again

I dived into the code, went to analogsea docs and found these issues/solutions by Hadley - pachadotdev/analogsea#81 pachadotdev/analogsea#88

After adding "C:\Program Files\RStudio\bin\msys-ssh-1000-18" to the PATH variable and getting excited, I went to another error:
Permission denied (publickey). Error: ssh failed ssh -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=C:\Users\Tomek\AppData\Local\Temp\Rtmpsdfqdz/hosts -i C:\Users\Tomek\Desktop\Projekty\shiny\my-ssh.ppk [email protected] "mkdir -p -m 0755 buildimage" In addition: Warning message: running command 'ssh -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=C:\Users\Tomek\AppData\Local\Temp\Rtmpsdfqdz/hosts -i C:\Users\Tomek\Desktop\Projekty\shiny\my-ssh.ppk [email protected] "mkdir -p -m 0755 buildimage"' had status 255, which seems to be similar to another analogsea issue: pachadotdev/analogsea#114

I'll be able to test the package on Mac next week and while I expect it to work, it would be great to find a way to fix the SSH connection on Windows.

[MarkEdmondson1234]

Do you have an SSH client installed on the same machine? That is how its "supposed" to work, but I don't have a Windows machine to test it upon. Its only not an issue on Mac/Linux as they have one pre-installed.

I'm not too familiar, but a quick Google gave this as a possibility: https://www.windows-commandline.com/windows-command-line-ssh/

The R script will call the system with "ssh" so if you can use ssh from the Windows Terminal, then it should work.

[MarkEdmondson1234]

Another alternative would be to launch an instance in GCE, log into that with something like gce_ssh_browser and use that instance to build and SSH into the production server, as that will have ssh installed.

[terzieff]

Okay, I think getting SSH to work properly on Windows is not worth the time ATM. But how would you go about building e.g. sample Shiny app after using gce_ssh_browser? I'd appreciate any guidance or pointing in the right direction.

[MarkEdmondson1234]

It should just be a case of downloading and installing a SSH client.

Without that, you can't use any SSH functions including the docker ones, but will be able to use the VM creation functions.

The alternative is to start up a build instance, use gce_ssh_browser to log in (which does SSH through the browser) then install R and this library, and use that to launch your own instance using Shiny.

That is a good candidate for a new template I could include, a bit meta, but I will look at making it easier for these situations.

[MarkEdmondson1234]

If RStudio installed on Windows, should have ssh client available via RSTUDIO_MSYS_SSH, = RStudio\bin\msys_ssh

[MarkEdmondson1234]

@terzieff @nickshffer do you have a Windows machine handy to test this?

It looks as if you can use the SSH bundled with RStudio and configure its SSH RSA Key in RStudio on Windows via the "Git/SVN" menu under Options.

I can't change the location of this key on OSX (is that the same on Windows?) but it could be that you can use this SSH key to connect. (or create a new one) That would require you to run first the gce_ssh_addkeys() function before any ssh (and docker) commands.

It doesn't specify but the public key should be saved in same folder as C://.ssh/id_rsa.pub, or if that doesn't work you can use the "View public key", copy-paste it into a file location you know.

Add the Windows ssh client to your PATH variable e.g.
"C:\Program Files\RStudio\bin\msys-ssh-1000-18"

Then upload the ssh keys to the VM:

vm <- gce_ssh_addkeys(vm, key.pub = "C:\\.ssh\id_rsa.pub", key.private = "C:\\.ssh\id_rsa")
gce_ssh(vm, "echo foo")

Let me know if this gets you past authentication.

[nickshffer]

Great solution, I can probably take a look later tonight.

[atauro]

I've tryed this solution, but i get this error...

Error: ssh failed
ssh -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=C:\Users\atauro\AppData\Local\Temp\RtmpyyqXoI/hosts -i c:\Users\USERNAME\.ssh\google_compute_engine.ppk [email protected] "docker tag rstudio-server gcr.io/PROJECT-GCE/rstudio-server"
In addition: Warning message:
running command 'ssh -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=C:\Users\atauro\AppData\Local\Temp\RtmpyyqXoI/hosts -i c:\Users\USERNAME\.ssh\google_compute_engine.ppk [email protected] "docker tag rstudio-server gcr.io/PROJECT-GCE/rstudio-server"' had status 255

R Installed on my Windows10 PC , authenticated with gcloud compute engine ssh

Someone can help me? thanx in advance.

[MarkEdmondson1234]

The issue for docker containers specifically is going to be sorted out when harbor moves to using the Docker API ( https://docs.docker.com/engine/api/ ) instead of SSH commands. wch/harbor#8

That won't solve general SSH connection issues, but that can then be mitigated by using gce_browser_ssh()

[MarkEdmondson1234]

Hi @atauro - it doesn't look like your SSH request is using the RStudio SSH keys - did you run this?

vm <- gce_ssh_addkeys(vm, key.pub = "C:\\.ssh\id_rsa.pub", key.private = "C:\\.ssh\id_rsa")
gce_ssh(vm, "echo foo")

[MarkEdmondson1234]

Also, may as well put this here since you can log in and build the image yourself whilst the fix is pending:

Upload the folder container the Dockerfile and supporting files. For the templates googleComputeEngineR supplies they are located in system.file("dockerfiles",package = "googleComputeEngineR")

For Windows follow these instructions for uploading to your instance, or use gcloud
https://cloud.google.com/compute/docs/instances/transfer-files

Log in to the instance via gce_browser_ssh().

Build the new docker image via:

docker build -t new_image_name location_of_dockerfile

[MarkEdmondson1234]

I've added a new template you can use to build images, rather than needing to SSH into a VM yourself.

Here is an example:

## this could be any Dockerfile, here it takes one that comes with package
my_file <- system.file("dockerfiles","googleAuthR-verse", "Dockerfile", package = "googleComputeEngineR")

gce_vm_template("builder", 
                dockerfile = my_file, 
                build_name = "test1builder", 
                name = "builder", 
                predefined_type = "n1-standard-1")

You supply the Dockerfile to build from and what to call the docker image on Google container engine, and it will launch a VM with the cloud-config configured to import the Dockerfile and build the image, then upload it. You should delete the build VM after it is done.

[MarkEdmondson1234]

I believe this specific issue is solved now, but the more general using SSH from Windows I'll track here: #35

[MarkEdmondson1234]

NTS - Adding Env in windows:

You need to locate the directory that ssh.exe is in.

You then need to go into System Properties (hit WINDOWS KEY+PAUSE), go to Advanced system settings, click Environment Variables...

Scroll down in the scrolling window at the bottom, until you see "Path" under "Variable". Click it, click Edit...

Go to the end of the value, add a semi-colon if one isn't there already, and type the path to the directory that ssh.exe is in.

[MarkEdmondson1234]

RStudio settings: Tools > Global Options > Git/SVN > Create RSA Key (leave password blank) it will appear in C:/Users/USERNAME/.ssh/id_rsa

Make VM object and add the SSH keys to it:

vm <- gce_vm("vm-name")
vm <- gce_ssh_addkeys(vm, key.pub = file.path("C:","Users", "USERNAME", ".ssh", "id_rsa.pub"), key.private = file.path("C:","Users", "USERNAME", ".ssh", "id_rsa"))

[MarkEdmondson1234]

Usernames with spaces won't work, set your own.