Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add rsync flag option to copy files using rsync #3143

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

olamilekan000
Copy link
Contributor

change adds rsync as an option to the copy command

issue #2198

@olamilekan000 olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch 6 times, most recently from a508328 to 4b57bc6 Compare January 23, 2025 17:09
Copy link
Member

@nirs nirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have rsync in the guest in all cases? Minimal vm images do not have rsync.

Also adding options is bad both for users and for testing. If it is better to copy with rsync, we should detect if rsync is available in the guest and host, and use it without adding new option. Otherwise fall back to scp.

If we always have rsync, better to use it instead of scp and minimize the testing matrix.

if err != nil {
return err
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This duplicate the lookup code. We could do:

tool := "scp"
if useRsync {
    tool = "rsync"
}

arg0, err = exec.LookPath(tool)
if err != nil {
    return err
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that makes sense. I was only following the discussion in the issue. I thought there was a consensus to go with having rsyn as a flag.

} else {
// for scp, append SSH options and scpArgs
cmdArgs = append(cmdArgs, append(sshArgs, scpArgs...)...)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change complicates the code by adding useRsync condition on every step. This is not maintainable.

A better way to do this is to add a function to create a scp command arguments, and a function to create rsync command arguments. Then use shared code to run the command.

@@ -67,6 +82,9 @@ func copyAction(cmd *cobra.Command, args []string) error {

if verbose {
scpFlags = append(scpFlags, "-v")
if useRsync {
scpFlags = append(scpFlags, "--progress")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot use variable scpFlags to keep rsync flags. This will confuse the next developer. Renaming the variable to toolFlags will avoid this.

@@ -34,6 +34,7 @@ func newCopyCommand() *cobra.Command {

copyCommand.Flags().BoolP("recursive", "r", false, "copy directories recursively")
copyCommand.Flags().BoolP("verbose", "v", false, "enable verbose output")
copyCommand.Flags().BoolP("rsync", "", false, "use rsync for copying instead of scp")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not flexible enough. What if we want to another tool later? If we add an option, adding a "tool" or "backend" option will be better, with possible values "scp", "rsync".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the tool option will default to rsync right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we add an option rsync sounds like a better default. But this is trivial to change if needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just remove the flag and only support rsync, unless there is a use case where scp is preferable over rsync

@nirs
Copy link
Member

nirs commented Jan 23, 2025

Also, do we have tests for code modified in this PR? if not we should not change it before adding tests. The new rsync flow should also have tests.

@jandubois
Copy link
Member

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

@olamilekan000
Copy link
Contributor Author

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

I'd appreciate it if you can help with steps to run locally. I did try but It didn't work as expected.

@AkihiroSuda
Copy link
Member

If we always have rsync, better to use it instead of scp and minimize the testing matrix.

Yes, rsync is always available on macOS and most Linux distros.
At least it is easily installable.

@nirs
Copy link
Member

nirs commented Jan 24, 2025

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

I'd appreciate it if you can help with steps to run locally. I did try but It didn't work as expected.

@olamilekan000, Can you describe how did you try to test and why it did not work well?

You can also join the #lima slack channel to get quicker help from other developers.

@nirs nirs requested a review from afbjorklund January 24, 2025 15:22
@olamilekan000 olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch from 4b57bc6 to aa5cf22 Compare January 24, 2025 20:45
@olamilekan000
Copy link
Contributor Author

olamilekan000 commented Jan 24, 2025

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

I'd appreciate it if you can help with steps to run locally. I did try but It didn't work as expected.

@olamilekan000, Can you describe how did you try to test and why it did not work well?

You can also join the #lima slack channel to get quicker help from other developers.

@nirs I have joined and shared my error logs on the channel.

@olamilekan000 olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch from aa5cf22 to 4c95f10 Compare January 25, 2025 00:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants