Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][CLI] Support working_dir is bigger than 500MB for ray job submit #2717

Open
1 of 2 tasks
kevin85421 opened this issue Jan 7, 2025 · 0 comments
Open
1 of 2 tasks
Labels
cli kubectl plugin enhancement New feature or request triage

Comments

@kevin85421
Copy link
Member

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

Currently, KubeRay CLI job submission relies on the working_dir API in the runtime environment. However, working_dir is stored in the KV store of GCS, which has a limitation of 500 MB.

Maybe we should support automatically uploading the directory to cloud storage, similar to the method used by the Spark operator.

https://docs.google.com/document/d/1hCJsrCFYPJLS3Zusdr8N_4Y5leWUMy4bQEbsqSQp2mw/edit?tab=t.0

Use case

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!
@kevin85421 kevin85421 added enhancement New feature or request triage cli kubectl plugin labels Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cli kubectl plugin enhancement New feature or request triage
Projects
None yet
Development

No branches or pull requests

1 participant