Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support execution in Windows using Local File System and NFS #996

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

rfung777
Copy link

@rfung777 rfung777 commented Aug 4, 2024

When running PyIceberg in Windows, pyiceberg.io.pyarrow.PyArrowFileIO.parse_location() incorrectly resolve Windows path, e.g. C:\foobar as a scheme of C and causes Unrecognized filesystem type in URI error later.

If using file:// scheme, the path is also incorrectly resolved to /C:\foobar. see https://stackoverflow.com/a/43925228

@Fokko
Copy link
Contributor

Fokko commented Aug 6, 2024

Thanks for working on this @rfung777 🙌

Ideally, we would also love to have Windows integration tests on Github Actions. WDYT?

@rfung777 rfung777 force-pushed the feature/support-windows-local-fs-and-nfs branch 2 times, most recently from 9c8d7de to e9cca42 Compare August 6, 2024 22:29
@rfung777
Copy link
Author

rfung777 commented Aug 6, 2024

Thanks for reviewing. New to the Windows integration test in GitHub Actions. I will do a bit of research to see how it is done.

@rfung777 rfung777 force-pushed the feature/support-windows-local-fs-and-nfs branch from e9cca42 to 5228f1c Compare August 6, 2024 22:41
@rfung777
Copy link
Author

hi @Fokko , I am not having much luck with setting up the Windows integration tests. It seems to require to install Make and Minio. The changes above seems to be localised in the parser function. Shall we push forward with this?

@rfung777 rfung777 force-pushed the feature/support-windows-local-fs-and-nfs branch from 5228f1c to 42a2e87 Compare August 14, 2024 21:39
@rfung777
Copy link
Author

I had an attempt of adding the Windows integration tests. Would you help review again please? Thanks.

@Fokko
Copy link
Contributor

Fokko commented Oct 28, 2024

@rfung777 Sorry, I was out for a while. Let's give it a try 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants