Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mapPar for collections #52

Merged
merged 8 commits into from
Nov 23, 2023
Merged

mapPar for collections #52

merged 8 commits into from
Nov 23, 2023

Conversation

amorfis
Copy link
Member

@amorfis amorfis commented Nov 22, 2023

Allows code like in README. Also adds extension to Iterable with .mapParWith.

There is ugly asInstanceOf in the code, any ideas how to do it better are welcome

@rucek
Copy link
Contributor

rucek commented Nov 22, 2023

Woult it make sense to re-use Source.mapPar for this, like:

val res = Source.fromIterable(iterable).mapPar(parallelism)(f)

and then transform res to the expected Iterable using the provided IterableFactory?

README.md Outdated Show resolved Hide resolved
@amorfis
Copy link
Member Author

amorfis commented Nov 23, 2023

@rucek I tried it, but it's ~3 times slower then directly using semaphores and forks

@adamw
Copy link
Member

adamw commented Nov 23, 2023

@amorfis @rucek yes making streams behave better is what I'm working on now :) but with Source.fromIterator there's always an extra fork

@adamw adamw merged commit 299e03f into master Nov 23, 2023
4 checks passed
@adamw
Copy link
Member

adamw commented Nov 23, 2023

Thanks!

@adamw adamw deleted the mapPar branch February 21, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants