You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This was just kind of a bad decision on my part. To simplify/optimize parallel generation, in MazeDataset.generate() we:
run _maze_gen_init_worker(config: MazeDatasetConfig), which initializes a global _GLOBAL_WORKER_CONFIG
imap the _generate_maze_helper(index: int) function
this is done to avoid passing the whole config around, and it might not be that bad of an idea for parallel generation, but this is done even when the generation is done on a single thread.
Two options:
if passing the config does not have that much of an overhead, then just pass it
don't use a global worker init for the non-parallel generation
use functools.partial instead (I think I originally tried to use a lambda but that of course does not pickle) and remove the need for a global worker config
I'm partial to the last one, since it feels cleanest.
The text was updated successfully, but these errors were encountered:
This was just kind of a bad decision on my part. To simplify/optimize parallel generation, in
MazeDataset.generate()
we:_maze_gen_init_worker(config: MazeDatasetConfig)
, which initializes a global_GLOBAL_WORKER_CONFIG
_generate_maze_helper(index: int)
functionthis is done to avoid passing the whole config around, and it might not be that bad of an idea for parallel generation, but this is done even when the generation is done on a single thread.
Two options:
functools.partial
instead (I think I originally tried to use a lambda but that of course does not pickle) and remove the need for a global worker configI'm partial to the last one, since it feels cleanest.
The text was updated successfully, but these errors were encountered: