parallel_processing.md

• Overview
• Select the pattern
• MPI
• Simulation

Overview

In this part, we show a case that uses Message Passing Interface (MPI) to archive multiple mosaic simulations in parallel. Here we chose openmpi and the mpi4py, both of them can be replaced by other MPI implementations and bindings on python.

The following Python script is in scripts/parallel_simulate.py. Then, you can run the mpirun command to begin the simulation process. For example, we use the script to generate 10 mosaics as,

mpirun -n 10 python scripts/parallel_simulate.py
# `mpirun` can be replaced by `mpiexec` or others, depending on your mpi tools
# `10` is the number of simulated cases during this process

Please check the OpenMPI document for more information.

Select the pattern

The first part of the Python script is to decide the Pattern, features, and output in simulation.

# Horizontal Cell Pattern
from examples.HorizontalCell import pattern as HC

# Mosaic Plane, and the interaction function
from examples.HorizontalCell import scope, h_func

# optimization features
features = ["NN", "VD"]

# The prefix of output files
save_prefix = "examples/simulated/HC/W1"
# chose a random seed
seed = 89375
# print the optimized process or not
verbose = True 

MPI

The second part is to import the MPI tool and decide the rank (ID) of the thread.

import numpy as np
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank() # the ID of this thread

# reseed to avoid the unique random generator 
np.random.seed(seed=rank*seed)

Simulation

The last step is simulating and saving the generated mosaic into local files so that output files that are not overlapped by each other.

# use the O-PIPP method to generate new mosaics
new_mosaic = HC.new_mosaic(scope=scope)
new_mosaic, losses = HC.simulate(new_mosaic, h_func, features=features, verbose=verbose)
# save the mosaic with the rank 
new_mosaic.save("%s_%d.points"%(save_prefix, rank), separate=False)
np.savetxt("%s_%d.losses"%(save_prefix, rank), losses)