CUDA updates for non-contiguous datatypes #5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
add cuda stream for submmitting multiple kernels. add suppot for predefined datatypes. Conflicts: opal/datatype/opal_datatype_unpack.c test/datatype/ddt_test.c
Add support for iovec and for pipeline iovec. a new way to compute nb_block and thread_per_block Conflicts: test/datatype/Makefile.am
Conflicts: test/datatype/Makefile.am
Improve the GPU memory management. Conflicts: opal/mca/mpool/gpusm/mpool_gpusm.h opal/mca/mpool/gpusm/mpool_gpusm_module.c
device 0, we now use the devices already opened.
issues, when 2 peers were doing a send/recv or when multiple senders were targetting the same receiver. Rolf provided a patch to solve this issue, by moving the IPC communication index from a global location onto each endpoint.
and will be populated with all the known information. Beware: one still has to manually set the CUDA lib and path as they are not available after configure (unlike the include which is). Conflicts: opal/datatype/cuda/Makefile
reason to have a copy of a locally generated file in the source.
minor cleanups.
Various other minor cleanups.
1. free code did not work right because we were computing the amount we freed after merging the list 2. we need to store original malloc GPU buffer in extra place because the one in the convertor gets changed over time Conflicts: opal/datatype/cuda/opal_datatype_cuda.cu opal/datatype/cuda/opal_datatype_pack_cuda_wrapper.cu
Conflicts: ompi/mca/pml/ob1/pml_ob1_cuda.c opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu
Conflicts: opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu opal/mca/btl/smcuda/btl_smcuda.c
Conflicts: opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu
we do a D2D copy Conflicts: opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu test/datatype/Makefile.am
Conflicts: opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu
Conflicts: opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu
iteration of the datatype based on a NULL pointer. This list will then contain the displacement and the length of each fragment of the datatype memory layout and can be used for any packing/unpacking purpose.
Conflicts: opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu opal/datatype/opal_datatype_unpack.c
Conflicts: opal/datatype/cuda/opal_datatype_pack_cuda_wrapper.cu
functions Conflicts: opal/datatype/cuda/opal_datatype_cuda.cu opal/datatype/cuda/opal_datatype_cuda_internal.cuh opal/datatype/cuda/opal_datatype_pack_cuda_kernel.cu opal/datatype/cuda/opal_datatype_pack_cuda_wrapper.cu opal/datatype/cuda/opal_datatype_unpack_cuda_kernel.cu opal/datatype/cuda/opal_datatype_unpack_cuda_wrapper.cu opal/datatype/opal_datatype_gpu.c
not work with realloc, so use malloc instead
check point use raw_cached, but cuda iov caching is not enabled check point, split iov into two version, non-cached and cached check point iov cache another checkpoint check point, cuda iov is cached, but not used for pack/unpack check point, ready to use cached cuda iov checkpoint, cached cuda iov is working with multiple send, but not for count > 1 checkpoint, fix a bug for partial unpack checkpoint, fix unpack size
checkpoint, during unpack, cache the entire iov before unpack another checkpoint checkpoint , remove unnecessary cuda stream sync use bit to replace % rollback to use %, not bit, since it is faster, not sure why
another checkpoint now convertor->count > 1 is woring
set_cuda_iov_position
not work with realloc, so use malloc instead
checkpoint, rewrite non-cached version fix for non cached iov fix the non cached iov, set position should be put at first move ddt iov to cuda iov into a function merge iov cached and non-cached for non cached iov, if there is no enough cuda iov space, break
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.