Add an option to disable disable measuring kudo buffer copy.

[liurenjie1024]

When used in shuffle cases with large partition numbers, each call to System.nanoTime is still quite expensive compared with a small memory copy. This pr adds an option to disable this measurement and disable it by default.

[liurenjie1024]

build

[liurenjie1024]

build

[liurenjie1024]

cc @abellina @revans2

[binmahone]

Just out of curiosity, how expensive is System.nanoTime with relative to the actual memory copy? The code change itself looks good to me

[revans2]

Can we please tie this into visibility of the copy buffer time metric? If this is off, but the metric still shows up on the UI it will be confusing for customers.

[liurenjie1024]

Can we please tie this into visibility of the copy buffer time metric? If this is off, but the metric still shows up on the UI it will be confusing for customers.

Hi, @revans2 I've submitted a pr to hide it in spark rapids: NVIDIA/spark-rapids#11996

The whole issue has been splitted into three steps, see NVIDIA/spark-rapids#11995

[liurenjie1024]

Just out of curiosity, how expensive is System.nanoTime with relative to the actual memory copy? The code change itself looks good to me

We have observed about 30% improvement in actual workload, where partition is quite small.