Skip to content

Latest commit

 

History

History
42 lines (26 loc) · 1.25 KB

README.md

File metadata and controls

42 lines (26 loc) · 1.25 KB
Raw

cluster-benchmark

How to use

  1. Download it.
git clone https://github.com/dharmeshkakadia/cluster-benchmark && cd cluster-benchmark
  1. Compile it.
mvn package
  1. Run it.
hadoop jar target/cluster-benchmark.jar [BenchmakrName]

Currently following benchmarks are available:

  • Sleep: Streeses YARN components (RM,NM,Historyserver) etc by creating a job with given number of tasks which sleeps for given amount of time.

  • Compute: Measures the compute capacity available on the cluster with a job which repeatedly generaetes random numbers.

  • Read: Measures the read capactiy of the file system.

  • DFSWrite: Measures the write capacity of the distributed file system (WASB,HDFS,ADLS, etc.)

  • Write: Measures the write capacity of local disk writes.

  • Shuffle: Measures the shuffle performance of the cluster.

  • ReadCompute: Generates a job that does given amount of read and compute.

  • ReadComputeWrite: Generates a job that does given amount of read, compute and write.

  • Mix: Generates a job which generates a mix of compute,read and write.

  • Mix2: Genereates a job which generates a mix of compute,read and write.