Code

Spark dfsioe and repartition tuning#724

Code

Open

jiafuzha wants to merge 16 commits intoIntel-bigdata:masterIntel-bigdata/HiBench:masterfrom jiafuzha:sparkDFSIOEjiafuzha/HiBench:sparkDFSIOECopy head branch name to clipboard

+303-39

Commits

Commits on Jan 19, 2021

• initial commit

  Show description for 6b7e5f8
  jiafuzha

  committed
  6b7e5f8

  Copy full SHA for 6b7e5f8

Commits on Mar 10, 2021

• use TeraInputFormat instead of TextInputFormat

  Show description for 036bb12
  jiafuzha

  committed
  036bb12

  Copy full SHA for 036bb12

Commits on Apr 14, 2021

• Use memory data instead of data reading from hadoop

  Show description for e316608
  jiafuzha

  committed
  e316608

  Copy full SHA for e316608

• Use memory data instead of data reading from hadoop

  Show description for 935d424
  jiafuzha

  committed
  935d424

  Copy full SHA for 935d424

Commits on Apr 30, 2021

• added In-memory repartition

  Show description for 0284845
  

  xuechendi

  authored and

  jiafuzha

  committed
  0284845

  Copy full SHA for 0284845

Commits on Jun 11, 2021

• removed warning of unbound variable SIZE

  Show description for 10e5c4a
  jiafuzha

  committed
  10e5c4a

  Copy full SHA for 10e5c4a

• enrich error message

  Show description for 2a360a5
  jiafuzha

  committed
  2a360a5

  Copy full SHA for 2a360a5

• enable output in InMemRepartition

  Show description for f8f6418
  jiafuzha

  committed
  f8f6418

  Copy full SHA for f8f6418

Commits on Aug 30, 2021

• Merge remote-tracking branch 'upstream/master'

  jiafuzha

  committed
  1680f53

  Copy full SHA for 1680f53

• make Java funtion compatible to both scala 2.11 and 2.12

  Show description for 11f9daf
  jiafuzha

  committed
  11f9daf

  Copy full SHA for 11f9daf

Commits on Nov 25, 2022

• separate read and write in sparkdfsioe

  Show description for f10ca5b
  jiafuzha

  committed
  f10ca5b

  Copy full SHA for f10ca5b

Commits on Dec 1, 2022

• added multiple rounds

  Show description for 36474a6
  jiafuzha

  committed
  36474a6

  Copy full SHA for 36474a6

Commits on Dec 2, 2022

• repartition tuning

  Show description for 56a525d
  jiafuzha

  committed
  56a525d

  Copy full SHA for 56a525d

Commits on Dec 12, 2022

• repartition multiple rounds

  Show description for 43a8621
  jiafuzha

  committed
  43a8621

  Copy full SHA for 43a8621

• Merge remote-tracking branch 'upstream/master'

  jiafuzha

  committed
  68e4861

  Copy full SHA for 68e4861

• merged with master

  Show description for f02ba7c
  jiafuzha

  committed
  f02ba7c

  Copy full SHA for f02ba7c