Status of the CI D4J

[martinezmatias]

General Summary

In this issue we report the status of the CI that executes D4J repair (currently executed in my folk)

Timeout Flacoco: M2 (Repaired Gzoltar),M5 (analyzed and closed: repaired both), M8 (analyzed and closed: repaired both),
M28 (analyzed and closed: repaired both), M2J (analyzed and closed: repaired both), M8J (analyzed and closed: repaired both), M28J (analyzed and closed: repaired both)

Timeout job: M40 (not repaired)

Unrepair only Gzoltar: M71 (analyzed)

Unrepair only Flacoco: M84 (analyzed and closed: repaired locally, in GH fails due to a large search space)

Unrepair both: Chart15 (analyzed, repaired by Gzoltar and Flacoco, closed) ,Chart15J (analyzed and repaired by Gzoltar and Flacoco, closed) , Time4, Time11, M95J (analyzed, repaired, closed) , Time4J, Time11J, M78J (analyzed and closed: not plausible),

Recently added in the CI: M32J (analyzed and closed: repaired both)

Particular analysis

Math 2

Repaired Gzoltar

time(sec)= 1955
operation: InsertBeforeOp
location= org.apache.commons.math3.distribution.HypergeometricDistribution
line= 321
lineSuspiciousness= 0.162
lineSuspiciousness= 10
original statement= return org.apache.commons.math3.util.FastMath.min(getNumberOfSuccesses(), getSampleSize())
buggy kind= CtReturnImpl|CtBlockImpl
fixed statement= numericalVarianceIsCalculated = true

In Gzoltar the suspiricious appears

FlacocoFaultLocalization - Suspicious: 12 line org.apache.commons.math3.distribution.HypergeometricDistribution l: 321, susp 0.16222142113076254

Math 84 (Repaired with Gzoltar not with Flacoco)

Bug applied at MP=org.apache.commons.math.optimization.direct.MultiDirectional line: 90

location= org.apache.commons.math.optimization.direct.MultiDirectional
line= 90
lineSuspiciousness= 0.707
lineSuspiciousness= 6

494397] DEBUG AstorCoreEngine - operation: OP_INSTANCE:
InsertAfterOp:(spoon.support.reflect.code.CtIfImpl) `if (comparator.compare(contracted, best) < 0) {    	return; } ` -topatch--> `break` (spoon.support.reflect.code.CtBreakImpl) 
[494397] DEBUG InsertAfterOp - ---MP=org.apache.commons.math.optimization.direct.MultiDirectional line: 90, pointed element: CtBreakImpl
[494397] DEBUG InsertAfterOp - ---OP_INSTANCE:
InsertAfterOp:(spoon.support.reflect.code.CtIfImpl) `if (comparator.compare(contracted, best) < 0) {    	return; } ` -topatch--> `break` (spoon.support.reflect.code.CtBreakImpl) 
[494397] DEBUG AstorCoreEngine - 
--Summary Creation: for variant [Variant id: 194, #gens: 89, #ops: 1, parent:193] gen mutated: 1 , gen not mut: 0, gen not applied  0
[494609] DEBUG AstorCoreEngine - -The child compiles: id 194
[494615] DEBUG main - -Running first validation
[495823] DEBUG main - Analyzing output from process
[495823] DEBUG main - TR: Success: true, failTest= 0, was successful: true, cases executed: 5] ,[]
[495823] DEBUG main - -Test Failing is passing, Executing regression
[553442] DEBUG main - Analyzing output from process
[553442] DEBUG main - TR: Success: true, failTest= 0, was successful: true, cases executed: 2054] ,[]
[553442] DEBUG AstorCoreEngine - -Valid?: |true|0|2054|[]|, fitness 0.0
[553442] INFO AstorCoreEngine - -Found Solution, child variant #194

Patch in log
https://github.com/martinezmatias/astor/runs/3954512867?check_suite_focus=true#step:8:4158

if (comparator.compare(contracted, best) < 0) {    	return; } ` -topatch--> `break` 

That patch is not evaluated in Flacoco:

The buggy location is in the list but in other place (11 vs 6) :
[677240] INFO FlacocoFaultLocalization - Suspicious: 11 line org.apache.commons.math.optimization.direct.MultiDirectional l: 90, susp 0.7071067811865475

Math78 by Jkali

Discussed in #103 (originally created for jGenProg Math78)

jKali patch

Status: The patch found in Dj4-experiments seems not plausible in the CI. patch, failing line in CI

Both in the CI and locally we have 11 failing test cases

testDecreasingSteps(org.apache.commons.math.ode.nonstiff.ClassicalRungeKuttaIntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 5.7, Values: 21.969, 0.544-, testEvents(org.apache.commons.math.ode.nonstiff.DormandPrince54IntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 5.165, Values: 18.004, 0.082-, testEvents(org.apache.commons.math.ode.nonstiff.DormandPrince853IntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 5.462, Values: 7.188, 0.37-, testUnstableDerivative(org.apache.commons.math.ode.nonstiff.DormandPrince853IntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 1, Values: -2, -1-, testDecreasingSteps(org.apache.commons.math.ode.nonstiff.GillIntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 5.224, Values: 18.701, 0.138-, testUnstableDerivative(org.apache.commons.math.ode.nonstiff.GillIntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 0.767, Values: -2, -1.233-, testEvents(org.apache.commons.math.ode.nonstiff.GraggBulirschStoerIntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 1.942, Values: -5.731, -0-, testUnstableDerivative(org.apache.commons.math.ode.nonstiff.GraggBulirschStoerIntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 0.781, Values: -2, -1.219-, testEvents(org.apache.commons.math.ode.nonstiff.HighamHall54IntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 5.145, Values: 17.108, 0.061-, testDecreasingSteps(org.apache.commons.math.ode.nonstiff.MidpointIntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 11.247, Values: 3.1, 0.286-, testDecreasingSteps(org.apache.commons.math.ode.nonstiff.ThreeEighthesIntegratorTest): function values at endpoints do not have different signs.  Endpoints: 0, 5.7, Values: 21.969, 0.544-, ...and 1 failures more

Nopol patch

Nopol is the only capable of repairing it: http://program-repair.org/RepairThemAll_experiment/#!/bug/Defects4J/Math/78

jKali is not able to repair it in that position (delta = 0.5 * dx):
https://github.com/martinezmatias/astor/runs/3954513758?check_suite_focus=true#step:8:2796

Failing the regression:
testCumulative(org.apache.commons.math.distribution.BetaDistributionTest): Maximal number of iterations (100) exceeded-, testQuinticZero(org.apache.commons.math.analysis.solvers.BrentSolverTest): null-, testIllConditioned(org.apache.commons.math.optimization.general.NonLinearConjugateGradientOptimizerTest): -

Found locally with JKall:

testQuinticZero(org.apache.commons.math.analysis.solvers.BrentSolverTest): null-, testCumulative(org.apache.commons.math.distribution.BetaDistributionTest): Maximal number of iterations (100) exceeded-, testIllConditioned(org.apache.commons.math.optimization.general.NonLinearConjugateGradientOptimizerTest): -]|, fitness 3.0

Chart-15 jKali

With gZoltar repaired similar to RthemAll experiment, https://github.com/martinezmatias/astor/runs/4076936247?check_suite_focus=true#step:8:5213: but other location that D4jExperiment https://github.com/Spirals-Team/defects4j-repair/blob/master/results/2015-august/Chart/15/Kali/stdout.log
We have it as modif point https://github.com/martinezmatias/astor/runs/4076936247?check_suite_focus=true#step:8:1620
and suspicious line https://github.com/martinezmatias/astor/runs/4076936247?check_suite_focus=true#step:8:586

Number of test:
https://github.com/martinezmatias/astor/runs/4076936247?check_suite_focus=true#step:8:128

Near the suspicious:
https://github.com/martinezmatias/astor/runs/4076936247?check_suite_focus=true#step:8:5938

It seems there is a difference on the test cases executed

https://github.com/martinezmatias/astor/runs/4076936247?check_suite_focus=true#step:8:129
[22995] INFO GZoltarFaultLocalization - Test failt: org.jfree.chart.plot.junit.PiePlot3DTests#testDrawWithNullDataset
[22997] INFO GZoltarFaultLocalization - Test failt: org.jfree.chart.plot.junit.PiePlot3DTests#testDrawWithNullDataset
[23006] INFO GZoltarFaultLocalization - Gzoltar Test Result Total:3579, fails: 2, GZoltar suspicious 22439, with positive susp 572
[23006] INFO GZoltarFaultLocalization - nr test results 3579

https://github.com/martinezmatias/astor/runs/4076936247?check_suite_focus=true#step:8:5383
INFO CoverageRunner - Tests found: 1789

Math-95

Repaired by D4JExperiment similar than in RthellAll https://program-repair.org/RepairThemAll_experiment/#!/bug/Defects4J/Math/95 but there it's not repaired by jKali

Problem: change on an operator of jKali: InsertBefore and replace

Math71

Buggy location only in Flacoco https://github.com/martinezmatias/astor/runs/4076935028?check_suite_focus=true#step:8:17266

It's different from the D4jExperiment https://github.com/Spirals-Team/defects4j-repair/blob/master/results/2015-august/Math/71/Genprog/stdout.log#L27

That location is in both susp lists https://github.com/martinezmatias/astor/runs/4076935028?check_suite_focus=true#step:8:16990

TIME-11

The suspicious is in the list. The problem is the compilation of the patch on the CI.
https://github.com/martinezmatias/astor/runs/4168385486?check_suite_focus=true#step:8:1429

Math-40

The suspicious is in both lists, but as the space is so large, it's not considered

[andre15silva]

Closing this as the paper has been submitted