Removing setup time from function timing for J-unit

Question

So I need to test how long it takes to get the kth element from sorted arrays of varying order. I would expect the run time to be O(N), but instead it seems to be growing more like O(2^N). This indicates to me that J-Unit is timing my setup() function even though it is annotated with @Before. What am I doing wrong?

import org.junit.Before;
import org.junit.Test;

public class GetKthTest {
int[] sorted1;
BubbleSort sort = new BubbleSort();
private int k = 50;

@Before
public void setUp() {
    RandomListGen gen1 = new RandomListGen(80000);
    sorted1 = sort.sort(gen1.getArray());

}

@Test
public void hundredThous() {
    System.out.println(sorted1[k]);

}
}

Doesn't junit tell you how long `hundredThous` ran, regardless of other functions (in the test tree)? — Eran Zimmerman Gonen, Sep 14 '11 at 06:26
Eran He may be correct I have a similar problem with no correct answers but with @AfterClass http://stackoverflow.com/questions/7338153/junit-afterclass-run-time-is-added-to-a-poor-testcase — , Sep 14 '11 at 06:51

score 1 · Accepted Answer · edited May 23 '17 at 12:12

Actually I don't think you are doing anything wrong I think there is a bug in JUnit.

For reference I have almost the exact problem with @AfterClass JUnit @AfterClass run time is added to a poor testcase :(.

I have replicated your problem with generic code / JUnit4 in Eclipse.

public class GetKthTest {
int[] sorted1;
int nbr = 8000000;
private int k = 50;

@Before
public void setUp() {
    sorted1 = new int[nbr];
    Random rand = new Random();
    for (int i = 0; i < nbr; i++) {
        sorted1[i] = rand.nextInt();
    }
    Arrays.sort(sorted1);

}

@Test
public void hundredThous() {
    long start = System.currentTimeMillis();
    System.out.println(sorted1[k]);
    System.out.println("Time in ms: " + (System.currentTimeMillis() - start));
}

}

Prints

-2147455505
Time in ms: 0

,

JUnit report 1,399s

score 1 · Answer 2 · edited May 23 '17 at 12:29

You're not doing anything wrong. Junit sends a testStarted before it executes the @Before and testFinished after it executes the @After.

If it is possible to move the setUp code into a @BeforeClass method, then you do have a solution available to you, to use your own RunListener, for example:

public class RunJunitTestRunListener {
    private static class MyListener extends RunListener {
        private long runStart = 0L;
        private long testStart = 0L;

        @Override
        public void testRunStarted(Description description) throws Exception {
            System.out.println("runStarted");
            runStart = System.currentTimeMillis();
            super.testRunStarted(description);
        }

        @Override
        public void testRunFinished(Result result) throws Exception {
            System.out.println("runFinished " + (System.currentTimeMillis() - runStart) + "ms");
            super.testRunFinished(result);
        }

        @Override
        public void testStarted(Description description) throws Exception {
            System.out.println("testStarted");
            testStart = System.currentTimeMillis();
            super.testStarted(description);
        }

        @Override
        public void testFinished(Description description) throws Exception {
            System.out.println("testFinished " + (System.currentTimeMillis() - testStart) + "ms");
            super.testFinished(description);
        }
    }

    public static void main(String[] args) {
        JUnitCore core= new JUnitCore();
        core.addListener(new MyListener());
        core.run(TimingTest.class);
    }
}

testRunStarted gets called before the @BeforeClass, and testRunFinished after the @AfterClass.

Also, see my answer to JUnit @AfterClass run time is added to a poor testcase.

As general advice, I would use a benchmarking library to do measuring, because it is suprisingly hard to do.

Removing setup time from function timing for J-unit

2 Answers2