Python unittest: test a test

Question

I'm writing some code that works with both very large and very small floating point numbers (for example, 1e-150 could be a valid answer). To unit test this I'd like to compare floats to a number of significant figures instead of decimal places, so I have the following.

import unittest as ut
from numpy.testing import assert_approx_equal

class newTestCase(ut.TestCase):
"""Extends the basic unittest TestCase."""

def assertSFAlmostEqual(self, a, b, places=7):
    """Uses numpy to test if two floats are the same but to a defined
    number of significant figures rather than decimal places.

    Args:
        a: float to be compared
        b: float to be compared
        places: number of significant figures to match. unittest default
        for assertAlmostEqual is 7, so 7 is the default here
    """
    if isinstance(a, float) != True or isinstance(b, float) != True:
        raise TypeError

    raised = False
    try:
        assert_approx_equal(a, b, significant=places)
    except:
        raised = True
    self.assertFalse(raised, "FLoats %g and %g are not equal to %i "
                     "significant figures" % (a, b, places))

Which seems to work fine, but I'm planning to use this in a lot of places, so I'd like to be certain that it really works correctly. My question is how can I do this most sensibly? Is there an proper mechanism to unit test a unit test?

I found what may be an answer here,

How to unittest unittest TestCases

but I don't understand how this works.

Thanks very much in advance!

Answer 1

A subclass of unittest.TestCase is like any other class, hence you can write a unittest.TestCase that checks whether its methods work as they should.

In particular you should build sets of pairs of numbers that should pass and fail the test, and then call the assertSFAlmostEqual method with these inputs and see whether the test pass or fail.

The answer you linked does this, even though it's probably a more complicated solution than what's required. For example, I'd simply have written something like:

import unittest


class MyBaseTestCase(unittest.TestCase):
    def assertSpec(self, thing):
        assert thing == 123


class TestMyTest(MyBaseTestCase):
    def test_failures(self):
        self.assertRaises(AssertionError, self.assertSpec, 121)

    def test_successes(self):
        self.assertSpec(123)

if __name__ == "__main__":
    unittest.main()

You simply subclass the test-case and al the tests simply call the assert* method you wrote with specific arguments that you know should pass/not pass the test.

---

Some notes on your current implementation of the assert* method:

if isinstance(a, float) != True or isinstance(b, float) != True:

Avoid comparing with True or False. In your case you can simply write:

if not isinstance(a, float) or not isinstance(b, float):
# or:
if not (isinstance(a, float) and isinstance(b, float))

Which is also clearer to read.

raised = False
try:
    assert_approx_equal(a, b, significant=places)
except:
    raised = True

Never catch exceptions using a plain except:. In this case you really want to catch only the AssertionError raised by assert_approx_equal, hence you should use:

raised = False
try:
    assert_approx_equal(a, b, significant=places)
except AssertionError:
    raised = True

Secondly, you can avoid using the raised flag. The try-except statement allows an else clause that is executed only when no exception was raised:

try:
    assert_approx_equal(a, b, significant=places)
except AssertionError:
    # here put the code to be executed when the assertion fails.
else:
    # here put the code to be executed when *no* exception was raised.

Answer 2

One way is TDD (Test Driven Development):

1. Write a failing test.
2. Make the code pass the test.
3. Refactor.
4. Goto 1.

The crux here is to write a failing test first.