Python testing: using a fake file with mock & io.StringIO

Question

I'm trying to test some code that operates on a file, and I can't seem to get my head around how to replace using a real file with mock and io.StringIO My code is pretty much the following:

class CheckConfig(object):
    def __init__(self, config):
        self.config = self._check_input_data(config)

    def _check_input_data(self, data):
        if isinstance(data, list):
            return self._parse(data)
        elif os.path.isfile(data):
            with open(data) as f:
                return self._parse(f.readlines())

    def _parse(self, data):
        return data

I have a class that can take either a list or a file, if it's a file it opens it and extracts the contents into a list, and then does what it needs to do to the resulting list.

I have a working test as follows:

def test_CheckConfig_with_file():
    config = 'config.txt'
    expected = parsed_file_data
    actual = CheckConfig(config).config
    assert expected == actual

I want to replace the call to the filesystem. I have tried replacing the file with io.StringIO but I get a TypeError from os.path.isfile() as it's expecting either a string, bytes or int. I also tried mocking the isfile method like so:

@mock.patch('mymodule.os.path')
def test_CheckConfig_with_file(mock_path):
    mock_path.isfile.return_value = True
    config = io.StringIO('data')
    expected = parsed_file_data
    actual = CheckConfig(config).config
    assert expected == actual

but I still get the same TypeError as the _io.StringIO type is causing the exception before isfile gets a chance to return something.

How can I get os.path.isfile to return True, when I pass it a fake file? Or is this a suggestion I should change my code?

Martijn Pieters · Accepted Answer · 2016-11-03T18:25:25.720

Just mock out both os.path.isfile and the open() call, and pass in a fake filename (you are not expected to pass in an open file, after all).

The mock library includes a utility for the latter: mock_open():

@mock.patch('os.path.isfile')
def test_CheckConfig_with_file(mock_isfile):
    mock_isfile.return_value = True
    config_data = mock.mock_open(read_data='data')
    with mock.patch('mymodule.open', config_data) as mock_open:
        expected = parsed_file_data
        actual = CheckConfig('mocked/filename').config
        assert expected == actual

This causes the if isinstance(data, list): test to be false (because data is a string instead), followed by the elif os.path.isfile(data): returning True, and the open(data) call to use your mocked data from the mock_open() result.

You can use the mock_open variable to assert that open() was called with the right data (mock_open. assert_called_once_with('mocked/filename') for example).

Demo:

>>> import os.path
>>> from unittest import mock
>>> class CheckConfig(object):
...     def __init__(self, config):
...         self.config = self._check_input_data(config)
...     def _check_input_data(self, data):
...         if isinstance(data, list):
...             return self._parse(data)
...         elif os.path.isfile(data):
...             with open(data) as f:
...                 return self._parse(f.readlines())
...     def _parse(self, data):
...         return data
...
>>> with mock.patch('os.path.isfile') as mock_isfile:
...     mock_isfile.return_value = True
...     config_data = mock.mock_open(read_data='line1\nline2\n')
...     with mock.patch('__main__.open', config_data) as mock_open:
...         actual = CheckConfig('mocked/filename').config
...
>>> actual
['line1\n', 'line2\n']
>>> mock_open.mock_calls
[call('mocked/filename'),
 call().__enter__(),
 call().readlines(),
 call().__exit__(None, None, None)]

Excellent, thankyou! This is exactly what I needed, and has helped me get a better grasp on mocking. Just out of interest, is there a reason you are using a context manager rather than a `@mock.patch` decorator. In previous tests I had been using the decorator, but here I couldn't find a way as I was using a `pytest.fixture` for the content of the `open` call, which wasn't available until inside the test function. — bordeltabernacle, Nov 03 '16 at 20:39
for anyone stumbling on this answer. mock_open currently doesn't support iterables so if instead of f.readlines() you used yield you will have to work around the iteration. https://stackoverflow.com/questions/24779893/customizing-unittest-mock-mock-open-for-iteration — Luis Meraz, May 02 '18 at 06:22
Echoing bordeltabernacle comment. Based on this answer I could mock file contents using either a decorator or context manager. However, the decorator did not support taking a parameter as identifier so I was in the unusual position of having a patch decorator and just `self` parameter. — John, Jun 03 '20 at 20:43
The only reason I used a context manager here is that it is easier to demo with the context manager. The decorator version work exactly the same; the decorator just sets up the context manager and passes in the patch object as an extra argument to the decorated function. — Martijn Pieters, Jun 04 '20 at 11:45

score 4 · Answer 2 · answered Oct 14 '20 at 08:15

4

In case you end up here wondering how to solve this using the pytest-mock library, here is how you do it:

def test_open(mocker):
    m = mocker.patch('builtins.open', mocker.mock_open(read_data='bibble'))
    with open('foo') as h:
        result = h.read()

    m.assert_called_once_with('foo')
    assert result == 'bibble'

This code example was found (but had to be adjusted) here.

answered Oct 14 '20 at 08:15

Manu CJ

2,629
1
18
29

Except this works for absolutely everything, including templates for errors and exceptions, which can lead to surprising results! :-) – Rmatt Nov 04 '20 at 12:30

Python testing: using a fake file with mock & io.StringIO

2 Answers2

Linked