Python - breaking and grouping user input

Question

I'm doing an exercise I came over in Learn Python the Hard Way(ex 48). The aim is to group user input by referring to our lexicon. I'm using nose to test my script but I get multiple errors. I get 5 failures out of 6 when I run nosetests. I don't understand why I'm getting these errors though. Any help?

errors

FAIL: tests.ex48_tests.test_verbs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\python\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Python\projects\skeleton2\tests\ex48_tests.py", line 13, in test_verbs
    assert_equal(scan("go").result, [('verb', 'go')])
AssertionError: <bound method scan.result of <ex48.lexicon.scan object at 0x03A8F3F0>> != [('verb', 'go')]

======================================================================
FAIL: tests.ex48_tests.test_stops
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\python\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Python\projects\skeleton2\tests\ex48_tests.py", line 21, in test_stops
    assert_equal(scan("the").result(), [('stop', 'the')])
AssertionError: Lists differ: [('stop', 'the'), ('error', 'the')] != [('stop', 'the')]

First list contains 1 additional elements.
First extra element 1:
('error', 'the')

- [('stop', 'the'), ('error', 'the')]
+ [('stop', 'the')]

======================================================================
FAIL: tests.ex48_tests.test_noun
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\python\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Python\projects\skeleton2\tests\ex48_tests.py", line 29, in test_noun
    assert_equal(scan("bear").result(), [('noun', 'bear')])
AssertionError: Lists differ: [('noun', 'bear'), ('error', 'bear')] != [('noun', 'bear')]

First list contains 1 additional elements.
First extra element 1:
('error', 'bear')

- [('noun', 'bear'), ('error', 'bear')]
+ [('noun', 'bear')]

======================================================================
FAIL: tests.ex48_tests.test_numbers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\python\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Python\projects\skeleton2\tests\ex48_tests.py", line 35, in test_numbers
    assert_equal(scan("1234").result(), [('number', 1234)])
AssertionError: Lists differ: [('error', '1234')] != [('number', 1234)]

First differing element 0:
('error', '1234')
('number', 1234)

- [('error', '1234')]
?     ---    -    -

+ [('number', 1234)]
?    ++++


======================================================================
FAIL: tests.ex48_tests.test_errors
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\python\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Python\projects\skeleton2\tests\ex48_tests.py", line 45, in test_errors
    ('noun', 'princess')])
AssertionError: Lists differ: [('no[20 chars]r', 'bear'), ('error', 'IAS'), ('noun', 'princ[24 chars]ss')] != [('no[20 chars]r', 'IAS'), ('noun', 'princess')]

First differing element 1:
('error', 'bear')
('error', 'IAS')

First list contains 2 additional elements.
First extra element 3:
('noun', 'princess')

+ [('noun', 'bear'), ('error', 'IAS'), ('noun', 'princess')]
- [('noun', 'bear'),
-  ('error', 'bear'),
-  ('error', 'IAS'),
-  ('noun', 'princess'),
-  ('error', 'princess')]

----------------------------------------------------------------------
Ran 6 tests in 0.027s

FAILED (failures=5)

lexicon.py

class scan(object):
    dirs = ['north','south','east','west','down','up','left','right','back']
    verbs = ['go','stop','kill','eat']
    stops = ['the','in','of','from','at','it']
    nouns = ['door','princess','bear','cabinet']
    numbers = ['0','1','2','3','4','5','6','7','8','9']

    def __init__(self, user_input):
        self.user_input = user_input

    def result(self):
        words = self.user_input.split()
        results = []

        for item in words:
            if item in scan.dirs:
                result = ('direction', item.lower())
                results.append(result)
            if item in scan.verbs:
                result = ('verb', item.lower())
                results.append(result)
            if item in scan.stops:
                result = ('stop', item.lower())
                results.append(result)
            if item in scan.nouns:
                result =('noun', item.lower())
                results.append(result)
            if item in scan.numbers:
                result = ('number', int(item))
                results.append(result)
            if item not in (scan.dirs or scan.verbs or scan.stops or
                            scan.nouns or scan.numbers):
                result = ('error', item)
                results.append(result)

        return results

lexicon_test.py

from nose.tools import *
from ex48.lexicon import scan


def test_direction():
    assert_equal(scan('north').result(), [('direction', 'north')])
    result = scan("north east south").result()
    assert_equal(result, [('direction', 'north'),
                          ('direction', 'east'),
                          ('direction', 'south')])

def test_verbs():
    assert_equal(scan("go").result, [('verb', 'go')])
    result = scan("go kill eat").result()
    assert_equal(result, [('verb', 'go'),
                          ('verb', 'eat')
                          ('verb', 'kill')])


def test_stops():
    assert_equal(scan("the").result(), [('stop', 'the')])
    result = scan("the in of").result()
    assert_equal(result, [('stop', 'the'),
                          ('stop', ' in'),
                          ('stop', 'of')])


def test_noun():
    assert_equal(scan("bear").result(), [('noun', 'bear')])
    result = scan("bear princess").result()
    assert_equal(result, [('noun', 'bear'),
                           ('noun', 'princess')])

def test_numbers():
    assert_equal(scan("1234").result(), [('number', 1234)])
    result = scan("3 91234").result()
    assert_equal(result, [('number', 3),
                          ('number', 91234)])

def test_errors():
    assert_equal(scan("ASDFADFASDF").result(), [('error', 'ASDFADFASDF')])
    result = scan("bear IAS princess").result()
    assert_equal(result, [('noun', 'bear'),
                          ('error', 'IAS'),
                           ('noun', 'princess')])

Answer 1

You have a couple of typos in your code and a couple of logic errors.

Here's a repaired version of your code, modified to run without the nose module (which I don't have).

class scan(object):
    dirs = ['north','south','east','west','down','up','left','right','back']
    verbs = ['go','stop','kill','eat']
    stops = ['the','in','of','from','at','it']
    nouns = ['door','princess','bear','cabinet']
    numbers = ['0','1','2','3','4','5','6','7','8','9']

    def __init__(self, user_input):
        self.user_input = user_input

    def result(self):
        words = self.user_input.split()
        results = []

        for item in words:
            if item in scan.dirs:
                result = ('direction', item.lower())
                results.append(result)
            elif item in scan.verbs:
                result = ('verb', item.lower())
                results.append(result)
            elif item in scan.stops:
                result = ('stop', item.lower())
                results.append(result)
            elif item in scan.nouns:
                result =('noun', item.lower())
                results.append(result)
            elif all(c in scan.numbers for c in item):
                result = ('number', int(item))
                results.append(result)
            else:
                result = ('error', item)
                results.append(result)

        return results

def assert_equal(u, v):
    print(u, v, u == v)

def test_direction():
    assert_equal(scan('north').result(), [('direction', 'north')])
    result = scan("north east south").result()
    assert_equal(result, [('direction', 'north'),
                          ('direction', 'east'),
                          ('direction', 'south')])

def test_verbs():
    assert_equal(scan("go").result(), [('verb', 'go')])
    result = scan("go kill eat").result()
    assert_equal(result, [('verb', 'go'),
                          ('verb', 'kill'),
                          ('verb', 'eat')])


def test_stops():
    assert_equal(scan("the").result(), [('stop', 'the')])
    result = scan("the in of").result()
    assert_equal(result, [('stop', 'the'),
                          ('stop', 'in'),
                          ('stop', 'of')])


def test_noun():
    assert_equal(scan("bear").result(), [('noun', 'bear')])
    result = scan("bear princess").result()
    assert_equal(result, [('noun', 'bear'),
                           ('noun', 'princess')])

def test_numbers():
    assert_equal(scan("1234").result(), [('number', 1234)])
    result = scan("3 91234").result()
    assert_equal(result, [('number', 3),
                          ('number', 91234)])

def test_errors():
    assert_equal(scan("ASDFADFASDF").result(), [('error', 'ASDFADFASDF')])
    result = scan("bear IAS princess").result()
    assert_equal(result, [('noun', 'bear'),
                          ('error', 'IAS'),
                           ('noun', 'princess')])

tests = (
    test_direction,
    test_verbs,
    test_stops,
    test_noun,
    test_numbers,
    test_errors,
)

for test in tests:
    print('\n' + test.__name__)
    test()

output

test_direction
[('direction', 'north')] [('direction', 'north')] True
[('direction', 'north'), ('direction', 'east'), ('direction', 'south')] [('direction', 'north'), ('direction', 'east'), ('direction', 'south')] True

test_verbs
[('verb', 'go')] [('verb', 'go')] True
[('verb', 'go'), ('verb', 'kill'), ('verb', 'eat')] [('verb', 'go'), ('verb', 'kill'), ('verb', 'eat')] True

test_stops
[('stop', 'the')] [('stop', 'the')] True
[('stop', 'the'), ('stop', 'in'), ('stop', 'of')] [('stop', 'the'), ('stop', 'in'), ('stop', 'of')] True

test_noun
[('noun', 'bear')] [('noun', 'bear')] True
[('noun', 'bear'), ('noun', 'princess')] [('noun', 'bear'), ('noun', 'princess')] True

test_numbers
[('number', 1234)] [('number', 1234)] True
[('number', 3), ('number', 91234)] [('number', 3), ('number', 91234)] True

test_errors
[('error', 'ASDFADFASDF')] [('error', 'ASDFADFASDF')] True
[('noun', 'bear'), ('error', 'IAS'), ('noun', 'princess')] [('noun', 'bear'), ('error', 'IAS'), ('noun', 'princess')] True

---

The first logic error I spotted was

if item not in (scan.dirs or scan.verbs or scan.stops 
    or scan.nouns or scan.numbers):

That doesn't test if item isn't in any of those lists. Instead, it first computes

scan.dirs or scan.verbs or scan.stops or scan.nouns or scan.numbers

using the standard rules for Python's or operator. scan.dirs is a non-empty list, so the result of that expression is simply scan.dirs.

So that if statement is equivalent to

if item not in scan.dirs:

which is clearly not what you intend to do.

For more info about how or and and work in Python, please see this answer I wrote earlier this year.

We could implement that test using

if not any(item in seq for seq in (scan.dirs, scan.verbs, scan.stops, 
    scan.nouns, scan.numbers)):

but we don't need to do that. Instead we change most of those ifs into elifs, and then anything that doesn't get successfully scanned must be an error, so we can handle that in the else block.

The second big logic error is your number test. You were trying to see if a multi-digit number string is a valid (positive) integer with

if item in scan.numbers:

but that test will only succeed if item is a single digit.

Instead we need to check that _all_digits of the number are, in fact, digits, and that's what

all(c in scan.numbers for c in item)

does.

However, there's a better way: we just use the str type's .isdigit method:

if item.isdigit():

I didn't use that in my code because I wanted to use your scan lists. Also, .isdigit can't handle negative numbers or decimal points, but you can easily add '-' and '.' to scan.numbers.