12

What's the best Python equivalent of Common Lisp's maplist function? From the maplist documentation:

maplist is like mapcar except that function is applied to successive sublists of the lists. function is first applied to the lists themselves, and then to the cdr of each list, and then to the cdr of the cdr of each list, and so on.

Example (pseudoy-code, not tested):

>>> def p(x): return x
>>> maplist(p, [1,2,3])
[[1, 2, 3], [2, 3], [3]]

Note: the arguments passed to p in the example above would be the lists [1, 2, 3], [2, 3], [3]; i.e., p is not applied to the elements of those lists. E.g.:

>>> maplist(lambda l: list(reversed(l)), [1,2,3])
[[3, 2, 1], [3, 2], [3]]
Jacob Gabrielson
  • 34,800
  • 15
  • 46
  • 64

9 Answers9

12

You can write a little function for that

def maplist(func, values):
    return [map(func, values[i:]) for i in xrange(len(values))]

>>> maplist(lambda a: a* 2, [1,2,3])
[[2, 4, 6], [4, 6], [6]]

[Edit]

if you want to apply the function on the sublists you can change the function to this:

def maplist(func, values):
    return [func(values[i:]) for i in xrange(len(values))]

>>> maplist(lambda l: list(reversed(l)), [1,2,3])
[[3, 2, 1], [3, 2], [3]]
Marcin
  • 48,559
  • 18
  • 128
  • 201
Nadia Alramli
  • 111,714
  • 37
  • 173
  • 152
  • 1
    You can't slice iterators. So the second parameter is badly named. – Aaron Maenpaa May 27 '09 at 17:36
  • I probably chose an ambiguous example in my question, unfortunately; 'maplist' should apply 'func' directly to the sublists [1,2,3], [2,3], [3], not the elements of those sublists. – Jacob Gabrielson May 27 '09 at 18:14
  • @Jacob, check the edit, I added another function that does what you want. – Nadia Alramli May 27 '09 at 18:27
  • -1 because that function isn't equivalent. LISP's maplist is a linear time algorithm. – Cybis May 27 '09 at 19:19
  • Couple of things: This doesn't appear to do what maplist does at all. The examples I've seen of it all show *multiple* lists, and the size of the output is equal to the size of the smallest list. For example: maplist(lambda a: a*2, [[1,2,3], [1,2]]) -> [ [[2,4,6], [2,4]], [[2,4], [2]] ]. – Kamil Kisiel May 27 '09 at 23:54
7

As @Cybis and others mentioned, you can't keep the O(N) complexity with Python lists; you'll have to create a linked list. At the risk of proving Greenspun's 10th rule, here is such a solution:

class cons(tuple):
    __slots__=()

    def __new__(cls, car, cdr):
        return tuple.__new__(cls, (car,cdr))

    @classmethod
    def from_seq(class_, l):
        result = None
        for el in reversed(l):
            result = cons(el, result)
        return result

    @property
    def car(self): return self._getitem(0)

    @property
    def cdr(self): return self._getitem(1)

    def _getitem(self, i):
        return tuple.__getitem__(self, i)

    def __repr__(self):
        return '(%s %r)' % (self.car, self.cdr)

    def __iter__(self):
        v = self
        while v is not None:
            yield v.car
            v = v.cdr

    def __len__(self):
        return sum(1 for x in self)

    def __getitem__(self, i):
        v = self
        while i > 0:
            v = v.cdr
            i -= 1
        return v.car

def maplist(func, values):
    result = [ ]
    while values is not None:
        result.append(func(values))
        values = values.cdr
    return result

Testing yields:

>>> l = cons.from_seq([1,2,3,4])
>>> print l
(1 (2 (3 (4 None))))
>>> print list(l)
[1, 2, 3, 4]
>>> print maplistr(lambda l: list(reversed(l)), cons.from_seq([1,2,3]))
[[3, 2, 1], [3, 2], [3]]

EDIT: Here is a generator-based solution that basically solves the same problem without the use of linked lists:

import itertools

def mapiter(func, iter_):
    while True:
        iter_, iter2 = itertools.tee(iter_)
        iter_.next()
        yield func(iter2)

Testing yields:

>>> print list(mapiter(lambda l: list(reversed(list(l))), [1,2,3]))
[[3, 2, 1], [3, 2], [3]]
Rick Copeland
  • 11,672
  • 5
  • 39
  • 38
  • 2
    +1 for showing that not even Python can escape Greenspun's 10th Rule. – Cybis May 27 '09 at 20:46
  • 1
    Your generator solution is giving this error on big lists: RuntimeError: maximum recursion depth exceeded – Nadia Alramli May 27 '09 at 22:04
  • And is 7 times slower than the regular list comprehension solution when running on a 100 entry list for 1000 times. – Nadia Alramli May 27 '09 at 22:08
  • @Rick, I tested your both solutions and both of them are very slow even for large lists. The first one is much slower than the second. And the second is 7 times slower than the regular list comprehension solution. – Nadia Alramli May 27 '09 at 22:16
  • @Nadia, thanks for the comments. I would expect both solutions to be fairly slow (and would expect the mapiter() solution to have a recursion depth problem.) The goal of this answer was simply to replicate the O(N) scaling of maplist, not necessarily to make it faster overall. – Rick Copeland May 27 '09 at 22:53
  • Most of the speed problems were due to the extravagant use of recursion. I have edited the solution to use iteration instead and now both solutions (linked list and generator-based) should beat list comprehensions on lists of more than 1000 elements. – Rick Copeland May 27 '09 at 23:45
  • @Rick, are you sure they beat list comprehensions? My results show otherwise. For 10000 elements list: the generator took 0m2.808s while list comprehension took 0m2.083s. I'm not arguing whether O(N) or O(N^2) is better. My point is that there might be hidden python overhead that we should take into account. – Nadia Alramli May 28 '09 at 00:01
  • The test I ran was using the identity function for "func" (lambda l:l). For my iterator solution, I used a list [None]*int(1e8) and completed 1000 runs in timeit in 0.00134s. When I tried the same for the list comprehension solution, I got a MemoryError, so I reduced the list to [None]*1000 and it still took 9.79s for 1000 runs. My linked list was also a memory hog, so I ran it on the 1000-element list and it completed in 1.4049s. All this is running on Python 2.5.2. To reproduce, here is maplist.py: http://pastebin.com/m54e69b8c and here is my timer script:http://pastebin.com/m7422c943 – Rick Copeland May 28 '09 at 00:35
  • I should mention that all my comparisons are with @Nadia's edited maplist() that applies func on all the sublists, if that makes a difference. – Rick Copeland May 28 '09 at 00:37
  • I did notice that my iterator timing was off because I didn't actually measure the time to exhaust the returned iterator. Fixing this (http://pastebin.com/m123068a), the generator solution is still faster than list comprehensions at 1.46s/1000 runs on a 1000-element list. – Rick Copeland May 28 '09 at 01:33
  • Do you know that deque is a linked structure? You don't have to create your own. – Marcin Mar 18 '12 at 10:05
4

Eewww... Slicing a list is a linear-time operation. All of the solutions posted thus far have O(n^2) time complexity. Lisp's maplist, I believe, has O(n).

The problem is, Python's list type isn't a linked list. It's a dynamically resizable array (i.e., like C++ STL's "vector" type).

If maintaining linear time complexity is important to you, it isn't possible to create a "maplist" function over Python lists. It would be better to modify your code to work with indices into the list, or convert the list into an actual linked list (still a linear-time operation, but would have a lot of overhead).

Cybis
  • 9,773
  • 2
  • 36
  • 37
  • That is a good point. In the case I was thinking of the lists would be short enough that it wouldn't be a concern. But in the general case it could. Maybe something involving generators would work, too? – Jacob Gabrielson May 27 '09 at 20:17
  • 1
    That's a good idea - if maplist accepted a function which itself accepted generators instead of lists. The implementation would be a bit convoluted, and there would be more overhead than LISP's maplist (from creating all those generators), but I think it could work. – Cybis May 27 '09 at 20:33
  • I've edited my answer below (hopefully soon above) to add a generator solution – Rick Copeland May 27 '09 at 20:49
  • Python has a linked list type, it's called deque. I have added an answer with a linear solution using that. – Marcin Mar 18 '12 at 10:15
  • here's [linear time algorithm](http://stackoverflow.com/a/925611/4279) (though it is non-practical) – jfs Oct 19 '13 at 20:27
2

Python has a linked list type, called deque:

from collections import deque

def maplist(f, lst):
    linked = deque(lst)
    results = []
    while linked:
        results.append(f(linked))
        linked.popleft() # assumes left is head, as in lisp

    return results

In [134]: maplist(lambda l: list(reversed(l))*2, [1,2,3])
Out[134]: [[3, 2, 1, 3, 2, 1], [3, 2, 3, 2], [3, 3]]

Linear time, does the same thing, will have the same performance characteristics as operations on lisp lists (i.e. modifying interior elements will be linear in the length of the list being modified), and the same semantics (i.e. if f modifies the list, it modifies it for all subsequent executions of f within that maplist call; however, because this creates a copy of lst, f cannot modify lst, unlike in Common Lisp).

Marcin
  • 48,559
  • 18
  • 128
  • 201
  • `deque` is implemented using doubly linked list but it is not LL itself e.g., how would you reference separate parts of it? – jfs Oct 19 '13 at 20:31
  • @J.F.Sebastian In fairness, you couldn't, except by numeric index. – Marcin Oct 19 '13 at 22:02
1

this works like your example (I've modified reyjavikvi's code)

def maplist(func, l):
    result=[]
    for i in range(len(l)):
        result.append(func(l[i:]))
    return result
Adrian Mester
  • 2,523
  • 1
  • 19
  • 23
1

You can use nested list comprehensions:

>>> def p(x): return x
>>> l = range(4)[1:]
>>> [p([i:]) for i in range(len(l))]
[[1, 2, 3], [2, 3], [3]]

Which you can use to define maplist yourself:

>>> def maplist(p, l): return [p([i:]) for i in range(len(l))]
>>> maplist(p, l)
[[1, 2, 3], [2, 3], [3]]
Aaron Maenpaa
  • 119,832
  • 11
  • 95
  • 108
  • I probably chose an ambiguous example by just using the identity function; the function "p" should just be passed the sublists themselves, not the sublists' elements. I added a clarification to the question. – Jacob Gabrielson May 27 '09 at 18:13
  • @Jacob That makes it easier :) – Aaron Maenpaa May 27 '09 at 18:56
  • I'm wondering how you tested your proposed solution to give the results you list. When I tried to replicate what you entered, the "p([i:])" gave me a syntax error for both places in your response. – Bill Evans at Mariposa Apr 09 '20 at 15:34
1

Modifying Aaron's answer:

In [8]: def maplist(p, l): return [p([x for x in l[i:]]) for i in range(len(l))]
   ...: 

In [9]: maplist(lambda x: x + x, [1,2,3])
Out[9]: [[1, 2, 3, 1, 2, 3], [2, 3, 2, 3], [3, 3]]
hbn
  • 1,846
  • 2
  • 12
  • 12
1

O(N) implementation of maplist() for linked lists

maplist = lambda f, lst: cons(f(lst), maplist(f, cdr(lst))) if lst else lst

See Python Linked List question.

Community
  • 1
  • 1
jfs
  • 399,953
  • 195
  • 994
  • 1,670
-2

I think there isn't, but the following function can work:

def maplist(func, l):
    for i in range(len(l)):
        func(l[i:])
Javier
  • 4,552
  • 7
  • 36
  • 46