How to monkeypatch dunder methods to existing instances?

Question

Context: I'd like to use heapq (and anything else) on objects I didn't create, which don't themselves have a __lt__ operator. Can I? (without a wrapper class).

the class:

class Node:
    def __init__(self, val):
        self.val = val

Now, at runtime in the interpreter, I am handed some collection of objects. I want to iterate over them, adding a dunder method (in my case lt), eg:

n = Node(4)
m = Node(5)

def myLT(self, other):
    return self.val < other.val

What I tried:

n.__lt__ = types.MethodType(myLT, n)
m.__lt__ = types.MethodType(myLT, m)

also

n.__lt__ = types.MethodType(myLT, n)
m.__lt__ = types.MethodType(myLT, n)

(on the off chance that binding the same functor-thing would improve matters)

>>> n < m
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '<' not supported between instances of 'Node' and 'Node'

even though:

>>> n.__lt__(m)
True

I can use a wrapper class, which is yucky in some ways (extra memory and traversal code gets uglier, but at least leaves the original objects untouched):

class NodeWrapper:
    def __init__(self, n):
        self.node = n
    def __lt__(self):
        return self.node.val

I'm just interested to know if I'm doing something wrong in adding the dunder method, or if this just doesn't work in python 3.x. I'm using 3.6.9 if that matters.

A wrapper is probably what you want, honestly. If you want to decrease the memory requirements, use `__slots__`. — juanpa.arrivillaga, Dec 26 '20 at 13:16

cs95 · Accepted Answer · 2020-12-26T13:17:56.920

2

You can try monkeypatching the dunder by changing the __class__ property of the instance. As explained in by docs section Special method lookup:

For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.

def patch_call(instance, func, memo={}):
    if type(instance) not in memo:
        class _(type(instance)):
            def __lt__(self, *arg, **kwargs):
               return func(self, *arg, **kwargs)
        memo[type(instance)] = _

    instance.__class__ = memo[type(instance)]

patch_call(m, myLT)
patch_call(n, myLT)

n < m
# True

Modified from reference.

Thanks to @juanpa.arrivilaga for recommending the classes be cached to improve performance.

edited Dec 26 '20 at 13:17

answered Dec 26 '20 at 12:29

cs95

379,657
97
704
746

You must be doing something different, or I misunderstood. I tried the above and get: `TypeError: myLT() missing 1 required positional argument: 'other'` on onlinegdb – wheelreinventor Dec 26 '20 at 12:59
@wheelreinventor It was a copy paste error in my code, will you take another shot at this now? – cs95 Dec 26 '20 at 13:02
I would cache based on `type(instance)`, or else you are creating a class for each instance. Which can be incredibly wasteful – juanpa.arrivillaga Dec 26 '20 at 13:12
fwiw, I observed this experimentally, monkeypatching using this method is about twice as slow, and ~25% more memory given the parameters vs a wrapper class. – wheelreinventor Dec 26 '20 at 13:14
@juanpa.arrivillaga interesting, that seems to address the performance issue OP just mentioned. How would the result be cached? Using `functools.lru_cache`? – cs95 Dec 26 '20 at 13:14
welll you would need to pass the type as the argument if you wanted to use `lru_cache` but you can just keep a dict, something like `seen_types` then `if type(instance) in seen_types: _ = seen_types[type(instance)] else: ...` – juanpa.arrivillaga Dec 26 '20 at 13:15
Great comments folks, thank you! @wheelreinventor I invite you to take a look at the revised answer based on juanpa's comment on caching the class. – cs95 Dec 26 '20 at 13:18
finally a real life use case for the mutable default argument! – timgeb Mar 07 '22 at 10:43

How to monkeypatch dunder methods to existing instances?

1 Answers1

Linked