Registering classes to factory with classes in different files

Question

I have a factory as shown in the following code:

class ClassFactory:
    registry = {}

    @classmethod
    def register(cls, name):
        def inner_wrapper(wrapped_class):
            if name in cls.registry:
                print(f'Class {name} already exists. Will replace it')
            cls.registry[name] = wrapped_class
            return wrapped_class
    return inner_wrapper

    @classmethod
    def create_type(cls, name):
        exec_class = cls.registry[name]
        type = exec_class()
        return type


@ClassFactory.register('Class 1')
class M1():
    def __init__(self):
       print ("Starting Class 1")


@ClassFactory.register('Class 2')
class M2():
    def __init__(self):
       print("Starting Class 2")

This works fine and when I do

if __name__ == '__main__':
    print(ClassFactory.registry.keys())
    foo = ClassFactory.create_type("Class 2")

I get the expected result of dict_keys(['Class 1', 'Class 2']) Starting Class 2

Now the problem is that I want to isolate classes M1 and M2 to their own files m1.py and m2.py, and in the future add other classes using their own files in a plugin manner. However, simply placing it in their own file m2.py

from test_ import ClassFactory
@MethodFactory.register('Class 2')
class M2():
    def __init__(self):
        print("Starting Class 2")

gives the result dict_keys(['Class 1']) since it never gets to register the class.

So my question is: How can I ensure that the class is registered when placed in a file different from the factory, without making changes to the factory file whenever I want to add a new class? How to self register in this way? Also, is this decorator way a good way to do this kind of thing, or are there better practices?

Thanks

Answer 1

How can I ensure that the class is registered when placed in a file different from the factory, without making changes to the factory file whenever I want to add a new class?

I'm playing around with a similar problem, and I've found a possible solution. It seems too much of a 'hack' though, so set your critical thinking levels to 'high' when reading my suggestion below :)

As you've mentioned in one of your comments above, the trick is to force the loading of the individual *.py files that contain individual class definitions.

Applying this to your example, this would involve:

1. Keeping all class implementations in a specific folders, e.g., structuring the files as follows:

.
└- factory.py     # file with the ClassFactory class
└─ classes/
  └- __init__.py
  └- m1.py        # file with M1 class
  └- m2.py        # file with M2 class

2. Adding the following statement to the end of your factory.py file, which will take care of loading and registering each individual class:

from classes import *

3. Add a piece of code like the snippet below to your __init__.py within the classes/ foder, so that to dynamically load all classes [1]:

from inspect import isclass
from pkgutil import iter_modules
from pathlib import Path
from importlib import import_module

# iterate through the modules in the current package
package_dir = Path(__file__).resolve().parent
for (_, module_name, _) in iter_modules([package_dir]):

    # import the module and iterate through its attributes
    module = import_module(f"{__name__}.{module_name}")
    for attribute_name in dir(module):
        attribute = getattr(module, attribute_name)

        if isclass(attribute):            
            # Add the class to this package's variables
            globals()[attribute_name] = attribute

If I then run your test code, I get the desired result:

# test.py

from factory import ClassFactory

if __name__ == "__main__":
    print(ClassFactory.registry.keys())
    foo = ClassFactory.create_type("Class 2")

$ python test.py
dict_keys(['Class 1', 'Class 2'])
Starting Class 2

Also, is this decorator way a good way to do this kind of thing, or are there better practices?

Unfortunately, I'm not experienced enough to answer this question. However, when searching for answers to this problem, I've came across the following sources that may be helpful to you:

• [2] : this presents a method for registering class existence based on Python Metaclasses. As far as I understand, it relies on the registering of subclasses, so I don't know how well it applies to your case. I did not follow this approach, as I've noticed that the new edition of the book suggests the use of another technique (see bullet below).
• [3], item 49 : this is the 'current' suggestion for subclass registering, which relies on the definition of the __init_subclass__() function in a base class.

If I had to apply the __init_subclass__() approach to your case, I'd do the following:

1. Add a Registrable base class to your factory.py (and slightly re-factor ClassFactory), like this:

class Registrable:
    def __init_subclass__(cls, name:str):
        ClassFactory.register(name, cls)

class ClassFactory:
    registry = {}

    @classmethod
    def register(cls, name:str, sub_class:Registrable):
        if name in cls.registry:
             print(f'Class {name} already exists. Will replace it')
        cls.registry[name] = sub_class

    @classmethod
    def create_type(cls, name):
        exec_class = cls.registry[name]
        type = exec_class()
        return type

from classes import *

2. Slightly modify your concrete classes to inherit from the Registrable base class, e.g.:

from factory import Registrable

class M2(Registrable, name='Class 2'):
    def __init__(self):
       print ("Starting Class 2")