I'm currently exploring a similar problem, and I think I may have found a solution. It is a bit of a 'hack' though, so take it with a grain of salt.
What why keeping them in the same file work while splitting them out doest
In order to make your classes self-register in the factory while keeping their definition in single .py files, we have to somehow force the loading of the classes in the .py files.
How can I make the separate file approach work?
In my case, I've came across this problem when trying to implement a 'Simple Factory', with self-registering subclasses to avoid having to modify the typical 'if/else' idiom in the factory's get() method.
I'll use a simple example, starting with the decorator method you've mentioned.
Example with decorators
Let's say we have a ShoeFactory as shown below, in which we register different 'classes' of shoes:
# file shoe.py
class ShoeFactory:
_shoe_classes = {}
@classmethod
def get(cls, shoe_type:str):
try:
return cls._shoe_classes[shoe_type]()
except KeyError:
raise ValueError(f"unknown product type : {shoe_type}")
@classmethod
def register(cls, shoe_type:str):
def inner_wrapper(wrapped_class):
cls._shoe_classes[shoe_type] = wrapped_class
return wrapped_class
return inner_wrapper
Examples of shoe classes:
# file sandal.py
from shoe import ShoeFactory
@ShoeFactory.register('Sandal')
class Sandal:
def __init__(self):
print("i'm a sandal")
# file croc.py
from shoe import ShoeFactory
@ShoeFactory.register('Croc')
class Croc:
def __init__(self):
print("i'm a croc")
In order to make Sandal self-register in the ShoeFactory while keeping its definition in a single .py file, we have to somehow force the loading of the Sandal class in .py file.
I've done this in 3 steps:
- Keeping all class implementations in a specific folder, e.g., structuring the files as follows:
.
└- shoe.py # file with the ShoeFactory class
└─ shoes/
└- __init__.py
└- croc.py
└- sandal.py
- Adding the following statement to the end of the
shoe.py file, which will take care of loading and registering each individual class:
from shoes import *
- Add a piece of code like the snippet below to your
__init__.py within the shoes/ foder, so that to dynamically load all classes [1]:
from inspect import isclass
from pkgutil import iter_modules
from pathlib import Path
from importlib import import_module
# iterate through the modules in the current package
package_dir = Path(__file__).resolve().parent
for (_, module_name, _) in iter_modules([package_dir]):
# import the module and iterate through its attributes
module = import_module(f"{__name__}.{module_name}")
for attribute_name in dir(module):
attribute = getattr(module, attribute_name)
if isclass(attribute):
# Add the class to this package's variables
globals()[attribute_name] = attribute
If we follow this approach, I get the following results when running some test code as follows:
# file shoe_test.py
from shoe import ShoeFactory
if __name__ == "__main__":
croc = ShoeFactory.get('Croc')
sandal = ShoeFactory.get('Sandal')
$ python shoe_test.py
i'm a croc
i'm a sandal
Example with __init_subclass__()
I've personally followed a slighly different approach for my simple factory design, which does not use decorators.
I've defined a RegistrableShoe base class, and then used a __init_subclass__() approach to do the self-registering ([1] item 49, [2]).
I think the idea is that when Python finds the definition of a subclass of RegistrableShoe, the __init_subclass__() method is ran, which in turn registers the subclass in the factory.
This approach requires the following changes when compared to the example above:
- Added a
RegistrableShoe base class to the shoe.py file, and re-factored ShoeFactory a bit:
# file shoe.py
class RegistrableShoe():
def __init_subclass__(cls, shoe_type:str):
ShoeFactory.register(shoe_type, shoe_class=cls)
class ShoeFactory:
_shoe_classes = {}
@classmethod
def get(cls, shoe_type:str):
try:
return cls._shoe_classes[shoe_type]()
except KeyError:
raise ValueError(f"unknown product type : {shoe_type}")
@classmethod
def register(cls, shoe_type:str, shoe_class:RegistrableShoe):
cls._shoe_classes[shoe_type] = shoe_class
from shoes import *
- Changed the concrete shoe classes to derive from the
RegistrableShoe base class and pass a shoe_type parameter:
# file croc.py
from shoe import RegistrableShoe
class Croc(RegistrableShoe, shoe_type='Croc'):
def __init__(self):
print("i'm a croc")
# file sandal.py
from shoe import RegistrableShoe
class Sandal(RegistrableShoe, shoe_type='Sandal'):
def __init__(self):
print("i'm a sandal")
