Metaclasses, the focal point of this chapter, can manipulate the way a new class is created by decorating the arguments without impacting the actual class definition itself. Metaclasses are not very frequently used in Python application development unless there is a need for more advanced implementations of frameworks or APIs that need features such as manipulation of classes or dynamic class generation and so on.
In the previous chapter, we looked at the concept of decorators with some examples. Understanding decorators helps in following metaclasses with more ease since both decorators and metaclasses deal with metaprogramming on Python 3 program objects by manipulating them externally.
In this chapter, we will cover the following main topics:
By the end of this chapter, you should be able to create your own metaclasses, implement inheritance on metaclasses, and reuse ones that are already created.
The code examples shared in this chapter are available on GitHub under the code for this chapter here: https://github.com/PacktPublishing/Metaprogramming-with-Python/tree/main/Chapter04.
Metaclasses are classes that can be created separately with certain features that can alter the behavior of other classes or can help in dynamically manufacturing new classes. The base class of all metaclasses is the type class and the object or instance of a metaclass will be a class. Any custom metaclass that we create will be inherited from the type class. type is the class of all data types in Python as well and everything else in Python 3 is an object of the type class. We can test this statement by checking the type of different program objects in Python, as follows:
class TestForType:
pass
type(TestForType)
type
type(int)
type
type(str)
type
type(object)
type
type(float)
type
type(list)
type
In this chapter, we will look at some examples of how to use these metaclasses, how to implement them, and how to reuse them. We will continue with our ABC Megamart examples to proceed further with the understanding of metaclasses.
A metaclass is like any other class, but it has the ability to alter the behavior of other classes that take it as their metaclass. Understanding the structure of a metaclass helps us create our own customized metaclasses, which can be used further in manipulating new classes. The superclass of a metaclass is the type itself. When we create a class with type as its superclass and override the __new__ method to manipulate the metadata of a class it returns, then we have created a metaclass. Let’s take a closer look with the help of some simple examples.
The __new__ method takes cls as its first argument, which is the class itself. The members of the class that has cls as its first argument can be accessed by the class name and the rest of the arguments as other metadata of the class, as seen here:
class ExampleMetaClass1(type):
def __new__(classitself, *args):
print('class itself: ', classitself)
print('Others: ', args)
return type.__new__(classitself, *args)
In the preceding code, we have created the class ExampleMetaClass1, which inherits the class type and overrides the __new__ method to print the class instance and its other arguments.
Let’s now create the class ExampleClass1 and add the preceding metaclass to it:
class ExampleClass1(metaclass = ExampleMetaClass1):
int1 = 123
str1 = 'test'
def test():
print('test')
Running the preceding code displays the following result:
class itself: <class '__main__.ExampleMetaClass1'>
Others: ('ExampleClass1', (), {'__module__': '__main__', '__qualname__': 'ExampleClass1', 'int1': 123, 'str1': 'test', 'test': <function ExampleClass1.test at 0x00000194A377E1F0>})
The first part of this output is the class instance <class '__main__.ExampleMetaClass1'> and the remaining arguments are the class name and the arguments of the class. A simple representation of the metaclass definition is as follows:
Figure 4.1 – Example metaclass definition
Let’s dive into a little more detail with another example in our next subsection.
We now will dig deeper into the arguments of the __new__ method of a metaclass. Analyzing the arguments of a metaclass will provide clarity on what information of a class can be customized using a metaclass. The data that can be manipulated in the classes that adds a metaclass while defining is represented in the following figure:
Figure 4.2 – Example metaclass with more arguments
Let’s now follow these steps to see how the behavior of arguments affects classes:
class ExampleMetaClass2(type):
def __new__(classitself, classname, baseclasses,
attributes):
print('class itself: ', classitself)
print('class name: ', classname)
print('parent class list: ', baseclasses)
print('attribute list: ', attributes)
return type.__new__(classitself, classname,
baseclasses, attributes)
class ExampleParentClass1():
def test1():
print('parent1 - test1')
class ExampleParentClass2():
def test2():
print('parent2 - test2')
class ExampleClass2(ExampleParentClass1,ExampleParentClass2, metaclass = ExampleMetaClass2):
int1 = 123
str1 = 'test'
def test3():
print('child1 - test3')
class itself: <class '__main__.ExampleMetaClass2'>
class name: ExampleClass2
parent class: (<class '__main__.ExampleParentClass1'>, <class '__main__.ExampleParentClass2'>)
attributes: {'__module__': '__main__', '__qualname__': 'ExampleClass2', 'int1': 123, 'str1': 'test', 'test3': <function ExampleClass2.test3 at 0x00000194A3994E50>}
This example shows us the highlighted arguments that are returned by the metaclass and gives an overview of which values can possibly be manipulated from a class using metaprogramming.
type(ExampleParentClass1)
type
type(ExampleParentClass2)
type
type(ExampleMetaClass2)
type
type(ExampleClass2)
__main__.ExampleMetaClass2
As we can see, the type of all other classes is the type itself whereas the type of ExampleClass2 is ExampleMetaClass2.
Now that you understand the structure of a metaclass, we can look further into applications of metaclasses on our ABC Megamart example.
In this section, we will look at an example where we will create a metaclass that can automatically modify the user-defined method attributes of any branch class that is newly created. To test this, let us follow these steps:
class BranchMetaclass(type):
def __new__(classitself, classname, baseclasses,
attributes):
import inspect
newattributes = {}
Iterate over the class attributes, check that the attributes start with __, and don’t change the value.
for attribute, value in attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif inspect.isfunction(value):
newattributes['branch' +
attribute.title()] = value for a
attribute, value in
attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif inspect.isfunction(value):
newattributes['branch' +
attribute.title()] = value
else:
newattributes[attribute] = value
return type.__new__(classitself,
classname, baseclasses,
newattributes)
def buy_product(product,unit_price,quantity,statetax_rate,promotiontype):
statetax_rate = statetax_rate
initialprice = unit_price*quantity
sales_price = initialprice +
initialprice*statetax_rate
return sales_price, product,promotiontype
The Brooklyn class has four variables, product_id, product_name, product_category, and unit_price. We will also create a method to calculate the maintenance cost and this method should be converted from maintenance_cost to branchMaintenance_cost due to the metaclass that alters the behavior of the newly created class. Here’s the new class:
class Brooklyn(metaclass = BranchMetaclass):
product_id = 100902
product_name = 'Iphone X'
product_category = 'Electronics'
unit_price = 700
def maintenance_cost(self,product_type, quantity):
self.product_type = product_type
self.quantity = quantity
cold_storage_cost = 100
if (product_type == 'Electronics'):
maintenance_cost = self.quantity * 0.25 +
cold_storage_cost
return maintenance_cost
else:
return "We don't stock this product"
dir(Brooklyn)
['__class__',
'__delattr__',
'__dict__',
‚__dir__',
‚__doc__',
‚__eq__',
‚__format__',
‚__ge__',
‚__getattribute__',
‚__gt__',
‚__hash__',
‚__init__',
‚__init_subclass__',
‚__le__',
‚__lt__',
‚__module__',
‚__ne__',
‚__new__',
‚__reduce__',
‚__reduce_ex__',
‚__repr__',
‚__setattr__',
‚__sizeof__',
‚__str__',
‚__subclasshook__',
‚__weakref__',
'branchMaintenance_cost',
'product_category',
'product_id',
'product_name',
'unit_price']
brooklyn = Brooklyn()
brooklyn.branchMaintenance_Cost('Electronics',10)
102.5
brooklyn.product_id
100902
brooklyn.product_name
'Iphone X'
brooklyn.product_type
'Electronics'
A simple representation of this example is as follows:
Figure 4.3 – Application of metaclass on ABC Megamart – Branch example
So far, we’ve looked at an overview of a metaclass, understood its structure, performed an analysis of its arguments, and applied our understanding by creating a custom metaclass on our core example. We will look at a few more applications in the following section.
In this section, we will walk through an example where we will inherit the metaclass to check whether it can be inherited as a regular parent class without altering the behavior of the new class that is being created. Take a look at the following code:
class Queens(BranchMetaclass):
def maintenance_cost(product_type, quantity):
product_type = product_type
quantity = quantity
if (product_type == ‹FMCG›):
maintenance_cost = quantity * 0.05
return maintenance_cost
else:
return "We don't stock this product"
Let's now create an object for the preceding class to check if an object can be created:
queens = Queens()
We get the following TypeError:
Figure 4.4 – Error while creating an object for the class inheriting a metaclass
This error occurred as __new__ is a static method that is called to create a new instance for the class and it expects three arguments of the class, which are not provided while creating the class object. However, there is another way of calling the newly created class, Queens. The class can be called directly, and its methods can be used without having to create an object:
Queens.maintenance_cost('FMCG',120)
6.0
The maintenance_cost method did not get modified into branchMaintenance_cost since the metaclass is not used as a metaclass but as a parent class. Since the metaclass is inherited, Queens also inherits the user-defined methods of BranchMetaclass as follows:
Queens.buy_product('Iphone',1000,1,0.04,None)
(1040.0, 'Iphone', None)
Let’s now look at what happens when we inherit a class as a parent and also add it as a metaclass while creating a new class:
class Queens(BranchMetaclass, metaclass = BranchMetaclass):
def maintenance_cost(product_type, quantity):
product_type = product_type
quantity = quantity
if (product_type == ‹FMCG›):
maintenance_cost = quantity * 0.05
return maintenance_cost
else:
return "We don't stock this product"
In the preceding code, we have added BranchMetaclass as the parent class for the class Queens and we have also added it as a metaclass. This definition should make the class Queens inherit the custom methods from BranchMetaclass and also change the maintenance_cost method into branchMaintenance_cost. Let’s see if it does:
Queens.branchMaintenance_Cost('FMCG',2340)
117.0
In the preceding code execution and output, the maintenance_cost method is converted into the branchMaintenance_cost method as expected. Now run the following command:
Queens.buy_product('Iphone',1500,1,0.043,None)
(1564.5, 'Iphone', None)
The buy_product method, which is a custom method from BranchMetaclass, is also inherited since it is a parent class.
Here is a simple representation of this example:
Figure 4.5 – Application of metaclass and also inheriting it on ABC Megamart branch example
Let us look further into examples of switching metaclasses from one class to another.
We can now look into the concept of switching metaclasses for a class. You may think, why do we need to switch metaclasses? Switching metaclasses reinforces the reusability concept of metaprogramming and in this case, it helps in understanding how a metaclass created for use on one class can also be used for a different class without impacting the class definition.
In the example for this section, we will be creating two meta classes – IncomeStatementMetaClass and BalanceSheetMetaClass. For the Malibu branch of ABC Megamart, we will create a class to capture the information required for its financial statements. The two financial statements relevant for this example are Income Statement attributes and Balance Sheet attributes for the Malibu branch. To differentiate where a particular attribute or method of a class should go, we will be creating two metaclasses that look at the names of the attributes and tag them under Income Statement or Balance Sheet accordingly.
The following is a simple representation of the attributes that will be manipulated by the aforementioned metaclasses:
Figure 4.6 – Finance attributes used in this metaclass example
Take a look at the following code snippet:
class IncomeStatementMetaClass(type):
def __new__(classitself, classname, baseclasses,
attributes):
newattributes = {}
for attribute, value in attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif («revenue» in attribute) or
("expense" in attribute) or
("profit" in attribute) or
("loss" in attribute):
newattributes['IncomeStatement_' +
attribute.title()] = value
else:
newattributes[attribute] = value
return type.__new__(classitself, classname,
baseclasses, newattributes)
Here, the new method is modified to check for attributes that have the key as one of the parameters that belong to an income statement such as revenue, expense, profit, or loss. If any of this terminology occurs in the method name or variable name, we will add a prefix of IncomeStatement to segregate those methods and variables.
To test this metaclass, we will be creating a new class, Malibu, with four variables and four methods, as follows:
class Malibu(metaclass = IncomeStatementMetaClass):
profit = 4354365
loss = 43000
assets = 15000
liabilities = 4000
def calc_revenue(quantity,unitsales_price):
totalrevenue = quantity * unitsales_price
return totalrevenue
def calc_expense(totalrevenue,netincome, netloss):
totalexpense = totalrevenue - (netincome + netloss)
return totalexpense
def calc_totalassets(cash,inventory,accountsreceivable):
totalassets = cash + inventory + accountsreceivable
return totalassets
def calc_totalliabilities(debt,accruedexpense,
accountspayable):
totalliabilities = debt + accruedexpense +
accountspayable
return totalliabilities
In the preceding code, we have added the metaclass IncomeStatementMetaClass and we see that the attributes of the class Malibu modify the behavior of variables and methods as follows:
Figure 4.7 – Malibu without metaclass (left) and Malibu with metaclass (right)
We will further add another metaclass, BalanceSheetMetaClass, to deal with the balance sheet-related attributes in the class Malibu. In the following metaclass, the new method is modified to check for attributes that have the key as one of the parameters that belong to a balance sheet such as assets, liabilities, goodwill, and cash. If any of these terms occur in the method name or variable name, we will add a prefix of BalanceSheet to segregate those methods and variables:
class BalanceSheetMetaClass(type):
def __new__(classitself, classname, baseclasses,
attributes):
newattributes = {}
for attribute, value in attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif («assets» in attribute) or
("liabilities" in attribute) or
("goodwill" in attribute) or
("cash" in attribute):
newattributes['BalanceSheet_' +
attribute.title()] = value
else:
newattributes[attribute] = value
return type.__new__(classitself, classname,
baseclasses, newattributes)
In the preceding code, we have added the metaclass BalanceSheetMetaClass and we see that the attributes of the class Malibu modify the behavior of variables and methods as follows:
Figure 4.8 – Malibu with IncomeStatementMetaClass (left) and Malibu with BalanceSheetMetaClass (right)
Now that you know why we need to switch metaclasses, let us look at the application of metaclasses in inheritance.
Inheritance, in a literal sense, means a child acquiring the properties of a parent and it means the same in the case of object-oriented programming too. A new class can inherit the attributes and methods of a parent class and it can also have its own properties and methods.
In this example, we will look at how inheritance works on metaclasses by creating two classes, California and Pasadena – California being the parent class and Pasadena the child class.
Let’s check these steps out to understand inheritance better:
class California(metaclass = IncomeStatementMetaClass):
profit = 4354365
loss = 43000
def calc_revenue(quantity,unitsales_price):
totalrevenue = quantity * unitsaleprice
return totalrevenue
def calc_expense(totalrevenue,netincome, netloss):
totalexpense = totalrevenue - (netincome + netloss)
return totalexpense
Here, we have defined only those attributes that can be modified by the IncomeStatement metaclass.
class Pasadena(California,metaclass = BalanceSheetMetaClass):
assets = 18000
liabilities = 5000
def calc_totalassets(cash,inventory,
accountsreceivable):
totalassets = cash + inventory +
accountsreceivable
return totalassets
def calc_totalliabilities(debt,accruedexpense,
accountspayable):
totalliabilities = debt + accruedexpense +
accountspayable
return totalliabilities
We have defined here only those attributes that can be modified by the BalanceSheet metaclass.
Figure 4.9 – Error while executing a child class that has a different metaclass
This error was thrown since Pasadena inherited the parent class California, which has a different metaclass, IncomeStatementMetaClass, which is inherited from type, and Pasadena’s metaclass BalanceSheetMetaClass is also inherited from type.
class BalanceSheetMetaClass(IncomeStatementMetaClass):
def __new__(classitself, classname, baseclasses,
attributes):
newattributes = {}
for attribute, value in attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif («assets» in attribute) or
("liabilities" in attribute) or
("goodwill" in attribute) or
("cash" in attribute):
newattributes['BalanceSheet_' +
attribute.title()] = value
else:
newattributes[attribute] = value
return type.__new__(classitself, classname,
baseclasses, newattributes)
class California(metaclass = IncomeStatementMetaClass):
profit = 4354365
loss = 43000
def calc_revenue(quantity,unitsales_price):
totalrevenue = quantity * unitsaleprice
return totalrevenue
def calc_expense(totalrevenue,netincome, netloss):
totalexpense = totalrevenue - (netincome +
netloss)
return totalexpense
class Pasadena(California,metaclass = BalanceSheetMetaClass):
assets = 18000
liabilities = 5000
def calc_totalassets(cash,inventory,
accountsreceivable):
totalassets = cash + inventory +
accountsreceivable
return totalassets
def calc_totalliabilities(debt,accruedexpense,
accountspayable):
totalliabilities = debt + accruedexpense +
accountspayable
return totalliabilities
Figure 4.10 – Pasadena class with inheritance
A simple representation of this application is as follows:
Figure 4.11 – Inheritance in metaclasses
In this case, we have redefined the parent class of BalanceSheetMetaClass to be IncomeStatementMetaClass since Python does not automatically resolve their parent classes while they were both inherited by type and instead throws a metaclass conflict. Redefining the parent class of BalanceSheetMetaClass not only resolves the error but will also not impact the overall functionality of the class since IncomeStatementMetaClass is in turn inherited from type.
Let us look at another example where we will be adding additional information to class attributes.
In this section, we will take an example to look at manipulating class variables further using metaclasses. We will be creating a metaclass named SchemaMetaClass and will define the __new__ method to manipulate attributes of a class if they are variables of data types that belong to integer, float, string, or boolean. Let’s go through the steps real quick:
class SchemaMetaClass(type):
def __new__(classitself, classname, baseclasses,
attributes):
newattributes = {}
for attribute, value in attributes.items():
if attribute.startswith("__"):
newattributes[attribute] = value
elif type(value)==int or type(value)==float:
newattributes[attribute] = {}
newattributes[attribute]['ColumnName']
= attribute.title()
newattributes[attribute]['Value']
= value
newattributes[attribute]['Type']
= 'NUMERIC'
newattributes[attribute]['Length'] = len(str(value))
elif type(value)==str:
newattributes[attribute] = {}
newattributes[attribute]['ColumnName']
= attribute.title()
newattributes[attribute]['Value']
= value
newattributes[attribute]['Type']
= 'VARCHAR'
newattributes[attribute]['Length']
= len(value)
elif type(value)==bool:
newattributes[attribute] = {}
newattributes[attribute]['ColumnName']
= attribute.title()
newattributes[attribute]['Value']
= value
newattributes[attribute]['Type']
= 'BOOLEAN'
newattributes[attribute]['Length']
= None
else:
newattributes[attribute] = value
return type.__new__(classitself, classname,
baseclasses, newattributes)
class Arizona(metaclass = SchemaMetaClass):
product_id = 200443
product_name = 'Iphone'
product_category = 'Electronics'
sales_quantity = 2
tax_rate = 0.05
sales_price = 1200
profit = 70
loss = 0
sales_margin = 0.1
promotion = '20%Off'
promotion_reason = 'New Year'
in_stock = True
def create_schema(self):
import pandas as pd
tableschema = pd.DataFrame([self.product_id,
self.product_name,
self.product_category,
self.sales_quantity,
self.tax_rate,
self.sales_price,
self.profit,
self.loss,
self.sales_margin,
self.promotion,
self.promotion_reason,
self.in_stock])
tableschema.drop(labels = ['Value'], axis = 1,
inplace = True)
return tableschema
We have added product details of an example product (in this case, an iPhone) and the variables are a combination of different data types – string, integer, float, and bool. We will define the method create_schema, which imports the pandas library to create a DataFrame that gives a table-like structure to the variables and returns the data frame as a table schema.
objarizona = Arizona()
objarizona.product_name
'Iphone'
objarizona = Arizona()
objarizona.product_name
{'ColumnName': 'Product_name',
'Value': 'Iphone',
'Type': 'VARCHAR',
'Length': 6}
objarizona.product_category
{'ColumnName': 'Product_category',
'Value': 'Electronics',
'Type': 'VARCHAR',
'Length': 11}
objarizona.sales_quantity
{'ColumnName': 'Sales_quantity', 'Value': 2, 'Type': 'NUMERIC', 'Length': 1}
objarizona.tax_rate
{'ColumnName': 'Tax_rate', 'Value': 0.05, 'Type': 'NUMERIC', 'Length': 4}
objarizona.create_schema()
We get the following table, which includes all of the variables defined in the class:
Figure 4.12 – Output of the method create_schema
These are some examples of how metaclasses can be used in developing applications. Metaclasses can further be used in more complex scenarios such as automated code generation and framework development.
In this chapter, we have learned how to create metaclasses and some applications of metaclasses.
We then saw how to switch metaclasses, reuse the functionalities, and how to implement inheritance on classes that use metaclasses. Finally, we also saw how to manipulate the variables of metaclasses further.
All of these concepts are part of Python metaprogramming and they are used to change the behavior of a class externally and without impacting the internal functionalities of the class itself.
In the next chapter, we will be looking at the concept of reflection with different examples.