In addition to the ones already mentioned in this chapter, Python comes bundled with a few other useful decorators. There are some that aren't in the standard library (yet?).
If you've used C++ or Java before, you're probably used to having ad hoc polymorphism available—different functions being called depending on the argument types. Python being a dynamically typed language, most people would not expect the possibility of a single dispatch pattern. Python, however, is a language that is not only dynamically typed but also strongly typed, which means we can rely on the type we receive.
Note
A dynamically typed language does not require strict type definitions. On the other hand, a language such as C would require the following to declare an integer:
int some_integer = 123;
Python simply accepts that your value has a type:
some_integer = 123
As opposed to languages such as JavaScript and PHP, however, Python does very little implicit type conversion. In Python, the following will return an error, whereas JavaScript would execute it without any problems:
'spam' + 5
In Python, the result is a TypeError. In Javascript, it's 'spam5'.
The idea of single dispatch is that depending on the type you pass along, the correct function is called. Since str + int results in an error in Python, this can be very convenient to automatically convert your arguments before passing them to your function. This can be useful to separate the actual workings of your function from the type conversions.
Since Python 3.4, there is a decorator that makes it easily possible to implement the single dispatch pattern in Python. For one of those cases that you need to handle a specific type different from the normal execution. Here is the basic example:
>>> import functools >>> @functools.singledispatch ... def printer(value): ... print('other: %r' % value) >>> @printer.register(str) ... def str_printer(value): ... print(value) >>> @printer.register(int) ... def int_printer(value): ... printer('int: %d' % value) >>> @printer.register(dict) ... def dict_printer(value): ... printer('dict:') ... for k, v in sorted(value.items()): ... printer(' key: %r, value: %r' % (k, v)) >>> printer('spam') spam >>> printer([1, 2, 3]) other: [1, 2, 3] >>> printer(123) int: 123 >>> printer({'a': 1, 'b': 2}) dict: key: 'a', value: 1 key: 'b', value: 2
See how, depending on the type, the other functions were called? This pattern can be very useful for reducing the complexity of a single function that takes several types of argument.
Note
When naming the functions, make sure that you do not overwrite the original singledispatch function. If we had named str_printer as just printer, it would overwrite the initial printer function. This would make it impossible to access the original printer function and make all register operations after that fail as well.
Now, a slightly more useful example—differentiating between a filename and a file handler:
>>> import json >>> import functools >>> @functools.singledispatch ... def write_as_json(file, data): ... json.dump(data, file) >>> @write_as_json.register(str) ... @write_as_json.register(bytes) ... def write_as_json_filename(file, data): ... with open(file, 'w') as fh: ... write_as_json(fh, data) >>> data = dict(a=1, b=2, c=3) >>> write_as_json('test1.json', data) >>> write_as_json(b'test2.json', 'w') >>> with open('test3.json', 'w') as fh: ... write_as_json(fh, data)
So now we have a single write_as_json function; it calls the right code depending on the type. If it's an str or bytes object, it will automatically open the file and call the regular version of write_as_json, which accepts file objects.
Writing a decorator that does this is not that hard to do, of course, but it's still quite convenient to have it in the base library. It most certainly beats a couple of isinstance calls in your function. To see which function will be called, you can use the write_as_json.dispatch function with a specific type. When passing along an str, you will get the write_as_json_filename function. It should be noted that the name of the dispatched functions is completely arbitrary. They are accessible as regular functions, of course, but you can name them anything you like.
To check the registered types, you can access the registry, which is a dictionary, through write_as_json.registry:
>>> write_as_json.registry.keys() dict_keys([<class 'bytes'>, <class 'object'>, <class 'str'>])
Using the contextmanager class, we can make the creation of a context wrapper very easy. Context wrappers are used whenever you use a with statement. One example is the open function, which works as a context wrapper as well, allowing you to use the following code:
with open(filename) as fh:
passLet's just assume for now that the open function is not usable as a context manager and that we need to build our own function to do this. The standard method of creating a context manager is by creating a class that implements the __enter__ and __exit__ methods, but that's a bit verbose. We can have it shorter and simpler:
>>> import contextlib >>> @contextlib.contextmanager ... def open_context_manager(filename, mode='r'): ... fh = open(filename, mode) ... yield fh ... fh.close() >>> with open_context_manager('test.txt', 'w') as fh: ... print('Our test is complete!', file=fh)
Simple, right? However, I should mention that for this specific case—the closing of objects—there is a dedicated function in contextlib, and it is even easier to use. Let's demonstrate it:
>>> import contextlib >>> with contextlib.closing(open('test.txt', 'a')) as fh: ... print('Yet another test', file=fh)
For a file object, this is of course not needed since it already functions as a context manager. However, some objects such as requests made by urllib don't support automatic closing in that manner and benefit from this function.
But wait; there's more! In addition to being usable in a with statement, the results of a contextmanager are actually usable as decorators since Python 3.2. In older Python versions, it was simply a small wrapper, but since Python 3.2 it's based on the ContextDecorator class, which makes it a decorator. The previous decorator isn't really suitable for that task since it yields a result (more about that in Chapter 6, Generators and Coroutines – Infinity, One Step at a Time), but we can think of other functions:
>>> @contextlib.contextmanager ... def debug(name): ... print('Debugging %r:' % name) ... yield ... print('End of debugging %r' % name) >>> @debug('spam') ... def spam(): ... print('This is the inside of our spam function') >>> spam() Debugging 'spam': This is the inside of our spam function End of debugging 'spam'
There are quite a few nice use cases for this, but at the very least, it's just a convenient way to wrap a function in a context without all the (nested) with statements.
While checking for types is usually not the best way to go in Python, at times it can be useful if you know that you will need a specific type (or something that can be cast to that type). To facilitate this, Python 3.5 introduces a type hinting system so that you can do the following:
def spam(eggs: int):
passSince Python 3.5 is not that common yet, here's a decorator that achieves the same with more advanced type checking. To allow for this type of checking, some magic has to be used, specifically the usage of the inspect module. Personally, I am not a great fan of inspecting code to perform tricks like these, as they are easy to break. This piece of code actually breaks when a regular decorator (one that doesn't copy argspec) is used between the function and this decorator, but it's a nice example nonetheless:
>>> import inspect >>> import functools >>> def to_int(name, minimum=None, maximum=None): ... def _to_int(function): ... # Use the method signature to map *args to named ... # arguments ... signature = inspect.signature(function) ... ... # Unfortunately functools.wraps doesn't copy the ... # signature (yet) so we do it manually. ... # For more info: http://bugs.python.org/issue23764 ... @functools.wraps(function, ['__signature__']) ... @functools.wraps(function) ... def __to_int(*args, **kwargs): ... # Bind all arguments to the names so we get a single ... # mapping of all arguments ... bound = signature.bind(*args, **kwargs) ... ... # Make sure the value is (convertible to) an integer ... default = signature.parameters[name].default ... value = int(bound.arguments.get(name, default)) ... ... # Make sure it's within the allowed range ... if minimum is not None: ... assert value >= minimum, ( ... '%s should be at least %r, got: %r' % ... (name, minimum, value)) ... ... if maximum is not None: ... assert value <= maximum, ( ... '%s should be at most %r, got: %r' % ... (name, maximum, value)) ... ... return function(*args, **kwargs) ... return __to_int ... return _to_int >>> @to_int('a', minimum=10) ... @to_int('b', maximum=10) ... @to_int('c') ... def spam(a, b, c=10): ... print('a', a) ... print('b', b) ... print('c', c) >>> spam(10, b=0) a 10 b 0 c 10 >>> spam(a=20, b=10) a 20 b 10 c 10 >>> spam(1, 2, 3) Traceback (most recent call last): ... AssertionError: a should be at least 10, got: 1 >>> spam() Traceback (most recent call last): ... TypeError: 'a' parameter lacking default value >>> spam('spam', {}) Traceback (most recent call last): ... ValueError: invalid literal for int() with base 10: 'spam'
Because of the inspect magic, I'm still not sure whether I would recommend using the decorator like this. Instead, I would opt for a simpler version that uses no inspect whatsoever and simply parses the arguments from kwargs:
>>> import functools >>> def to_int(name, minimum=None, maximum=None): ... def _to_int(function): ... @functools.wraps(function) ... def __to_int(**kwargs): ... value = int(kwargs.get(name)) ... ... # Make sure it's within the allowed range ... if minimum is not None: ... assert value >= minimum, ( ... '%s should be at least %r, got: %r' % ... (name, minimum, value)) ... ... if maximum is not None: ... assert value <= maximum, ( ... '%s should be at most %r, got: %r' % ... (name, maximum, value)) ... ... return function(**kwargs) ... return __to_int ... return _to_int >>> @to_int('a', minimum=10) ... @to_int('b', maximum=10) ... def spam(a, b): ... print('a', a) ... print('b', b) >>> spam(a=20, b=10) a 20 b 10 >>> spam(a=1, b=10) Traceback (most recent call last): ... AssertionError: a should be at least 10, got: 1
However, as demonstrated, supporting both args and kwargs is not impossible as long as you keep in mind that __signature__ is not copied by default. Without __signature__, the inspect module won't know which parameters are allowed and which aren't.
Generally when writing Python, warnings are very useful the first time when you're actually writing the code. When executing it, however, it is not useful to get that same message every time you run your script/application. So, let's create some code that allows easy hiding of the expected warnings, but not all of them so that we can easily catch new ones:
import warnings
import functools
def ignore_warning(warning, count=None):
def _ignore_warning(function):
@functools.wraps(function)
def __ignore_warning(*args, **kwargs):
# Execute the code while recording all warnings
with warnings.catch_warnings(record=True) as ws:
# Catch all warnings of this type
warnings.simplefilter('always', warning)
# Execute the function
result = function(*args, **kwargs)
# Now that all code was executed and the warnings
# collected, re-send all warnings that are beyond our
# expected number of warnings
if count is not None:
for w in ws[count:]:
warnings.showwarning(
message=w.message,
category=w.category,
filename=w.filename,
lineno=w.lineno,
file=w.file,
line=w.line,
)
return result
return __ignore_warning
return _ignore_warning
@ignore_warning(DeprecationWarning, count=1)
def spam():
warnings.warn('deprecation 1', DeprecationWarning)
warnings.warn('deprecation 2', DeprecationWarning)Using this method, we can catch the first (expected) warning and still see the second (not expected) warning.