Python's two built-in higher-order functions, map() and filter(), generally handle almost everything we might want to throw at them. It's difficult to optimize them in a general way to achieve higher performance. We'll look at functions of Python 3.4, such as imap(), ifilter(), and ifilterfalse(), in Chapter 8, The Itertools Module.

We have three, largely equivalent ways to express a mapping. Assume that we have some function, f(x), and some collection of objects, C. We have three entirely equivalent ways to express a mapping; they are as follows:

• The map() function:

map(f, C) 

• A generator expression:

(f(x) for x in C) 

• A generator function with a yield statement:

def mymap(f, C): 
    for x in C: 
        yield f(x) 
mymap(f, C) 

Similarly, we have three ways to apply a filter function to a collection, all of which are equivalent:

• The filter() function:

filter(f, C) 

• A generator expression:

(x for x in C if f(x)) 

• A generator function with a yield statement:

def myfilter(f, C): 
    for x in C: 
        if f(x): 
            yield x 
myfilter(f, C) 

There are some performance differences; often the map() and filter() functions are fastest. More importantly, there are different kinds of extensions that fit these mapping and filtering designs, which are as follows:

• We can create a more sophisticated function, g(x), that is applied to each element, or we can apply a function to the whole collection prior to processing. This is the most general approach and applies to all three designs. This is where the bulk of our functional design energy is invested.
• We can tweak the for loop inside the generator expression or generator function. One obvious tweak is to combine mapping and filtering into a single operation by extending the generator expression with an if clause. We can also merge the mymap() and myfilter() functions to combine mapping and filtering.

Profound changes that alter the structure of the data handled by the loop often happen as software evolves and matures. We have a number of design patterns, including wrapping, unwrapping (or extracting), flattening, and structuring. We've looked at a few of these techniques in previous chapters.

We need to exercise some caution when designing mappings that combine too many transformations in a single function. As far as possible, we want to avoid creating functions that fail to be succinct or expressive of a single idea. As Python doesn't have an optimizing compiler, we might be forced to manually optimize slow applications by combining functions. We need to do this kind of optimization reluctantly, only after profiling a poorly performing program.