Managing contexts and the with statement

Contexts and context managers are used in several places in Python. We'll look at a few examples to establish the basic terminology.

Python defines context using the with statement. The following program is a small example that parses a log file to create a useful CSV summary of that log. Since there are two open files, this will use the nested with contexts. The example uses a complex regular expression, format_1_pat. We'll define this shortly.

We might see something similar to the following in an application program:

from pathlib import Path
import gzip
import csv

source_path = Path.cwd()/"data"/"compressed_data.gz"
target_path = Path.cwd()/"data"/"subset.csv"

with target_path.open('w', newline='') as target: wtr= csv.writer( target ) with gzip.open(source_path) as source: line_iter = (b.decode() for b in source)
row_iter = Counter(format_1_pat.match(line) for line in line_iter)
non_empty_rows: Iterator[Match] = filter(None, row_iter)
wtr.writerows(m.groups() for m in non_empty_rows)

Two contexts with two context managers are part of this example:

  • The outermost context starts with the with target_path.open('w', newline='') as target: statement. The path.open() method opens a file that is also a context manager and assigns it to the target variable for further use.
  • The inner context starts with the with gzip.open(source_path, "r") as source: statement. This gzip.open() function opens the given path and also behaves as a context manager. 

When the with statements end, the contexts exit and the files are properly closed; this means that all of the buffers are flushed and the operating system resources are released. Even if there's an exception in the body of the with context, the context manager's exit will be processed correctly and the file will be closed.

Always use a with statement around a path.open() and related file-system operations

Since files involve operating system (OS) resources, it's important to be sure that the entanglements between our applications and the OS are released as soon as they're no longer needed. The with statement ensures that resources are used properly.

Just to complete the example, the following is the regular expression used to parse the Apache HTTP server log files in the Common Log Format:

import re 
format_1_pat= re.compile( 
    r"([d.]+)s+" # digits and .'s: host 
    r"(S+)s+"     # non-space: logname 
    r"(S+)s+"     # non-space: user 
    r"[(.+?)]s+" # Everything in []: time 
    r'"(.+?)"s+'   # Everything in "": request 
    r"(d+)s+"     # digits: status 
    r"(S+)s+"     # non-space: bytes 
    r'"(.*?)"s+'   # Everything in "": referrer 
    r'"(.*?)"s*'   # Everything in "": user agent 
) 

The preceding expression located the various log format fields used in the previous example.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset