GZIP compression

The first simple thing you can do to reduce the bandwidth is to use GZIP compression, so everything that is sent over the wire gets smaller. Web servers such as Apache or nginx provide native support to compress responses on the fly, and it's better to avoid implementing your ad hoc compression at the Python level.

For example, this nginx configuration will enable GZIP compression for any response produced by the Flask app on port 5000, with an application/json content type:

    http { 
gzip on;
gzip_types application/json;
gzip_proxied any;
gzip_vary on;

server {
listen 80;
server_name localhost;

location / {
proxy_pass http://localhost:5000;
}
}

From the client-side, making an HTTP request to the nginx server at localhost:8080 proxying for the application at localhost:5000 with an Accept-Encoding: gzip header will trigger the compression:

$ curl http://localhost:8080/api -H "Accept-Encoding: gzip" 
<some binary output>

In Python, request responses will automatically decompress responses that are ;gzip ;encoded, so you don't have to worry about doing it when your service is calling another service. Unzipping the data adds some processing, but Python's gzip module relies on the zlib (http://www.zlib.net/), which is very fast (and massively spiffy).

>>> import requests 
>>> requests.get('http://localhost:8080/api', headers={'Accept-Encoding': 'gzip'}).json()
{'Hello': 'World!', u'result': 'OK'}

To compress the data you're sending to the server, you can use the gzip module and specify a Content-Encoding header:

>>> import gzip, json, requests 
>>> data = {'Hello': 'World!', 'result': 'OK'}
>>> data = bytes(json.dumps(data), 'utf8')
>>> data = gzip.compress(data)
>>> headers = {'Content-Encoding': 'gzip'}
>>> requests.post('http://localhost:8080/api',
... headers=headers,
... data=data)

<Response [200]>

In that case, however, you will get the zipped content in your Flask application, and you will need to decompress it in your Python code unless you implement something in nginx with Lua to handle it. Apache, on the other hand, can decompress it for you with the mode_deflate module and its SetInputFilter option.

To summarize, setting up GZIP compression for all your service responses is a no-brainer with nginx or Apache, and your Python client can benefit from it by setting the right header. Handling GZIP compression in HTTP requests is a little trickier because if you don't use Apache, you need to implement decompression of incoming data in Python code or somewhere else.

If you want to further reduce the size on HTTP request/response payloads, another option is to switch to binary payloads rather than JSON payloads compressed with gzip . That way, you don't have to deal with unzipping the data and will get a speedup. But we'll see that the compression is not as good.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset