Let us send our HTTP request through a public domain proxy server.
Listing 4.5 explains proxying web requests across a proxy server as follows:
#!/usr/bin/env python # Python Network Programming Cookbook -- Chapter - 4 # This program requires Python 3.5.2 or any later version # It may run on any other version with/without modifications. # # Follow the comments inline to make it run on Python 2.7.x. import urllib.request, urllib.parse, urllib.error # Comment out the above line and uncomment the below for Python 2.7.x. #import urllib URL = 'https://www.github.com' PROXY_ADDRESS = "165.24.10.8:8080" # By Googling free proxy server if __name__ == '__main__': proxy = urllib.request.ProxyHandler({"http" : PROXY_ADDRESS}) opener = urllib.request.build_opener(proxy) urllib.request.install_opener(opener) resp = urllib.request.urlopen(URL) # Comment out the above 4 lines and uncomment the below
for Python 2.7.x. #resp = urllib.urlopen(URL, proxies = {"http" : PROXY_ADDRESS}) print ("Proxy server returns response headers: %s " %resp.headers)
If you run this script, it will show the following output:
$ python 14_5_proxy_web_request.py Proxy server returns response headers: Server: GitHub.com Date: Thu, 22 Jun 2017 14:26:52 GMT Content-Type: text/html; charset=utf-8 Transfer-Encoding: chunked Connection: close Status: 200 OK Cache-Control: no-cache Vary: X-PJAX X-UA-Compatible: IE=Edge,chrome=1 Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Mon, 22 Jun 2037 14:26:52 -0000; secure; HttpOnly Set-Cookie: _gh_sess=eyJzZXNzaW9uX2lkIjoiNzNiODUwNjg1M2M1ZWQ3NjQxZmE2ODI5NTY5Y2UxNmUiLCJsYXN0X3JlYWRfZnJvbV9yZXBsaWNhcyI6MTQ5ODE0MTYxMjA2NCwiX2NzcmZfdG9rZW4iOiJmdlM1ME5oUGUyUU1hS0pVQ29EZnlTL1Bab0pZOHM1ZlpBT2JoRUhYL1NRPSJ9--9db8c2d5bd3e75a1ec5250192094de38937398f8; path=/; secure; HttpOnly X-Request-Id: 88704a9ed7378c7d930cdff660739693 X-Runtime: 0.033159 Content-Security-Policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src render.githubusercontent.com; connect-src 'self' uploads.github.com status.github.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com wss://live.github.com; font-src assets-cdn.github.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; img-src 'self' data: assets-cdn.github.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; media-src 'none'; script-src assets-cdn.github.com; style-src 'unsafe-inline' assets-cdn.github.com Strict-Transport-Security: max-age=31536000; includeSubdomains; preload Public-Key-Pins: max-age=5184000; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="RRM1dGqnDFsCJXBTHky16vi1obOlCgFFn/yOhI/y+ho="; pin-sha256="k2v657xBsOVe1PQRwOsHsw3bsGT2VzIqz5K+59sNQws="; pin-sha256="K87oWBWM9UZfyddvDfoxL+8lpNyoUB2ptGtn0fv6G2Q="; pin-sha256="IQBnNBEiFuhj+8x6X8XLgh01V9Ic5/V3IRQLNFFc7v4="; pin-sha256="iie1VXtL7HzAMF+/PVPR9xzT80kQxdZeJ+zduCB3uj0="; pin-sha256="LvRiGEjRqfzurezaWuj8Wie2gyHMrW5Q06LspMnox7A="; includeSubDomains X-Content-Type-Options: nosniff X-Frame-Options: deny X-XSS-Protection: 1; mode=block X-Runtime-rack: 0.036536 Vary: Accept-Encoding X-Served-By: 29885c8097c6d503a86029451b2e021c X-GitHub-Request-Id: 9144:282AF:FE7E37:17F7695:594BD3AB