Sitemap files are provided bywebsites to help crawlers locate their updated content without needing to crawl every web page. For further details, the sitemap standard is defined at http://www.sitemaps.org/protocol.html. Many web publishing platforms have the ability to generate a sitemap automatically. Here is the content of the  Sitemap file located in the listed robots.txt file:

<?xml version="1.0" encoding="UTF-8"?> 
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> 
  <url><loc>http://example.webscraping.com/view/Afghanistan-1</loc></url> 
  <url><loc>http://example.webscraping.com/view/Aland-Islands-2</loc></url> 
  <url><loc>http://example.webscraping.com/view/Albania-3</loc></url> 
  ... 
</urlset> 

This sitemap provides links to all the web pages, which will be used in the next section to build our first crawler. Sitemap files provide an efficient way to crawl a website, but need to be treated carefully because they can be missing, out-of-date, or incomplete.