Extracting data from Web Files
by Ferran Garcia Pagans, Neeraj Kharpate, Philip Hand, James Richardson, Dr. Henric
Qlik Sense: Advanced Data Visualization for Your Organization
- Qlik Sense: Advanced Data Visualization for Your Organization
- Table of Contents
- Qlik Sense: Advanced Data Visualization for Your Organization
- Credits
- Preface
- 1. Module 1
- 1. Qlik Sense® and Data Discovery
- 2. Overview of a Qlik Sense® Application's Life Cycle
- 3. Empowering Next Generation Data Discovery Consumers
- 4. Contributing to Data Discovery
- 5. Authoring Engaging Applications
- 6. Building Qlik Sense® Data Models
- 7. Qlik Sense® Apps in the Cloud
- 8. Extending the Qlik® Analytic Platform
- 9. Administering Qlik Sense®
- 10. Sales Discovery
- 11. Human Resource Discovery
- 12. Travel Expense Discovery
- 13. Demographic Data Discovery
- 2. Module 2
- 1. Getting Started with the Data
- Introduction
- Extracting data from databases and data files
- Extracting data from Web Files
- Activating the Legacy Mode in Qlik Sense® desktop
- Extracting data from custom databases
- Invoking help while in the data load editor or the expression editor
- Previewing data in the Data model viewer
- Creating a Master Library from the Data model viewer
- Using a Master Library in the Edit mode
- 2. Visualizations
- Introduction
- Creating Snapshots
- Creating and adding content to a story
- Adding embedded sheets to the story
- Highlighting the performance measure in a bar chart
- Associating persistent colors to field values
- Using the ColorMix1 function
- Composition
- Relationships
- Comparison
- Distribution
- Structuring visualizations
- 3. Scripting
- Introduction
- Structuring the script
- Efficiently debugging the script
- Packaging the code in script files
- How to use sub routines in Qlik Sense®
- Optimizing the UI calculation speed
- Optimizing the reload time of the application
- Using a For Each loop to load data from multiple files
- Using the Concat function to store multiple field values in a single cell
- 4. Managing Apps and User Interface
- Introduction
- Publishing a Qlik Sense® application created in Qlik Sense® desktop
- Creating private, approved, and community sheets
- Publishing a Qlik Sense® application to Qlik Sense® cloud
- Creating geo maps in Qlik Sense®
- Reference lines in Sales versus Target gauge chart
- Effectively using the KPI object in Qlik Sense®
- Creating Tree Maps
- Creating dimensionless bar charts in Qlik Sense®
- Adding Reference Lines to trendline charts
- Creating text and images
- Applying limitations to charts
- Adding thumbnails – a clear environment
- Navigating many data points in a scatter chart
- 5. Useful Functions
- Introduction
- Using an extended interval match to handle Slowly Changing Dimensions
- Using the Previous() function to identify the latest record for a dimensional value
- Using the NetworkDays() function to calculate the working days in a calendar month
- Using the Concat() function to display a string of field values as a dimension
- Using the Minstring() function to calculate the age of the oldest case in a queue
- Using the Rangesum() function to plot cumulative figures in trendline charts
- Using the Fractile() function to generate quartiles
- Using the FirstSortedValue() function to identify the median in a quartile range
- Using the Declare and Derive functions to generate Calendar fields
- Setting up a moving annual total figure
- Using the For Each loop to extract files from a folder
- Using the Peek() function to create a currency Exchange Rate Calendar
- Using the Peek() function to create a Trial Balance sheet
- 6. Set Analysis
- Introduction
- Cracking the syntax for Set Analysis
- Using flags in Set Analysis
- Using the = sign with variables in Set Analysis
- Point in time using Set Analysis
- Using comparison sets in Set Analysis
- Using embedded functions in Set Analysis
- Creating a multi-measure expression in Set Analysis
- Using search strings inside a set modifier
- Capturing a list of field values using a concat() function in Set Analysis
- Using the element functions P() and E() in Set Analysis
- 7. Extensions in Qlik Sense®
- 8. What's New in Version 2.1.1?
- Introduction
- Using the visual exploration capability in Qlik Sense® 2.1.1
- Defining variables in Qlik Sense®
- Exporting stories to MS PowerPoint
- Using the Qlik Dev Hub in Qlik Sense® 2.1.1
- Using Extension editor in Qlik Dev Hub
- Using Qlik Dev Hub to generate mashups
- Embedding Qlik Sense® application on a website using a single configurator
- Using the Qlik DataMarket
- Creating dynamic charts in Qlik Sense®
- Using Smart Search
- Using smart data load profiling
- Conclusion
- A. Appendix
- 1. Getting Started with the Data
- 3. Module 3
- 1. Getting Ready with Predictive Analytics
- 2. Preparing Your Data
- 3. Exploring and Understanding Your Data
- 4. Creating Your First Qlik Sense Application
- 5. Clustering and Other Unsupervised Learning Methods
- 6. Decision Trees and Other Supervised Learning Methods
- 7. Model Evaluation
- 8. Visualizations, Data Applications, Dashboards, and Data Storytelling
- 9. Developing a Complete Application
- A. Bibliography
- Index
Often, the data required for the purpose of reporting is not stored in a database, but instead needs to be fetched from a website. For example, customer location information specifically the geographic co-ordinates used in mapping analysis is not available internally within an organization. This information may be available on the web and can be extracted from there.
When extracting the data from a web file:
- Open an existing Qlik Sense application or create a new one.
- Open the data load editor.
- Click on Create New Connection and select Web file.
- The Select web file window will open.
- Insert the following URL from which you can fetch the data:
http://www.csgnetwork.com/llinfotable.html
- Name the connection as
QlikSense Cookbook Webfile, as shown:
- In the list under Data Connections, select QlikSense Cookbook Webfile and click on Select Data. This will open up a preview window listing out all the tables from the web page. When you carefully examine the table contents, you realize that it is the second table @2 that contains the location information.
- Check the box next to @2 and ensure that it is selected, so the correct table is shown in the preview. The user will need to change the value under Field names to embedded field names.
- The preview of the table will look like the following screenshot:
- Select all the fields from the table in the preview window. Click on Insert script to load the web data in the application.
- Name the table as
Country_Locationand the script will read as follows:Country_Location: LOAD Country, Capital, Latitude, Longitude FROM [lib://QlikSense Cookbook Webfile] (html, codepage is 1252, embedded labels, table is @2);
- Save and load the data. Once the script is successfully loaded, the data model viewer will show the loaded table.
Qlik Sense connects to the web file using the stored data connection. Once connected it identifies the tables in the HTML source and lists them in the preview window.
Certain external websites require authentication in order to be accessed and Qlik Sense is unable to cope with websites that are secured in this manner. In order to get over this issue, we can use a third party data extraction tool. The extracted data can be stored in a data file, such as a qvd. The qvd file can then be used as a data source in the Qlik Sense application.
Qlik Sense can also extract data from other data formats, such as XML. The underlying principles remain the same as explained in the preceding recipes.
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.