Data is fundamental to the ways that modern business, communications, science, and even our personal lives are changing. The information we create from our online shopping, social media posts, search engine queries, and location data is collected, managed, and analyzed and can be used for a number of reasons, including to track consumer patterns, to train artificial intelligence algorithms, or even to study the geographic distribution of particular events such as diseases.
Learn about Qt’s Model/View architecture for creating GUIs that work with data
Use the QTableView class to build data-handling applications
See how to work with CSV files in PyQt
Introduce the QtSql module for creating and managing SQL relational databases
Before we begin, let’s think a little more about data’s usefulness.
Thinking About Data
Data analysis , or the process of organizing, modifying, and modeling data, is an important process, and this chapter will have a look at working with structured data for GUI development. Data can be stored in many different formats, including textual, visual, and multimedia.
In order to analyze data, we need to organize it into structures that we can store and then access electronically through a computer system. Sometimes, you may only be working with a small dataset consisting of one or two files. Other times, you may need to access certain portions of an entire database filled with private information. A database is an organized collection of multiple datasets.
We generally view the data from files and databases in tables. The rows and columns of a table typically work best for handling the style of data in data files. If we had a dataset of employees in a company, each row might represent an individual employee in the company, while each column depicts the different types of attributes for each employee, such as their age, salary, and employee ID number.
This chapter will focus on using PyQt’s table classes for displaying and manipulating data. We will see how to use tables to work with CSV files and to build and interact with the SQL database management language. Of course, there are also other formats that you can use for viewing data, namely, lists and trees, should they better fit your application’s requirements.
Introduction to Model/View Programming
Qt, and therefore PyQt, needs a system to access, display, and manage data that can be presented to the user. An older technique used for managing the relationship between data and its visual representation for user interfaces is the Model-View-Controller (MVC) software design pattern. MVC divides a program’s logic into three interlinked components: a model, a view, and a controller.
Qt utilizes a similar design pattern that is based on MVC: the Model/View paradigm.
The Components of the Model/View Architecture
Model – The class that communicates with the data source, accessing the data, and provides a point of connection between the data and the view and delegate.
View – The class that is responsible for displaying the data to the user, either in list, table, or tree formats, and for retrieving items of data from the model using model indexes. The view also has similar functionality to the controller in the MVC pattern, which handles the input from a user’s interaction with items displayed in the view.
Delegate – The class that is in charge of painting items and providing editors in the view. The delegate also communicates back to the model if an item has been edited.
Using the model/view structure has quite a few benefits, specifically being ideal for developing large-scale applications, giving more flexibility and control over the appearance and editing of data items, simplifying the framework for displaying data, and offering the ability to display multiple views of a model at the same time.
PyQt’s Model/View Classes
QTreeWidget – Creates a table of items
QListWidget – Displays a list of items
QTreeWidget – Provides a hierarchical tree-like structure
What these widgets provide are all the tools necessary to work with data and already include the view, model, and delegate classes grouped together into single classes. However, these classes are more focused on item-based interfaces and are less flexible than working with the Model/View structure. It is also worth noting that each of these widgets inherits behavior from the abstract item view class, QAbstractItemView, creating the behavior for selecting items and managing headers.
An abstract class provides the points of connection, referred to as an interface, between other components, providing the class methods, functionality, and default implementation of features. They are the basis used for creating other classes. Qt’s abstract data classes can also be used to create custom models, views, or delegates.
Models – All models are based on the QAbstractItemModel class, defining the interface used by both views and delegates to access data. They can be used to handle lists, tables, or trees. Data can take on a number of forms, including Python data structures, separate classes, files, or databases. Some other model classes are QStandardItemModel, QFileSystemModel, and SQL-related models.
Views – All views are based on QAbstractItemView and are used to display data items from a data source, including QListView, QTableView, and QTreeView.
Delegates – The base class is QAbstractItemDelegate, responsible for drawing items from the model and providing an editor widget for modifying items. For example, while editing a cell in a table, an editor widget, such as QLineEdit, is placed directly on top of the item.
Communication between the models, views, and delegates is handled by signals and slots. The model uses signals to notify the view about changes to the data. The view generates signals that provide information about how a user interacts with items. For a simple GUI, you may not need to interact with a delegate, but it is important to know that signals from the delegate are emitted while editing an item in the view. This, in turn, informs the model and view about the state of the editor widget.
Explanation for Introduction to Model/View
Code for setting up the MainWindow class in the introductory Model/View example
Tables are great for organizing and displaying various types of textual (and sometimes graphical) data, such as employee or inventory information.
We begin by importing classes, including QTableView from the QtWidgets module and the QStandardItemModel and QStandardItem classes from QtGui. QStandardItemModel will supply the item-based model we need to work with the data; QStandardItem creates the items that are used in the model.
Setting Up the Model, View, and Selection Modes
Code for the setUpMainWindow() method in the introductory Model/View example
SingleSelection – A user can select only a single item at any given time. The item that was previously selected will become unselected.
ExtendedSelection – Allows for normal selection and also for a user to select multiple items by pressing the Ctrl key (Cmd on MacOS) while clicking an item in the view or to select several items using the Shift key.
ContiguousSelection – Allows for normal selection and also for a user to select multiple items by pressing the Shift key.
MultiSelection – The user can select and deselect multiple items by clicking and dragging the mouse in the table.
NoSelection – Selection of items is disabled.
To set up the view to display data from the model, you’ll need to call the setModel() method and pass the model you instantiated. The model used for this example is QStandardItemModel.
The table_view widget is added to the QVBoxLayout.
Working with CSV Files
Code for the loadCSVFile() method in the introductory Model/View example
We’ll open the file, set up the reader object to read the sequences in the file, get the header labels, and skip to the next line using next(). For this example, we'll assume that the CSV file will have header labels. The horizontal labels for model are set using the list of items from the first row in the file, which are stored in header_labels.
For the remaining rows, we use a list comprehension to read the items for each row into a list. Items created for QStandardItemModel need to be instances of QStandardItem. The insertRow() method is used to insert the list of items into the ith row.
With a fundamental understanding of how to create a model and a view, we can move onto creating GUIs that handle larger datasets. While the rest of this chapter will focus on SQL-based classes and models, you do not have to use SQL. All you need is some form of structured data in order to use the Model/View classes.
Working with SQL Databases in PyQt
Now that we have looked at PyQt’s Model/View architecture and the QTableView class, let’s see how to use SQL for handling structured data.
What Is SQL?
Structured Query Language (SQL) is a programming language designed for communication with databases. The data stored in databases is organized into a set of tables. The rows of the tables are referred to as records, and the columns are referred to as fields. Each column can only store a specific kind of information, such as names, dates, or numbers.
With SQL, we can query the data stored in relational databases – a collection of data items that have predefined relationships across multiple tables, marked by a unique identifier known as a foreign key. In a relational database, multiple tables comprise a schema, more than one schema makes up a database, and those databases are stored on a server. Relational databases allow for multiple users to handle the data at the same time. For this reason, accessing a database often requires a user to log in with a username and password in order to connect to the database.
This section will focus solely on using SQL along with classes from PyQt’s QtSql module for creating a very basic database management system interface.
Working with Database Management Systems
The QtSql module provides drivers for a number of Relational Database Management Systems (RDBMS) , including MySQL, Oracle, Microsoft SQL Server, PostgreSQL, and SQLite versions 2 and 3. An RDBMS is the software that allows users to interact with relational databases using SQL. More information about Qt SQL drivers can be found at https://doc-snapshots.qt.io/qt6-dev/sql-driver.html.
For the following examples, we will be using SQLite 3 since the library already comes shipped with Python and is included with Qt. SQLite is not a client-server database engine, so we do not need a database server. In addition, SQLite operates on a single file and is mainly used for small desktop applications.
Getting Familiar with SQL Commands
A list of common SQLite keywords and functions that can be found in this chapter1
SQLite Keywords | Description |
---|---|
AUTOINCREMENT | Generates a unique number automatically when a new record is inserted into the table |
CREATE TABLE | Creates a new table in the database |
DELETE | Deletes a row from the table |
DROP TABLE | Deletes a table that already exists in the database |
FOREIGN KEY | Constraint that links two tables together |
FROM | Specifies the table to interact with when selecting or deleting data |
INTEGER | Signed integer datatype |
INSERT INTO | Inserts new rows into the table |
MAX() | Function that finds the maximum value of a specified column |
NOT NULL | Constraint that ensures a column will not accept NULL values |
PRIMARY KEY | Constraint that uniquely identifies a record in the table |
REFERENCES | Used with FOREIGN KEY to specify another table that has relation with the first table |
SELECT | Selects data from a database |
SET | Identifies which columns and values should be updated |
UNIQUE | Constraint that ensures all values in a column are unique |
UPDATE | Updates existing values in a row |
VALUES | Defines the values of an INSERT INTO statement |
VARCHAR | Variable character datatype for strings |
WHERE | Filters the results of a query to include only records that satisfy specific conditions |
In the following sections, we will work toward creating a user interface that can be used to view and manage a database’s information in a table view.
Project 14.1 – Account Management GUI
For this project, we are going to take a different approach to designing the account management GUI. This section builds up to the final project by working through a number of smaller example programs. There is a good deal of information to unpack, and if this is your first time working with SQL, especially to build an interface in PyQt, then the process for working with databases can become a little unclear.
Imagine you have a business and you want to create a database to keep track of your employees’ information. You want to include information such as their first and last names, employee IDs, email addresses, departments, and the countries where they work. (This could be extended to include more information such as salaries, phone numbers, and dates of hire.) In the beginning, a small database is okay. However, as your workforce builds, so will the information. Some employees may have the same first or last name or even work in the same country. You need a way to manage all of those employees so that fields in the database are populated with the correct information and data types.
- 1.
Introduce how to use QSqlDatabase to connect to databases and QSqlQuery for creating queries.
- 2.
A few examples of how to use QSqlQuery for editing database items.
- 3.
Introduce QSqlTableModel for creating editable data models that work with tables that do not contain foreign keys.
- 4.
Show how use to QSqlRelationalTableModel for creating editable data models that work with tables that do have foreign key support.
- 5.
Create the account management GUI.
Let’s get started!
Explanation for Working with the QtSql Module
In this first example, we are going to see how to use QSqlQuery to create a small database that we will be able to view in the account management GUI. The database has two tables, accounts and countries. The two tables are linked together through the country_id field in accounts and the id field in countries.
This program does not create a GUI. Rather, it demonstrates how to get started with QSqlDatabase for connecting to a database and how to use QSqlQuery to create entries and in a database. It also creates the database, accounts.db, that is used throughout the remainder of the chapter.
Creating a Connection to a Database
Since this program does not create a GUI, we’ll only need to import the QSqlDatabase and QSqlQuery classes from QtSql. We will use QSqlDatabase to create the connection that allows access to a database; QSqlQuery will be used to perform SQL statements in PyQt.
Connecting to a database with QSqlDatabase
We begin by creating a connection to the database in the CreateEmployeeData class. The addDatabase() function allows you to specify the SQL driver that you want to use. The examples in this chapter use SQLite 3 so we pass QSQLITE. Once the database object is created, we can set the other connection parameters, including which database we are going to use, the username, password, host name, and the connection port. For SQLite 3, we only need to specify the name of the database with setDatabaseName(). You can also create multiple connections to a database by passing an additional argument, a connection name, to addDatabase() after the driver argument.
A connection is referenced by its name, not by the name of the database. If you want to give your database a name, pass it as an argument after the driver in the addDatabase() method. If no name is specified, then a default connection will be used.
If accounts.db does not already exist, then it will be created. Once the parameters are set, you must call open() to activate the connection to the database. A connection cannot be used until it is opened.
Building a Dataset with QSqlQuery
Building a dataset with QSqlQuery in the CreateEmployeeData class
Each field, such as employee_id or first_name, is associated with one of the placeholders. Since we used AUTOINCREMENT for id, we do not have to include the field or a placeholder in the query.
The prepare() method gets the query ready for execution. If the query is prepared successfully, then values can be bound to the fields using the addBindValue() method. For information about executing SQL statements in Qt, have a look at https://doc.qt.io/qt-6/sql-sqlstatements.html. Different approaches for binding values are found at https://doc.qt.io/qt-6/sql-sqlstatements.html.
Creating the values for the example dataset in the CreateEmployeeData class, part 1
Creating the values for the example dataset in the CreateEmployeeData class, part 2
Once the tables are populated, we call sys.exit(0) to exit the program. There is no QApplication instance since there is no GUI.
Visualizing SQL Data Using SQLite
If you want to create a readable SQLite file from accounts.db in order to visualize the data, there are a few options. The first is to use available tools, such as SQLiteStudio at https://sqlitestudio.pl/, for browsing databases.
- 1.
On the SQLite download page, www.sqlite.org/download.html, download the Precompiled Binaries for Windows. Look for the option that includes command-line tools. Locate where the files have downloaded on your computer. Inside the folder, you’ll notice three files, one of which is sqlite3.exe.
- 2.
Open a shell window and navigate to C:>. Next, mkdir sqlite.
- 3.
Move the three files from step 1 to your new sqlite folder. One way to do this is by opening the folder with start .sqlite and dragging and dropping the files.
- 4.
So that you’ll be able to use SQLite no matter what folder you are in, you’ll need to add the sqlite folder you made to your PATH environment variable. On the command line, enter $env:Path += ";C:sqlite".
- 5.
Finally, run sqlite3 in the command line, and you should enter the SQLite shell environment. To exit, type .quit.
You’ll see a new file, accounts.sql, created in the files folder. Also, worth a mention is that SQLite 3 is included as part of the standard Python library, so you can import sqlite in your applications if necessary.
In the following section, you’ll see how to use QSqlQuery to do more than just create tables.
Explanation for Querying a Database with QSqlQuery
The program created from Listings 14-8 to 14-10 is not necessary for the accounting manager GUI, but it does give a few more examples for understanding how to input, update, and delete records with SQL in a PyQt application. The purpose of the section is to demonstrate how to open an existing database and modify its contents. We’ll do this for the database created in the “Explanation for Working with the QtSql Module” section.
createConnection() – Establishes the connection to the database
exampleQueries() – Queries the database to acquire and modify existing entries
Creating the connection for the QueryExamples class
We start by adding a database using the SQLite 3 driver and a default connection since no connection name is passed to addDatabase(). Next, set the database created in the previous program, accounts.db. Next, we’ll complete the connection using open().
Demonstrating how to access SQL databases in PyQt
We create a new QSqlQuery instance to search for the first and last names of the employees whose employee ids are greater than 2000.
With that query, we could use the values from first_name and last_name to update or delete records. To cycle through the results of the query, we use the QSqlQuery method next(). Other methods that could be used to navigate the results include next(), previous(), first(), and last().
Demonstrating how to insert, update, and delete records using SQL and PyQt
To insert a single record, we can use the INSERT SQL command. In this query, we insert specific values for each field. You could also add multiple records into the database. Refer back to the “Explanation for Working with the QtSql Module” section to see how.
To update records, use UPDATE. We update the department value for the employee that was just inserted. Finally, to delete a record, use DELETE.
This example also has no GUI window. To see the changes, you could run this program after running the program in the “Explanation for Working with the QtSql Module” section and then use the GUI in the next section to visualize the results in a table.
Working with the QSqlTableModel Class
We could use QSqlQuery to do all of the database work, but combining the class with the Model/View paradigm allows us to design GUIs that make the data management process simpler.
Explanation for Working with QSqlTableModel
Get started by using the basic_window.py script from Chapter 1 and then import the PyQt classes we need, including QSqlTableModel, in Listing 14-11. QHeaderView is the class that provides both horizontal and vertical headers for item view classes.
Code for the MainWindow class using QSqlTableModel
Code for the createConnection() method in the QSqlTableModel example
Code for the setUpMainWindow() method in the QSqlTableModel example
This line also handles stretching the table when the window resizes.
Finally, the model is populated with data using select(). If you have made changes to the table but have not submitted them, then select() will cause the edited items to return back to their previous states.
In the next section, you’ll find out how to create and display relations set by foreign keys in the table view.
Working with the QSqlRelationalTableModel Class
Next, we are going to see how to use the QSqlRelationalTableModel class for working with relational databases. The QSqlRelationalTableModel class provides a model for managing and editing data in a SQL table, with additional support for using foreign keys. A foreign key is a SQL constraint used to link tables together.
Explanation for Working with QSqlRelationalTableModel
Start with basic_window.py script from Chapter 1. This time we need to import QSqlRelationalTableModel since we are working with relational databases and foreign keys. Also, QSqlRelation is included because we’ll need to use the class to store the information about SQL foreign keys. The QSqlRelationalDelegate is also needed because we’ll need to display editor widgets in the columns that pertain to foreign keys.
Code for the MainWindow class using QSqlRelationalTableModel
Code for the createConnection() method in the QSqlRelationalTableModel example
Code for the setUpMainWindow() method in the QSqlRelationalTableModel example
The country_id field in accounts is mapped to countries table’s field, id. For the QSqlRelationalTableModel method setRelation(), we’ll need to pass the index of the column that contains a foreign key (done with fieldIndex()) and a QSqlRelation object that defines the relationship. For QSqlRelation, the field id of table countries maps to country_id in the accounts table. The final argument, country, specifies which field should be displayed in the accounts table.
If you compare Figure 14-8 to Figure 14-6, you’ll notice that data in the last column, country, has been updated to display the names of the countries and that the header has also been changed to country.
Adding Delegates to Edit Relational Data
The purpose of delegates when using Model/View classes becomes more obvious when you either start creating your own custom classes or when you need to use relational classes to select values for fields with foreign keys. With a delegate, an editor widget such as QLineEdit or QComboBox will appear when a user is editing data. You may not have even realized that every time you edit values in the cells of QTableView, you have been using a delegate all along. That’s because of how Qt seamlessly blends the view and delegates.
For SQL relational databases, QSqlRelationalDelegate data from a QSqlRelationalTableModel can be viewed and edited.
Adding delegates in the QSqlRelationalTableModel example
The last step is to create and set the layout for the main window.
By this point, you should have a fundamental understanding of how to use model and view classes, use QtSql classes to perform queries, and display foreign key relationships in a table using relational classes. We are now ready to build the account management GUI.
Explanation for the Account Management GUI
The account management GUI uses the QSqlRelationalTableModel for managing the accounts and countries tables. We'll use the concepts we learned in the previous sections to design a GUI with features for managing the database directly rather than programmatically. Refer back to Figure 14-4 to see the interface.
The application lets a user add, delete, and sort the contents of the table. Rows added or deleted will also update the database.
Also, be sure to download the icons folder from GitHub for this project.
Code for the MainWindow class in the account management GUI
For this GUI, a separate method, createModel(), will create the QSqlRelationalTableModel that is used by the QTableView object in setUpMainWindow(). This is done to help organize the code.
Code for the createConnection() method in the account management GUI
Code for the createModel() method for the account management GUI
Code for the setUpMainWindow() method in the account management GUI, part 1
The add_product_button and del_product_button objects are used to add and delete items from the table and the model. Each button is connected to a slot using the clicked signal. The slots are created in Listing 14-23.
Items in a view can also be sorted. The sort_combo provides various ways to organize the table’s data based on the column names. For the QComboBox, when the selection has changed, the widget can emit a currentTextChanged signal. The signal also passes text that we can use to determine how to set the view’s order for displaying records in the setSortingOrder() slot. This is handled in Listing 14-24. The widgets are then arranged in buttons_h_box and added to edit_container.
Code for the setUpMainWindow() method in the account management GUI, part 2
SelectRows – Only rows can be selected
SelectColumns – Only columns can be selected
SelectItems – Only individual items can be selected
In Model/View programming, the delegate provides the default tools for painting item data in the view and for providing editor widgets for item models. The appearance and editor widgets of the item delegate can be customized. For the account management GUI, the delegate used is the QSqlRelationalDelegate. This class provides a combo box for editing data in fields that are foreign keys for other tables.
An example of the combo box used by the delegate can be seen in the bottom-right corner of Figure 14-4. The widget appears whenever the user needs to select a country from the countries table that will be displayed in the view.
Code for addItem() and deleteItem() slots in the account management GUI
For deleteItem(), we get the currently selected row’s index and delete the row with removeRow(). Then we update the model using select().
Code for setSortingOrder() slot in the account management GUI
Finally, select() is called to update the model and view with the data.
At this point, you should run the application and test it out. If you want to tinker around with the code, first have a look at the different selection modes and selection behaviors. From there, you could go back to the SQL database and try implementing additional fields or create new foreign keys to test out the relational classes.
Summary
PyQt provides convenience classes for lists, tables, and trees. QListWidget, QTableWidget, and QTreeWidget are useful when you need to view data for general situations. While they are practical for creating quick interfaces for editing data, if you need to have more than one widget for displaying a dataset in an application, you must also create a process for keeping the datasets and the widgets in agreement. A better option is to use PyQt’s Model/View architecture.
There are different formats available for storing and managing data. One example is the CSV format that is convenient for reading, parsing, and storing smaller datasets. However, for large databases that contain multiple tables with relational characteristics, a Relational Database Management System that uses the SQL language is a more preferable option for managing the data. SQL allows users to select desired information that might be shared between tables as well as insert, update, and delete existing records easily.
Model/View is very useful for working with SQL databases, providing the tools necessary for connecting to a database and viewing its content. Qt provides three models for working with SQL databases. For an editable data model without foreign key support, use QSqlTableModel. If you have tables with relational properties, use QSqlRelationalTableModel. Finally, the QSqlQueryModel is beneficial when you only need to read the results of a query without editing them.
Over the course of this book, we took a look at a few applications that could have benefited greatly from data management. The login GUI in Chapter 3 could connect to a database to retrieve usernames and passwords. There is also the pizza ordering GUI from Chapter 6. You could implement a database for storing customer’s information, using a relational database for adding new customers, updating existing ones, and preventing data from being duplicated.
In Chapter 15, we will take a brief look at multithreading in PyQt.