Building the Data Layer

You start off creating this game by designing the data layer. For this game you need only a simple text file to hold the state of the game so that it can be saved or resumed. A tic-tac-toe game consists of a board with nine squares. Each square can be empty or have the letter ‘X’ or ‘O’ in it. You can represent those three options with a simple list of characters. For storage you convert that list into a simple character string.

The only other piece of data needed is which player is due to move next but, in a computer versus human game, you can assume the human is always next to go. So your data layer interface consists of only two exposed, or published, methods:

  saveGame()
  restoreGame()

Because you want to keep your layers separate, you should put these methods into a module. To create this module, type the following code and save it as oxo_data.py (or load it from the OXO folder of the Chapter4.zip download):

  ''' oxo_data is the data module for a tic-tac-toe (or OXO) game.
  It saves and restores a game board. The functions are:
  		 saveGame(game) -> None
  		 restoreGame() -> game
  Note that no limits are placed on the size of the data.
  The game implementation is responsible for validating
  all data in and out.'''

  import os.path
  game_file = ".oxogame.dat"

  def _getPath():
 	''' getPath -> string
 	Returns a valid path for data file.
 	Tries to use the users home folder, defaults to cwd'''

 	try:
  		 game_path = os.environ['HOMEPATH'] or os.environ['HOME']
  		 if not os.path.exists(game_path):
 			 game_path = os.getcwd()
 	except (KeyError, TypeError):
  		 game_path = os.getcwd()
 	return game_path

  def saveGame(game):
 	''' saveGame(game) -> None

 	saves a game object in the data file in the users home folder.
 	No checking is done on the input, which is expected to
 	be a list of characters'''

 	path = os.path.join(_getPath(), game_file)
 	with open(path, 'w') as gf:
  		 gamestr = ''.join(game)
  		 gf.write(gamestr)

  def restoreGame():
 	''' restoreGame() -> game

 	Restores a game from the data file.
 	The game object is a list of characters'''

 	path = os.path.join(_getPath(), game_file) 
 	with open(path) as gf:
  		 gamestr = gf.read()
  		 return list(gamestr)

  def test():
 	print("Path = ", _getPath())
 	saveGame(list("XO XO OX"))
 	print(restoreGame())

  if __name__ == "__main__": test()

The first function, _getPath(), is a helper function that uses the os module to try to determine the user’s home folder and, if that fails, use the current folder. By convention, functions that are not intended to be called by module users have a leading underscore in front of their name, like _getPath(). The saveGame() function uses _getPath() to create a new file containing the string representation of the game. The final function restoreGame() also uses _getPath() to locate the saved file and open it, reading back the stored game data.

Ideally, you would include a more sophisticated test function (or a set of unit tests, like those described in Chapter 6, “Python on Bigger Projects”). In the interest of brevity, these are not shown here.

Building the Core Logic layer

You now create the core logic of the game. For that you need to define a number of functions that are used throughout the course of a game. However, to know what those functions are, you first need to think about how the game will be played. So, before diving into logic code, you need to map out the sequence of play and, for that, you can use a sequence diagram.

The game starts by presenting the user with a menu of options. These will include the options to start a new game or restore a saved game. In either case, once the game has been set up, the board will be displayed and the user prompted to select a cell. The computer then analyzes the move and responds with its own. The board is then shown again until a winner is found. The sequence diagram for this is as shown in Figure 4.1.

The sequence diagram is a simplified version of a UML sequence diagram. Each of the “objects” (modules in this case), along with the user, are represented by a vertical line. The arrows indicate messages flowing between the modules (and user). The messages have descriptive titles. Some messages are optional, and the conditions that cause them are indicated in square brackets (called guards). Some sequences of arrows are enclosed in boxes, and these represent loops or conditional blocks with a description given in the upper-left corner. In this case the main game-play sequence is repeated until there is a win, or draw, or until the user selects quit. If the user selects quit, then the sequence in the box at the bottom is executed. Sequence diagrams are a very powerful analysis and design tool.

From the sequence diagram, you can see that you need to provide functions to handle the user’s menu choices (only new is shown in the diagram), as well as a function to play the game. The latter would need to return different results depending on the outcome, which is usually a bad idea. However, if you create separate functions for the user’s and computer’s moves, you can then use the same analysis helper function for both the user and the computer. With that in mind you need to write the following functions:

  newGame()
  saveGame()
  restoreGame()
  userMove()
  computerMove()

You need helper functions to generate a random move and analyze whether a given move wins the game. The list of helper functions is therefore:

  _generateMove()
  _isWinningMove()

Once again, to keep the separation between the layers, you should put the logic code into a separate module, this time called oxo_logic.py. It looks like this:

  ''' This is the main logic for a tic-tac-toe game.
  It is not optimised for a quality game it simply
  generates random moves and checks the results of
  a move for a winning line. Exposed functions are:
  newGame()
  saveGame()
  restoreGame()
  userMove()
  computerMove()
  '''

  import os, random
  import oxo_data

  def newGame():
 	return list(" " * 9)

  def saveGame(game):
 	oxo_data.saveGame(game)

  def restoreGame():
 	try:
  		 game = oxo_data.restoreGame()
  		 if len(game) == 9:
 			 return game
  		 else: return newGame()
 	except IOError:
  		 return newGame()

  def _generateMove(game):
 	options = [i for i in range(len(game))
  		 if game[i] == " "]
  	 return random.choice(options)

  def _isWinningMove(game):
 	pass

The newGame() function simply returns a new list of nine spaces.

The saveGame() function calls the oxo_data function of the same name. The restoreGame() function is marginally more complex in that it catches any errors arising because the data file cannot be found and validates the length of the restored game. It could do more data validation on the content of the data, but you can add that later if you wish.

The _generateMove() function looks for the unused cells in the current game and then randomly selects a cell to place the computer’s move. This is not optimal, and a more intelligent algorithm would greatly improve the quality of the game.

The _isWinningMove() method has been left unfinished because it is the most complex of the logic functions. This is where the real processing takes place. The approach you take is very simple and relies on the fact that there are only eight possible wining lines. They can be listed in terms of the indices of the game cells involved:

  wins = ((0,1,2), (3,4,5), (6,7,8),
 	(0,3,6), (1,4,7), (2,5,8),
 	(0,4,8),(2,4,6))

To assess whether a move has resulted in a win, you need to check each winning line. You can extract the character in each cell of the candidate line and construct a three-character string. For a win all three characters need to be either ‘X’ or ‘O’. The function looks like this:

  def _isWinningMove(game):
 	wins = ((0,1,2), (3,4,5), (6,7,8),
  		 (0,3,6), (1,4,7), (2,5,8),
  		 (0,4,8), (2,4,6))

 	for a,b,c in wins:
  		 chars = game[a] + game[b] + game[c]
  		 if chars == 'XXX' or chars == 'OOO':
  		 return True
 	return False

Finally, you need a pair of functions that can analyze a user move and a computer move. The former takes a cell value input by the user; the latter needs only the game because it uses _generateMove() internally. They return the outcome of the move as one of four character codes. An empty string means the game is still on, an ‘X’ or ‘O’ signifies the victor, and a ‘D’ means it’s a draw. These functions look like this:

  def userMove(game,cell):
 	if game[cell] != ' ':
  		 raise ValueError('Invalid cell')
 	else:
  		 game[cell] = 'X'
 	if _isWinningMove(game):
  		 return 'X'
 	else:
  		 return ''

  def computerMove(game):
 	cell = _generateMove(game)
 	if cell == -1:
  		 return 'D'
 	game[cell] = 'O'
 	if _isWinningMove(game):
  		 return 'O'
 	else:
  		 return ''

You could have implemented all of these functions as methods of a Game class. That would have removed the need to pass the game data into each function.

Finally, you need a test function:

  def test():
 	result = ''
 	game = newGame()
 	while not result:
  		 print(game)
  		 try:
 			 result = userMove(game, _generateMove(game))
  		 except ValueError:
 			 print("Oops, that shouldn't happen")
  		 if not result:
 			 result = computerMove(game)
  		  
  		 if not result: continue
  		 elif result == 'D':
 			 print("Its a draw")
  		 else:
 			 print("Winner is:", result)
  		 print(game)

  if __name__ == "__main__":
 	test()

The test function keeps on generating moves until either somebody wins or the board fills up (at which point _generateMove() returns -1). The moves are generated entirely randomly, so there is no intelligence in the selections. You can run the code, and you should see something like this (the actual results you get are random because you use the random module to generate the moves):

  [' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ']
  [' ', ' ', ' ', ' ', ' ', 'O', ' ', 'X', ' ']
  [' ', 'X', ' ', 'O', ' ', 'O', ' ', 'X', ' ']
  Winner is: X
  [' ', 'X', ' ', 'O', 'X', 'O', 'O', 'X', ' ']

It’s not very pretty, but then, it’s not supposed to be. Presentation is in the user interface, and that’s what you build next.

Building the User Interface

In the previous section, you started to think about how the user would perceive the application, that is, the user experience. The user experience is the key factor in driving the user interface design. You need to consider the flow of control from entering the application through normal use to exit. In a command-line application, the most common approaches are to offer menus, often nested to several levels, or to accept commands typed at a prompt. In this section you build a set of very simple menus and create a simple prompt-based solution. This process illustrates both techniques while keeping the code size small.

If you walk through the user experience in playing a game of tic-tac-toe, the game starts by offering a menu that allows the user to start a new game, resume a saved game, request help, or quit. (The ability to quit easily is an important, but often overlooked, feature of good user interface design.)

The quit option just exits the program. The help screen displays a page of explanatory text. Both of the other options take the user into a tic-tac-toe game. Once in that game, the user can either select a cell, save the game, or quit. If they choose to quit while the game is still in progress, they should be asked if they want to save the game first. If they choose a cell, then the computer analyzes the move, makes its own move, analyzes that, and presents the result. If either move results in a win or in all cells being used, then a message is displayed to the user and an option to quit or return to the main menu.

The user interface presents the information and manages the flow from screen to screen. It does not perform any of the computational logic; that is provided by the core logic layer. The user interface simply calls the functions that you defined in the oxo_logic module.

You start by defining a function to display a menu and return a valid user selection. The function could be specific to the menu and include the menu definition within it; however, it is not much harder to write a function that takes a menu as input and is therefore reusable for all menus in the system. You could even extract it to a module for reuse across projects.

The menu code, which you can save in oxo_ui.py (or load it from the zip file), looks like this:

  ''' CLI User Interface for Tic-Tac-Toie game.
 	Use as the main program, no reusable functions'''

  import oxo_logic

  menu = ["Start new game",
  		 "Resume saved game",
  		 "Display help",
  		 "Quit"]

  def getMenuChoice(aMenu):
 	''' getMenuChoice(aMenu) -> int

  		 takes a list of strings as input,
  		 displays as a numbered menu and
  		 loops until user selects a valid number'''

 	if not aMenu: raise ValueError('No menu content')
 	while True:
  		 print("

")
  		 for index, item in enumerate(aMenu, start=1):
 			 print(index, "	", item)
  		 try:
 			 choice = int(input("
Choose a menu option: "))
 			 if 1 <= choice <= len(aMenu):
 				 return choice
 			 else: print("Choose a number between 1 and", len(aMenu))
  		 except ValueError:
 			 print("Choose the number of a menu option")

  def main():
 	print(getMenuChoice(menu))
 	getMenuChoice([]) # raise error

  if __name__ == "__main__": main()

Notice that the enumerate function is used to generate the menu option numbers and that they start at 1 because most users find 0 a strange option choice. The function keeps repeating until the user selects a valid choice, with prompts to correct invalid choices.

Having presented some choices, you now need to write code to do all of those things. The methods are named after the menu options and are shown as follows:

  def startGame():
  	return oxo_logic.newGame()

  def resumeGame():
  	return oxo_logic.restoreGame()

  def displayHelp():
  	print('''
  	Start new game: starts a new game of tic-tac-toe
  	Resume saved game: restores the last saved game and commences play
  	Display help: shows this page
  	Quit: quits the application
  	''')

  def quit():
  	print("Goodbye...")
  	raise SystemExit

Your next step is to write a function to process the user’s choice. That can be done using an if/elif chain, but that can become difficult to maintain if there are many options. Although the number of possible choices in this project is small, you use a dispatch table because this is a powerful, efficient, and flexible technique. You also need to change the main() function to loop over the menu and game code until the user quits. Make the following modifications to your program:

  def executeChoice(choice):
  	''' executeChoice(int) -> None

  		 Execute whichever option the user selected.
  	If the choice produces a valid game then
  	play the game until it completes.'''
  	dispatch = [startGame, resumeGame, displayHelp, quit]
  	game = dispatch[choice-1]()
  	if game:
  		 # play game here
  		 pass

  def main():
  	while True:
  		 choice = getMenuChoice(menu)
  		 executeChoice(choice)

That only leaves the task of actually playing the game. You need to write a function that takes a starting game position and interacts with the user until the game completes, either because there are no more moves or a winner is found. In addition, it helps if you have a function that displays the game in the usual grid layout rather than the flat data format that you are using internally. These are shown here:

  def printGame(game):
  	display = '''
  		 1 | 2 | 3 {} | {} | {}
  		 ---------- ------------
  		 4 | 5 | 6 {} | {} | {}
  		 ---------- ------------
  		 7 | 8 | 9 {} | {} | {}'''
  	print(display.format(*game))
  def playGame(game):
  	result = ""
  	while not result:
  		 printGame(game)
  		 choice = input("Cell[1-9 or q to quit]: ")
  		 if choice.lower()[0] == 'q':
 			 save = input("Save game before quitting?[y/n] ") if
 			 save.lower()[0] == 'y':
 				 oxo_logic.saveGame(game)
 			 quit()
  		 else:
 			 try:
 				 cell = int(choice)-1
 				 if not (0 <= cell <= 8): # check range
 					 raise ValueError
 			 except ValueError:
 				 print("Choose a number between 1 and 9 or 'q' to quit ")
 				 continue

  		 try:
 			 result = oxo_logic.userMove(game,cell)
  		 except ValueError:
 			 print("Choose an empty cell ")
 			 continue
  		 if not result:
 			 result = oxo_logic.computerMove(game)
  		 if not result:
 			 continue
  		 elif result == 'D':
 			 printGame(game)
 			 print("Its a draw")  
  		 else:
 			 printGame(game)
 			 print("Winner is", result, "
")

The printGame() function uses formatting to insert the game values into the display. Note the use of the asterisk (*game) that expands the list into its individual elements.

The playGame() function displays the user interface prompt for the game screen. You ask for a number representing the cell that the user wishes to place an ‘X’ into. You also provide the option to quit, and if it is selected, offer the chance to save the game. If the user’s choice is valid, you then use the oxo_logic functions to determine the outcome of the selection, and if not, a finishing move to get the computer to take a turn. Notice how all of the rules of the game and the data management are in the lower layers. The user interface layer is dealing with presentation and control flow only. One slight quirk of the design is that the board is passed into both the userMove() and computerMove() functions, but the updated board is not returned by the functions. That is because the board object is a mutable list and, as such, can be changed by the function and the original variable will reflect those changes.

Later in this chapter, you revisit this game and see how easily you can create a new GUI-based user interface layer on top of the existing oxo_logic and oxo_data modules. Before you do that, there are a couple of interesting options you can use to enhance your command-line applications. You look at them in the next sections.

Introducing Key GUI Principles

Virtually all GUIs are event driven. That means you need to write your code to respond to certain events generated by the GUI toolkit. GUIs come with a whole language of their own in terms of the objects from which a GUI is built. There are windows, frames, controls, and so on. These objects are all connected by something called a containment tree. You see what each of these concepts means and how they fit together to form a GUI in the following sections.

The Containment Tree

Every GUI application is constructed in a tree-like manner with a top-level window containing other windows that in turn contain more windows until you eventually reach the lowest level controls and widgets. This hierarchy can be represented as a tree structure and is known as the containment tree.

Events arrive at a child widget, which, if it is unable to handle it, passes the event to its parent and so on up to the top level. Similarly, if a command is given to draw a widget, it will send the command on down to its children; thus a draw command to the top-level widget redraws the entire application, whereas one sent to a button likely only redraws the button.

This concept of events percolating up the tree and commands being pushed down is fundamental to understanding how GUIs operate at the programmer level. It is also the reason that you always need to specify a widget’s parent when creating it, so that it knows where it sits in the containment tree. An example containment tree is shown in Figure 4.3.

This illustrates the top-level widget containing a single Frame that represents the outermost window border. This in turn contains two more Frames; the first contains a text Entry widget and a Label, and the second contains the two Buttons used to control the application. You should refer back to this diagram when you get ready to build a simple GUI in the next section.

Building a Simple GUI

It’s time to turn this discussion into real code. You start by building the application illustrated in Figure 4.3 in the previous section. The code looks like this (and can be loaded from the file demo1.py in the Tkinter folder of the Chapter4.zip file):

  	import tkinter as tk

  	# create the top level window/frame
  	top = tk.Tk()
  	F = tk.Frame(top)
  	F.pack(fill="both")

  	# Now the frame with text entry
  	fEntry = tk.Frame(F, border=1)
  	eHello = tk.Entry(fEntry)
  	eHello.pack(side="left")
  	lHistory = tk.Label(fEntry, text=" ", foreground="steelblue")
  	lHistory.pack(side="bottom", fill="x")
  	fEntry.pack(side="top")

You start by importing tkinter and creating a top-level widget. (You already saw this in the earlier section on using message boxes.) Next, you create a Frame to hold all of the other widgets. The first parameter of all widget creation methods is the parent widget, so in this case you specify the parent as being top. The next step calls F.pack(). The pack() call invokes a simple layout manager that, by default, simply packs components into the containing object starting at the top and working down. You are packing your Frame object into its parent, top, that represents the main window. The fill option tells the widget to expand to fill the window in both vertical and horizontal directions.

You then create a second Frame to hold the Entry and Label widgets. You use named parameters of the constructor to set a border around the entry widget and set the foreground color of the label font. Notice also that you are using arguments to tell the packer to place the widgets at the sides of the frame rather than its default vertical stacking arrangement. You can see the usage pattern developing—create a widget then pack it. Finally, you pack the new fEntry frame itself.

The next step is to create the buttons and associate some behavior with them. For that you need to create an event handler that is activated when the user presses a button:

  # create the event handler to clear the text
  def evClear():
  	lHistory['text'] = eHello.get()
  	eHello.delete(0,tk.END)

The event handler sets the text of the lHistory label to the contents of the eHello entry field and then deletes the text from the eHello field itself. You use a dictionary style access to set the Label’s text. This technique works for any of the attributes of the widget. The delete() method takes 0 as the first argument. That indicates the start of the text, and the special value tk.END, used as the second argument, means the end of the text.

You create the buttons and connect the event handler with the following code:

  	# Finally the frame with the buttons.
  	# sink this one for emphasis
  	fButtons = tk.Frame(F, relief="sunken", border=1)
  	bClear = tk.Button(fButtons, text="Clear Text", command=evClear)
  	bClear.pack(side="left", padx=5, pady=2)
  	bQuit = tk.Button(fButtons, text="Quit", command=F.quit)
  	bQuit.pack(side="left", padx=5, pady=2)
  	fButtons.pack(side="top", fill="x")

  	# Now run the eventloop
  	F.mainloop()

Once again, you create a frame to hold the buttons and then pack the buttons using side arguments. You also added some padding to the buttons so that you can create some space around them. The bClear button is connected to your event handler function by specifying the function name, evClear, as the command argument. The bQuit button uses the predefined quit method on the top-level frame, F.

Finally, you start the Tkinter mainloop() running and wait for the user to do something. If you run the code, the resulting window should look like Figure 4.4, which shows before and after versions of the application.

You’ll find that you can resize, iconify, and move the window just like any other window in your desktop. Pressing the Quit button closes the application (as does the usual close icon on the title bar) and pressing the Clear button calls your evClear function.

Referring back to the containment tree diagram from the previous section, you see that it describes the layout of this application with three frames, a label and entry box, and two buttons.

Now that you’ve seen the basics in action, you can move onto a more realistic example using more widgets and linking the controls to more substantial event handling functions. It’s time to revisit your tic-tac-toe game.

Building a Tic-Tac-Toe GUI

In this section you build a GUI for your tic-tac-toe game using exactly the same logic and data layers as previously. This GUI is much closer to the kind of GUI you would expect to see on a modern desktop application, complete with menus, buttons, and mouse interaction rather than relying on keyboard input. By using the same tic-tac-toe logic as before, you can focus on the structure of the GUI without thinking too much about how the application itself works.

Sketching a UI Design

When building a substantial GUI application, it often helps to sketch out roughly what you want it to look like before jumping into code. For a tic-tac-toe game, you want a menu bar with File and Help menus. The File menu has New, Resume, Save, and Exit menu items. (You could have called the menu the Game menu, but File is the conventional choice for GUI applications and consistency of style is one of the advantages of using GUIs.) The Help menu has Help and About options.

The board itself will be represented by nine buttons laid out in the usual grid style. When a button is clicked, its label will display the player’s mark.

A status bar on the bottom will display messages to the user and the final results will be presented using message boxes. It should look something like Figure 4.5.

With a clear idea of the layout, you now want to start thinking about the code structure. The GUI essentially consists of three areas: the menu bar, the board, and the status bar. The board needs to be treated as a group and centered in its frame so you could make the board itself another frame inside the central frame. The status bar just displays text so it will consist only of a label widget. That just leaves the menu bar.

  import tkinter as tk
  import tkinter.messagebox as mb
  import oxo_logic

  top = tk.Tk()

  def buildMenu(parent):
  	menus = (
  		 ("File", (("New", evNew),
 			 ("Resume", evResume),
 			 ("Save", evSave),
 			 ("Exit", evExit))),
 			 ("Help", (("Help", evHelp),
 			 ("About", evAbout)))
  		 )

  	menubar = tk.Menu(parent)
  	for menu in menus:
  		 m = tk.Menu(parent)
  		 for item in menu[1]:
 			 m.add_command(label=item[0], command=item[1])
  		 menubar.add_cascade(label=menu[0], menu=m)

  	return menubar

  def dummy():
  	mb.showinfo("Dummy", "Event to be done")

  	evNew = dummy
  	evResume = dummy
  	evSave = dummy
  	evExit = top.quit
  	evHelp = dummy
  	evAbout = dummy

  	mbar = buildMenu(top)

  	top["menu"] = mbar

  	tk.mainloop()

  def evClick(row,col):
  	mb.showinfo("Cell clicked", "row:{}, col:{}".format(row,col))

  def buildBoard(parent):
  	outer = tk.Frame(parent, border=2, relief="sunken")
  	inner = tk.Frame(outer)
  	inner.pack()

  	for row in range(3):
  		 for col in range(3):
 			 cell = tk.Button(inner, text=" ", width="5", height="2",
 				 command=lambda r=row, c=col : evClick(r,c) )
 			 cell.grid(row=row, column=col)
  	return outer

  mbar = buildMenu(top)
  top["menu"] = mbar

  board = buildBoard(top)
  board.pack()
  status = tk.Label(top, text="testing", border=0,
  		 background="lightgrey", foreground="red")
  status.pack(anchor="s", fill="x", expand=True)

  tk.mainloop()

Connecting the GUI to the Game

Having gotten the basic GUI structure in place, you can now turn your attention to writing the game play code and hooking up the various menus to the final event handling functions. You can tackle the game code first as it sits mainly in the evClick event handler and is aided by a couple of helper functions that you can call cells2game and game2cells.

The modifications look like this (and can be loaded from the file oxo_gui_game.py in OXO folder of the zip file):

  gameover = False
  def evClick(row,col):
  	global gameover
  	if gameover:
  		 mb.showerror("Game over", "Game over!")
  		 return
  	game = cells2game()
  	index = (3*row) + col
  	result = oxo_logic.userMove(game, index)
  	game2cells(game)

  	if not result:
  		 result = oxo_logic.computerMove(game)
  		 game2cells(game)
  	if result == "D":
  		 mb.showinfo("Result", "It's a Draw!")
  		 gameover = True
  	else:
  		 if result == "X" or result == "O":
 			 mb.showinfo("Result", "The winner is: {}".format(result))
 			 gameover = True

  def game2cells(game):
  	table = board.pack_slaves()[0]
  		 for row in range(3):
 			 for col in range(3):
 				 table.grid_slaves(row=row,column=col)[0]['text'] = game[3*row+col]

  def cells2game():
  	values = []
  	table = board.pack_slaves()[0]
 	for row in range(3):
 		 for col in range(3):
 			 values.append(table.grid_slaves(row=row, column=col)[0]['text'])
  	return values

If you compare the evClick code to the original playGame() function, you will see that there are many similarities. The game2cells() function is analogous to the original printGame() function. The cells2game() function uses some widget methods to retrieve the child widgets, in this case the buttons. You could have, as an alternative, stored the lists of buttons in a global data structure that would have given more direct access. The game logic of userMove and computerMove is unchanged even though the user interface is vastly different.

The last thing to do is fill in the menu event handlers. These are almost trivial to complete, and the finished program looks like this (it is found as oxo_gui_complete.py in the OXO folder of the zip file):

  import tkinter as tk
  import tkinter.messagebox as mb
  import oxo_logic

  top = tk.Tk()

  def buildMenu(parent):
 	menus = (
 		("File",( ("New", evNew),
 			("Resume", evResume),
 			("Save", evSave),
 			("Exit", evExit))),
 			("Help",( ("Help", evHelp),
 			("About", evAbout)))
 		)

 	menubar = tk.Menu(parent)
 	for menu in menus:
 		 m = tk.Menu(parent)
 		for item in menu[1]:
 			m.add_command(label=item[0], command=item[1])
 		menubar.add_cascade(label=menu[0], menu=m)

 	return menubar

  def evNew():
  		 status['text'] = "Playing game"
  		 game2cells(oxo_logic.newGame())

  def evResume (): 
  		 status['text'] = "Playing game"
  		 game = oxo_logic.restoreGame()
  		 game2cells(game)

  def evSave():
  		 game = cells2game()
  		 oxo_logic.saveGame(game)
  

  def evExit ():
  		 if status['text'] == "Playing game":
 			 if mb.askyesno("Quitting","Do you want to save the game before
 			 quitting?"):
 				 evSave()
  		 top.quit()
  

  def evHelp ():
  	mb.showinfo("Help",'''
  	File->New: starts a new game of tic-tac-toe
  	File->Resume: restores the last saved game and commences play
  	File->Save: Saves current game.
  	File->Exit: quits, prompts to save active game
  	Help->Help: shows this page
  	Help->About: Shows information about the program and author''')

  def evAbout():
  	mb.showinfo("About","Tic-tac-toe game GUI demo by Alan Gauld")

  def evClick(row,col):
  	if status['text'] == "Game over":
 		mb.showerror("Game over", "Game over!")
 		return

 	game = cells2game()
 	index = (3*row) + col
 	result = oxo_logic.userMove(game, index)
 	game2cells(game)

 	if not result:
 		result = oxo_logic.computerMove(game)
 		game2cells(game)
 	if result == "D":
 		mb.showinfo("Result", "It's a Draw!")
  		 status['text'] = "Game over"
 	 else:
 		if result =="X" or result == "O":
 			mb.showinfo("Result", "The winner is: {}".format(result))
 			 status['text'] = "Game over"

  def game2cells(game):
 	table = board.pack_slaves()[0]
 	for row in range(3):
 		for col in range(3):
 			table.grid_slaves(row=row,column=col)[0]['text'] = game[3*row+col]

  def cells2game():
 	values = []
 	table = board.pack_slaves()[0]
 	for row in range(3):
 		for col in range(3):
 			values.append(table.grid_slaves(row=row, column=col)[0]['text'])
 	return values

  def buildBoard(parent):
 	outer = tk.Frame(parent, border=2, relief="sunken")
 	inner = tk.Frame(outer)
 	inner.pack()

 	for row in range(3):
 		for col in range(3):
 			cell = tk.Button(inner, text=" ", width="5", height="2",
 				command=lambda r=row, c=col : evClick(r,c) )
 			cell.grid(row=row, column=col)
 	return outer
 	mbar = buildMenu(top)
 	top["menu"] = mbar

 	board = buildBoard(top)
 	board.pack()
  	status = tk.Label(top, text="Playing game", border=0,
  		 background="lightgrey", foreground="red")
 	status.pack(anchor="s", fill="x", expand=True)

 	tk.mainloop()

The event functions mirror the original functions in that they mostly just call the oxo_logic functions and then use the game2cells() function to display the board. You now use the status text instead of the global gameover flag. The Help menus simply display text in a showinfo dialog.

The final working game looks like Figure 4.6.

There is a lot more you could do to add polish to this game, but it provides enough to show what can be done using Tkinter as a user interface toolkit and illustrates once again the power of separating the logic and data layers from the presentation layer. You have now written more than 600 lines of code to play the various versions of tic-tac-toe. That’s enough for anyone, so it’s time to move onto new pastures.

Tkinter has many other widgets and tricks for you to play with. Experiment with the simple GUI application we started with and add or modify the different options that affect layout and appearance. You haven’t even looked at how to display formatted text or images or plot graphs or build complex dialogs. All of these things are possible and build on the foundation you saw here. There are many Tkinter tutorials and sample programs around, including IDLE, the default IDE for Python. Reading the code and seeing how these programs control appearance and use widgets is a great way to learn.

Tkinter has a couple of extension modules that are included in the standard library (although some Linux distributions omit them for some reason). In the next section, you see what these extension modules can add to your programs.

Extending Tkinter

The two biggest criticisms leveled at Tkinter are that it doesn’t have enough widgets and it looks ugly. In comparison to the other GUI toolkits, these are valid issues. However, in recent releases, Tkinter has been fighting back with the introduction of two new modules built on top of Tkinter. These are tix, which adds several new widgets, and ttk, which enables theming, which basically, just means the GUI can look more like the native OS GUI.

Unfortunately, the documentation for these modules is not as comprehensive as you would want, and at the time of writing, the Python documentation often just contains links to the Tcl/Tk documentation. Once you get used to it, you can usually figure out what options you need from there, but it’s not ideal. However, the power of the Python interactive prompt comes to the rescue once again, because you can play with them and experiment to find out what they have to offer. Both modules support most of the same Tkinter widgets that you have used so far; so you should be able to take your Tkinter program and convert it to using ttk or Tix fairly easily. For Tix it is as simple as changing the tkinter import line as shown:

  import tkinter.tix as tk

This is one benefit of using the tk alias when importing—you don’t need to change all your tkinter prefixes to tix; you simply set the import to use the same alias. You can try that on your tic-tac-toe game if you like. It functions identically to the Tkinter version except that the title bar of the window displays tix instead of Tk.

For ttk it’s marginally more complicated because ttk uses the tkinter mainloop and top-level window so you need to import both. You then refer to ttk when creating widgets and tkinter when controlling the top window and event loop. You see this in practice later, in the “Using ttk” section.

Using Tix

Because tix is so similar to tkinter, you can translate all you know about tkinter into tix and jump straight into learning about the new widgets. There are more than 40 of them, but some are very poorly documented, even in the Tcl/Tk community. If you stick to the subset listed on the Python documentation page, you should be fine. You only dip a toe in the water here, but hopefully it is enough to demonstrate that tix is a valuable addition to the Tkinter family.

The widgets that you look at here are the ComboBox, the ScrolledText, and the Notebook.

You can see how a ComboBox input control can be used to set a label’s text by typing the following code at the Python interactive prompt:

  >>> import tkinter.tix as tix
  >>> top = tix.Tk()
  >>> lab = tix.Label(top)
  >>> lab.pack()
  >>> cb = tix.ComboBox(top,command=lambda s:lab.config(text=s))
  >>> for s in ["Fred","Ginger","Gene","Debbie"]:
  ... cb.insert("end",s)
  ...
  >>> cb.pick(0)
  >>> lab['text'] = "Pick any item"
  >>> cb.pack()
  >>> top.mainloop()
  >>>

There are several things to note here. First, you used config() to set the text attribute rather than the dictionary style access you used previously. The advantage of config() is that you can set multiple attributes at once just by passing them as named arguments. Second, the event handler lambda function uses a string argument, s, which is passed in by the widget event. The string holds the currently selected value. Figure 4.7 shows the resulting window in action.

The ScrolledText widget is an extension of the standard Text widget. As such it can display images as well as formatted text. The Tix version adds scrollbars automatically, which is a useful addition that involves quite a lot of work using the standard toolkit. In use it is very much like the other Tkinter widgets. You can play with it by typing the following code:

  >>> top = tix.Tk()
  >>> st = tix.ScrolledText(top, width=300, height=100)
  >>> st.pack(side='left')
  >>> top.mainloop()

Figure 4.8 shows the resulting text box with enough text to activate the vertical scrollbar.

You can simply type into the text box manually or insert text programmatically, like so:

  >>> t = st.subwidget('text')
  >>> t
  <tkinter.tix._dummyText object at 0x019186D0>
  >>> t.insert('0.0',"Some inserted text")
  >>> t.insert('end',"
 more inserted text")
  >>>

Notice that you used the subwidget() method to get a reference to the underlying text widget and then used its insert() method to insert the text. This technique of fetching the underlying standard widget is quite common when working with Tix widgets.

You can also select areas of text in a text widget. Carrying on the previous example, you can change the font and weight of the first line of the previous text like so:

  >>> t.tag_configure('newfont', font=("Roman", 16, "bold"))
  >>> s = t.get('1.0','1.end')
  >>> t.delete('1.0','1.end')
  >>> t.insert('1.0',s,'newfont')

The tag_configure() method creates a new tag, that is to say a style that you can apply to text. The style is called newfont and uses a Roman font, of size 16 points and weight bold. This triplet font specifier format is standard across Tkinter (and therefore Tix and ttk).

You then used get() to fetch the text from the first character through to the end of line 1. You then deleted the existing text from the widget and replaced it with the same text but using the newfont tag as a third argument of the insert() method.

The result is as shown in Figure 4.9. Note that all of this is being displayed in a Tix ScrolledText widget, but you are actually using the underlying standard Tkinter Text widget to manipulate the text.

The final widget you look at is an altogether more complex contraption. It is a NoteBook widget. It consists of a number of pages, each with an associated tab that the user can select to activate the page. The page is just a Tkinter container that can be populated with whatever controls you want to use. Often it is a text window or a form. You create a two-page notebook, the first pane containing a ScrolledText widget and the other a set of buttons that launch various message box dialogs.

To see the Notebook in action, type the following code into a file and execute it from the command line or your IDE (or load it from the file tix-notebook.py in the Tkinter folder of the zip file, if you prefer):

  	import tkinter.tix as tix
  	import tkinter.messagebox as mb

  	top = tix.Tk()

  	nb = tix.NoteBook(top, width=300, height=200)
  	nb.pack(expand=True, fill='both')

  	nb.add('page1', label="Text")
  	f1 = tix.Frame(nb.subwidget('page1'))
  	st = tix.ScrolledText(f1)
  	st.subwidget('text').insert("1.0", "Here is where the text goes...")
  	st.pack(expand=True)
  	f1.pack()

  	nb.add('page2', label="Message Boxes")
  	f2 = tix.Frame(nb.subwidget('page2'))
  	tix.Button(f2, text="error", bg="lightblue",
  		 command=lambda t="error", m="This is bad!":
 			 mb.showerror(t,m) ).pack(fill='x',expand=True)
  	tix.Button(f2, text="info", bg='pink',
  		 command=lambda t="info", m="Information":
 			 mb.showinfo(t,m) ).pack(fill='x',expand=True)
  	tix.Button(f2, text="warning", bg='yellow',
  		 command=lambda t="warning", m="Don't do it!":
 			 mb.showwarning(t,m) ).pack(fill='x',expand=True)
  	tix.Button(f2, text="question", bg='green',
  		 command=lambda t="question", m="Will I?":
 			 mb.askquestion(t,m) ).pack(fill='x',expand=True)
  	tix.Button(f2, text="yes-no", bg='lightgrey',
  		 command=lambda t="yes-no", m="Are you sure?":
 			 mb.askyesno(t,m) ).pack(fill='x',expand=True)
  	tix.Button(f2, text="yes-no-cancel", bg='black', fg='white',
  		 command=lambda t="yes-no-cancel", m="Last chance...":
 			 mb.askyesnocancel(t,m) ).pack(fill='x',expand=True)

  	f2.pack(side='top', fill='x')

  	top.mainloop()

In this example you imported the modules and created the top-level widget as usual. You then created a tix.Notebook object called nb. To this you added a page and called it page1. You then created a frame and made its parent the page1 page that you just created. You added a text widget and some text, and then packed the widget and frame.

Next, you created a second page, called page2, and added a frame to that page as before. You then created a bunch of buttons and linked them to the various message boxes using lambda functions as commands. You modified the pack() options of both the buttons and the frame to make the buttons occupy the full width of the page, and you gave them different colors to make them stand out.

Notice that for the notebook, buttons, and ScrolledText widgets, we specified the expand option to the packer. expand tells the layout manager to expand the widget when the window is resized. By using expand in combination with fill and anchor, you can very precisely control how your widgets behave when resizing the window. (It is well worth experimenting with the options to get a feel for them.)

Finally, you started the mainloop() function. When you ran it, the result should have looked like Figure 4.10, which shows both pages of the notebook in action.

Using ttk

As mentioned earlier, the ttk module brings the concept of themes to Tkinter and Python. A theme is a graphical style and enables the same GUI structure to take on the look and feel of the native operating system. There are several themes that ship with ttk, but you can also create bespoke themes of your own.

The predefined themes vary by operating system, and you can find the names for your platform by looking at the output of ttk.Style().theme_names(). On Windows they include Classic (the default), winnative, vista, and xpnative.

ttk comes with its own versions of 11 of the standard Tkinter widgets that are theme aware as well as 6 new widgets of its own, including a ComboBox and NoteBook. You can change the look of applications just by changing the theme. Figure 4.11 shows a very simple Tkinter GUI presented first using classic, then using vista, and then using classic again, but this time on the Ubuntu Linux platform. Notice that the top button doesn’t change too much, but the new ttk button is different in each image. The code looks like this:

  >>> import tkinter as tk
  >>> import tkinter.ttk as ttk
  >>> top = tk.Tk()
  >>> s = ttk.Style()
  >>> s.theme_use('classic')
  >>> tk.Button(top,text="old button").pack()
  >>> ttk.Button(top,text="new button").pack()
  >>>

Obviously, you need to swap vista for classic in the style object to get the Vista theme. It should be obvious that you need to do a small amount of extra work to define the style object, but otherwise the use of ttk mostly looks like normal Tkinter programming. You should notice when you run the code that the differences are more than simple tweaks to the appearance of the button. The behavior is different too. For example, when you mouse over the button, it changes color differently to the old style button object.

That’s all you really need to know about ttk for now. There are lots of options that you can play around with; you can even define your own styles and themes. Mostly, you just use it as shown, and you get small but significant improvements to your Tkinter program’s look and feel.

Revisiting the Lending Library

You’ve now looked at several parts of the Tkinter toolset. It is time to wrap things up by bringing these pieces together in a larger example based on something less frivolous than a tic-tac-toe game. You revisit the lending library database that you created in Chapter 3 and build a GUI front end. This will reinforce much of what you have already done, but also introduces some new elements and techniques:

• The ScrolledListBox widget
• How to capture low-level events such as mouse double-click and window-level events
• How to create and use custom dialog boxes
• How to set fonts
• How to activate/deactivate widgets using the state attribute
• How to build a GUI using object-oriented techniques

You’ll build this in the following Try It Out.

You’ve now covered enough Tkinter to see how it can wrap a GUI around your application logic and data. It’s not the most powerful toolkit around, but it is quick and easy to get started, and it comes out of the box with Python. In the next section, you review some of the more powerful third-party GUI toolkits available.

Exploring Other GUI Toolkits for Python

There are many GUI toolkits around, ranging from the very specific native toolkits for Windows, MacOS X, and X windows, to more generic, multiplatform toolkits. Most of them have a wrapper layer of some kind available for Python. They all have the same core ideas and concepts that you saw in Tkinter, although some require an object-based approach while others, like Tkinter, permit a procedural style of programming, too. If Tkinter is not working for you, or if your main area of interest is GUI development, then these other toolkits may hold the answer. In the following sections, you find out about the strengths and weaknesses of each, and for the platform independent toolkits a very short “hello world” style sample program. If you want to run these, you need to install the toolkits because they are all provided by third parties.

That having been said, there are no absolute best or worst toolkits here. Each has its fans, and different programmers prefer different toolkits. It is worth taking some time to try out each toolkit of interest and at least work through their introductory tutorial to see whether it fits your personal style of coding. There are also some useful online videos that introduce their features, too.

  import wx

  app = wx.App(False)
  frame = wx.Frame(None, wx.ID_ANY, size=(320,240), "Hello World")
  frame.Show(True)

  app.MainLoop()

  import sys
  from PyQt4 import QtGui

  app = QtGui.QApplication(sys.argv)
  win = QtGui.QWidget()
  win.resize(320, 240)
  win.setWindowTitle("Hello World!")
  win.show()

  sys.exit(app.exec_())

  import sys
  from PySide import QtGui

  app = QtGui.QApplication(sys.argv)
  win = QtGui.QWidget()
  win.resize(320, 240)
  win.setWindowTitle("Hello World!")
  win.show()

  sys.exit(app.exec_())

  from gi.repository import Gtk

  win = Gtk.Window(title="Hello World")
  win.resize(320,240)
  win.connect("delete-event", Gtk.main_quit)
  win.show_all()

  Gtk.main()

Storing Application-Specific Data

Application specific data could be stored on the local computer, or it could be stored in a local network location that all instances of the application can reach. This raises the question of how the computer knows where to look. The usual solution to this problem is to set an environment variable or use a local configuration file stored in the startup folder of the application. This can then be set as part of the installation procedure. It can even be provided as a startup parameter.

The advantage of storing this kind of information on the network is that it is shared so that any changes made, for example if the database is moved, can be detected immediately by all of the installed instances on the network without having to manually reconfigure each machine or user configuration. It also allows for a backup configuration to be available in the event of a system failure and by changing one environment variable or configuration setting on the local computers the new central configuration can be accessed with minimal downtime.

The physical storage medium is relatively unimportant because the data are relatively static and normally only read once when the application starts up. A simple configuration file using plaintext, XML, or even Windows INI format will likely suffice. Python provides tools for creating and reading all of these; refer to Chapter 2 for details.

Storing User-Selected Preferences

User preferences are nearly always stored in the local computer, and often in a configuration file stored in the user’s home directory. Occasionally, applications store user preferences in the main database, especially if the database contains a significant amount of user data anyway. The disadvantage of using the database is that the application can access the preferences only if the database is accessible, which may not be the case if the user is mobile. It is disconcerting for a user to find that the application appears or functions differently depending on whether they are connected to their network or not. Local storage is definitely the preferred option for this kind of data.

Locating the data should be straightforward if a standard filename is used and the location is the home directory, because the home directory is nearly always obtainable either as an environment setting or as a user database value. (See Chapter 2 for guidance on how to determine user details such as the home directory.)

The format is normally a text file using either Windows INI or XML format. If the number of settings is very large, a small local database using SQLite might be appropriate, but this would be separate from the main application data store.

One other factor to consider when dealing with user preferences is how these are set and modified. If the settings are few and simple in nature (for example boolean or integer values), then it might be acceptable to generate a default preferences file and ask the user to manually edit the file. This does carry a risk if the user is not familiar with text editors and uses a rich format word processor to edit the file. This can render the configuration file unreadable by the application. However, if the users are likely to be experienced in editing text, such as developers or system administrators, then this approach can work well. If the configuration data is not simple or is stored in a structured file using XML or similar, then user editing is much less suitable and error prone, and a preferences dialog needs to be included in the application itself.

Storing Application State

Storing application state is the least standardized form of local data storage. The location and format of the storage is down to the developer. Some applications make the choice of whether to save-state a user preference; others do it automatically, while most do not save state (except perhaps for a list of recently accessed files). You need to decide how much state information you want to store, where to store it, and what format to use.

To keep application behavior consistent, you should choose a local storage option so that the application behaves as expected even when disconnected from the network. However, you need to be careful with error handling because, if the application was closed while online but then opened offline, many of the resources previously used may not be accessible. You need to have a working fallback configuration that can be used when things go wrong.

The format of the state data is likely to be quite complex because it may involve multiple windows and even tabs and control settings within windows. This almost inevitably requires a rich storage format such as XML. On the other hand, if you are only saving the open file history, a simple text file may suffice, or you could even append it to the user’s preference data in their configuration file.

Logging Error Information

You often want to keep a record of unexpected events or inputs so that you can analyze them later. This could be during testing, following a system failure, or even as a continuous improvement activity. The usual way to do this is to record messages in a log file. The log file might be a single file that just grows continuously or, more commonly, a file whose name is based on the date. Housekeeping (archiving or deleting of old files) of old files might be done manually, automatically, or via a shell script.

Python provides the logging package to assist in this process. It can generate standard information in a standard file with different levels of logging (that is it can flag a message with different category markers: debug, info, warning, error, and critical). The package is very flexible and allows for many different configuration options to control how it works. The basic usage is straightforward and you can extend its functions as you need them.

At the most basic level you import the module and then call one of several logging methods corresponding to the categories mentioned. For example:

  >>> import logging
  >>> logging.basicConfig(level=logging.DEBUG)
  >>> logging.info('Heres some info')
  INFO:root:Heres some info
  >>> logging.error('Oops, thats bad')
  ERROR:root:Oops, thats bad
  >>> logging.critical('AAARGH, Its all gone wrong!')
  CRITICAL:root:AAARGH, Its all gone wrong!

It’s important that you call basicConfig() before using any of the logging methods; otherwise, it has no effect, and you use the default values. The level setting indicates the minimum level of message that is displayed; DEBUG is the lowest level, so everything gets printed. In addition to setting the level, you can also specify an output filename.

You can also specify the format of the log message including things like the date of the message, the file and function where it was generated, and so on. (The options are all documented in the LogRecord section of the logging documentation.) Because the default format doesn’t include any date or time information, you usually want to set something like this:

  >>> import logging
  >>> logging.basicConfig(format="%(asctime)s %(levelname)s : %(message)s")
  >>> logging.error('Its going wrong')
  2014-04-24 16:12:44,832 ERROR : Its going wrong
  >>> logging.error('Its going wrong')
  2014-04-24 16:12:54,415 ERROR : Its going wrong
  >>>
  >>> logging.critical('Told you...')
  2014-04-24 16:13:08,431 CRITICAL : Told you...

You can also use a datefmt argument to basicConfig() to change the date format using the same options as you used in time.strftime(). Here is a short example:

  >>> import logging
  >>> logging.basicConfig(format="%(levelname)s:%(asctime)s %(message)s",
  		 datefmt="%Y/%m/%d-%I:%M")
  >>> logging.error("It's happened again")
  ERROR:2014/04/24-04:21 It's happened again
  >>>

There are many other things you can do, but the basic usage described here should be enough for all but the largest projects.

Using Locales

Python supports different locales via the locale module. The way the module works is quite complex and uses a layered approach; however, mostly you don’t need to know about that. You can use a very small subset and generally you pick up the correct locale for your user.

When your program starts, it is usually set to the “C” locale by default (although that may not always be the case, and local configuration settings may have changed it). However, you usually want to set the locale that your user has chosen. The way to do that is to call the locale
.setlocale() function with an empty locale argument. This causes the system locale to be selected. Most of the time, that’s all you need to do. You are strongly recommended to do this only once in your program and to do so near the start of your code.

You can then use locale.getlocale() to fetch the local details if you need to find out what has been set up (you see that in action in the next section where you look at how to translate your program’s strings into the local language).

Unfortunately, setting the locale is not the end of the story. If you want that change to take effect, there are some changes you need to make to your code. Specifically, there are some type conversion and comparison operations that are not locale-aware in the standard library and built-in functions. To get around that, the locale module provides alternatives. In other cases the standard functions do understand locales if you give them the right hints. For example, the time.strftime() function can format times to the local style if you use the appropriate formatting specifications such as %x for the localized date and %X for the localized time. Similarly, to print() numbers in the locale specific format, you need to specify the n style instead of d or f or g in the string format() method.

The following interpreter session demonstrates some of these features:

  >>> import locale as loc
  >>> import time
  >>> loc.setlocale(loc.LC_ALL,'')
  'English_United States.1252'
  >>> loc.currency(350)
  '$350.00'
  >>> time.strftime("%x %X", time.localtime())
  '4/22/2014 7:44:57 PM'
  >>>

Repeating those in a cygwin session set to the en_GB locale, you can see some differences:

  >>> import locale as loc
  >>> import time
  >>> loc.setlocale(loc.LC_ALL, '')
  'en_GB.UTF-8'
  >>> loc.currency(350)
  '£350.00'
  >>> time.strftime("%x %X", time.localtime())
  '22/04/2014 19:32:23'

The locale itself is clearly very differently specified. The currency uses the appropriate symbols, and the dates and times are quite different with the UK version using a 24-hour clock format and with the day and month of the date transposed.

You can now try some of the conversion and comparison functions. These work the same in both UK and U.S. English, so there is no point in doing a comparison this time.

  >>> print("{:n}".format(3.14159))
  3.14159
  >>> print("{:n}".format(42))
  42
  >>> loc.strcoll("Spanish", "Inquisition")
  1
  >>> loc.strcoll("Inquisition", "Spanish")
  -1
  >>> loc.strcoll("Spanish","Spanish")
  0

These examples show how the string formatting n specifier works for both integers and floating-point numbers. The locale.strcoll() string comparison examples are useful because they take account of locale specific ideas on character ordering. The return values are 1 if the first string is “higher” valued, -1 if it is “lower” valued, and 0 if the two arguments are the same.

Locale provides the following conversion functions that are useful in particular situations: atoi(), atof(), str(), format(), and format_string().

Using Unicode in Python

Computers store data as binary numbers. Characters are mapped onto these numbers so that when the computer prints a string of characters it maps the numeric data in memory to a set of character representations on screen. Back in the dawn of computing, characters were represented by as few as 5 bits and more commonly using 7 or 8 bits. All of these encodings could fit into a single 8-bit byte of storage, so it was very compact. Unfortunately, a single byte can only cater to 256 different combinations, which is fine for the Latin alphabets, used in the Western nations where modern computing originated, but nowhere near enough for all the alphabets in the world as a whole. Over time each country and corporation invented its own encoding system, and software engineers had to write lots of code to cater to these if their software was to be used in different localities. The solution was the Unicode standard.

Unicode is a 32-bit character catalog that can store a huge number of possible characters. Unicode characters are represented by entities called code points that are the numeric values that map onto characters. Code points are described using the format U+xxxx. The U+ indicates that it is a Unicode code point and the xxxx is the hexadecimal number representing the location of the code point in Unicode (there can be up to 8 hex digits not just 4). Code points are then mapped to characters that also have descriptive names. Thus the character “A” is listed as “U+0041 LATIN CAPITAL LETTER A”. The Python representation of the Unicode encoding is uxxxx for 4 digits or Uxxxxxxxx for 8 digits, which is a fairly obvious translation of the Unicode format, simply replacing U+ with u or U. The codes and names can all be found on the Unicode website at http://www.unicode.org/Public/UNIDATA/NamesList.txt. It is important to realize that Unicode defines the character only, not its appearance. What you see as a character on your computer screen, or printed out on paper, is known as a glyph, which is a graphical representation of that character in some font or other. Unicode does not specify the font family, weight, size, or any other details about appearance, only the actual character.

The Unicode data has to be stored on the computer as a set of bytes that, as you recall, can only store values from 0–255. The simplest translation, or encoding, of Unicode is known as UTF-32 and is a one-to-one mapping from the Unicode code point value to a 32-bit number. This is simple to understand, but requires 4 bytes for every character, making it very memory and bandwidth hungry. To conserve space two other encodings are used. UTF-16 uses 16-bit blocks to represent most characters, but with an option to extend that to two 16-bit blocks for some rarely used characters. Microsoft Windows uses UTF-16 by default.

UTF-8 stores the most commonly used characters in a single 8-bit block, but can be extended to use 2, 3, or 4 blocks for less commonly used characters. This makes UTF-8 the most compact format if you are using the right set of characters—specifically the Latin alphabet. UTF-8 also has the convenient feature of having the original ASCII character set as its lowest set of bytes, making interworking with older non-Unicode applications much easier. UTF-8 is the default encoding for Python version 3.

That is all pretty complicated, so how does Python help you handle all of this? Python version 3 uses Unicode strings with UTF-8 as its default encoding, although you can change that if you need to. The way to change the encoding is to put a special comment as the very first line of your code, like this,

  # -*- coding: <encoding name> -*-

where the encoding name is whatever encoding you choose to use, typically utf-16 or ascii.

You can also use the Unicode characters in literal strings. You can even use their long names, for example:

  >>> print('A')
  A
  >>> print('u0041')
  A
  >>> print("N{LATIN CAPITAL LETTER A}")
  A

They all print the same character, A.

You can use the encode() method of a string to get the raw bytes used to store the data:

  >>> print("u00A1Help!")
  ¡Help!
  >>> print("u00A1Help!".encode('utf-8'))
  b'xc2xa1Help!'
  >>> b'xc2xa1Help!'.decode('utf-8')
  '¡Help!'
  >>>

Notice that the second version printed the UTF-8 two-byte representation of the inverted exclamation point represented by u00A1. The third line converted the encoding back into a string using encode()’s complementary method decode().

So far you’ve seen how to represent single Unicode characters, how to change the default encoding used by Python, and how to convert a string into its byte representation using encode() and how to turn bytes into a Unicode string using decode(). Mostly that is all you need to know about Unicode and Python. The interpreter does most of the work for you. You can go on to define your own encodings, too, but that requires some more detailed knowledge and a study of the codecs module. You really shouldn’t need to do that in most situations.

To conclude this section, you should be aware of the unicodedata module that is particularly useful at the interactive prompt as a way of finding out about Unicode characters. Combined with the data published on the Unicode website you should be able to answer most questions that arise during regular coding.

You can tell the Unicode name and category of a given character using the unicodedata module and that can then be used to look up the website for more details. Assume you have just stored some data that you believe to contain a Unicode string and you want to know about the characters it contains. You might try this:

  >>> data = b'xd9x85xd8xb1xd8xadxd8xa8xd8xa7 xd8xa3xd9x84xd8xa7
 	xd9x86'
  >>> print(data.decode('utf-8'))

  >>> for ch in data.decode('utf-8'):
  ...	print(ord(ch), ud.name(ch))
  ...
  1605 ARABIC LETTER MEEM
  1585 ARABIC LETTER REH
  1581 ARABIC LETTER HAH
  1576 ARABIC LETTER BEH
  1575 ARABIC LETTER ALEF
  32 SPACE
  1571 ARABIC LETTER ALEF WITH HAMZA ABOVE
  1604 ARABIC LETTER LAM
  1575 ARABIC LETTER ALEF
  1606 ARABIC LETTER NOON
  >>>

Here you have a byte string that you suspect is UTF-8 characters and you want to find out what kind of data it is. You can try decoding it, to check that it is valid UTF-8, and that works, but you don’t recognize the printed character set. You then import the unicodedata module and run a for loop over the data printing out the long Unicode name of each character. From this it is obvious that the data is in Arabic.

Using gettext

To translate your program strings using the gettext mechanism, there is a standard set of steps you need to take. First, you have to use gettext functions to identify the strings in your code that you want translated. Second, you run a utility to extract those strings into a template file, typically called messages.po. After that you have to produce translation files based on messages.po, ideally by hiring a set of translators or perhaps by trusting Google translate or similar tools! Third, use another tool to convert the translation files to the language specific .mo format used by getttext, for example, messages_en.mo for the English version. Finally, you need to ship the folder with the .mo files in it along with your translation. There are various different tools available depending on the operating system. For Windows users there are a couple of scripts in the Tools/i18n folder of your Python distribution. For UNIX-like systems there are operating system utilities available that are Python aware.

You now walk through a simple example—the ubiquitous “Hello world” script. The first step is to use the gettext module and its functions to mark your program strings that require translation. You start by creating a new Python code file called gettext_demo.py (or load it from the gettext folder of the zip file):

  	import gettext
  	import locale as loc

  	# Set up the locale and translation mechanism
  	#############################
  	loc.setlocale(loc.LC_ALL,'')
  	filename = "res/messages_{}.mo".format(loc.getlocale()[0][0:2])

  	trans = gettext.GNUTranslations(open(filename,'rb'))
  	trans.install()

  	# Now the main program with gettext markers
  	#############################
  	print(_("Hello World"))

This sets the locale as discussed in the previous section and uses it to dynamically create the name of the translation file to be used. You are only creating an English translation so you could have hard-coded the name, but using getlocale() demonstrates how it would be done if you had more than one language available.

Next, you instantiate a gettext.GNUTranslations object and install() it. This activates the magic function _() that you then use to surround all the strings you need translated. In this case that’s the single string "Hello World".

Your next step is to generate the messages.po template file. If you are on a UNIX-like operating system, you can run the tool xgettext like this:

  $ xgettext gettext_demo.py

You should now find a messages.po file in the same folder as your Python file. Open this in your text editor and replace the string CHARSET with UTF-8 and insert a translation of "Hello world" in the empty string at the bottom. Just to prove that the translation is being picked up, you should use something like "Hello beautiful world" but normally, for English, you would just repeat the same string. (There are a bunch of other metadata fields that you could fill in, but they are not needed for this demonstration so you can just ignore them.) Now save the file as messages_en.po. It should look like this:

  # SOME DESCRIPTIVE TITLE.
  # Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
  # This file is distributed under the same license as the PACKAGE package.
  # FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
  #
  #, fuzzy
  msgid ""
  msgstr ""
  "Project-Id-Version: PACKAGE VERSION
"
  "Report-Msgid-Bugs-To: 
"
  "POT-Creation-Date: 2014-04-22 18:05+0100
"
  "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE
"
  "Last-Translator: FULL NAME <EMAIL@ADDRESS>
"
  "Language-Team: LANGUAGE <[email protected]>
"
  "Language: 
"
  "MIME-Version: 1.0
"
  "Content-Type: text/plain; charset=UTF-8
"
  "Content-Transfer-Encoding: 8bit
"

  #: gettext-demo.py:14
  msgid "Hello World"
  msgstr "Hello beautiful world"

Next, you want to create the res (for resource) folder that you specified in the filename in the Python script. The name is not critical, but res is a reasonable convention. You now run another utility called msgfmt like this:

  $ msgfmt -o res/messages_en.mo messages_en.po

Note the different file endings! This creates the translation file that you told gettext to read when running your script. You are now ready to run the demo:

  $ python3 gettext_demo.py

You should find that the translated string "Hello beautiful world" is printed instead of the original "Hello World" that you had in your code. This proves that the translation worked. You can go on to make more translation files by repeating the steps after running xgettext for each new language. To test them you need to change the local language settings.

Localization is a complex and growing area of study. However, if you plan on distributing your applications to multiple language users or in different countries, it is something you cannot afford to ignore.