Interactive debugging

Now that we have discussed basic debugging methods that will always work, we will look at interactive debugging for some more advanced debugging techniques. The previous debugging methods made variables and stacks visible through modifying the code and/or foresight. This time around, we will look at a slightly smarter method, which constitutes doing the same thing interactively, but once the need arises.

Console on demand

When testing some Python code, you may have used the interactive console a couple of times, since it's a simple yet effective tool for testing your Python code. What you might not have known is that it is actually simple to start your own shell from within your code. So, whenever you want to drop into a regular shell from a specific point in your code, that's easily possible:

import code


def spam():
    eggs = 123
    print('The begin of spam')
    code.interact(banner='', local=locals())
    print('The end of spam')
    print('The value of eggs: %s' % eggs)


if __name__ == '__main__':
    spam()

When executing that, we will drop into an interactive console halfway:

# python3 test_code.py
The begin of spam
>>> eggs
123
>>> eggs = 456
>>>
The end of spam
The value of eggs: 123

To exit this console, we can use ^d (Ctrl + d) on Linux/Mac systems and ^z (Ctrl + Z) on Windows systems.

One important thing to note here is that the scope is not shared between the two. Even though we passed along locals() to share the local variables for convenience, this relation is not bidirectional. The result is that even though we set eggs to 456 in the interactive session, it does not carry over to the outside function. You can modify variables in the outside scope through direct manipulation (for example, setting the properties) if you wish, but all variables declared locally will remain local.

Debugging using pdb

When it comes to actually debugging code, the regular interactive console just isn't suited. With a bit of effort, you can make it work, but it's just not all that convenient for debugging since you can only see the current scope and can't jump around the stack easily. With pdb (Python debugger), this is easily possible. So let's look at a simple example of using pdb:

import pdb


def spam():
    eggs = 123
    print('The begin of spam')
    pdb.set_trace()
    print('The end of spam')
    print('The value of eggs: %s' % eggs)


if __name__ == '__main__':
    spam()

This example is pretty much identical to the one in the previous paragraph, except that this time we end up in the pdb console instead of a regular interactive console. So let's give the interactive debugger a try:

# python3 test_pdb.py
The begin of spam
> test_pdb.py(8)spam()
-> print('The end of spam')
(Pdb) eggs
123
(Pdb) eggs = 456
(Pdb) continue
The end of spam
The value of eggs: 456

As you can see, we've actually modified the value of eggs now. In this case, we used the full continue command, but all the pdb commands have short versions as well. So, using c instead of continue gives the same result. Just typing eggs (or any other variable) will show the contents and setting the variable will simply set it, just as we would expect from an interactive session.

To get started with pdb, first of all, a list of the most useful (full) commands with shorthands is shown here:

Command

Explanation

h(elp)

This shows the list of commands (this list).

h(elp) command

This shows the help for the given command.

w(here)

Current stack trace with an arrow at the current frame.

d(own)

Move down/to a newer frame in the stack.

u(p)

Move up/to an older frame in the stack.

s(tep)

Execute the current line and stop as soon as possible.

n(ext)

Execute the current line and stop at the next line within the current function.

r(eturn)

Continue execution until the function returns.

c(ont(inue))

Continue execution up to the next breakpoint.

l(ist) [first[, last]]

List the lines of source code (by default, 11 lines) around the current line.

ll | longlist

List all of the source code for the current function or frame.

source expression

List the source code for the given object. This is similar to longlist.

a(rgs)

Print the arguments for the current function.

pp expression

Pretty-print the given expression.

b(reak)

Show the list of breakpoints.

b(reak) [filename:]lineno

Place a breakpoint at the given line number and, optionally, file.

b(reak) function[, condition]

Place a breakpoint at the given function. The condition is an expression that must evaluate to True for the breakpoint to work.

cl(ear) [filename:]lineno

Clear the breakpoint (or breakpoints) at this line.

cl(ear) breakpoint [breakpoint ...]

Clear the breakpoint (or breakpoints) with these numbers.

Command

List all defined commands.

command breakpoint

Specify a list of commands to execute whenever the given breakpoint is encountered. The list is ended using the end command.

Alias

List all aliases.

alias name command

Create an alias. The command can be any valid Python expression, so you can do the following to print all properties for an object:

alias pd pp %1.__dict__

unalias name

Remove an alias.

! statement

Execute the statement at the current point in the stack. Normally the ! sign is not needed, but this can be useful if there are collisions with debugger commands. For example, try b = 123.

Interact

Open an interactive session similar to the previous paragraph. Note that variables set within that local scope will not be transferred.

Breakpoints

It's quite a long list, but you will probably use most of these quite regularly. To highlight one of the options shown in the preceding table, let's demonstrate the setting and use of breakpoints:

import pdb


def spam():
    print('The begin of spam')
    print('The end of spam')


if __name__ == '__main__':
    pdb.set_trace()
    spam()

So far, nothing new has happened, but let's now open the interactive debugging session, as follows:

# python3 test_pdb.py
> test_pdb.py(11)<module>()
-> while True:
(Pdb) source spam  # View the source of spam
  4     def spam():
  5         print('The begin of spam')
  6         print('The end of spam')

(Pdb) b 5  # Add a breakpoint to line 5
Breakpoint 1 at test_pdb.py:5

(Pdb) w  # Where shows the current line
> test_pdb.py(11)<module>()
-> while True:

(Pdb) c  # Continue (until the next breakpoint or exception)
> test_pdb.py(5)spam()
-> print('The begin of spam')

(Pdb) w  # Where again
  test_pdb.py(12)<module>()
-> spam()
> test_pdb.py(5)spam()
-> print('The begin of spam')

(Pdb) ll  # List the lines of the current function
  4     def spam():
  5 B->     print('The begin of spam')
  6         print('The end of spam')

(Pdb) b  # Show the breakpoints
Num Type         Disp Enb   Where
1   breakpoint   keep yes   at test_pdb.py:5
        breakpoint already hit 1 time

(Pdb) cl 1  # Clear breakpoint 1
Deleted breakpoint 1 at test_pdb.py:5

That was a lot of output, but it's actually not as complex as it seems:

  1. First, we used the source spam command to see the source for the spam function.
  2. After that, we knew the line number of the first print statement, which we used to place a breakpoint (b 5) at line 5.
  3. To check whether we were still at the right position, we used the w command.
  4. Since the breakpoint was set, we used c to continue up to the next breakpoint.
  5. Having stopped at the breakpoint at line 5, we used w again to confirm that.
  6. Listing the code of the current function using ll.
  7. Listing the breakpoints using b.
  8. Removing the breakpoint again using cl 1 with the breakpoint number from the previous command.

It all seems a bit complicated in the beginning, but you'll see that it's actually a very convenient way of debugging once you've tried a few times.

To make it even better, this time we will execute the breakpoint only when eggs = 3. The code is pretty much the same, although we need a variable in this case:

import pdb


def spam(eggs):
    print('eggs:', eggs)


if __name__ == '__main__':
    pdb.set_trace()
    for i in range(5):
        spam(i)

Now, let's execute the code and make sure that it only breaks at certain times:

# python3 test_breakpoint.py
> test_breakpoint.py(10)<module>()
-> for i in range(5):
(Pdb) source spam
  4     def spam(eggs):
  5         print('eggs:', eggs)
(Pdb) b 5, eggs == 3  # Add a breakpoint to line 5 whenever eggs=3
Breakpoint 1 at test_breakpoint.py:5
(Pdb) c  # Continue
eggs: 0
eggs: 1
eggs: 2
> test_breakpoint.py(5)spam()
-> print('eggs:', eggs)
(Pdb) a  # Show function arguments
eggs = 3
(Pdb) c  # Continue
eggs: 3
eggs: 4

To list what we have done:

  1. First, using source spam, we looked for the line number.
  2. After that, we placed a breakpoint with the eggs == 3 condition.
  3. Then we continued execution using c. As you can see, the values 0, 1, and 2 are printed as normal.
  4. The breakpoint was reached at value 3. To verify this we used a to see the function arguments.
  5. And we continued to execute the rest of the code.

Catching exceptions

All of these have been manual calls to the pdb.set_trace() function, but in general, you are just running your application and not really expecting issues. This is where exception catching can be very handy. In addition to importing pdb yourself, you can run scripts through pdb as a module as well. Let's examine this bit of code, which dies as soon as it reaches zero division:

print('This still works')
1/0
print('We shouldnt reach this code')

If we run it using the pdb parameter, we can end up in the Python Debugger whenever it crashes:

# python3 -m pdb test_zero.py
> test_zero.py(1)<module>()
-> print('This still works')
(Pdb) w  # Where
  bdb.py(431)run()
-> exec(cmd, globals, locals)
  <string>(1)<module>()
> test_zero.py(1)<module>()
-> print('This still works')
(Pdb) s  # Step into the next statement
This still works
> test_zero.py(2)<module>()
-> 1/0
(Pdb) c  # Continue
Traceback (most recent call last):
  File "pdb.py", line 1661, in main
    pdb._runscript(mainpyfile)
  File "pdb.py", line 1542, in _runscript
    self.run(statement)
  File "bdb.py", line 431, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "test_zero.py", line 2, in <module>
    1/0
ZeroDivisionError: division by zero
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> test_zero.py(2)<module>()
-> 1/0

Tip

A useful little trick within pdb is to use the Enter button, which, by default, will execute the previously executed command again. This is very useful when stepping through the program.

Commands

The commands command is a little complicated but very useful. It allows you to execute commands whenever a specific breakpoint is encountered. To illustrate this, let's start from a simple example again:

import pdb


def spam(eggs):
    print('eggs:', eggs)


if __name__ == '__main__':
    pdb.set_trace()
    for i in range(5):
        spam(i)

The code is simple enough, so now we'll add the breakpoint and the commands, as follows:

# python3 test_breakpoint.py
> test_breakpoint.py(10)<module>()
-> for i in range(3):
(Pdb) b spam  # Add a breakpoint to function spam
Breakpoint 1 at test_breakpoint.py:4
(Pdb) commands 1  # Add a command to breakpoint 1
(com) print('The value of eggs: %s' % eggs)
(com) end  # End the entering of the commands
(Pdb) c  # Continue
The value of eggs: 0
> test_breakpoint.py(5)spam()
-> print('eggs:', eggs)
(Pdb) c  # Continue
eggs: 0
The value of eggs: 1
> test_breakpoint.py(5)spam()
-> print('eggs:', eggs)
(Pdb) cl 1  # Clear breakpoint 1
Deleted breakpoint 1 at test_breakpoint.py:4
(Pdb) c  # Continue
eggs: 1
eggs: 2

As you can see, we can easily add commands to the breakpoint. After removing the breakpoint, these commands obviously won't be executed anymore.

Debugging using ipdb

While the generic Python console is useful, it can be a little rough around the edges. The IPython console offers a whole new world of extra features, which make it a much nicer console to work with. One of those features is a more convenient debugger.

First, make sure you have ipdb installed:

pip install ipdb

Next, let's try the debugger again with our previous script. The only small change is that we now import ipdb instead of pdb:

import ipdb


def spam(eggs):
    print('eggs:', eggs)


if __name__ == '__main__':
    ipdb.set_trace()
    for i in range(3):
        spam(i)

Then we execute it:

# python3 test_ipdb.py
> test_ipdb.py(10)<module>()
      9     ipdb.set_trace()
---> 10     for i in range(3):
     11         spam(i)

ipdb> b spam  # Set a breakpoint
Breakpoint 1 at test_ipdb.py:4
ipdb> c  # Continue (until exception or breakpoint)
> test_ipdb.py(5)spam()
1     4 def spam(eggs):
----> 5     print('eggs:', eggs)
      6

ipdb> a  # Show the arguments
eggs = 0
ipdb> c  # Continue
eggs: 0
> test_ipdb.py(5)spam()
1     4 def spam(eggs):
----> 5     print('eggs:', eggs)
      6

ipdb>   # Repeat the previous command, so continue again
eggs: 1
> test_ipdb.py(5)spam()
1     4 def spam(eggs):
----> 5     print('eggs:', eggs)
      6

ipdb> cl 1  # Remove breakpoint 1
Deleted breakpoint 1 at test_ipdb.py:4
ipdb> c  # Continue
eggs: 2

The commands are all the same, but the output is just a tad more legible in my opinion. The actual version also includes syntax highlighting, which makes the output even easier to follow.

In short, you can just replace pdb with ipdb in most situations to simply get a more intuitive debugger. But I will give you the recommendation as well, to the ipdb context manager:

import ipdb


with ipdb.launch_ipdb_on_exception():
    main()

This is as convenient as it looks. It simply hooks ipdb into your exceptions so that you can easily debug whenever needed. Combine that with a debug flag to your application to easily allow debugging when needed.

Other debuggers

pdb and ipdb are just two of the large number of debuggers available for Python. Some of the currently noteworthy debuggers are as follows:

  • pudb: This offers a a full-screen command-line debugger
  • pdbpp: This hooks into the regular pdb
  • rpdb2: This is a remote debugger that allows hooking into running (remote) applications
  • Werkzeug: This is a web-based debugger that allows debugging of web applications while they are running

There are many others, of course, and there isn't a single one that's the absolute best. As is the case with all tools, they all have their advantages and their fallacies, and the one that is best for your current purpose can be properly decided only by you. Chances are that your current Python IDE already has an integrated debugger.

Debugging services

In addition to debugging when you encounter a problem, there are times when you simply need to keep track of errors for later debugging. Especially when working with remote servers, these can be invaluable to detect when and how a Python process is malfunctioning. Additionally, these services offer grouping of errors as well, making them far more useful than a simple e-mail-on-exception type of script, which can quickly spam your inbox.

A nice open source solution for keeping track of errors is sentry. If you need a full-fletched solution that offers performance tracking as well, then Opbeat and Newrelic are very nice solutions; they offer both free and paid versions. Note that all of these also support tracking of other languages, such as JavaScript.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset