When your program really goes kablooey, the best thing to do is not to let Perl try to run it, even through your test programs. If things are going that badly wrong, take a tip from the Haskell world and convince yourself that if it at least compiles, it has to be fairly okay.^[1] Just make it compile already!

Go to the command line and tell Perl to compile the program with warnings and then stop. If your program is what_went_wrong.pl, use:

$ perl -wc what_went_wrong.pl
            

If there’s no error, Perl will report a happy okay message. Great! Go on to making your tests pass. Otherwise, grab the first error message and figure out how to solve it.

This technique also works decently for figuring out where an error occurs, especially if you don’t have logging or tracing statements in the code. Instead of commenting out code selectively, dump appropriate data with YAML or another serialization module at appropriate places to narrow down the error.

This approach often works, but it can fail when you do odd things, such as using a source filter. Beware.

A very common mistake is to forget to chomp( ) data read from external sources. Suppose that you’re processing a list of files read from another file:

while (<$file_list>)
{
    warn "Processing $_";
    next unless -e $_;
    process_file( $_ );
}

All of the files look correct in the warn( ) output, but the process_file( ) code never occurs.

Change the debugging line to make the filename more visible:

while (<$file_list>)
{
    warn "Processing '$_'";
    next unless -e $_;
    process_file( $_ );
}

Adding single quotes (or any other visible character) around the filename will likely reveal that all of the filenames within the loop have newlines at the end (or whatever the current input record separator, $/, contains). The solution is obvious:

while (<$file_list>)
{
    chomp;
    next unless -e $_;
    process_file( $_ );
}

The previous technique only works well on scalar variables. Lists and arrays are more tricky. Fortunately, the special punctuation variable $" controls the separator used when interpolating a list into a string.

Suppose that you’re writing a ranking system for a table-tennis league. You’ve come to the end of the program and you want to display the top ten players:

my @ranked;

for my $rank ( 1 .. 10 )
{
    $ranked[$rank] = get_player_by_rank( $rank );
}

Of course, players may tie—leaving two players at the third rank and no players at the fourth rank. The naïve approach of assuming the array contains exactly ten entries may fail, especially if get_player_by_rank( ) always only returns a single player or potentially returns an array reference of multiple players.

Printing the array may be no help:

for my $rank ( 1 .. 10 )
{
    $ranked[$rank] = get_player_by_rank( $rank );
}

warn "Ranks: [@ranked]\n";

Everything interpolates into a single string, leaving you to count to see which is missing.

Instead, set $" to a nice, visible string:

local $" = '] [';
warn "Ranks: [@ranked]\n";

This puts the delimiters between the entries, making it much easier to see which slot is empty.

If this is too much for you to handle manually, bring in the big guns of a serialization module to do your debugging for you. Data::Dumper has enjoyed a long reign as a nice debugging aid, but YAML provides even more readability and conciseness. To see a nested data structure (or even a scalar, array, or hash without having to worry about the right delimiters), use the Dump( ) function:

my $user = User->load( id => 54272 );
warn Dump( $user );

This prints a nice, compact representation of the data in the $user object without the excessive indentation and indirection that Data::Dumper can often provide.

Another good, if complex option, is using Devel::Peek to see exactly what Perl thinks of your variable. When you need it, you really need it. The rest of the time, take a breath, and then spend two minutes writing the test case.

Suppose you’ve just learned that Perl’s default sorting algorithm changed from unstable to stable for Perl 5.8.0. The Internet reveals that, with a stable sorting algorithm, elements that have the same position in the sorting order will retain the positions relative to each other that they had in the input.

my @elements =
(
    [ 2, 2 ], [ 2, 1 ], [ 2, 0 ],
    [ 1, 0 ], [ 1, 1 ], [ 1, 2 ],
);

my @sorted   = sort { $a->[0] <=> $b->[0] } @elements;

local $"     = ', ';
print "[ @$_ ]\n" for @sorted;

[ 1, 0 ]
[ 1, 1 ]
[ 1, 2 ]
[ 2, 2 ]
[ 2, 1 ]
[ 2, 0 ]

use Test::More tests => 4;

my @elements =
(
    [ 2, 2 ], [ 2, 1 ], [ 2, 0 ],
    [ 1, 0 ], [ 1, 1 ], [ 1, 2 ],
);
my @sorted   = sort { $a->[0] <=> $b->[0] } @elements;

is( $sorted[0][0], 1, 'numeric sort should put 1 before 2'     );
is( $sorted[0][1], 0, '... keeping stability of original list' );
is( $sorted[2][1], 2, '... through all elements'               );
is( $sorted[3][1], 2, '... not accidentally sorting them'      );

Ideally, someone’s already tested this sort of code—the Perl 5 porters. If you have access to the source code of Perl (in this case) or the library you’re testing, you can skim the test suite for examples to borrow and modify or learn from outright. In this case, code in t/op/sort.t tests Perl’s stable sort. Even just skimming the test descriptions can reveal a lot of information about the ideas behind the implementation.

“Something left in the code, but ignored” is pretty much the definition of a comment, so it’s no surprise that you can use comments to turn off debugging statements. There’s a much more interesting alternative, however: using comments to turn on debugging statements. The Smart::Comments CPAN module does just that: it turns comments into debugging statements.

my $call       = "26, 17, 22, hut!";
my @play_calls = split /\s*,?\s*/, $call;

# make '###' magical...
use Smart::Comments;

my $call       = "26, 17, 22, hut!";

### $call

my @play_calls = split /\s*,?\s*/, $call;

### @play_calls

$ perl play_book.pl

### $call: '26, 17, 22, hut!'

### @play_calls: [
###                '2',
###                '6',
###                '1',
###                '7',
###                '2',
###                '2',
###                'h',
###                'u',
###                't',
###                '!'
###              ]

$

use Smart::Comments;

my $call       = "26, 17, 22, hut!";

### input: $call

my @play_calls = split /\s*,?\s*/, $call;

### split to: @play_calls

$ perl play_book.pl

### input: '26, 17, 22, hut!'

### split to: [
###             '2',
###             '6',
###             '1',
###             '7',
###             '2',
###             '2',
###             'h',
###             'u',
###             't',
###             '!'
###           ]

$

use Smart::Comments;

my $call       = "26, 17, 22, hut!";

my @play_calls = split /\s*,?\s*/, $call;

#### require: @play_calls = = 4

$ perl play_book_with_assertion.pl

### @play_calls = = 4 was not true at play_book_with_assertion.pl line 7.
###     @play_calls was: [
###                        '2',
###                        '6',
###                        '1',
###                        '7',
###                        '2',
###                        '2',
###                        'h',
###                        'u',
###                        't',
###                        '!'
###                      ]

$

# use Smart::Comments;

my $call       = "26, 17, 22, hut!";

### input: $call

my @play_calls = split /\s*,?\s*/, $call;

### split to: @play_calls

$ perl play_book.pl

$

use Smart::Comments '####';   # Only ####... comments are "smart"
                              # Any ###... comments are ignored

my $call       = "26, 17, 22, hut!";

### $call

my @play_calls = split /\s*,?\s*/, $call;

### @play_calls

#### require: @play_calls = = 4

$ perl -MSmart::Comments split_test.pl
               

$ perl -MSmart::Comments="" split_test.pl
               

The code to do this is pretty easy, if unsubtle:

package SourceCarp;

use strict;
use warnings;

sub import
{
    my ($class, %args) = @_;

    $SIG{__DIE__}  = sub { report( shift, 2 ); exit } if $args{fatal};
    $SIG{__WARN__} = \&report                         if $args{warnings};
}

sub report
{
    my ($message, $level)  = @_;
    $level               ||= 1;
    my ($filename, $line)  = ( caller( $level - 1 ) )[1, 2];
    warn $message, show_source( $filename, $line );
}

sub show_source
{
    my ($filename, $line) = @_;
    return '' unless open( my $fh, $filename );

    my $start = $line - 2;
    my $end   = $line + 2;

    local $.;
    my @text;
    while (<$fh>)
    {
        next unless $. >= $start;
        last if     $. >  $end;
        my $highlight   = $. = = $line ? '*' : ' ';
        push @text, sprintf( "%s%04d: %s", $highlight, $., $_ );
    }

    return join( '', @text, "\n" );
}

1;

The magic here is in three places. report( ) looks at the call stack leading to its current position, extracting the name of the file and the line number of the calling code. It’s possible to call this function directly with a message to display (and an optional level of calls to ignore).

show_source( ) simply reads the named file and returns a string containing two lines before and after the numbered line, if possible. It also highlights the specific line with an asterisk in the left column. Note the localization and use of the $. magic variable to count the current line in the file.

import( ) adds global handlers for warnings and exceptions, if requested from the calling module. The difference between the handlers is that when Perl issues a lexical warning, it doesn’t affect the call stack in the same way that it does when it throws an exception.

This short program shows all three ways of invoking SourceCarp:

#!/usr/bin/perl

use strict;
use warnings;

use lib 'lib';
use SourceCarp fatal => 1, warnings => 1;

# throw warning
open my $fh, '<', '/no/file';
print {$fh}...

# report from subroutine
report_with_level( );

sub report_with_level
{
    SourceCarp::report( "report caller, not self\n", 2 );
}

# throw error
die "Oops!";

There’s no reason to limit your error and warning reporting to showing the file context around the calling line. caller( ) offers much more information, including the variables passed to each function in certain circumstances.^[2] It’s possible to provide and present this information in a much more useful manner.

Overriding the global __WARN__ and __DIE__ handlers is serious business as it can interfere with large programs. A more robust implementation of this hack might work nicely with Carp, not only because it is more widely compatible, but also because that module offers more features. Another possibility is to integrate this code somehow with Log::Log4perl.

Suppose that you’ve written a simple filter subroutine which returns all of the lines from a file handle that match your filter criteria.

sub filter
{
    my ($filter) = @_;

    if ('Regexp' eq ref $filter)
    {
        return sub
        {
            my $fh = shift;
            return grep { /$filter/ } <$fh>;
        };
    }
    else
    {
        return sub
        {
            my $fh = shift;
            return grep { 0 <= index $_, $filter } <$fh>;
        };
    }
}

Using the subroutine is simple. Pass it a precompiled regex and it will return lines which match the regular expression. Pass it a string and it will return lines which contain that string as a substring.

Unfortunately, later on you wonder why the following code returns every line from the file handle instead of just the lines which contain a digit:

my $filter = filter(/\d/);
my @lines  = $filter->($file_handle);

Data::Dumper is of no use here:

use Data::Dumper;
print Dumper( $filter );

This results in:

$VAR1 = sub { "DUMMY" };

Using the Data::Dump::Streamer serialization module allows you to see inside that subroutine:

use Data::Dump::Streamer;
Dump( $filter );

Now you can see the body of the subroutine more or less as Perl sees it.

my ($filter);
$filter = undef;
$CODE1 = sub {
           my $fh = shift @_;
           return grep({0 <= index($_, $filter);} <$fh>);
         };

From there, it’s pretty apparent that Perl didn’t recognize that you were trying to pass in a regular expression and the bug is trivial to fix:

my $filter = filter(qr/\d/);
my @lines  = $filter->($file_handle);

Behind the scenes, Data::Dump::Streamer uses the core module B::Deparse. In essence it does the following:

use B::Deparse;
my $deparse = B::Deparse->new( );
print $deparse->coderef2text($filter);

which outputs:

{
    my $fh = shift @_;
    return grep({0 <= index($_, $filter);} <$fh>);
}

The primary difference is that Data::Dump::Streamer also shows the values of any variables that the subroutine has closed over. See “Peek Inside Closures” [Hack #76] for more details. This technique is also good for displaying diagnostics when you eval code into existence or receive a subroutine reference as an argument and something goes wrong when you try to execute it.

The B::Deparse documentation gives more information about the arguments that you can pass to its constructor for even better control over the output.

Creating an anonymous subroutine creates a glob named *__ANON__ in the current package. When caller( ) and the rest of Perl’s guts look for names for anonymous subroutines, they look there. Using carp and croak will quickly reveal this.

The solution is therefore to override this name temporarily. The easy way is to have the parent subroutine name the anonymous one:

sub divide_by
{
    my $numerator = shift;
    my $name      = (caller(0))[3];
    return sub
    {
        local *__ANON__ = "__ANON__$name";
        my $denominator = shift;
        croak "Denominator must not be zero!" unless $denominator;
        return $numerator / $denominator;
    };
}

Running the program now produces the output:

Denominator must not be zero! at anon_subs.pl line 12
      __ANON__main::divide_by(0) called at anon_subs.pl line 18

While that’s better and it may fit your needs, it’s not the most flexible solution. If you create several anonymous subroutines, they will all have the same name. It’s more powerful to name the anonymous subroutines by passing the creator subroutine a name—or taking it from an argument, as appropriate.

use Carp;

sub divide_by
{
    my ($name, $numerator) = @_;
    return sub
    {
        local *__ANON__ = "__ANON__$name";
        my $denominator = shift;
        croak "Denominator must not be zero!" unless $denominator;
        return $numerator / $denominator;
    };
}

my $three_divided_by = divide_by( 'divide_by_three', 3 );
my $answer           = $three_divided_by->(0);

The output looks like you expect:

Denominator must not be zero! at anon_subs.pl line 12
      __ANON__main::divide_by_three(0) called at anon_subs.pl line 18
               
            

my $old_p;
BEGIN { $old_p = $^P; $^P &= ~0x200; }

sub divide_by
{
     # ...
}

BEGIN { $^P = $old_p; }

You could muck around in your symbol table [Hack #72] and use introspection to find the CV and check its STASH information [Hack #78], but Rafael Garcia-Suarez’s Sub::Identify does this for you (using the invaluable B backend module internally).

Just use the stash_name( ) function:

package My::Package;

use Sub::Identify ':all';
use HTML::Entities 'encode_entities';
print stash_name( \&encode_entities );

Run this code; it will print HTML::Entities. Even if another module has re-exported &encode_entities into your namespace, Sub::Identify will still report HTML::Entities as the source of the subroutine.

For descriptions of the class hierarchy of these objects and the methods that you can call on them, see OVERVIEW OF CLASSES and SV-RELATED CLASSES in perldoc B. Unfortunately, much of the documentation is rather sparse and reading the source code of this module and the header files of the various Perl data structures, as well as pestering P5P with questions, is often the best way to figure out what you’re doing. See also Chapter 8.

Writing a command is simple. There are only a few things to remember:

A “hello, world” command looks like:

package Devel::Command::HelloWorld;
use base 'Devel::Command';

sub command
{
    print DB::OUT "Hello world!\n";
    1;
}

1;

Devel::Command defaults to using the command( ) as the actual command code. Run this by putting it somewhere in your @INC and then start the debugger:

flatbox ~ $ perl -de0
Default die handler restored.
Patching with Devel::Command::DBSub::DB_5_8

Loading DB routines from perl5db.pl version 1.07
Editor support available.

Enter h or Qh h' for help, or Qman perldebug' for more help.

main::(-e:1):    0
  DB<1> cmds
cmds
helloworld
  DB<2> helloworld
Hello world!
  DB<3> q
flatbox ~ $

The message that begins Patching with... lets you know that Devel::Command has successfully activated. cmds lists the commands and typing helloworld runs your command.

package Devel::Command::X;

use base 'Devel::Command';

sub command
{
    my ($cmd) = @_;

    if ($cmd  =~ /x marks/)
    {
        print DB::OUT "Arrrrr....\n";
        return 1;
    }
    else
    {
        return 0;
    }
}

1;

flatbox ~ $ perl -de0
Default die handler restored.
Patching with Devel::Command::DBSub::DB_5_8

Loading DB routines from perl5db.pl version 1.07
Editor support available.

Enter h or Qh h' for help, or Qman perldebug' for more help.

main::(-e:1):    0
  DB<1> $x = [1,2,3]

  DB<2> x $x
0  ARRAY(0x804e2f4)
   0  1
   1  2
   2  3
  DB<3> x marks the spot
Arrrrr....
  DB<4> q
flatbox ~ $

Create a module in the Devel::Command:: namespace. Install Devel::Command from CPAN, and then tell the debugger to load it by adding one line to your debugger initialization file, .perldb (or perldb.ini, for non-Unix systems):

use Devel::Command;

That’s it. This makes the debugger automatically search @INC for modules in the Devel::Command:: namespace, load them, and install them as commands. By default, it picks a name for the command by downcasing the last namespace qualifier (so, for example, Devel::Command::My::DoStuff ends up as the dostuff command).

Devel::Command also installs its own cmds command, which lists all commands that it found and loaded, and dynamically patches the debugger’s command processing subroutine with a modified version which knows how to find the commands installed by Devel::Command.

To develop tests while using the debugger, try the Devel::Command::Tdump module on CPAN. This module loads Test::More for you and lets you actually write tests and save them from the debugger.

If you want to see drawings of your data structure in the debugger, Devel::Command::Viz and the graphviz package will let you do it. Install those, then use the viz command on a variable:

flatbox ~ $ perl5.8.5 -de0
Patching with Devel::Command::DBSub::DB_5_8_5

Loading DB routines from perl5db.pl version 1.27
Editor support available.

Enter h or Qh h' for help, or Qman perldebug' for more help.

main::(-e:1):    0
  DB<1> use WWW::Mechanize

  DB<2> $m = WWW::Mechanize->new( )

  DB<3> viz $m
            

You’ll see a graphical depiction of the WWW::Mechanize object in a pop-up window.