Sample API Application

This chapter is all about how to debug APIs and to do that, we’ll use an absolutely trivially simple API as our example. The idea is to show a very simple use case with a minimal amount of code to illustrate the techniques that can be scaled up and applied to your real-world applications.

The example API uses the Slim microframework as a lightweight way of quickly starting up a new application. This API is very simple and only has a handful of endpoints:

• /, the root, just returns a list of endpoints.

• /list can be accessed via GET in which case it returns a list of items, or via POST in which case it adds the supplied item to the list.

The code for this is in Example 11-1 and you’ll find the formatter just a little further along in Example 11-2. The only initial dependency is the Slim framework, so composer.json looks like this:

{
    "require": {
        "slim/slim": "^3.0"
    }
}

<?php

require "../vendor/autoload.php";
require "Formatter.php";

$app = new SlimApp();

$container = $app->getContainer();
$container['formatter'] = function ($c) {
    return new Formatter($c->get('request'));
};

$app->get(
    '/',
    function ($request, $response) {
        $data = ["home" => "/", "list" => "/list"];
        $response = $this->formatter->render($response, $data);
        return $response;
    }
);

$app->get(
    "/list",
    function ($request, $response) {
        // fetch items
        $items = [];
        $fp = fopen('../items.csv', 'r');
        while(false !== ($data = fgetcsv($fp))) {
            $items[] = current($data);
        }

        $data = ["items" => $items, "count" => count($items)];
        $response = $this->formatter->render($response, $data);
        return $response;
    }
);

$app->post(
    "/list",
    function($request, $response) {
        $data = $request->getParsedBody();

        if(isset($data) && isset($data['item']) && !empty($data['item'])) {
            $this->logger->addInfo("Adding data item: " . $data['item']);
            // save item
            $fp = fopen('../items.csv', 'a');
            fputcsv($fp, [$data['item']]);

            $response = $response
                ->withStatus(201)
                ->withHeader("Location", "/list");

            $response = $this->formatter->render($response);
            return $response;
        }

        // if we got this far, something went really wrong
        throw new UnexpectedValueException("Item could not be parsed");
    }
);

$app->run();

This very simple example forms the basis upon which we’ll add some particular features that will make an API maintainable and easy to work with in development and for a long and happy maintenance window.

Consistent Output Formats

This is the golden rule: always respond in the format that the client was expecting. This means that it is never acceptable to return an HTML error message when the client expected JSON—but beware that many of the frameworks will do exactly this by default when an error occurs! If your system does return HTML messages when things go wrong, that is a bug and needs fixing. If an unexpected format is sent, the client will not be unable to understand the response and any error information contained in it.

One of the side effects of using the Slim framework is that it offers the getParsedBody() method on the Request object, which essentially means that our API already supports multiple input formats. This method reads the Content-Type header of the incoming request, and parses the body data accordingly. As a result, if you POST data from a form, or send JSON data with an appropriate Content-Type header, then Slim will parse that accordingly.

The example code handles multiple output formats by using an output renderer, which is the Formatter class added to the container and then used in each endpoint. The code for the Formatter class is in Example 11-2, and it takes care of getting the right output format (the HTML version is quite basic in an attempt to keep the code samples shorter) and the right headers in accordance with the Accept header that was received.

<?php

class Formatter
{
    protected $request;

    public function __construct($request) {
        $this->request = $request;
    }

    public function render($response, $data = [])
    {
        if ($data) {
            $format = $this->getFormatFromAcceptHeader();

            switch ($format) {
                case 'html':
                    $body = $response->getBody();
                    // very ugly output but you get the idea
                    $body = $response->getBody();
                    $body->write(var_export($data, true));
                    break;

                case 'json':
                default:
                    $body = $response->getBody();
                    $body->write(json_encode($data));
                    $response = $response
                        ->withHeader("Content-Type", "application/json");
            }
        }
        return $response;
    }

    protected function getFormatFromAcceptHeader() {
        $accept = explode(
            ',',
            $this->request->getHeaderLine("Accept")
        );

        // we prefer JSON
        $format = 'json';

        // we also support HTML
        if (in_array("text/html", $accept)
            || in_array("application/xhtml+xml", $accept)) {
            $format = 'html';
        }

        return $format;
    }
}

There’s a method here that does basic Accept header parsing (for a more comprehensive approach revisit the content negotiation section in Chapter 3) so we need to supply the $request object in order to make that information available. Once the correct format has been identified, the supplied data array is converted and set on the $response object along with appropriate headers.

There are a few Slim-specific features in the code shown here that might look different in another framework or without a framework at all, so it’s worth identifying those and giving the “vanilla” PHP equivalents in case you need them. All of these have been seen in examples throughout the rest of the book.

• $response->withStatus() in Slim simply sets the status code, so use http_response_code() in PHP.

• Each call to $response->withHeader() could be replaced in a PHP application with a call to header(), in which case the two arguments are put into one string and separated by a colon, e.g., Content-Type: application/json.

• The Slim function $request->getParsedBody() is a very neat feature that wraps up a read from php://input, checks the incoming Content-Type header, and in this case identifies JSON and does a json_decode() accordingly

Each framework will have its own way of doing the same things, but you’ll find that the modern frameworks all use a very similar approach since the shape of the request and response objects is related to the standards laid out in PSR-7, which covers HTTP Messaging.

With this simple sample app (and some data already in the low-tech storage solution items.csv) it is possible to look at ways that debugging techniques can be used with an API.

Debug Output as a Tool

Every PHP developer will have used print_r() or var_dump() at some point to return some additional information to the client during the course of the server processing a request. This technique is quick, easy, approachable, and can often be all that is needed to spot a typo or missing value.

When working with APIs, this can still sometimes be useful, but it does carry health warnings! If standard debug output is included with a response, and the client is expecting valid JSON, XML, or some other format, then your client will not be able to parse the response. Instead, try making the call from another tool such as the ones we saw in Chapter 10.

A great example would be to add a var_dump() call to the /list endpoint of the API, so that the route now looks like this:

$app->get(
    "/list",
    function ($request, $response) {
        // fetch items
        $items = [];
        $fp = fopen('../items.csv', 'r');
        while(false !== ($data = fgetcsv($fp))) {
            $items[] = current($data);
        }
        var_dump($items);
        exit;
    }
);

I can inspect this in any one of a number of ways:

• Use cURL, HTTPie, Postman, or any similar tools to make the API call that I want to debug. The preceding example can be seen in Example 11-3.

• If it’s easy to replicate from a client that can’t display the response, use Wireshark, Charles, or mitmproxy to inspect the response; just ignore the client errors since we know we’re sending back invalid JSON. Figure 11-1 shows the previous example inspected via Charles.

$ curl -v http://localhost:8080/list
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /list HTTP/1.1
> User-Agent: curl/7.38.0
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Host: localhost:8080
< Connection: close
< X-Powered-By: PHP/5.6.4-4ubuntu6
< Content-type: text/html; charset=UTF-8
<
array(4) {
  [0]=>
  string(5) "bread"
  [1]=>
  string(4) "eggs"
  [2]=>
  string(4) "milk"
  [3]=>
  string(6) "apples"
}
* Closing connection 0

Figure 11-1. Using Charles to inspect debug output from an API

Don’t be afraid to use the fast-and-dirty approach to debug APIs exactly as you might take a first look at any other PHP problem. It’s a little harder to see when the output doesn’t just appear in the browser, but this section illustrated how to take the extra step to make the output visible.

Effective Logging Techniques

When it is important to continue returning clean responses, more information can be acquired from an API, as it processes requests, by adding logging. This just means that, rather than sending debug information along with the output, it is sent somewhere else to be inspected (usually to a file on the server).

By default, PHP will write errors to the location specified in the configuration directive error_log in php.ini. If this is left empty, then PHP defaults to writing to Apache’s error log (or stderr, if you’re not using Apache). It is possible to write other information to this log, as well as the errors generated by PHP itself, using the error_log() function:

<?php

error_log("this is an error!");

Perhaps this looks like a rather oversimplified example, but at its most basic level this is all that is needed to add logging. When I look in the Apache error log on the server (the exact file location varies between platforms), I see this:

[Wed Dec 26 14:49:36 2015] [error] [client 127.0.0.1] this is an error!, referer: http://localhost:8080/
[Wed Dec 26 14:49:36 2015] [error] [client 127.0.0.1] File does not exist: /var/www/favicon.ico

A couple of errors can be seen in the previous output. The first was sent by the code sample, which deliberately wrote a message to the error log, and the other is what happened when my browser requested a favicon, but none existed. Using this approach, error_log() calls can be added into a project to help debug a particular issue. The output from the error log can then be checked to discover the additional information needed, rather than sending the additional error information back to the client.

Logging is a powerful technique; there are many more tricks available to make it even more effective. Log messages can be directed to a specific file, for example, rather than to the generic error log. To do this, use the error_log() function but with some additional arguments. The first argument is the message, as before, the second argument is where to send the message (3 means “to a file”; for more detail see PHP’s error log documentation), and the final argument is the filename to use:

<?php

error_log("all gone wrong", 3, "log.txt");

The file should be writeable by the user that the web server represents, and then the error message will appear in the file (beware: it doesn’t add a new line after each message). Specifying a file means that all the debug information can be kept in one place and will be easy to follow. The file could be truncated between test runs to make it even clearer exactly what happened in any given scenario.

There are lots of excellent libraries around to make logging easier, and if you’re using a framework, it will probably offer some great logging functionality. There are some great features in dedicated logging tools or modules that will help keep track of what’s happening in your application without resorting to a var_dump() call in the middle of your JSON output. When selecting a logging solution, look out for:

Multiple storage options

Many logging libraries support more ways to store log entries than just email or files. Usually it’s possible to log in to many different kinds of databases, use various file formats, and set other options. Depending on how you want to use the data, this can be very useful indeed.

Configurable logging levels

Logging libraries usually allow you to state the level of error that is being logged; this is comparable to the PHP approach of having ERROR, WARN, NOTICE, and so on. The application allows you to set what level of logging should be performed. This means you can change the logging levels on a lower-traffic test platform when you want to see more detail, or increase them temporarily to see more detail during a particular set of operations. As a result, the log files don’t become too huge when things are going well, but more detail can be obtained when required.

Standards compliant

There is a PHP standard interface for logging tools called PSR-3. Choosing a logging tool that complies with this means that you can change between logging tools in the future if you should wish to.

{
    "require": {
        "slim/slim": "^3.0",
        "monolog/monolog": "^1.17"
    }
}

<?php

require "../vendor/autoload.php";

$app = new SlimApp();

// create the logger, add to DI container
$container = $app->getContainer();
$container['logger'] = function($c) {
    $logger = new MonologLogger('my_logger');
    $file_handler = new MonologHandlerStreamHandler("../app.log");
    $logger->pushHandler($file_handler);
    return $logger;
};

$app->post(
    "/list",
    function($request, $response) {
        $data = $request->getParsedBody();

        if(isset($data) && isset($data->item) && !empty($data->item)) {
            $this->logger->addInfo("Adding data item: " . $data->item);
            // save item
            $fp = fopen('../items.csv', 'a');
            fputcsv($fp, [$data->item]);

            $response = $response
                ->withStatus(201)
                ->withHeader("Location", "/list");

            $response = $this->formatter->render($response);
            return $response;
        }
    }
);

[2015-09-05 16:41:27] my_logger.INFO: Adding data item: cheese [] []

Error Handling with PHP Exceptions

PHP offers really excellent error handling with its Exception class. This works similarly to exceptions in other object-oriented languages: you can throw an exception in your code, and it will then “bubble” up the stack by swiftly returning from each level of function call until it lands in either a catch() block or an exception handler.

While there are many great resources on PHP error handling around, there’s one particular feature in PHP that I make extensive use of, and that’s the ability to set a top-level exception handler. Whether that’s setting the error-handling feature in a framework (as will be seen in a Slim example momentarily) or using the standard set_exception_handler(), this ability to handle errors in the same way throughout the application is very helpful in APIs, in particular where a common output handler is often used.

When something goes wrong in an application, we throw an exception. While it’s possible to use a generic Exception class in PHP, it’s also very easy to extend that class and include the new class (often with no additional functionality) in your own code base. The big advantage of taking this approach is that you can then distinguish between the exception type you were expecting and something completely unexpected happening.

Exceptions always include a descriptive error message, but it’s important to consider where this error message will be seen. This chapter already covered some techniques for logging information, and as a general rule of thumb it is useful to include enough information to understand what went wrong in the exception; any information that should not be displayed to a user or other consumer can instead be logged.

A simple example of throwing an exception can be seen at the very end of Example 11-6 where in the event that we don’t get valid data, we use the exception handler to return that. The exception here is an UnexpectedValueException, which is one of PHP’s built-in exception types.

$app->post(
    "/list",
    function($request, $response) {
        $data = $request->getParsedBody();

        if(isset($data) && isset($data->item) && !empty($data->item)) {
            $this->logger->addInfo("Adding data item: " . $data->item);
            // save item
            $fp = fopen('../items.csv', 'a');
            fputcsv($fp, [$data->item]);

            $response = $response
                ->withStatus(201)
                ->withHeader("Location", "/list");

            $response = $this->formatter->render($response);
            return $response;
        }

        // if we got this far, something went really wrong
        throw new UnexpectedValueException("Item could not be parsed");
    }
);

In addition to the message, an exception can also have a code. Since HTTP has status codes, it can be useful to supply the code as well as the message in the exception, since in this case the exception handler will be formatting the error output.

So how can we craft an exception handler that will parse this Exception object and return the response that the API consumer can understand? The key things here are to be consistent, and to always return in the format (e.g., JSON, XML) that the client is expecting, even through we’re returning unexpected content.

The example we’ve just reviewed would want an exception handler to catch any Exceptions that haven’t been dealt with in any other way. Mine looks something like the example in Example 11-7. It takes the exception, sets the given status code or a generic 400 if it’s missing, and returns an error message in the body (in the Slim framework, you need to unset the $container[errorHandler] to be able to add your own standard PHP exception handler; other frameworks will also have their own ways of doing things).

set_exception_handler(function ($exception) {
    if($exception->getCode()) {
        http_response_code($exception->getCode());
    } else {
        http_response_code(400);
    }

    header("Content-Type: application/json");
    echo json_encode(["message" => $exception->getMessage()]);
});

With the added Exception from Example 11-6, my output looks like this when I make a POST request with no data:

$ curl -v -X POST -H "Content-Type: application/json" http://localhost:8080/list
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /list HTTP/1.1
> User-Agent: curl/7.38.0
> Host: localhost:8080
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 400 Bad Request
< Host: localhost:8080
< Connection: close
< X-Powered-By: PHP/5.6.4-4ubuntu6
< Content-Type: application/json
<
* Closing connection 0
{"message":"Item could not be parsed"}

The status code and headers are important, but this also includes a readable message so even if we were consuming it without the -v switch, we’d see something. For example, if I just pass the body output through jq, I’d see this:

$ curl -s -X POST -H "Content-Type: application/json" http://localhost:8080/list | jq "."
{
      "message": "Item could not be parsed"
}

In PHP 5, the exception handler will catch any uncaught object extending Exception, regardless of whether it came from your code, from a library, or from PHP itself. In PHP 7, the exception handler will catch anything that is both uncaught and that implements the new Throwable interface. This will probably behave as you expect in that it will still catch your Exception objects as before, but it will also catch any uncaught Error objects. You may see changes to when the exception handler is fired after you upgrade your existing applications, and it’s worth bearing in mind that this could mean catching objects whose messages you might not automatically want to expose to the outside world.

Using this exception handler pattern is a great way to ensure consistency of output so that your error messages are always delivered in a predictable way and in the expected format so that a client can understand them.

With error handling in place, it’s important to have test coverage of those failure cases as well as the successful ones; you can read more about testing tools in “Automated Testing Tools” along with other key delivery tools for APIs. At this point, we have covered the tools needed to add diagnostic, logging, and error-handling features to an API—we will discuss some of the finer points of API design in the following chapters.