Testing

There are multiple kinds of automated tests that can be applied to Flutter apps:

• Unit testing, the most common, which tests a smaller piece of code and should comprise the majority of the testing you should do on an app.

• Integration testing, which tests a big section of integrated pieces of code and, in the case of Flutter, runs the app in an emulator or physical device to check whether it behaves as it should when the user interacts with it in a certain way.

• Widget testing, which tests UI elements without firing up the app on an emulator or device, instead just keeping track of everything that is supposed to be on screen and letting us interact with it in a way that is much simpler and faster than is done in integration tests.

In this section we are going to use as an example the calculator app we built earlier in the book (Chapter 3, ​ Building a Calculator App ​) and write some tests that make sure each section of the app is working correctly. We are then going to test the XKCD comic browsing app built in the previous chapter to introduce you to mock objects and dependency injection in Flutter.

Dart Unit Testing

Unit testing is really simple: you run a piece of code (for example, a class constructor and a few member methods) and check whether the return value is what is expected.

To perform unit testing, we need to add the test package to pubspec.yaml’s dev_dependencies, which is the section where we list dependencies only needed for development and debugging/testing and not needed for the execution of the app on an user’s device.

Writing Unit Tests for the Calculator App

Let’s start by taking a look at the app tree created by flutter create: it contains a directory called test. That’s where we’re going to add any tests we want to be performed by Flutter.

Here we’ll create a file called calculation_test.dart, inside which, as the name says, we are going to test the Calculation class which, if you remember (or go back to ​Implement the Calculations​), allows us to add little pieces of an expression (as Strings) at a time and then get either the result as a double number or a String representing the expression.

There are multiple tests that can be done: we’ll test whether it performs all the different operations and return the right result, then whether it can handle multiple operations and then whether it respects operator precedence. Additionally, we need to check both string values and numeric values returned by it.

We’ll start writing the test by importing the needed classes. In this case we’ll need to import the test package and the calculator.dart file in which we have defined the Calculation class:

Then we call the Calculation constructor to get an object on which to operate:

Everything else will go into the main function:

The first thing to do is to define a setUp function that will get called before each test, creating a brand-new calculation object by calling the constructor in order to make sure each test is run in isolation.

We are going to call the test function, which takes a string and a callback function as positional arguments. The string is a text description of the test, so that we know what tests failed (if any fail) and the callback is the code that is going to be run when that test is fired. Inside the callback we’ll use the expect function to specify what value a certain instruction should return. For example, if we wanted to test a simple addition operation in our Calculation we could write the following:

If we want to test more than one addition/subtraction operation in the same test, we just add the following after it:

We also need to test whether multiplication works, as we do in the following:

Division is also important, and we also get a chance to check whether we can properly return floating-point values with a decimal part:

Operator precedence can’t be taken for granted, so we need to check whether it is respected:

The last thing to check is whether the getString operation works correctly:

The entire calculation_test.dart file adds up to the following:

At this point, every time you try to build your app these tests will be run. You can also run the tests using flutter test on the command line. However, probably the best way to run tests is by using Visual Studio Code, which shows a Run button for each test, allowing you to run each test and get the results shown in an intuitive UI. Android Studio also shows a right arrow button next to the line number to run a test, but I found its UI slightly less intuitive and consistent. You might feel differently, though, especially if you prefer the Android Studio/IntelliJ UI over the Visual Studio UI; opt for whichever you’re more comfortable using.

Flutter Widget and Integration Testing

Unit testing was simple and straight-forward enough, but widget and integration testing need more attention since they rely on UI, which isn’t exactly as dry and simple as the calculation processing code.

Each little piece of low-level logic in our app may be correct, but there might be something wrong in the UI or the combination of some of the pieces of code we already tested. This is why widget testing and integration testing exist. It’s to test whether a single widget or a bigger part of the app work correctly.

Flutter Widget Testing

Flutter provides a way to test whether widgets work correctly and to programmatically interact with widgets in our app.

This might be really hard to imagine if you’re not used to similar testing infrastructure, so here’s an example of how you might test whether pressing the 5 button will actually make the 5 number appear on the calculator screen after importing main.dart:

​①​

The testWidgets function is very similar to the test function, but with a big difference: it passes a WidgetTester object to its callback, and that object is what makes widget testing special.

​②​

pump means build or render. tester.pumpWidget renders a widget for which we have a definition.

It’s important to keep in mind that, even though the pumpWidgets gives the impression that it allows to pump any kind of widget, if you have a Scaffold in your widget it needs to be wrapped in a MaterialApp (or the more generic WidgetsApp class, which doesn’t rely on the Material Design classes) because it needs to have a MediaQuery widget in its scope, and such a widget is provided by the aforementioned classes.

​③​

This is the first expect call. When we build the app for the first time, just one widget containing the 5 string should exist (the 5 button).

find is a constant. We’ll look at its use in more detail later in this section.

find.text is a Finder and findsOneWidget is a flutter_test-provided Matcher constant.

We’ll list all Finders and Matchers in more detail in the next two paragraphs after this code’s explanation. All you need to know now is that a Matcher is used to judge the results delivered by a Finder, which looks for widgets based on a certain characteristic.

​④​

This is quite self-explanatory: we tap that widget.

​⑤​

When testing, Flutter widgets aren’t rebuilt automatically when setState is called: to do that, we need to call tester.pump, which re-renders the next frame the app is waiting to render.

There is also another method called tester.pumpAndSettle, which keeps rendering frames until there is no re-render waiting to happen. This is useful when certain actions trigger an animation that takes several frames to complete.

​⑥​

This is just like the other find.text check, but this time there should be two widgets with that text: one is the button, and the other is the screen, which should now show that number.

As you can see, the interface is quite flexible and intuitive, and it is really useful to check whether the UI actually functions correctly.

All of the Finders

Let’s get a bigger picture of Flutter widget testing: there are multiple ways to find widgets using the find constant, which is a collection (of class CommonFinders^[43]) of the most commonly used Finders:

• find.text(text), which finds Text widgets that display a string that is equal to its text argument.

• find.byIcon(icon), which looks for Icon widgets that show an IconData (something such as Icons.not_interested) equal to its icon argument.

• find.byKey(key), which takes us back to ​The Key​: it matches the widget that has the Key you give it as an argument.

• find.byElementPredicate(predicate), which is very flexible: the predicate argument (of type WidgetPredicate) is simply a callback that receives a Widget as its argument, and returning either true or false will, respectively, consider the widget being examined as matching or not matching the requirements to be included in the Finder’s result, which gives the developer significantly more freedom in deciding matching conditions than the other Finders.

• find.byWidget(widget), which is the most specific of the Finders: it matches just the Widget given as an argument.

There are other, less common Finder provided by the find constant:

• find.byTooltip;
• find.byElementType;
• find.byType;
• find.widgetWithIcon;
• find.widgetWithText;
• find.byDescendant.

All of the Matchers

In addition to findsNothing, findsOneWidget, and findsNWidgets(n) there is also findsWidgets, which is the opposite of findsNothing (equal to !findsNothing).

More Calculator Testing

Let’s actually get into a more complete and realistic example of widget testing our calculator app.

Checking the Grid

The first part of the 5 press test can actually be turned into something very useful: checking whether all of the grid is actually being rendered and the user has a button for each button. The numbers, the deletion buttons, the . button and all of the operation buttons except the division button contain a Text to show their use. This means that we can match them using find.text:

​①​

There should be two widgets containing the 0 string: the calculator screen at the start and the zero button.

​②​

The numbers from 1 to 10 can be found by simply using a for loop.

​③​

We haven’t used forEach much in this book, but it is very useful: it executes the same callback on each member of a List or Iterable. In this case, it is a good replacement for a for loop iterating over the list of symbols or just writing the instruction to look for each symbol separately.

​④​

We can’t use find.text for images obviously, and find.byWidget might come up as a viable alternative. The issue with that, though, is that it doesn’t actually work in our case: it looks for an exact match, and it wouldn’t find our image if we tried since the resulting object usually contains extra arguments that control widget position and other properties that we don’t set directly while creating the widget.

That’s where find.byElementPredicate comes to our rescue: we can judge ourselves whether the widget matches what we want, which means we can focus just on what we need: we need the widget in question to be an image and we want it to be displaying icons/divide.png from the assets.

What gets passed to the callback closure is an Element (an instance of a widget along with some context on its position in the widget tree, which we don’t need for what we’re doing right now). We can access the Widget that Element was spawned from by accessing its widget member property, so that’s what we’re doing when using element.widget.

If the widget isn’t an instance of an Image (that’s what is is for, it’s checking whether the widget is an instance of an Image or a subclass), we need to ignore it (return false to exclude it from the found widgets). If it is, we need to check whether the ImageProvider it’s taking the image from is actually the icons/divide.png file from the assets.

Please note that accessing assets in unit and widget tests is only available starting from Flutter version 1.5.

Flutter Integration Testing

Integration testing is a level above widget testing: for mobile apps, it is performed in an emulator and is meant to test whether a big part of app functionality works correctly. The calculator app is really simple and we don’t really get much from integration testing when compared with widget testing, whereas one app that requires integration testing is the previous chapter’s XKCD comics app.

Before we get to running integration tests on it, we need to figure out how to also run unit tests and widget tests on it without being dependent on a network connection to the XKCD servers being available and without being able to access the filesystem, as is the case during unit and widget testing.