Reprinted with permission of the Everett Collection.
Modern Times with Charlie Chaplin.
A Private Workspace (6) allows you, as a developer, to insulate yourself from external changes to your environment. But your changes need to work with the rest of the system too. To verify this, you need to build the system consistently, including building with your changes. This pattern explains how you can check whether your code will still be consistent with the latest published code base when you submit your changes.
Note
How do you verify that your changes do not break the build or the system before you check them in?
In a development team with liberal codeline policies, changes happen very fast. You change existing code, add new modules to the codeline, and perhaps change the dependencies.
The only true test of whether changes are truly compatible is the centralized integration build. But checking in changes that are likely to break the build wastes time. Other developers will have you suffer through mistakes that you could have fixed quickly, and, unless the system build turnaround is very short, it will be harder for you to recall the source of an error because you may have lost the context in the meantime. Because the system build incorporates other changes as well as yours, as Figure 8-1 shows, it is the true test of whether your code integrates with the current state of the work. But this also makes it harder to isolate the source of problems.
At times your precheck-in build may work just fine, but the nightly build fails. Or your copy of the system that you get fresh from source control works just fine, but the product install made from the nightly build does not work the way you expect. You could always start debugging from a product install, analyzing logs and other runtime debugging facilities, but debugging from your development environment gives you more information. Sometimes the problem in this case is that the product build and install did not incorporate a new file or resource that you added to source control. Having release engineering maintain a list of what is built and installed adds a sense of reliability and reproducibility to the build, but if developers are the ones adding components to the version tree and they don't have visibility or control over what gets on the list, changing the build involves an added layer of communication, which implies an additional chance for error.
Often organizations have very well-established formal build procedures, but they don't scale down to the developers. Separate developer and release builds can make things simpler for some developers, but it also means that significant problems can't be found until the system build, which can be as infrequent as daily. This wastes time for anyone who needs a working codeline and makes it harder to get product release candidates to the testers.
To be able to do a reasonable test of the effect of the changes, you must be able to build all parts of the system that your code affects. This means building components in your own workspace. You can work by patching your workspace with your built objects—for example, by building only the components you changed and altering the system PATH or CLASSPATH to use the new components first. But software systems are complicated, and you may not see the interactions of a “normal” build and execution process. Maintaining two procedures in parallel is difficult and error prone.
Note
Before making a submission to source control, build the system using a Private System Build similar to the nightly build.
The private system build should have the following attributes.
Be like the Integration Build (9) and product builds as much as possible, although some details related to release and packaging can be omitted. It should at least use the same compiler, versions of external components, and directory structure.
Include all dependencies.
Include all the components dependent on the change (for example, various application executables).
The architecture will help you determine what is a sufficient set of components to build. An architecture that exhibits good encapsulation makes it easier to do this build with confidence. Figure 8-2 illustrates what goes into a build.
The build should not differ significantly from the nightly build. Wingerd and Seiwald suggest that “developers, test engineers, and release engineers should all use the same build tools” to avoid wasting time because you are not able to reproduce a problem (Wingerd and Seiwald 1998). In The Pragmatic Programmer: From Journeyman to Master (Hunt and Thomas 2000), Andy Hunt and Dave Thomas say, “If you do nothing else, make sure that every developer on a project compiles his or her software the same way, using the same tools, against the same set of dependencies. Make sure that the compilation process is automated.”
If it must, it can differ from the product build in the following ways.
It can be done in an IDE or other development environment as long as you know that the compiler is compatible with the one used in the product build process. Beware of differences that cause inconsistencies. And make an integration build script available to enable debugging.
It can skip steps that insert identifying information into the final product—for example, updating version resources. Even steps like this can cause problems, so it is best to include these steps as well and just not check in changes that happen because of the build process. For example, if your changes did not change a version resource, do not commit the automatic change to the resource.
It can skip some packaging steps, such as building installation packages, unless this is what the developer is trying to test.
It is important that the private build mechanism replicate the production build mechanism semantically as much as possible while still being usable by a single developer.
Examine how you decide what is put into the build and what is put into each component, such as a jar file or a library. Some common approaches follow.
Build everything in version control. This has the advantage of making it easy to decide what to build and include. The disadvantage is that it can discourage developers from using the version control area to share files that may not yet be ready for use because, if they do not build, they can create errors. You still need to know how to package files into deliverable units.
Build everything except items marked to exclude. This allows you to put anything in source control so that it is included by default but allows the option of excluding certain files.
Build only parts of the version control tree that are explicitly included in a release list. The parts can be parts of a directory structure or individual files. The advantage of this approach is that it makes it easier to put experimental code into version control without worrying about breaking the build.
Either approach can work well. The build-everything approach is simpler. The include/exclude approach can work well if the include/exclude list is maintained in version control and can be built by developers, because the developers drive what code and components go into the product. As long as developers and the nightly build build the same files and “install” them in the same places, you will be able to debug problems easily. Any approach that has two different systems will cause conflict and delays.
When rebuilding in your workspace, you need to decide whether to do a full build or an incremental build. A full build is best to ensure that you are not missing any dependencies. But a full build may be impractical for active development because it can take a long time. So under most circumstances, you can do an incremental build if your dependencies are set up correctly. You should do a clean build under the following circumstances.
When you are adding new files to the source control system. In this case, you also should start with an empty workspace. This is the only way to check that you added the file to the correct place. It's not unknown for people to forget to check in a file but for their builds and tests to pass because the file was in their workspace.
When you make extensive changes involving key functionality. This may be overcautious but is best to do if you suspect that your dependency checking is in error.
You can also do a “clean” build of an individual component (for example, a library file or a jar).
They key thing to remember is that you should do this process repeatedly. Requiring a clean build all the time will make the process too slow to be useful, but never doing a clean build will expose you to any flaws in the way your tools handle dependency checking. An Integration Build should catch any problems of this sort, but the earlier you catch the problem, the less expensive it is to fix. When in doubt, do a clean build if time permits.
Once you have made sure that your code works in your current environment, update your workspace with the latest versions of code from the codeline you are working on, and repeat the build-and-test procedure.
A Private System Build (8) does take time, but this is time spent by only one person rather than each member of the team, should there be a problem. If building the entire system is prohibitive, build the smallest number of components that your changes affect.
If you change a component that other components depend on as an interface, it can become very difficult to cover every case. Ideally, you would build all clients. In this situation, let the Smoke Test (13) determine what executables to build.
As in many aspects of software development, this is a situation where communication is very helpful. If you think you will be making a sweeping change, run all the tests and then announce the pending change widely. This will enable people to let you know about their dependencies and to identify your change as a potential conflict if they see a problem later.
Related to the question of clean versus incremental build is the question of “what” to build. Start with whatever you need to run smoke tests. If you are developing a component that is used by one or more applications, consider building one or more of these executable applications. Ideally, you would build all the applications you know about. You don't need to be exhaustive though. If you miss something, your integration build and related testing will find it. This approach is not “passing the buck”; although each team member needs to attend to quality, everyone cannot take an infinite amount of time to do this. Consider the time in the release cycle, the reliability of your incremental build tools, and the time it takes for a full build.
Regardless of what approach you take daily, you should be able to start from an empty workspace and re-create the products of the nightly build when necessary.
Once you know you can build the system, you still need to know whether you are breaking the functionality. To make sure the system still works, do a Smoke Test (13). This pattern enables you to do a smoke test.
If the system is very large, it may not be efficient to build every component that uses your comoponents. These leftover dependencies will be validated in an Integration Build (9).
What do you do when you find a build error in some other code that is related to your changes? Ideally, you should merge your own changes if you can identify them and they are not too extensive. If you change a widely used interface and the team has agreed to the change beforehand, it may make sense to communicate the timing of your change so that other team members can change the code they are responsible for. Your team dynamics will best decide the answer to this question.
Steve McConnell discusses the need to do a build before checking in code in Rapid Development (McConnell 1996).
The Pragmatic Programmer: From Journeyman to Master (Hunt and Thomas 2000) has much good advice about build automation.