C H A P T E R  9

Quality Assurance

Delivery of high-quality working software on a fast and regular basis is the goal of Agile software development. This book has discussed how to incorporate Agile practices in your APEX development process but so far has not specifically addressed quality. This chapter focuses on the principle of continuous attention to technical excellence and good design. And of course beyond pure technical excellence and good design, the software has to do what it is supposed to–not just be technically elegant. This chapter gives an overview of Agile quality assurance and then highlights APEX features and best practices to build quality into your application throughout the development and delivery process.

Agile Quality Assurance

In Agile software development, quality assurance is an integral part of the culture. Chapter 8 mentions applying standards until they become habits. Regarding quality, habits must become culture. Every team member is responsible for quality, and every team member is responsible for testing.

This is a big change for those who are used to traditional waterfall development methods, where the ping-pong process of testing predominates:

At some point in the game, someone runs out of bugs or fixes, or time’s up and deployment happens. Each side focuses on clearing their list and tossing it back to the other team, but communication and collaboration between teams is not encouraged.

At one site, we were told that no interaction between development and quality assurance was best, so that developers would not taint the testers. The net result was that the entire process took longer. At each volley, bugs and estimates and fixes were meticulously recorded; meetings tracked numbers, percents, graphs, and slopes; everyone was focused and stressed; and progress was slow but inevitable. We got there. Eventually Agile quality assurance is not like that.

Agile quality assurance is the practice of building quality into every step of your design and development process. It is not a before or after action, it is a during action.

Every member of the Agile team is responsible for quality, starting on day 1 and continuing through deployment. Customer, project manager, developer, business analyst, and yes, the quality assurance person all have an equal responsibility for quality, including testing. Each is equally tasked with answering the question, “Is our software working or not?” Testing is as much the responsibility of the developer as it is the business analyst or the tester or any other member of the Agile development team. All viewpoints are valuable and important to the process.

Agile quality assurance comes in many flavors, depending on the Agile methodology and the nature of the project and the organization. It is beyond the scope of this book to discuss them all, and it is not our intent to promote one over the other. However, several concepts are common to most Agile quality assurance viewpoints and are most certainly applicable to APEX quality assurance:

• Quality is a team culture. The whole team is responsible.
• Follow rules and guidelines.
• Create tests first. Know the answer to the question, “How do we know we are done?”
• Automate tests.
• Integrate continuously.
• Promote and use feedback.

The following sections address each of these points.

Quality as Culture

The whole team is responsible for quality assurance. Close daily co-operation between business people and developers throughout the process means that every team member, every day, has not just the chance but the responsibility to speak up and cooperate. Although it isn’t an agenda item at every meeting, quality is an integral aspect of every sprint, every conversation, and every task. To ensure a quality end product, every team member needs to speak when something is not working as it should. This includes cases of

• It doesn’t work as described. (Bug: let’s fix it.)
• It works as described, but that’s not what I meant. (Miscommunication: let’s clarify what you meant.)
• It works as described, but I see now I really don’t need that; I need this instead. (Discovery: let’s build what you really need.)
• It works as described, but I don’t need that sort of thing at all. (Throw away.)

It is common in Agile software development to encounter each of these cases. There is nothing right or wrong here; it is part of the process. The sooner these cases are flushed out, the better. The only way to flush them out is through constant communication between all members of the team. Code, demonstrate, and ask the question, “Are we done?” or, “Is this what you mean?” The key is for the team to accept this interaction and feedback as a normal process, adjust, and move on.

As a simple example, in Friday’s Scrum meeting, I demonstrated the latest User Admin module. The security folks asked for a Y/N flag instead of a date for ease of querying. Managers need the date. Security wanted User Role added as a column. A business user wanted “Maintenance” instead of “Administration” or “Admin”. We talked, settled on date and flag, agreed that “Maintenance” or “Maint” was fine for all, and added User Role. By the end of the 30-minute call, the changes were in place. Done. On to the next task. Oh yes: they need a Login Failures page from which they can restore logins. Got it; it’s ready for Monday’s meeting. Everyone pointed out their needs, the developer was not mortally offended by any of it, consensus was reached quickly, changes were implemented and demonstrated, and everyone went back to work. Face-to-face (well, Net Meeting) communication accomplished in under 30 minutes what could have been days. That’s Agile.

This is not to say every member of the team must have a testing background or a quality assurance background. In reality, team member expertise will overlap. Regarding testing and quality, a customer brings a different perspective to testing than does a developer, a business analyst, or the project manager. The customer is looking for usability: they want to get their work done. The business analyst is looking to ensure that a business process is complete. The project manager is looking to sign off on a task or create a new one. The developer’s brain may be stuck behind the code and less focused on the user interface. All perspectives are needed. It is the overlapping of skill sets and perspectives that adds value.

Note that a key part of this culture is trust. Team members must trust that their word will be heard, that they are an equal with every other member of the team, and that collaboration is open and unmeasured. There are no negative consequences for pointing out cases of “It works as described, but I don’t need that anymore.” If your team cannot develop this trust, then your process will be hampered (at best), or it will fail.

If one or more team members is stubborn or indecisive, your process will be hampered–talking points will not be settled in one meeting or will require longer meetings. Longer and subsequent meetings take time (resources) and may necessitate extending the project schedule (more resources). Such indecision can quickly become expensive. In APEX, development is fast, and developers can quickly move on to the next task. Missing bugs or issues, minor or major, early in the development process due to fear of embarrassment or ego-bruising negatively affects the quality of the software. When team dynamics break down, software quality suffers. This point is not unique to Agile software development; it is just more painfully noticeable. There must be trust for there to be an environment that promotes true quality. Your team must be a team, and quality must be part of the culture.

Build Quality into Rules and Guidelines

Rule and guidelines serve to improve code quality. For example, the technical design standard of following third normal form for relational integrity promotes solid database design. Good database design promotes good data. A guideline of separating business logic from presentation logic makes an application easier to debug and test, among other values. The topic of rules and guidelines is addressed in Chapter 3.

Create Tests First

Before developing anything, before starting any task, the team needs to know, “How do we know we are done?” That question is best answered with this one: “How do we test it?” The team may need to collaborate on how best to test certain features or tasks. At these times, the process of defining the test serves to clarify the task. At minimum, team members need to agree on the test success criteria, because this is what answers the “How do we know we are done?” question. This is what tells you when you’re “done”.

The practice of test-driven development involves building tests, usually true/false assertions, to validate requirements before coding, and then adjusting and expanding the tests as development evolves. The premise is that to write a test, the developer must completely understand the requirement. The Agile team works together to define and agree on the test, thus clarifying the task at hand. Developers need write only enough code to pass the test, following the Keep It Simple Stupid (KISS) and You Ain’t Gonna Need It (YAGNI) principles. In this way, test-driven development, and variations of this practice, serve to keep development lean and focused, serving the end goal of delivering working software.

Remember that testing a web application involves testing for several aspects, not just a simple true/false “it works or it doesn’t.” When you’re planning tests, consider testing that covers all these areas:

• Functionality: Do all the items, pages, links, and images work? Is all business logic correct? Does the HTML/CSS validate? Are all database interactions correct?
• Usability: Is the application simple and intuitive to operate? Are all features operable? Is content clear? Is help readily accessible?
• Interface: Are all database and application server interactions as designed?
• Compatibility: Does your application work in all supported browsers? On all operating systems? What about mobile clients? Do all charting and printing operations operate on all browsers and platforms?
• Performance: Does the application perform well under data load? Under user load? Do all page operations perform as expected?
• Security: Are all protected pages and processes protected? Is data secured? Are web directories accessible when they should be locked down? Do manual URL changes allow access to privileged operations?

This list is by no means comprehensive. The point is to think about what type of tests, and what depth of tests, are essential for your application. These questions are food for thought in developing your own test plan to cover your application.

Automate Tests Where Possible

Each story or module should include a task for unit-test development. Following the “Create tests first” point, this task should be early in the cycle, before coding. The cost of that task should be in proportion to the size and complexity of your deliverable. Over time, you will have a suite of unit-test scripts. As your test suite grows, it only makes sense to automate the execution of these tests, to save time, to save effort, for repeatability, and to not disrupt current and future development. Strive for working software delivered frequently. If the team spends more time testing, they are doing less development, and they are delivering less useful software less frequently. It’s simple: spend as little time testing as possible. Do just enough (sound familiar?) to be effective and repeatable. Balance the cost of building the tests with the return on investment.

As of this writing, there are no automated test suites tailored specifically for APEX development. We suspect that will change in the near future. There are, however, several automated test products that assist in testing PL/SQL program units and in testing web application screens. A combination of these types of products can cover both the business logic (the PL/SQL modules) and the presentation logic (the page flow and operation) of your APEX application.

For PL/SQL unit testing, consider products like SQL Developer PL/SQL Unit Testing, Quest Code Tester for Oracle, utPLSQL (the precursor to Quest Code Tester), DbFit, and others that may be embedded in your favorite PL/SQL integrated development environment (IDE).

For scripting web-application page flow, consider products like Selenium. Selenium is an open source suite of tools for testing web applications. The Selenium IDE has a recording feature that records user actions and saves them as a reusable script that can later be executed to validate an application’s operation. Many commercial products provide similar functionality.

Also consider scripting for life-cycle management: for promoting your application from development to your testing environment to production, conditional on passing an automated suite of acceptance or regression tests. Chapter 9 of Expert Oracle Application Express (Apress, 2011) describes a proven approach for testing and the promotion process.

The one exception to the “Automate tests” principle is when your application and the tests to validate it are so simple that the process of automating those tests would take more time than running them repeatedly manually. In that case, do not waste resources automating. Tests are important–they reduce bugs and eliminate waste. Take care not to introduce waste when building your tests.

Above all, keep your tests as simple as possible. We have seen incredibly beautiful and complex test worksheets that record every test and quantify results through charts and graphs. This is wonderful, but how much time did it take away from development? Unless you have a requirement to formally document test suites and test results, restrain from spending as much time developing tests as you do developing your application. No one cares how elegant your test scripts are. They do care that your software works.

Continuous Integration

Continuous integration is a core Agile practice. The principle Continuous attention to technical excellence and good design reminds you that you are continually integrating quality. Close, daily cooperation between business people and developers means you are integrating code frequently, preferably on a daily basis (or more frequently), so that business users and developers can communicate about whether it works. If it works, you move on. If it doesn’t, you fix it now.

Feedback, Feedback, Feedback

Feedback is essential to Agile software development, no less so where quality is concerned. The principle of Close daily cooperation between business people and developers encourages interactions. Feedback–information exchange–ensures that the team stays on the target of producing quality working software. Feedback allows the team to know when it is not on target, and to adapt as needed. Feedback makes the entire Agile development concept work. Without it, the team is proceeding (or not) blindly. The delivery of quality working software suffers.

Team members should partake in honest, open exchanges, stating early and clearly when code is not producing results or is producing unintended results. All team members should learn to ask questions to draw out feedback when it is not forthcoming. As a developer, ask if the customer’s needs are being met. As a customer, ask if the programmers have enough information to begin coding. Ask these questions in planning meetings and retrospectives as well as at scheduled daily meetings.

Feedback can take many forms–verbal or written, formal or informal. E-mail, voicemail, sketches, video, phone calls, drawings, instant messages, text messages, notes, spreadsheets, and documents are all valid forms of feedback. Whenever possible, make feedback visible to all team members. Preserve feedback only if it is essential to document the progression of the project. When feedback must be preserved, do so in the most efficient mode possible. The point is to enable and encourage open information exchange toward improving the software product. Successful Agile software development doesn’t just happen: it only happens because the interactions between team members enable it. Collaborative feedback throughout the lifecycle of the product is essential.

APEX Features for Quality Assurance

APEX incorporates many tools for quality assurance, ranging from the application builder wizards to the Debug Repository to the full suite of APEX application utilities. The following sections highlight the more useful APEX tools and utilities that promote the concepts of building in quality and validating that it exists during the development process. A more complete treatment of each of these tools can be found in the APEX documentation.

Wizards

You may not normally think of development wizards as a quality assurance tool, but think again. The APEX wizards create all the elements required for the object being created. For a simple form on a table or view, that means page items; validations; fetch, insert, update, and delete processes; buttons; and branches. Add a report on the same table, and the wizard also creates navigational elements for moving between pages and the report, including all default report column, pagination, and search settings. You can build those elements manually and produce the same result, but why? Using the wizard ensures that everything is created exactly as it should be, with no mistakes. The wizard enables error-free development of moderately complex components in a matter of seconds. That not only promotes quality but also economizes on development time.

When an APEX wizard is available, use it. It is most always more economical to use a wizard and customize the result as needed than it is to build from scratch. The result is fewer typos, fewer mistakes, and faster time to working software. That’s what you want.

User Interface Defaults

User Interface Defaults promote a common look, feel, and operation of page items and report columns, and minimize developer time spent individually setting element attributes. Commonality looks professional and makes your end users happy because the look and operation of your application is consistent.

User Interface Defaults allow a developer to synchronize page-item and column settings with underlying table and column settings, using the Table Dictionary or the Attribute Dictionary, for propagation of those settings to like page items and columns. You save time in both cases. User Interface Defaults and the Table and Attribute Dictionaries are discussed in greater detail in Chapter 8.

Unit Testing

Gotcha. There is no APEX-supplied unit-testing framework. This section is here to remind you that even though APEX does not currently include a unit-testing module, unit tests need to be created and executed, the sooner the better in the development cycle.

User Acceptance Tests

Gotcha again. APEX has no built-in tool for acceptance tests. The recommendation is to use the same methodology used for unit and regression tests, plus ad-hoc testing by real users.

In our opinion, user acceptance testing (UAT) is mandatory. At some point during development, team members become too close to the code and the product for unbiased acceptance testing. They know too well how the code is supposed to work, and they know all the exception cases built into the unit tests, but they are not end users and just don’t have the uncanny ability to find a bug no matter what. End users have that knack. Certain end users excel at it. Make note of the bug-finding, gifted end users, because they are your best friends (or your worst enemies) when it comes to UAT time.

Acceptance tests don’t just execute the unit test suites again. UAT must include users exercising the system as they will normally be using it. Normal use is not scripted–it involves all kinds of unexpected clicks, starts, stops, and hiccups that most testing teams do not plan for.

Acceptance tests are best done just after successful unit tests, at the end of each sprint, when features of the functionality just developed are fresh in the development team’s mind. Yes, performing acceptance tests means there will be some overlap as issues from the previous sprint in UAT must be dealt with while the next sprint is underway. This small amount of overlap is far preferable to waiting until the end of development, when discovering an error will be more costly. Errors found at the end of development are always more costly–we have yet to find an exception to that axiom. If time spent addressing issues found in UAT becomes significant, add a catch-up task to bring the project back on track.

If you have built and saved your unit tests, they need to be executed as part of or just prior to UAT. The key thing about UAT is the user part–have real live users test your application the way they, the real live users, will use it. It’s just common sense to do so, but in many cases–and we’re sure you know a few–it doesn’t get done.

Acceptance tests are of course more critical for rocket science O-ring monitoring than for a three-page application that manages the in-office soccer pool. That only means the amount of time spent on unit tests and user acceptance tests for more critical applications is appropriately longer than for less critical applications. Excellence is the goal in both cases, balanced with the cost of attaining it and the cost of not attaining it.

APEX Debugging Tools

Although it isn’t usually considered a quality assurance tool, the APEX debug module certainly promotes quality assurance by making it easier for the developer to deliver working code. The APEX team has provided a sizeable suite of debug mechanisms and approaches for tracking and solving most APEX errors and performance issues. Tools for code instrumentation, reports on page-rendering and page-processing events, an interactive debug mode, and a whole suite of APEX utilities to view debug information are a few of the APEX-supplied debugging options.

The Debug Repository records a wealth of information about the session state and page-rendering processes at the time of the bug. This information is invaluable for tracking errors and leads to addressing them more quickly. Faster debug time means faster delivery of working software. And that is the point: to deliver high-quality software quickly. Debugging in APEX is covered extensively in Chapter 6 of Expert Oracle Application Express. This is a recommended read for any developer building an APEX application.

APEX Advisor

The APEX Advisor, new in APEX 4.0, is APEX application quality assurance central. The Advisor is a utility that scans your APEX application for a select series of known best practice and value-added practices and reports when and exactly where your application falls short. It sounds a bit intimidating, and in fact it can be. It’s a health check. Run the Advisor, and it tells you exactly what and where in your application you are not following the selected quality criteria. The Advisor report includes links to the offending page, so you can readily correct the deficiency.

Access the APEX Advisor from the Application Builder Utilities menu. Figure 9-1 shows this menu, including the Advisor option.

Figure 9-1. Accessing the APEX Advisor through the APEX Utilities menu

The Advisor interface allows you to select which checks to perform and a list of pages to check. Checks fall into these categories:

• Errors
• Security
• Warnings
• Performance
• Usability
• Quality Assurance

Each check has a tooltip that contains a brief explanation of what the check does. Not all checks make sense for all applications. Determining which make sense for your application and your organization may take a few tries. Don’t be intimidated–all of these checks add value, yet some of them may seem picky. Only you can decide if it is essential to address all of them. The checks applied should coincide with the standards documented in your rules and guidelines. Our recommendation is to at minimum address all errors and security failures. Figure 9-2 shows the Advisor Check to Perform and Check Pages elements. Figure 9-3 shows a sample Advisor results page.

On the Advisor Results page, you can filter the results by check to narrow the scope of results for ease in addressing them. To address a failed check, simply click the View link at the bottom of the listing to navigate to the page of the failed check. Once there, resolve the issue, and you are one step closer to quality.

Figure 9-2. APEX Advisor: selecting checks to perform, and Check Page(s)

Figure 9-3. Advisor result

Team Development

The Team Development module, discussed in detail in Chapter 6, offers Feedback, Milestones and Releases, Features, Bugs, and To Do’s. Each of these modules facilitates communication and efficiency, key values in the Agile process. The Team Development utilities promote quality because they facilitate the Agile development process. Of these, the Feedback and Bugs modules are the most pertinent to quality assurance; but they all play a part in keeping the team organized, keeping developers on schedule, and tracking project features, bugs, and enhancements. Whether you use one or a combination of these modules, the simple, lightweight Team Development framework is wise addition to an Agile APEX development project. The following sections supplement Chapter 6, with a focus on quality assurance.

Feedback

One of the most obvious modes of quality assurance in APEX is the Team Development Feedback utility. The Feedback module is invaluable for immediate, complete, documented communication among all members of the team. Such communication enhances the Agile process and thereby improves quality. The under-the-hood values automatically captured by the Feedback module enable thorough analysis of the issues and comments entered. This information is essential for bug tracking, because you cannot count on getting it from your users. It is a common phenomenon for end users to loudly declare, “It doesn’t work,” “I got an error,” or “It’s broken,” and yet have absolutely no recall of any additional details, such as what didn’t work, what error message they saw, or how it’s broken. The highly motivated members of your development team may be excellent, but they are not mind readers. Feedback is essential, however you can get it. The extra values that APEX passes on automatically are invaluable.

In a recent Agile project, we encountered a web interface error that was initially reported by only one user. The user passed on screenshots of the problem. The developer was unable to reproduce the error in any browser; the project manager was unable to produce the error in the standard IE 8 browser; and an on-site team developer was unable to reproduce the error. When the bug was reported in UAT of another application, the team realized the issue had to be addressed. But how could they address it if it couldn’t be reproduced? The developer dove in and spent hours researching such behavior, examining the theme, the pages templates, and all custom CSS and HTML, basically searching for possible causes. He finally traced the issue to the use of IE 7 compatibility mode, with IE 7 document standards. Had this issue been reported with Feedback, the extra information regarding the reporter operating system and browser settings would have been available. Having this information would have saved hours of development time–the IE7 browser would have stood out as non-standard for the corporation. The development time and team brainpower in researching this problem was lost. The less tangible but perhaps more significant loss was in credibility. How could the developers deliver such buggy code? Customers do not expect–or deserve–significant user interface bugs, and first impressions are lasting, even in a development or test environment. Eventually the issue was resolved and credibility was restored, but the loss of any hour in an Agile project is significant. In this case, the lost time amounted to the equivalent of an entire task. That’s the loss of a deliverable!

Our advice is to use the most efficient tools available to capture as much feedback (data) as possible as easily as possible. There is no doubt that the APEX Feedback utility is one of those efficient, useful, easy tools. For a developer, it is like feedback heaven delivered to your feet. Just pick it up and use it!

Note that not all users have access to Feedback by default. Non-developer, non-APEX Administrator team members must have access to APEX to be able to use it. It is possible to have Team Development–only APEX users: create a user who is not a developer and not an Administrator, and select Yes for Team Development Access. Make the appropriate selections in the Create User interface, in the Account Privileges region, as shown in Figure 9-4. Your feedback moderator may be one such user.

Figure 9-4. Setting account privileges for a Team Development–only user

Consider also that not every enterprise will embrace the use of Feedback and/or the Team Development module in general. There may be a corporate issue-tracking and change-management system, or something as simple as Microsoft Excel spreadsheets that are embedded in corporate culture and are the preferred manner of issue tracking. In these cases, team members can still use Feedback to rapidly enter issues as they are encountered. The feedback moderator can then download the feedback settings into the corporate issue-tracking system. The problem with this approach is that it is one-way–there is no “Import Feedback” utility for tracking responses to feedback items that are external to APEX. Once exported, the Feedback loop is interrupted. Our advice in these cases is to demonstrate the value of Feedback and the other Team Development features, emphasizing the value added versus the resource cost (how much time does it take you to maintain issues in the corporate system versus through APEX Team Development?). Of course, this value-add needs to be weighed across all team members, not just the developer.

The Feedback form is a plain-old APEX form page. This entire form may be customized as desired, following corporate development practices, of course. The Feedback module allows for up to eight custom fields that may be added to the Feedback pop-up form. These let you capture data or settings specific to your application or user environment. The point is to get information from the user to the development team, and vice versa, with the least pain possible and the least impact to the goal of delivering working software.

Bugs

The Bugs module is another important feature for Agile quality assurance. Feedback that is classified as a bug is automatically tracked within the Team Development network. Bugs can be tied to a release and assigned to a developer, who in turn closes the bug when it is completed. Although the Bugs module is not as complete as many off-the-shelf issue-tracking programs, for APEX development it is sufficient and lightweight, and it does the job. This follows the Agile practice of doing just enough, and no more.

It is up to your team to decide whether Team Development provides all the issue-tracking features and documentation required for your project or your enterprise. Remember that the APEX development team used Team Development in developing APEX 4, demonstrating that the Bugs module certainly contains the essentials for tracking issues in a sizeable APEX project.

Summary

Agile quality assurance is a during action–it occurs during the development process and is the responsibility of all team members, at every phase of the project. Although there are many approaches to ensuring quality in Agile software development projects, they all include these aspects:

• Instill quality as part of the culture.
• Follow rules and guidelines.
• Create tests first.
• Automate tests wherever possible.
• Integrate continuously.
• Promote and use feedback.

APEX provides many built-in tools for building quality into an application, all of which are free and completely integrated with the APEX instance. These tools include the create wizards in the Application Builder, the Debug Repository, the Team Development suite of tools, and the APEX utilities, highlighted by the APEX Advisor. In many cases, data is automatically collected by the APEX engine, as with the Feedback interface and debug mode. In other tools, such as Team Development milestones and To Do’s, minimal user input is required.

Although APEX does not include built-in unit-test and acceptance-test modules, commercial and open source options are readily available for PL/SQL and web application testing that integrate well with APEX development. Because the end goal of your project is to deliver quality working software, it makes sense to adopt as many of these tools as practical to streamline the development process.