4 methodologies every test engineer should know

by Jess Ingrassellino

Testers are often bombarded with questions about four key test methodologies: test-driven development (TDD), behavior-driven development (BDD), acceptance test-driven development (ATDD), and design-driven testing (DDT). The broad scope of each, and the fact that all are not exclusive to software testing, makes it difficult to get a handle on what methodology works best for a given situation, or when you might need to combine methodologies. 

As a test engineer, you need to understand all of these test methodologies if you want to speak intelligently with the development team about how they should test. Sharing this expertise is also a good way to build a strong relationship with the development team. Here's what you need to know about each methodology, and how to determine what will work best in your organization.

Test-driven development

With the TDD development pattern you write the tests first, before the code. After testing, developers write the code needed to make the tests pass. The initial tests are essentially a blueprint for the application’s components.

This method is applied primarily to unit tests, which means it’s primarily done by developers. While test-driven development is generally not within the purview of software testers, you should know how TDD is practiced and what it means for your organization.

TDD has had mixed reviews. Proponents argue that, by writing tests before the production code, your code  will be clean, clear, and testable at the integration and user interface (UI) levels. Bob Martin writes about how test-first development also helps with thoughtful decoupling, which keeps production code flexible. Meanwhile, Ruby on Rails creator and test-driven development opponent David Heinemeier Hansson notes that while TDD provided useful contributions to developer testing, its dogmatic usage was too strict in many instances, resulting in poor code design for the sake of writing tests.

TDD has remained popular, however, because:

  • Dev shops actively using test-driven development have a suite of unit tests developed alongside production code.
  • A well-maintained unit test suite provides useful information about the health of the code.
  • TDD methods can help companies address problems with legacy code over time.

Software testers may be unsure about how test-driven development impacts them, since they are not tasked with writing unit tests, but it’s important that they stay informed about testing and quality as a holistic activity.

Behavior-driven development

Behavior-driven development, as defined by Dan North, evolved from test-driven development. It takes TDD and adds a domain-specific language (DSL) to express tests in the style of a human-readable language. Using a human-readable language to express the tests gives business stakeholders and other nontechnical stakeholders the ability to understand what is happening in the unit tests. This helps keep the software aligned with business requirements. Here are the main points about BDD you need to remember:

  • It is a practice rooted in developing unit tests and then writing production code based on those unit tests (TDD).
  • In addition to TDD practices, BDD seeks clarity of design and intent by using human-readable specifications.
  • Use behavior-driven development if your situation calls for communicating what is being tested to the business in clear terms. This might be necessary in heavily regulated industries such as medical and financial, where the DSL allows the code to become part of the audit trail in a manner that's useful to both developers and non-developers.

Acceptance test-driven development

Elizabeth Hendrickson defines acceptance test-driven development as "a practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins." Teams that practice ATDD often include nontechnical stakeholders such as business developers. The major goal of this work is to create a shared understanding of a feature or product among all stakeholders before coding starts.

A team that practices acceptance test-driven development :

  • Develops acceptance tests while developing production code, creating automated UI tests from the beginning, when it makes sense to do so.
  • Requires fully automated user stories that are defined up front by the business team, developers, and testers working together.
  • Avoids scope creep by clarifying expectations and narrowing down the expectations of the final feature, reducing the likelihood of extra asks or stress on the development team.
  • Uses exploratory testing to find holes in the acceptance criteria.

Acceptance test-driven development is a test methodology that explicitly includes software testers in every phase of design and development, from initial planning meetings to the performing of exploratory testing and the creation of test charters to provide feedback.

Design-driven testing

Design-driven testing is like test-driven design implemented backwards: You write tests based on what is designed, rather than basing product design on the results of tests. DDT has four test artifacts: unit tests, controller tests, scenario tests, and business requirement tests. Each has a  test artifact that corresponds with each aspect of the software development lifecycle:

Unit tests in DDT are written by the programmers who write the feature. They are tests on the smallest elements of the code.

Controller tests are integration tests. Written by developers, these tests exercise logical functions in groups of code that might work as individual units of code (and pass unit testing) but could still have errors when working together.

  • It is not necessary to write both unit and controller tests. Exploratory analysis helps determine which tests are best for which pieces of code.
  • The "Top Ten" list in chapter 6 of the book Design Driven Testing: Test Smarter, Not Harder provides an excellent walkthrough of the controller test process.

Scenario tests, or behavioral tests, are generated from narrative use cases, then scripted and conducted manually. The scenarios are generally written by developers and then passed on to and conducted by software testers.

  • These tests take the point of view of a user manual, with knowledge of how the product should have been implemented.
  • The tests can be automated but do not need to be automated for efficacy. The return on investment for automation depends on the scope of the project and the stability of the codebase.
  • Automated test scripts are likely created and maintained by automation engineers or software development engineers in test (SDET)—people with testing and programming skills.

Business requirement tests validate whether the product fulfills the user expectations and business requirements. These tests are highly scripted, rather than exploratory. These tests:

  • Take the point of view of the customer who has expectations for what the product should do, without knowledge of how the product has been implemented.
  • Are owned by the entire team. Product, development, and QA/testing teams all participate in their creation.

Additional resources

Each test methodology has an abundance of books, blogs, papers, presentations, and other materials available for the interested tester to pursue independently. Now that you've read up on each methodology, it should be easier to analyze your organization's practices and make recommendations to your team. You might decide to mix and match various aspects of these methodologies, or use just one.

Want to know more? The resources below should help.

Test-driven development

Acceptance test-driven development

Behavior-driven development

Design-driven development