Stop Thinking, Just Do!

TDD Best practices

Practices have been separated into following categories:

• Naming Conventions
• Processes
• Development practices
• Tools

Naming Conventions

Naming conventions help organize tests better so that it is easier for developers to find what they’re looking for. Another benefit is that many tools expect that those conventions are followed. There are many naming conventions in use and those presented here are just a drop in the sea. The logic is that any naming convention is better than none. Most important is that everyone on the team knows what conventions are used and is comfortable with them. Choosing “more popular” conventions has the advantage that newcomers to the team can get up to speed fast since they can leverage existing knowledge to find their way around.

Separate the implementation from the test code

Benefits: avoids accidentally packaging tests together with production binaries; many build tools expect tests to be in a certain source directory.

Common practice is to have at least two source directories. Implementation code should be located in src/main/java and test code in src/test/java. In bigger projects number of source directories can increase but the separation between implementation and tests should remain.

Build tools like Maven and Gradle expect source directories separation as well as naming conventions.

[GRADLE: build.gradle]

apply plugin: 'java'
 
repositories {
mavenCentral()
}
 
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.11'
}

The build.gradle can be found in the GitHub TechnologyConversations repository.

[COMMAND PROMPT]

gradle test jar

You’ll notice that we are not specifying what to test nor what classes to use to create a jar file. Gradle assumes that tests are in src/test/java and that the implementation code that should be packaged to the jar file is in src/main/java.

Place test classes in the same package as implementation

Benefits: helps finding tests.

Knowing that tests are in the same package as the code they test helps finding them faster. For example, examples in this article are in the package com.wordpress.technologyconversations.tddbestpractices. As stated in the previous practice, even though packages are the same, classes are in the separate source directories.

Name test classes in a similar fashion as classes they test

Benefits: helps finding tests.

One commonly used practice is to name tests the same as implementation classes with suffix Test. If, for example, implementation class is StringCalculator, test class should be StringCalculatorTest.

Often, number of lines in test classes is bigger than number of lines in corresponding implementation class. There can be many test methods for each implementation method. To help locate methods that are tested, test classes can be split. For example, if StringCalculator has methods add and remove, there can be test classes StringCalculatorAddTest and StringCalculatorRemoveTest.

Use descriptive names for test methods

Benefits: helps understanding the objective of tests.

Using method names that describe tests is beneficial when trying to figure out why some test failed or when the coverage should be increased with more tests. It should be clear what conditions are set before the test, what actions are performed and what is the expected outcome.

There are many different ways to name test methods. Our prefered method is to name them using the Given/When/Then syntax used in BDD scenarios. Given describes (pre)conditions, When describes actions and Then describes the expected outcome. If some test does not have preconditions (usually set using @Before and @BeforeClass annotations), Given can be skipped.

An example of BDD format for naming test methods would be:

@Test
public final void whenSemicolonDelimiterIsSpecifiedThenItIsUsedToSeparateNumbers() {
Assert.assertEquals(3+6+15, StringCalculator.add("//;n3;6;15"));
}

The whole class can be found in the GitHub TechnologyConversations repository.

Do NOT rely only on comments to provide information about test objective. Comments do not appear when tests are executed from your favorite IDE nor do they appear in reports generated by CI or build tools.

In the example screenshot, both failed tests have the same code inside. The only difference is in the name of the method. Test1 does not give much info regarding the failure. Method name whenSemicolonDelimiterIsSpecifiedThenItIsUsedToSeparateNumbers is much more descriptive and provides information even without going deeper into the log or the test code.

Processes

TDD processes are the core set of practices. Successful implementation of TDD depends on practices described in this section.

Write the test before writing the implementation code

Benefits: ensures that testable code is written; ensures that every line of code gets tests written for it.

By writing or modifying test first, developer is focused on requirements before starting to work on a code. This is the main difference when compared to writing tests after the implementation is done. Additional benefit is that with tests first we are avoiding the danger that tests work as quality checking instead of quality assurance.

Only write new code when test is failing

Benefits: confirms that the test does not work without the implementation

If tests are passing without the need to write or modify the implementation code then either the functionality is already implemented or test is defective. If new functionality is indeed missing then test always passes and is therefore useless. Test should fail for the expected reason. Even though there are no guarantees that test is verifying the right thing, with fail first and for the expected reason, confidence that verification is correct should be high.

Rerun all tests every time implementation code changes

Benefits: ensures that there is no unexpected side-effect caused by code changes.

Every time any part of the implementation code changes, all tests should be run. Ideally, tests are fast to execute and can be run by developer locally. Once code is submitted to version control, all tests should be run again to ensure that there was no problem due to code merges. This is specially important when more than one developer is working on the code. Continuous Integration tools like Jenkins, Hudson, Travis and Bamboo should be used to pull the code from the repository, compile it and run tests.

All tests should pass before new test is written

Benefits: focus is maintained on a small unit of work; implementation code is (almost) always in working conditions.

It is sometimes tempting to write multiple tests before the actual implementation. In other cases, developers ignore problems detected by existing tests and move towards new features. This should be avoided whenever possible. In most cases breaking this rule will only introduce technical debt that will need to be paid with interests. One of the goals of TDD is that the implementation code is (almost) always working as expected. Some projects, due to pressures to reach the delivery date or maintain the budget, break this rule and dedicate time to new features leaving fixing of the code associated with failed tests for later. Those projects usually end up postponing the inevitable.

Refactor only after all tests are passing

Benefits: refactoring is safe

If all implementation code that could be affected has tests and they are all passing, it is relatively safe to refactor. In most cases there is no need for new tests. Small modifications to existing tests should be enough. Expected outcome of refactoring is to have all tests passing both before and after the code is modified.

Development practices

Practices listed in this section are focused on the best way to write tests.

Write the simplest code to pass the test

Benefits: ensures cleaner and clearer design; avoids unnecessary features

The idea is that the simpler the implementation the better and easier to maintain is the product. The idea adheres to the “keep it simple stupid” (KISS) principle. It states that most systems work best if they are kept simple rather than made complex; therefore simplicity should be a key goal in design and unnecessary complexity should be avoided.

Write assertions first, act later

Benefits: clarifies the purpose of the requirement and test early.

Once assertion is written, purpose of the test is clear and developer can concentrate on the code that will accomplish that assertion and, later on, on the actual implementation.

Minimize assertions in each test

Benefit: avoids assertion roulette; allows execution of more asserts.

If multiple assertions are used within one test method, it might be hard to tell which of them caused a test failure. This is especially common when tests are executed as part of continuous integration process. If the problem cannot be reproduced on a developer’s machine (as may be the case if the problem is caused by environmental issues) fixing the problem may be difficult and time-consuming.

When one assert fails, execution of that test method stop. If there are other asserts in that method, they will not be run and information that can be used in debugging is lost.

Last but not least, having multiple asserts creates confusion about the objective of the test.

This practice does not mean that there should always be only one assert per test method. If there are other asserts that test the same logical condition or unit of functionality, they can be used within the same method.

Few examples:

@Test
public final void whenOneNumberIsUsedThenReturnValueIsThatSameNumber() {
Assert.assertEquals(3, StringCalculator.add("3"));
}
@Test
public final void whenTwoNumbersAreUsedThenReturnValueIsTheirSum() {
Assert.assertEquals(3+6, StringCalculator.add("3,6"));
}

This code contains 2 tests that clearly define what is the objective of those tests. By reading method name and looking at the assert it should be clear what is being tested.

@Test
public final void whenNegativeNumbersAreUsedThenRuntimeExceptionIsThrown() {
RuntimeException exception = null;
try {
StringCalculator.add("3,-6,15,-18,46,33");
} catch (RuntimeException e) {
exception = e;
}
Assert.assertNotNull("Exception was not thrown", exception);
Assert.assertEquals("Negatives not allowed: [-6, -18]", exception.getMessage());
}

This test has more than one assert but they are testing the same logical unit of functionality. First assert is confirming that exception exists and the second that its message is correct. When multiple asserts are used in one test method, they should all contain messages that explain the failure. This way debugging of the failed assert is easier. In case of one assert per test method, messages are welcome but not necessary since it should be clear from the method name what is the objective of the test.

@Test
public final void whenAddIsUsedThenItWorks() {
Assert.assertEquals(0, StringCalculator.add(""));
Assert.assertEquals(3, StringCalculator.add("3"));
Assert.assertEquals(3+6, StringCalculator.add("3,6"));
Assert.assertEquals(3+6+15+18+46+33, StringCalculator.add("3,6,15,18,46,33"));
Assert.assertEquals(3+6+15, StringCalculator.add("3,6n15"));
Assert.assertEquals(3+6+15, StringCalculator.add("//;n3;6;15"));
Assert.assertEquals(3+1000+6, StringCalculator.add("3,1000,1001,6,1234"));
}

This test has many asserts. It is unclear what is the functionality and if one of them fails it is unknown whether the rest would work or not. It might be hard to understand the failure when this test is executed through some of CI tools.

Do not introduce dependencies between tests

Benefits: tests work in any order independently whether all or only subset is run

Each test should be independent from others. Developers should be able to execute any individual test, set of tests or all of them. Often there is no guarantee that tests will be executed in any particular order. If there are dependencies between tests they might easily be broken with introduction of new tests.

Tests should run fast

Benefits: tests are used often

If it takes a lot of time to run tests, developers will stop using them or run only a small subset related to the changes they are making. Benefit of fast tests, besides fostering their usage, is fast feedback. Sooner the problem is detected, easier it is to fix it. Knowledge about the code that produced the problem is still fresh. If developer already started working on a next feature while waiting for the completion of the execution of tests, he might decide to postpone fixing the problem until that new feature is developed. On the other hand, if he drops his current work to fix the bug, time is lost in context switching.

Use mocks

Benefits: reduced code dependency; faster tests execution.

Mocks are prerequisites for fast execution of tests and ability to concentrate on a single unit of functionality. By mocking dependencies external to the method that is being tested developer is able to focus on the task at hand without spending time to set them up. In case of bigger teams, those dependencies might not even be developed. Also, execution of tests without mocks tends to be slow. Good candidates for mocks are databases, other products, services, etc. Mock objects are a big topic and will be described in more details in a future article.

Typical reasons for Mock Objects

• Real object hasn’t been written yet
• What you’re calling has a UI / needs human interaction
• Slow or difficult to set up
• External resource: file system, database, network, printer
• Non-deterministic behavior

Fake Objects vs. Mock Objects

• Fake Objects
  • Match the original (or intended) method implementations
  • Return pre-arraged results
• Mock Objects
  • Assert the expected values from unit under test

Mock Object Framework

• Provide structure for defining mock objects
  • Sometimes removing need to create a custom class
• Can aid in generating (or autogenerating) method stubs
• Often provide prearraged mock objects
  • For file stream, console, network, printer equivalents
• jMock: http://jmock.org
• EasyMock: http://easymock.org
• mockito: http://code.google.com/p/mockito

Use setup and tear-down methods

Benefits: allows setup and tear-down code to be executed before and after the class or each method.

In many cases some code needs to be executed before test class or before each method in a class. For that purpose JUnit has @BeforeClass and @Before annotations that should be used as the setup phase. @BeforeClass executes the associated method before the class is loaded (before first test method is run). @Before executes the associated method before each test is run. Both should be used when there are certain preconditions required by tests. Most common example is setting up test data in the (hopefully in-memory) database. On the opposite end are @After and @AfterClass annotations that should be used as tear-down phase. Their main purpose is to destroy data or state created during the setup phase or by tests themselves. As stated in one of the previous practices, each test should be independent from others. More over, no test should be affected by others. Tear-down phase helps maintaining the system as if no test was previously executed.

Do not use base classes

Benefits: test clarity.

Developers often approach test code in the same way as implementation. One of the common mistakes is to create base classes that are extended by tests. This practice avoids code duplication at the expense of tests clarity. When possible, base classes used for testing should be avoided or limited. Having to navigate from the test class to its parent, parent of the parent and so on in order to understand the logic behind tests introduces, often unnecessary, confusion. Tests clarity should more important than avoiding code duplication.

Tools

TDD, coding and testing in general are heavily dependent on other tools and processes. Some of the most important are following. Each of them is a too big of a topic to be explored in this article so they will be described only briefly.

Code coverage

Benefit: assurance that everything is tested.

Code coverage practice and tools are very valuable in determining that all code, branches and complexity is tested. Some of the tools are JaCoCo, Clover and Cobertura.

EMMA: code coverage tool for Java http://emman.sourceforge.net/index.html

Eciplse plugin for EMMA: http://www.eclemma.org

Continuous integration (CI)

Continuous Integration (CI) tools are a must for all but most trivial projects. Some of the most used tools are Jenkins, Hudson, Travis and Bamboo.

Use TDD together with BDD

Benefits: both developer unit test and functional customer facing tests are covered.

While TDD with unit tests is a great practice, in many cases it does not provide all the testing projects need. TDD is fast to develop, helps the design process and gives confidence through fast feedback. On the other hand, BDD is more suitable for integration and functional testing, provide better process for requirements gathering through narratives and is the better way of communication with clients through scenarios. Both should be used and together they provide the full process that involves all stakeholders and team members. TDD and BDD should be driving the development process. Recommendation is to use TDD for high “code coverage” and fast feedback and BDD as automated acceptance tests. While TDD is mostly oriented towards white-box, BDD often aims at black-box testing (more info on black-box vs white-box testing). Both TDD and BDD are trying to focus on quality assurance instead quality checking.

References

[1] TEST DRIVEN DEVELOPMENT (TDD): BEST PRACTICES USING JAVA EXAMPLES, 2013.