Red, Green, Refactor: An introduction to TDD

Recently at Enfuse, we are actively hiring and part of our interview process is a mock-pairing Test-Driven Development (TDD) session. During these interviews, there has been a wide variety of experience with TDD and unit testing, which has prompted me to write a basic article about what TDD is, some of the benefits, some best practices and tips, and how you can get started super easily. 

Testing 101

If you have never heard of testing, there are many types of tests. They all involve writing code that executes your code and asserts that it has the intended effect. The most basic test is a unit test, which tests a single class or function. Building on that you have integration tests that cover a larger portion of the system (generally including an API and persistence layer) and end-to-end tests that cover your entire application and the flow a user may go through. If you want to read more about testing, search for testing articles about inside-out vs outside-in testing, the testing pyramid, and diamond. 

What is Test-Driven Development?

Test-Driven Development is a test-forward paradigm in which you start by writing tests (normally unit tests) that define the expected behavior of your system and only write the code necessary to make a test pass. This is an approach where the tests drive the development of your code/product/platform. 

Software development expert Martin Fowler sums up the TDD paradigm with three basic rules to follow [2]:

• Write a test for the next bit of functionality you want to add
• Write the functional code until it passes
• Refactor both new and old code to make it well structured 

Another simple way to imagine this TDD approach is to remember the mantra “Red, Green, Refactor”: 

1. RED: Start by writing a test. Since you have written no code to implement your code it should fail. This test should be a small section of expected user behavior and output. For example, if you were implementing a “GetMovie Service” your first test may be that it returns the title of the movie.
2. GREEN: Write the minimum code to make the test pass. 
3. REFACTOR: Now that you have defined the behavior and are confident that your code is working, you can safely refactor. If you see areas of code duplication or areas for simplification or extraction of static methods: now is the safest time to make changes. If your tests stay green you know your refactor was successful and the behavior is unchanged. 

Benefits of Testing & TDD?

Any benefit you get from testing is a benefit of TDD. Tests act as a form of insurance, like any insurance there is good and bad insurance. Well-written tests help protect you against inadvertently breaking existing behavior in your system as you continue development. Good tests (which test behavior not implementation) allow you to confidently refactor your system, since if your tests pass you know the behavior has not changed. Tests help catch bugs and mistakes earlier, and the earlier you catch an issue the cheaper and easier it is to fix. Tests act as a form of living documentation, tests with good names and values help you understand how you expect users to interact with your system. 

TDD helps you write better tests. By writing the tests before you write the code it forces you to understand the use case you are trying to solve and the behavior you expect. It encourages leaner code as you are writing only the code you know you need to create the behavior you expect. It pushes us to focus on production-ready code and deploying it. TDD minimizes time spent in test-case design and instead on use-case design. In addition, TDD can drastically reduce the iteration cycle of developing new code. TDD puts the focus where it should be: on the user and developing features as they will use it.

When and when not to TDD?

As this graphic illustrates, in a perfect world, you would start your code base with TDD. If you are in an existing repository your best bet is to start writing new features with TDD and write new tests only as you touch that functionality. This is for two main reasons: firstly, pragmatically it’s easier to just write tests for new features it’s easier to agree to; secondly, if you aren’t touching that code the relative value of a test is low. Tests are most valuable in code that is changing often as it guarantees behavior, if the code is never going to be changed – well that’s another type of guarantee in the behavior. 

So when should I not use TDD?

Surprisingly, some things do not need to be test-driven, or more accurately do not need to be tested. Any static pages, boiler-plate, and templatized code do not need to be tested (e.g. I would not test a model object, DTO, getters or setters, or Java JPA class that has no custom logic). One-off or limited-use scripts do not need to be tested. 

Another time when not to TDD is when you don’t know enough about the user behavior or the technical implementation. In either of these cases, there may be room for a “SPIKE”. A spike is a time-boxed period of research to learn about a problem or possible solution. The result of a spike is generally working, if scrappy, piece of code, or understanding about a problem. Once you have that, you should “throw away” your spike code and start again using TDD knowing the solution you are going for. This may seem like doubling up your work, but as an engineer would tell you after you build – well almost anything, “I’d build it differently next time.” We are giving that opportunity at the start and believe the end product will be much better when you start with the tests and use the knowledge from your spike. The process would look like this:

How to TDD?

If you have gotten this far you are probably interested enough in TDD (at least to keep reading) but wondering how you can start TDD. As mentioned above the mantra is RED-GREEN-REFACTOR. TDD is that simple, write a failing test, make it pass, refactor and make sure the tests still pass. It is important to be disciplined about writing *just enough* code to get a test to pass. This allows the rest of the TDD flow to progress effectively, and creates consistent, “self-testing” code. TDD is simple to understand and hard to master and if you have never written a test before and don’t know where to get started I have shared some resources below about libraries for Java and Javascript. But you can start by creating a new method that implements an existing method and throws an error if the result is not what you expected. If you want a basic answer I like this stack-overflow answer and you can see some examples at the bottom.

To help you on your journey to mastery here are some tips from Enfuse experts:

• Run the test before you implement the test to avoid false positives (where there is a mistake in your test)
• Ensure the test passes before you start refactoring
• Keep it simple, minimize dependencies, and mocks
• Follow a coherent testing strategy throughout your codebase (whether that is the Testing Pyramid, ice-cream cone, diamond, etc)
• Feel free to split your code into smaller mini-classes as this will help both testing and maintaining code long term
• When possible, avoid mocks in unit tests and especially avoid mocking your services (obvious caveats include external services where you pay to trigger and databases where the in-memory option is not feasible)
• Tests should reflect intended user behavior and so act as documentation. It’s important to give them sensible names and values that are possible from the user.
  • Test names are so important they deserve their section, when you run all your unit tests it will show you which test failed and why. Ideally, looking only at the function name you would know which function has failed and what behavior is not working as expected. A good start is:
    • functionName_givenCondition_returnsExpectation, so in our above example: getMovie_givenAvatar_returnsSimpleMovieDetails
    • If possible, consider a Gherkin[1] format for names & test descriptions
• It can help if you run through the test in your head before writing the test, what is the context and what is the user expecting
• Make sure you write the test first (duh)
• Try not to stub your own, if you find yourself doing this there may be bigger problems within your code, focus on mocking external dependencies. 
• Avoid coupling your tests and test data with implemented behavior. Should be able to refactor without breaking the tests. 

Frontend Testing

Using TDD for front-end development is somewhat of a controversial topic. Like any type of software development, there are advantages and costs. However, specifically, In the context of frontend development, arguments against TDD often follow this logic:

“Unlike simple backend APIs, pipelines and services, I don’t know what UI elements or behavior I am going to use when I implement the UI, TDD would be a waste of time”. 

To this point, I would argue:
1. Though not a luxury all teams have, a good design should have MOST elements worked out beforehand, and with the gaining popularity of UI components a la React, Material UI, etc – this is far less of a concern.
2. At the end of the day, if we have something like a contact form – this has inputs, outputs, and behavior. Therefore – not only can it be effectively tested with a TDD approach, but it also greatly benefits from it, especially since the UI is generally more susceptible to regression issues.

In terms of frontend testing, it’s important to test functionality not form. This means you should test whether a button is visible, clickable and the behavior, but should not test the position, color, etc of the button. TDD is best driven by user behavior, it should not drive UI design. 

Downsides of TDD

Of course, no framework or implementation is perfect and I don’t want to leave you thinking that TDD is perfect. Much like pairing, testing – and test-driven development, is a skill that you can better at. You will learn the right way to implement classes, services, and interfaces but in the meantime, there will be a learning curve. Even before that though, there are some potential downsides to well-done TDD. 

• Since tests are like insurance, they are dead weight to your live product. What I mean by dead-weight is they do not directly add value or help your customer. They can help you feel confident making changes and deploying your code (indirectly helping your customer). 
• TDD may initially seem to slow down your development cycles, as it first comes across as writing ‘extra’ code. Though this can initially be true to the extent any new process creates overhead, it’s a style of development that pays huge time dividends with a little practice and discipline. When implemented consistently, TDD can go a long way in preventing ‘throwaway’ code, as each step and behavior of code is derived from the prior piece. As mentioned before – this helps lead to what’s often called “self-testing” code. This benefit is even more pronounced when successfully applied to large teams! Over time, you will see the benefits of TDD with fewer bugs, faster resolution of bugs, and more readable & extensible code.  
• Tradeoffs when testing integrations can either introduce bugs (if tested too lightly) or increase the cost of changing 3rd party providers (if tested too extensively). Especially if your implementation needs to change to match the 3rd party provider as your interface was not as generic as you’d hoped.
• Tests, especially frontend tests, can be coupled with bad implementations making it harder to implement the right behavior as you have to change the tests too. Ideally, when writing tests you would catch some of this and try to test behavior instead of implementation. 
• Longer deployment times as you wait for the test suite to complete. It’s very important to have a fast test suite (faster than it takes you to make a cup of coffee and get back to the test). If your test suite is taking more than 10 minutes, you either have too complex an app, or an overblown test suite or there is an opportunity to parallelize tests to improve performance. Having a fast, reliable test suite is essential to well-run teams.

All that said, I would recommend finding out more about TDD and your use case to see if it’s a good fit for you. In my opinion, as you get better at TDD it will help you write better code, and better tests; resulting in a cleaner, more maintainable code base that is easier to add on to and plug-n-play 3rd party libraries. Though there may be some painful learning curves in the early days as you hit your head on failing tests, testing library issues, and intermittent failing tests, and realize you were testing your implementation and not behavior. My advice: pause, reflect, learn and try again. 

Tools for Testing

Now everyone has their favorite tools, frameworks, and libraries for testing and many of these are language-specific I will upset a lot of people by suggesting any so I will start with those that I am most familiar with, for Java and Javascript and as you suggest some I will append this list with your suggestions.

Java

• Mockito
• JUnit
• Hamcrest (originally Java, now has Python, Ruby, GO… matchers)
• Spring Testing

Javascript

• Jest (React)
• Karma (Angular)
• Selenium / Playwright (End to end)

Other

• PyTest (Python)
• tSQLt (SQL)
• TerraTest (Terraform)

Conclusion

Tests and TDD create a code baseline with insurance, protecting your code from unintentional breakage. In my opinion, TDD is the best insurance carrier, not only does it give excellent tests (insurance) the added benefits of cleaner, simpler, more maintainable code. Less time developing and more code that ends up in production. So, that said – go out and RED, GREEN, REFACTOR. 

Notes

[1] https://martinfowler.com/bliki/TestDrivenDevelopment.html

[2] Gherkin is a specific English-like language, that you can use for writing requirements and acceptance criteria. Gherkin has four main sections, Scenario (function), Given (context of the scenario), WHEN (action the user takes), THEN (testable expectation). Read more here: https://lazaroibanez.com/agile-development-use-gherkin-to-write-better-user-stories-62bbff72145a