This is the third chapter of our three-part Q&A blog series with J. B. Rainsberger. In this chapter he adresses questions about „Questions About Test Frameworks„. The first chapter and the second chapter is in our blog in case you missed it.
On June 3, 2020 J.B. Rainsberger spoke in our remote Intro Talk about managing the various kinds of uncertainty that we routinely encounter on projects that involve legacy code. He presented a handful of ideas for how we might improve our practices related to testing, design, planning, and collaboration. These ideas and practices help us with general software project work, but they help us even more when working with legacy code, since legacy code tends to add significant uncertainty and pressure to every bit of our work. Fortunately, we can build our skill while doing everyday work away from legacy code, then exploit that extra skill when we work with legacy code.
If the code base is too old even for any available test frameworks, how you handle it?
**Testing does not need frameworks. Testing never needed frameworks.** You can always start by just writing tests and refactoring them. If you do this long enough, you will extract a testing framework. If you’ve never tried it, then I recommend it! Kent Beck’s _Test-Driven Development: By Example_ included this exercise.
Every test framework began life with `if (!condition) { throw Error(„Test failure.“) }`. If you can write this, then you can build a testing framework; if this suffices, then you don’t need a testing framework. Start there!
If you can execute one part of the system in isolation from the rest, then you can write unit tests. In the early days of web browsers, we could only execute Javascript in the browser, because even so, we could (and did!) write unit tests without frameworks. We merely had to run those tests in a browser window. Eventually, someone decided to run Javascript outside the browser, which made it easier to write microtests for Javascript code. This made it _easier_ to write tests, but we were writing tests long before NodeJS existed.
If you can invoke a function (or procedure or division or block of code) and you can signal failure (such as by raising an error), then you can write tests without waiting for someone else to build a framework.
In addition, you don’t need to write your tests in the same language or environment as the running system. Golden Master technique helps us write tests for any system that offers a text-based interface. Any protocol could help us here: for example, think of HTTP as „merely“ a special way of formatting requests and responses with text. If you have (or can easily add) this kind of interface or protocol to your system, then you can write tests in any language that might offer a convenient test framework. Use Python to test your COBOL code. Why not?
Finally, not all testing must be automated. As I wrote earlier, programmers have a strong habit of forgetting alternatives to techniques that they’ve found helpful. If you don’t know how to automate your tests easily, then don’t automate them yet. Instead, make them repeatable and document them. One day, someone will have a good idea about how to automate them.
You may have to write your own test framework but it can prove a daunting task.
In addition to what I wrote in the previous answer, I encourage you to follow the general advice about building any software with a Lightweight (Agile, Lean, …) approach: build the first feature that you need, then start using it, then add more features one at a time. You don’t need to build a fully-featured testing framework before you start to benefit from it. Start with `if (!assertion) throw Error()` and then use it! The testing framework SUnit was built incrementally. All the testing frameworks you know began from there. You can do it, too. Merely start, then take one step at a time.
You also need this refactoring-without-tests skill, to effectively refactor your tests!
Maybe! I don’t say you _need_ it, but it would probably help you. Your production code helps you to refactor your tests: if you change your tests and they now expect the wrong behavior, then your production code will fail that test for „the right reasons“. It doesn’t provide perfect coverage, but it helps more than you might expect. In that way, the production code helps to test the tests.
There are testing frameworks for COBOL and NATURAL. What could be older?
Indeed, the „framework“ portion of testing relates to identifying tests, collecting test results, and reporting them in a unified way, as well as adding standard mechanisms for „set up“ and „tear down“. We don’t need those things to start writing tests, although eventually we will probably want to have them. **Simply start writing tests, then remove duplication in any way that your programing language allows.** I don’t know what might be older than COBOL or NATURAL.
➡️ Also read our last two Q & A Blogposts with J.B. Rainsberger Part #1 „Managing the Uncertainty of Legacy Code“ and Part #2 „The Risks Related to Refactoring Without Tests„! Follow us on Twitter or LinkedIn to get new posts.
