reflexive cat Photo from Code Construct

Your test suite can’t fail if you don’t have any tests

vs

100% test coverage is REQUIRED

Let’s address the elephant in the room by saying neither one of them is true.

No tests, no problems 🤦🏽‍♂️

This has been pretty much the state of the art for the last number of years in all the software factories (at least the ones I’ve worked on) and I can tell for experience it’s no fun at all. You end up shipping code with close to zero confidence whenever there’s no a full manual regression test. Fair to say this is rarely the case so stress is constantly ramping up.

The scenario described above makes the entire CI/CD process unattainable due to software’s exponential complexity. The more code we write, the more states’ permutations we have to keep track of. This is simply not an scalable endeavor by any measure without involving some form of automation.

Every single line of code HAS TO be covered 😰

Then there’s the other end of the spectrum: 100% coverage dreamland. This goal might be something achievable whenever we’re starting a project from scratch with the mindset it will reach production state no matter what AND will have a lifespan of months or longer. Think real hard for a moment about those two conditions.

Most software projects start either as a side hustle/hobby or an MVP to validate market-fit as soon as possible. Suffice to say neither one of them have unit tests in its initial scope, nor they should if you ask me. At such early stage, there’s no point to heavily invest time in designing a robust architecture -let alone setting a test suite with unit, integration, E2E tests and so forth- without any certainty that the project will live long enough to reap the benefits of said investment.

cost graph

Source: great talk about the economics of software design by J.B. Rainsberger

Then, there is the real world. The harsh reality the vast majority of us face is that we inherit legacy code bases every time we enter a new job. Adding test coverage to an already productive software project is a noble but often futile effort since there’s no real value to the business in it by itself.

Weren’t you in favor of testing Mauri, WTF? 🤨

When a measure becomes a target, it ceases to be a good measure. – Goodhart’s law

If you’ve been reading my blog for awhile now, all of this might sound as a hard left turn coming from me. I get it, but the reason I’m hammering on 100% test coverage absolute is because there’s no real point in doing so.

As stated above: more code = more complexity so % of test coverage by itself is no guarantee of proper operation. Wrong incentives are often the source of many evils. For instance a more Jr developer might be tempted to write tests that add no real value nor cover any actual use cases only to keep coverage “high”.

An old college used to call this practice “testing setters and getters”. That is just going through code via the test suite without actually asserting anything important (a business rule, validation policy or state change). This is a poor use of everyone’s time and it might be much better spent documenting or learning a new framework that will make us more productive.

What are test useful for then? 🧐

Unit tests are a great tool to automate and validate atomic behaviors in our objects 🤖

Let us picture the time-consuming and boring task of launching an app, doing all the ceremony involved in reaching a desired context (depending of the feature/bug we’re dealing with, this might include login/registering and a bunch of non-related steps to what we actually want to check) and finally getting to the view we’re working in to verify for ourselves the changes we made.

Sounds like a lot for any simple change, right? And remember, as the system evolves, so does complexity. All that boilerplate won’t go away, on the contrary: we’ll need to add previous steps whenever a new feature is included and A/B experiments are running.

Unit tests get ride of all that boilerplate by simply invoking the desired object –SUT– with its required dependencies, execute the desired change we want to test and finally asserting the outcome is the expected one. All of this is now automated.

They aid in the clean design of systems 🧼

A popular strategy for writing test is following the given, when, verify approach. Fair enough, when we follow this simple recipe and start noticing that any part is getting larger than it should then we might have caught a code smell.

  • We might be in presence of a god object, requiring too many dependencies to be initialized on the given part
  • Our logic might be scattered around multiple places when there has to occur multiple steps at the when part in order to get the object to the desired state
  • There are way too many asserts in a verify part which could reveal unrelated things being affected by a single state change.

All of this is of course circumstantial and the rules aren’t set on stone but again, unit tests aid in revealing these sort of code smell rather sooner than later.

They enable proper and constant refactoring

Refactor is a widely misused and unpopular term but the truth of the matter is that without automated testing there’s simply no way of constantly performing it. One of the ideals behind well crafted software is, as it evolves, more tasks become automated. It shouldn’t need more human power to directly monitor each and every single step we make or feature we add.

Unit tests should, at the very minimum, validate the simplest, most boring use cases and context.

Conclusion

As most things in life, there’s rarely an absolute truth and testing isn’t the exception. We should aim to apply them whenever they make sense to do so. I want to leave you guys with a quick reflection below I remember all the time when dealing with time-crunched folks.

Uncle Bob tells, at one of his books (don’t remember if it was Clean code or The clean coder), the history of the first doctor to realize that hand washing before surgery could help saving lives. His colleges at the time routinely discarded this recommendation, claiming “they didn’t have time to do so because were too busy”.

The reality is that principles should remain as constants as possible at all times. These automatics systems are the ones that will make our lives easier tomorrow. We’ll thank ourselves by doing the right thing early on.