A git codebase off the shelf — A cautionary tale
Imagine a Github repository was a physical product in a store. Just like a phone, TV or any other piece of technology in a nice package.
On the main shelf, there is a Python code-base selling at $99.9. Seems pretty nice for the price. I could easily use it for my E-Commerce startup.
All product specifications and decisions seem documented right. The Readme file is extremely attractive with a beautiful logo and descriptions. There are many checklists. This code has hundreds of features! I am getting excited!
React+MobX, Flask, Celery, Postgres, Docker, AWS-ready. The UX/UI is good too. I try to check it out a bit, open different files, try to run a few commands from the terminal. All good so far. There is even a 3-page wiki!
Seems like a straight-forward buy!
But wait… there is no guarantee in the package! How do I know this product (code) works? The list of features is super-long, and I don’t have 7 days to test this thing (I never used it before. I wish I knew how to test it properly anyway).
A store clerk noticed my suspicion and he tells me “It was purchased and used by my girlfriend yesterday for her college project and it worked really well. It has 5,000 stars and likes in our store. Very popular code if you ask me!”
Should I trust this? It seems to have worked for many people. Is popularity relevant here at all? The number of stars seems more relevant for Instagram than for code.
I ask the clerk: “Is there a guarantee for this product”?
The clerk says: “How do you mean a guarantee?”
And I reply: “I mean, automated tests. The only way to guarantee software works for specific use-cases.”
The clerk replies: “Well, unfortunately, no. The team that developed it didn’t write any automated tests. It was developed as a YOLO project and tested by the product owner and developers. There is a linter, though, and the team did code-reviews with +1 comments in Pull-requests! They did use Jenkins and CI, though.”
And I tell him promptly: “Man, there is no CI/CD without continuous testing. You should seriously read the Continuous Delivery book and get up to speed to advise customers properly.”
It seems there is no way I can objectively validate and know this code works. I get frustrated since I expected it to be an easy purchase. Why are software engineers so irresponsible?
The question now is: Who do I trust more, humans or automated tests?
I tend to think humans are fallible and irrational. I intuitively trust emotionless machines more than humans, at least in regards to static code analysis and code quality.
At that point, the clerk mentions another code-base: “Look, there is this other code. It’s not that sexy and it’s $120 since it was more expensive to produce. It’s a Java + MySQL Server database. Standard stuff. However, there is a 3-year guarantee since it has a strong layer of automated tests and code-coverage is above 80%. There are BDD executable specifications using Cucumber. All major user stories are automatically tested.”
And then he adds: “It has a similar set of features. It matches your needs and it’s a very SOLID product.”
At that point, I realize that I really need a system that I can build upon, and having sexy technology is less important than usability (maintainability).
I take the product for a spin. All the code looks nice, it follows SOLID principles. I open a terminal and execute the test suite. I feel a warm sensation of safety seeing the feature-tests execute all-green.
At this point, I confirm my trust in machines over humans. The clerk’s girlfriend was not a relevant tester anyway, and I will never meet the product-owner of the Python code-base in Zimambwe to ask him if he did a good manual test before shipping.
I decide to buy the Java code with tests, even with a higher price. After all, I can always return it if it doesn’t work by the executable specifications. Guarantee wins!
…
As good old Michael C. Feathers would say: “Every code without tests is legacy code”. Never trust anyone guaranteeing code works without automated-tests, especially when you inherit a code-base.
Inheriting a bad code-base is the equivalent of buying a broken iPhone without a guarantee. You are officially F*Ck!d.