Dashboards vs. Trashboards

Say you’re sitting in your car and you see the water temperature is 95 degrees Celsius (203 degrees Fahrenheit). What do you do? Then it rises to 105 degrees Celsius (221 degrees Fahrenheit). What do you do? Do you stop the car because it’s overheating?

It depends. In a modern car, this can be perfectly normal; however, in a prewar car, for instance, it’s not. We’re not all experts on car technology, so that’s why the car manufacturer puts a redline in your temperature indicator. That’s the temperature where you have a problem. Same holds for the redline of your rev-counter. Most cars will electronically prevent you from going beyond that redline. Some modern cars even have the redline change depending on engine temperature. You probably couldn’t care less and may have no idea why all this is relevant. You’re not a trained mechanic, nor do you want to be. You don’t need to understand the complex model that’s behind all this, because you can trust it.

And that’s a perfect example of an actionable dashboard. It tells you what to do (and what not to do) and actively stops you from doing the wrong thing. Extra meters with g-forces, boost pressure, and so on are useless for 99% of drivers (and for most of the rest as well, for that matter).

That’s the definition of a proper dashboard. Clear and actionable, especially for non-experts. It’s based on a model: this is good and that is not good.

But that’s not what the typical business dashboard looks like. Most often, it’s a bunch of nice-looking graphs and numbers. More graphs are supposedly better. But the interpretation needs to come from experts.

Most business dashboards are therefore mere noise. So perhaps it’s better to call them “trashboards” instead.

That’s exactly the problem in software assurance. Why are we measuring software quality in dozens and even hundreds of metrics? Some parts of the code are really long, is this good or bad? McCabe is 17, is this good or bad? Maybe you have no idea who or what McCabe even is.

Let’s be clear: this is exactly the same as informing you that your car’s water temperature is 105 degrees Celsius without the redline. Unless you’re a car mechanic and know what car you’re operating, this is absolutely useless information.

Measurements must be based on a model, otherwise you need a combination of luck and a whole lot of experts. These experts should be building your software, not turning your data noise into something that’s perhaps useable, probably even only for that precise situation.

You can save yourself a lot of effort by no longer measuring things without a model behind it. It’s useless. Throw away the tools that do: they may be even free, but they cost a lot money. If your water temperature gauge doesn’t have a redline, I’m sure you would ignore it as well. Why bother looking at metrics, spending hours and hours on building a fancy “trashboard?” Use a proper model, it really is the only way forward. It takes away discussion, interpretation and makes software quality actionable. It helps to focus, as a proper model understands that it’s OK to have some long pieces of code (but not too much) and some complex parts of the code (but not too much of that, either). A proper model is based on deep understanding of software engineering and is properly benchmarked against a large representative database of code.

Want to know more about what a good model should look like? Read this blog from my colleague, Magiel Bruntink, Head of Research at SIG.