Attempting, and to some degree, failing, to prevent a user from accruing technical debt

We strive to do right by our customers. Sometimes this involves telling them unpleasant truths about choices they are going to make in the future, or have made in the past. I try not to overly sugar coat things … I won’t be judgemental … but I will be frank, and sometimes, this doesn’t go over well. During these discussions, I often see people insisting that their goal is X, but the steps Y to get there, will lead them to Z, which is not coincident with X. That is, they are optimizing for a different thing. I ask about this. As often is the case, the optimization is constrained, so Z may be the best they can actually achieve. I understand that. I point out the differences, and ask them to help me understand what are the most important features, so we can look at lower level optimizations that might be able to move closer to the business goal. X is usually a business goal. Z is usually an endpoint which isn’t at X, and there are ramifications in terms of being there at Z versus being at X. If those ramifications will have little to no impact on the business goals and objectives, then Z is fine. If they do, then what are the most important factors to consider? What will have the largest overall impact upon business? Basically I am asking what do you need versus what do you want, and then how can we deliver on what you need, or a very close approximation to it? So where does technical debt accrual come in? Technical debt is first and foremost an opportunity cost. It is the cost of choosing an alternative path (the steps Y' when you should choose Y). The debt is back loaded, in that you don’t start paying interest and premium until it really starts to matter. And then the technical debt has both a large ticking clock, and a real actual expenditure associated with it. Technical debt arises in picking other optimizations over the one you actually need for your business. It arises in making alternative choices to the choice you should make for reasons that do not wind up returning much on the difference in up-front cost of those choices. It arises in using other non-tangible, non-measurable, and to an extent, largely irrelevant parameters than suitability to task as the basis for a decision (or step Y). We see technical debt accrual happen when the optimization changes from “lets build the system we need” to “lets build the cheapest possible system.” Or “lets build the system we need” to “lets build it in the cloud.” I am not saying the cloud is bad here. I am saying that each new endpoint changes what you should expect as the outcome, and your performance, workflow processes, and costing model will also change, sometimes drastically so (more often than not, not for the better). I could write a few posts on people whom have made such decisions, and then run away screaming later. But that’s not the focus of this. As you make decisions, your paths to these endpoints, and your accumulation of this technical debt, that you are going to have to repay at some point (you can’t escape this), often times, the objective gets lost in the process. I won’t go into specifics, but I’ve seen (recently) some terrible payback of technical debt, and corresponding negative expectations as a result of this. All because of specific front end optimizations that sought to maximize or minimize one particular feature, which, over the lifetime of the project, was basically irrelevant. But it scored political points for the people. The backloaded payback … not so much fun. Getting calls at 2am to help fix (self inflicted) problems that the technical debt payback resulted in? Also not so much fun. Basically I am saying, don’t optimize for the wrong thing. Keep the business objectives in mind when you chart out the path you need. Avoid hyperoptimization of one aspect, as it will have ramifications further down the line. In grad school, my advisor would have me hyperoptimize our parts ordering (I built many of our machines by hand in the early 90s). I would spend a few hours to shave a few dollars. One thing I learned during this time was that not all parts are created equal. That price optimization can have surprising (and very negative) impacts when you want warranty coverage, or support, or … . I learned that this focus on one aspect (price) when we cared about another aspect (performance and reliability) was misguided. I also learned that brand names were irrelevant. What mattered was the commitment of the vendor to provide a good product, stand behind it, and help you when you needed it. You don’t get that when you hyperoptmize. There is a cost to every decision. I spent many many hours making the initial decisions, and many weeks/months regretting some of them. All in all a bad use of my time. Yet I was “rewarded” for this. I see many of the same things happening now. With many of the same outcomes. And I try to warn people of the risks of their choices. As long as the go in with open eyes, hey, they can make their choices. I don’t like the blame game when the technical debt comes due. So I like the paper trail my emails provide. And I try my best to provide some mechanism so they can escape the ravages of the worst of the debt. But I don’t always win those arguments. And sometimes they have to pay the debt, in full, and quite quickly. “I told you so” isn’t the right course, but “hey, here’s a plan to mitigate these issues” is.