Why DevOps is failing: It’s Not You, It’s The Tools

We all know the adage — “Bad workers blame their tools.” But here’s five reasons why the software tools involved in DevOps in general, and the CI/CD pipeline in particular, have become part of the problem rather than the solution. 

1. They take too long to deploy

This, it must be said, is a massive irony. When software development started down the agile path in the last millennium, it was to break with waterfall models – not because these were wrong per se, but because they were too slow. Just as a two-to-three year development cycle is far too long as the (tech) world will have changed, so is a similar period to deploy a tool across an enterprise. 

As well as the fundamental timescales, decision making attention spans simply don’t stretch that far. The result is that tools are often half-deployed before the next wave of tools comes in, adding to the complexity rather than reducing it. 

2 . They are still being sold as magic bullets

In just about every solution-oriented webinar I have ever been on, the “but it’s not a magic bullet” line is used (fair enough, if nobody else says it, I will). Nonetheless, we’re still being told that software tools are the answer – they can transform how you deliver software, they have everything you need, and so on. 

It would genuinely be lovely if these marketing statements were true, but they are only partially correct. Unlike financial instruments, however, tools vendors are not required to include disclaimers on their front pages – “Success requires customer due diligence and change management,” for example. 

3. New ones appear all the time

This is as ironic as 1. A reasonably standard path is when smart engineers working for a bigger company express their frustration with their own pipelines and end up building a cool solution. Equally smart, they realize they’re on to something, form a startup, find a couple of customers and go get some investment, spending not unreasonable sums on marketing and, indeed, analyst advisory. 

In the DevOps space, which is inevitably full of developers, this situation is more common than (say) networking infrastructure – people don’t generally start telecommunications companies as side projects. The trouble is, this falls into the trap of assuming that nobody else out there has ever solved the same problem, creating additional sides up the same mountain to work through. 

4. They’re sold without controls

In my experience of deploying software development tools (which isn’t bad), they tend to fall into two camps. Some are tactical – a little performance testing widget, dashboard or integration capability. The rest, almost to a fault, have some kind of strategic expectation on how things are done. Development environment managers need teams to work with the notion of environments. Testing tools need a testing methodology. 

Whilst many tools may have an associated framework, way of thinking or approach, they don’t often lead with this. However, let’s be clear, all will require management-level changes in how things are done, or even higher. Exceptions exist – some vendors sell at board level – but see also 5. 

5 . They’re sold to, and bought by engineers

This does have overlaps with 4, but it is more about the go-to-market, which is often a freemium or open-source-plus model. Essentially tools are presented as tactical, in the direct knowledge that sooner or later, they will have to be perceived as strategic. 

In the industry, this sales model is called “land and expand” – just get in any which way, and grow from there. However, the reality is more “land, expand and then start to hit problems” – organizations end up with pockets of tooling deployed in different ways, and a very fragmented environment. Vendors, as well, can claim customer logos but then struggle to turn tactical deployments into more strategic customers. 

All of these issues can be summed up as, “Strategic tools, sold and bought tactically.” I’m not going to point the finger exclusively at vendors (though I am reminded of a conversation I had about sales tactics, in a different domain. Me: “Why do you do it that way, it’s not nice.” Them: “Because it works.”). 

And then newer capabilities, such as feature flags, are presented as making things even better, when in fact they’re the opposite. I think feature flags are great for the record, but sold/bought without controls, they are a route to even more fragmentation and despair. 

Is there an answer? Given that software tools vendors are hardly going to adopt a “let’s sell less and qualify out any organization that isn’t organized enough to use our stuff” approach, the buck has to stop with end-user organizations, specifically with their engineering teams and how they are managed. Whilst vendors need to be held to account, it takes two to tango. 

The clue is with the word “tools” itself. We need to stop thinking about tools like we might see screwdrivers and mallets, and start seeing them as we might manufacturing systems – we’re building a software factory, not a hipster-esque workshop.

Organizations need to see their internal software supply chains in the same way they might a fabrication plant (third irony alert – that’s exactly the point made in the Phoenix Project, written 15 years ago).

This also directly means restricting developers from making unquestioned tooling decisions. I’m sorry, but I don’t fully buy the “developers are the new kingmakers” line, as I speak to too many operations and infrastructure people who have to shovel up the piles of manure they create if left unchecked. 

We all need to be protected against ourselves – and the consequence of an anything-goes-in-the-name-of-innovation culture is a series of fragmented fiefdoms, not great empires. The “prototype/PoC becomes the platform” issue is as true for tooling as it is for bespoke software if time is restricted, which it inevitably is. 

From a vendor perspective, this means focusing on a smaller list of vendors, potentially giving them more money, and working with them to understand the cultural changes required to adopt their capabilities fully. 

And enterprise leaders, as long as you allow the situation to pervade, you are encouraging inefficiency and waste. Good things only come out of chaos by exception. On the upside, particularly relevant in these recessionary this, this fragmentation creates major opportunities to reduce waste and increase efficiency, releasing funds for innovation. 

Above all, we have created a jungle by not paying attention up front. It’s never too late to start tackling this, but bluntly, if all businesses are software businesses, they need to start acting like it.