In Defense of Cargo Culting


In discussions on tech forums, you’ll sometimes hear people refer to cargo cults Cargo cult - Wikipedia en.wikipedia.org/wiki/Cargo_cult . The term was originally coined to describe indigenous tribes who came into contact with more technologically advanced colonizers (most famously, the US military in the South Pacific during World War II). After the colonizers left, the tribes would try to mimic their “rituals”, building airstrips and marching in formation in a futile attempt to attract more of the same supplies the colonizers summoned from the sky.1

With regard to software, cargo cult programming Cargo cult programming - Wikipedia en.wikipedia.org/wiki/Cargo_cult_programming (or just “cargo culting”) is a similarly pejorative term that describes someone who uses a technology or follows a pattern without understanding why. Often, it’s related to over-engineering: building software to handle a scale the developers anticipate, rather than the scale they’re actually at. Another common accusation is that developers mindlessly mimic large tech companies, because if something works for Google or Facebook then it must work for them too.2

There’s some truth to all that, but the discussion tends to myopically focus on how well a particular tool fits a problem’s engineering constraints.

Popularity

Cargo cults tend to form around popular or trendy tools. Developers see people around them using something and conclude that there must be a good reason, so they should use it as well. Or so the thinking goes.

Some people choose based on the number of job opportunities a tool affords (“résumé-driven development”) and while that gets a bad rap, it’s a perfectly rational way to choose a stack for a personal project.3 On the flip side, for companies hiring people to write code, the ability to find developers who are already familiar with their stack might be more important than picking the absolute best tools for the job.

There are a lot of advantages to using popular tools! There’s probably documentation, tutorials and Stack Overflow answers readily available. Practices have been tried out and established. Plugins have been developed. Bugs have been found and fixed. If you’re working on a team, it’s much easier to arrive at a shared understanding of what’s idiomatic, which patterns to use and what pitfalls to avoid. It can also help when onboarding new members, who can rely on an external community rather than the (lack of) documentation for a bespoke tool or niche framework.

One important consideration here is whether something is is popular or merely trendy. A decent metric for this is longevity: if it’s been already around for a while, then it’ll probably stick around for a while longer Lindy effect - Wikipedia en.wikipedia.org/wiki/Lindy_effect . By picking a tool that a lot of people have used for a long time, you can rest assured that if you run into an issue, someone else will have run into it before you, and (hopefully) will have also written words on the internet about how to fix it.

Inexperience

Another reason people rely on popularity is inexperience: it’s not clear to them what the actual best solution is, so they just pick one and get to work.

This is pretty common! Most of us have one or maybe two areas of deep expertise. But many projects we take on will involve new domains, or unexplored corners of familiar ones. Often, we specifically seek out such projects, since solving the puzzle sharpens our skills. This means we’re constantly being placed in situations where we have to make technology choices without fully understanding the constraints or the consequences.

But before we develop that domain knowledge, we still have a job to do. At that stage, cargo culting is a reasonable way to make a decision. We don’t have the experience to make an educated choice on our own, so we turn to the wisdom of the crowd. A more charitable way to describe this sort of cargo culting is “following best practices Best Practices Exist for a Reason If you've ever used node.js, you've probably also used Isaac Schlueter's npm, the node package manager. By maintaining a central repository to which authors can quickly publish their JavaScript modules, npm has made it easy to get started building node apps and its popularity has exploded over the past year. Unfortunately, two months ago, [the hashed passwords of all npm accounts were leaked.](https://gist.github.com/2001456) npm uses CouchDB for its backend, and the default security model is to grant administrator access to all databases, but only when connections originate from the same machine. It appears that in this case, the CouchDB server was made accessible to the world over HTTP with the default access settings left in place. tomdale.net/2012/04/best-practices-exist-for-a-reason/ ”.

Knowledge Reuse

Conversely, people sometimes use tools they know well to solve problems for which they’re ill-suited.4 This is the same sort of pragmatism that leads people to choose unfamiliar tools: they have a job to do, and getting it done quickly (or at all) is better than getting it done perfectly, eventually.

In Choose Boring Technology Choose Boring Technology mcfunley.com/choose-boring-technology , Dan McKinley uses “innovation tokens” as a metaphor:

Let’s say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet. Clearly this model is approximate, but I think it helps.

If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens. If you choose to use service discovery tech that’s existed for a year or less, you just spent one of your innovation tokens. If you choose to write your own database, oh god, you’re in trouble.

The thing is, you don’t just spend your tokens on technologies that are new and shiny. You pay a price whenever you use something with a lot of unknowns to you, personally. Sometimes, that’s unavoidable — if you’re building a web app, you probably need a database even if you don’t know the first thing about SQL — but if you can avoid it, you should.

In practice, this means reducing risk by choosing tools you know well No One Ever Got Fired for Choosing React | jakelazaroff.com If you spend a lot of time on Hacker News, it’s easy to get taken by the allure of building a project without a framework. jakelazaroff.com/words/no-one-ever-got-fired-for-choosing-react/ . This is why so many people tend to use the same stacks they use at work for their side projects: sure, it might be overkill, but they already know exactly how to use it.


Ultimately, people choose technologies for a lot of reasons. From afar, it’s easy to judge technology choices based on how well they fit the engineering problem. But “the problem” is usually multidimensional, and the engineering dimension not always the most important.

Footnotes

  1. I wish there were a less colonial name for this. Anthropologists now argue that the term and its connotation says more about Western ideology than the people it allegedly describes Who Is John Frum? In the 20th century, anthropologists fell over themselves to study the “cargo cult” phenomenon in the South Pacific. But was it really a new religion—or just a Western fantasy? www.topic.com/who-is-john-frum .

  2. Some examples, like using Kubernetes for a blog or React for a marketing website, have basically become memes dex on Twitter “Deployed my blog on Kubernetes” twitter.com/dexhorthy/status/856639005462417409 .

  3. Side projects should be entirely exempt from the cargo culting discussion. Why do you care about the stack of this app I made for fun? If you want me to build something in a specific way, hire me and pay me money.

  4. Technically, this isn’t cargo culting, but the resulting discourse tends to take the same shape so I’m including it here anyway.