The Infinite Manager: Duplication is evil

The following diagram shows how:

It’s extremely hard to manage an information-based company without creating a disorganized mess. One insidious cause of such mess is duplicated information.

Software engineers know this well. They use the term duplication is evil. They insist that “every piece of knowledge must have a single, unambiguous, authoritative representation within a system—a Single Source Of Truth.”

We have already described how Genchi Genbutsu is the most powerful technique we know. The principle duplication is evil runs close behind it. Whenever we have violated it, we have ended up in a world of pain. (And we have seen many companies that have been broken by violating it.)

Why duplication is evil

The following steps describe how a seemingly innocuous act—the duplication of information—backfires and creates a wasteful tangled mess.

Step 1: Information gets duplicated

The problem begins when someone allows the same piece of information to exist in more than one place, either because…

1. They create a new version and don’t delete the old one. For example, they collaborate on a Microsoft Word document by sending the file back and forth by email. Or they create a new document (like “Plans for ABC project on January 10”) without checking first whether there’s an existing one that could have been used (“Plans for ABC project”).
2. They allow the information to exist in multiple places or formats. For example, someone might take a process that’s written in Microsoft Word, then turn it to a checklist in a to-do app, and also turn it into a nice-looking PDF version for clients to see, and then turn it into a PowerPoint slide for the training course for new team members, which they then record as a video. That’s five different formats.

Step 2: Without a single standard, you no longer have a winning control

The duplication becomes a problem when someone then updates only one version of the information, either because they forget (or don’t know) that multiple versions exist, or because they don’t have the time or discipline to update them all. The different versions then evolve separately, drifting apart. Neither version is correct; each may contain useful updates that the others don’t have.

Step 3: A substandard version gets used (because all versions are substandard)

Someone then uses a version that didn’t include important updates. For example, a prospect receives a price list that contains old prices; a client is sent an old version of a contract that’s missing an important clause; or a document is sent to the recipient’s old email address.

Step 4: Deduplication becomes hard

As time passes, it becomes almost impossibly difficult to consolidate the different versions. It becomes hard even to work out which version should be the official one (what we call the “canonical” one), and which versions should be cannibalized into it. Deduplication becomes an arduous game of spot-the-difference. For each difference, you need to identify which variation should be the winning control and which should be deleted.

If someone lacks the time or discipline to avoid duplication, they almost certainly lack the time and discipline to deduplicate—because it’s much harder—so things move to Step 5…

Step 5: The company breaks

It’s surprising how quickly duplication leads to a shambles. Folders become cluttered. No one can find things. Team members, understandably, avoid the documentation. The best way to find information becomes via word-of-mouth notifications.

Most companies have an upsetting amount of duplication. There’s so much, they become blind to it. You can tell such companies because they have the following symptoms:

• The company has call centers the size of small cities, staffed by people whose job it is to cover for the fact that the company’s flows don’t make sense. The call center staff members are just as in-the-dark as the customers—because they are having to struggle with the same documentation and flows.
• Most work gets done via emails, phone calls, meetings, and instant messages—things that most people say they want less of.

When you’re in the midst of all that mess, it becomes hard to see how much waste there is. Often, the extent of the problem becomes clear only when the business is disrupted by a company that does a good job of documentation, like Amazon. Often, the disruptive company replaces a huge amount of activity with a small number of great flows. (This article describes how each department in Amazon documents its inputs and outputs.)

Avoiding duplication is the best way to prevent a company from deteriorating in this way.

Be allergic to duplication. Be offended when you encounter it.

“Always leave the campground cleaner than you found it”—The Boy Scout Rule

Two examples of how we have suffered when we have allowed duplication

Here are just two examples:

• On our website, there’s a sentence about how many clients we have worked with. It is mentioned on several pages. Sometimes, when we updated one version of it, we forgot to update the other. The mistakes then propagated to other documents, like sales materials. We tried to remove the duplication altogether, but different pages are seen by different people. So instead, we now use pre-defined “snippets” for all the sentences we re-use.
• Our process folder used to contain files with the following names:
  • Procedures for Ordering Work from a Contractor.doc
  • Procedures for Ordering Work from a Contractor.pdf
  • Procedures for Ordering Work from a Contractor (Contractor Version).doc
  • Procedures for Ordering Work from a Contractor (Contractor Version).pdf
  • Procedures for Ordering Work from a Contractor (CRE Consultant Version).doc
  • Procedures for Ordering Work from a Contractor (CRE Consultant Version).pdf

It was hard to know whether the PDF versions were up to date with their “.doc” counterparts—and whether the three versions of the file were “in sync” with one another. At best, it took a lot of discipline to keep them from turning into chaos. So we consolidated them all into one document and highlighted in yellow the “conditionals”—the parts that needed to be edited by each type of user.

At this point, you may want to pause reading

The remainder of this document describes ways to avoid duplication problems. At this point, though, you may want to spend a few days looking for duplication within your activities, to see what problems you need to fix. The following questions can help:

• How many of your activities only happen because you can’t easily find information that the company, somewhere, already knows? How many meetings wouldn’t be needed if you were able to find the information quickly?
• Do your team members spend most of their time improving the company—and the “winning controls”—or do they spend their time re-inventing the wheel?
• Consider whether your questions get answered via “notification media” or “documentation media”:

  • Notification media are ones that are used for conversations. They include email, chat, and spoken conversations. Whenever you email a file to someone, for example, you create many duplicates: the version on your desktop; the version in your email client’s “Sent Items” folder; the version in each recipient’s email client; and the version that each recipient downloads. Notifications tend to be unmanageable and impossible to edit retrospectively. If you want to get up to speed on a project, you have to re-read every conversation and decision change, like you’re fast-forwarding through twelve months of a particularly dry soap opera.

  • Documentation media are ones for which there’s “a place for everything and everything in its place.” The place can be accessed by everyone who needs it. It’s always up to date.

How to avoid duplication problems

Once you start spotting duplicated information, you become painfully aware of the problems it will cause. The following tips will help you avoid duplication-is-evil problems:

• As far as possible, all information needs to have a canonical instance (that’s a fancy way of saying “an official version”). All other instances should be just pointers to it. Examples of pointers include…
  • When a document needs to contain information that is present elsewhere, use a pointer in plain text. In other words, just describe where the information is, rather than repeating it.
  • On a website, you can use a “file include” to insert a piece of information that needs to appear in several places on the site. (The principle of embedding a document into another document is known as “transclusion.”)
  • When you need to keep a file in two separate locations, delete all but one instance of it, then use shortcut files (on Mac) or aliases (on Windows) to point to it from the other locations. In Google Docs, you can use links to refer to information that’s stored elsewhere.
• Whenever you copy information to a new location, delete the version in the old location (or at the very least, add the word “Old” to the start of the filename, then create a folder called “Old” and drag it into that.) Better still, delete the information from the old place, and put a “pointer” there instead.
• If you ever spot information that’s present in two locations, but you are too busy to combine them, just put a pointer in both locations, so that a future reader will be aware of the duplication.
• If you are keeping multiple versions of a file for backup reasons, that’s a sign that you don’t trust your backup systems and your version control system. Study why you don’t trust them.
• Keep similar information together. In our company training manual, for example, we group advice into sections. That way, it’s much easier to spot when information is duplicated. Duplication often happens because items of information that are similar to one another are stored far apart, so the users don’t notice the duplication.
• This one is crucial, and few people do it: Try hard to avoid creating new documents. Challenge yourself hard about whether a new document is really necessary. Imagine that it costs $100 to create a new document. You can find and organize information much easier when it’s within one long document than when it’s fragmented between many different documents. Create a new document only in the following circumstances:
  • When information needs to have a different access permissions (different people are allowed to see and edit it)
    …or…
  • When a particular section becomes so large that it ruins the document it’s in—when it becomes “too big for its room and needs to move out.” For example, in our list of how we manage different web apps, the section on Google Workspace takes up as much space as all the other apps put together, so we gave it its own document and linked to that. Note that its siblings didn’t have to move out—that is, we didn’t need to create separate documents for every app.
• When you create a flow that supersedes an existing one, it’s your responsibility to decommission the existing one—even if the existing one belongs to someone else. Decommissioning sometimes just means deleting the old version, but more often it means redirecting all pointers to the old version, and checking that the new version contains everything useful that the old one contained. If you aren’t yet sure that your new system will supersede the existing one, set yourself a reminder so you don’t forget about it. Example: If you create a new feedback survey for our clients, delete the old one, or—at the very least—add the word “Old” to the start of its name, and add to the top of it a notification saying that it’s obsolete, and a link to its replacement. Decommissioning is tedious work—particularly if both versions were run in parallel for a duration—so do it as soon as possible. (Even better, always try to update the existing version incrementally.)
• Google Docs and Google Sheets solve duplication incredibly well. Each viewer sees the canonical version. The Revision History feature saves every change, so there’s no need to create documents with filenames like “Presentation Slides FINAL VERSION (ignore all the others) 3b—updated MASTER (pending approval) FINALFINALc.”
• The Mac software Kaleidoscope is excellent for playing “spot the difference” between text files, image files—or even whole folders.

Sometimes, we can’t find a way to avoid duplication

Duplication is always evil, but sometimes it’s unavoidable:

• Sometimes, you need to have an audit trail.
• Sometimes, you need to share a document with an external party who can’t access the data in its canonical format—either for technical, legal, or policy reasons.
• Sometimes, the information is in a technology that doesn’t allow you to share on a level that’s granular enough. For example, you may wish to share particular financial data, but your financial package’s sharing permissions are all-or-nothing.

Conclusion

If any piece of information exists in two separate places, you will end up with a mess. Next time you encounter clutter, disorganization, confusion, deterioration, and errors, look for the root cause. You’ll often find that it’s duplication.

(As you may have guessed, the moment we published this article, we deleted the corresponding section from our onboarding flow for new team members—and replaced it with a link to this article.)

How much did you like this article?

What’s your goal today?

1. Hire us to grow your company

We’ve generated hundreds of millions for our clients, using our unique CRE Methodology™. To discover how we can help grow your business:

• Read our case studies, client success stories, and video testimonials.
• Learn about us, and our unique values, beliefs and quirks.
• Visit our “Services” page to see the process by which we assess whether we’re a good fit for each other.
• Schedule your FREE website strategy session with one of our renowned experts.

Schedule your FREE strategy session

2. Learn how to do conversion

Download a free copy of our Amazon #1 best-selling book, Making Websites Win, recommended by Google, Facebook, Microsoft, Moz, Econsultancy, and many more industry leaders. You’ll also be subscribed to our email newsletter and notified whenever we publish new articles or have something interesting to share.

Browse hundreds of articles, containing an amazing number of useful tools and techniques. Many readers tell us they have doubled their sales by following the advice in these articles.

Download a free copy of our best-selling book

3. Join our team

If you want to join our team—or discover why our team members love working with us—then see our “Careers” page.

4. Contact us

We help businesses worldwide, so get in touch!

© 2024 Conversion Rate Experts Limited. All rights reserved.