To Update or Not to Update?

about | archive

[ 2020-September-07 16:07 ]

We build software on a huge stack of pre-existing software. This raises the question of when do we update those dependencies? The policy that emerges if you do nothing is that dependencies are updated when you have a clear need, such as requiring a feature that is only available in the latest version. Otherwise, you keep using the version of the operating system, libraries, and tools you installed when you first needed them. In my opinion, this is a bad policy. For projects actively being developed, I believe it is well worth the effort to use the latest stable release of your dependencies. This means you need to constantly spend some effort to stay up to date. However, I believe this saves effort in the long run, by avoiding bugs and reducing the total time spent applying updates. In the rest of this article, I will explain my position. As with most software engineering topics, I'm going to argue based on my own experience, and without any data.

Disadvantages of updating dependencies

There are two disadvantages to updating your dependencies. First, any change to a software system has some risk of introducing errors. Even if the update is a bug fix (e.g. going from version 1.0.2 to 1.0.3 in semantic versioning), it might break your program. Your code might depend on some behaviour that was changed by the "fix," causing your program to break (see: Hyrum's Law). Hence, the least risky thing to do when maintaining an application is change it as little as possible. That means not updating dependencies.

The second disadvantage is that it takes effort, and does not provide any user-visible improvement. The argument is that instead of spending the time to update your dependencies, you should spend that time doing something that directly benefits users instead. While the effort may be small, there is usually some work involved, even if it is just changing version numbers. If your process is automated, then you've spent the effort in the past to build and maintain the automation, so it still cost something. Even worse, on the rare case that an update breaks your software, you will need to debug it, which might be the worst of all, since you don't suspect it and usually can't plan for it. At least incompatible major versions usually give you a hint that you may need to spend some effort, so you can plan for it. At the extreme, you might be depending on a feature that has been removed in the latest version, which means you cannot update without substantial effort.

Advantages of updating dependencies

The first advantage of updating is that you avoid some unknown bugs. A well-managed software project generally improves over time. While that "improvement" may be uneven, and some releases may regress, if a project isn't improving, it dies. Hence, if you stay current, you are avoiding some bugs in your dependencies. Unfortunately, you will never know which bugs you avoided. On the other hand, you always know exactly what things broke by updating, because you will need to fix it. The fact that the benefits are invisible is what contributes to the perspective that updates don't provide value. However, on rare occasions you can notice the opposite problem: you will discover a bug in a dependency, and find out it has already been fixed in the latest version.

The second advantage to staying up to date is that it is easier to get support for the most recent release. In this context, I mean "support" in the broadest sense. You will find better documentation, it is easier to search for answers on the Internet, and can more easily get help with bug reports. If you are using an older version, it is common for someone to ask you to try it again with the latest version. This "support" cost is very small, and spread across your entire development effort.

The final advantage of updating is that I believe applying regular smaller updates actually takes less total effort than applying big updates less often. This is a bit counter-intuitive: If there is some amount of fixed "per update" effort (e.g. running the "update my dependencies" tool), then shouldn't it be less effort to do one big update instead of a sequence of small updates? In particular, you get to skip updates, which should safe effort. However, I believe the sequence of small updates is more efficient. One big reason is that when you find a problem, it is isolated to a smaller update, so finding and fixing the problem takes less time. When you find a problem in a big batch of updates, the first step tends to try the sequence of smaller updates anyway. Another reason smaller updates are more efficient is if you do it regularly, you get better at it. It forces you to document the process, or write some basic tests or automation for it.

Opinionated advice about how to do updates

In my opinion, a well-run software project should regularly update all their dependencies to the latest stable version. However, the exact schedule for how to do this, or how often, will depend on the project and the organization. At the extreme end, you can fully automate the process, which is what parts of the Chrome web browser seem to do Skia's AutoRoll documentation). A middle ground is to provide a script that generates the update change, but requires some manual approval or shepherding of the change through a build process (e.g. see Chrome's DEPS file instructions). I would say that you probably need to be doing this more often than once a year, to avoid some dependencies from not being the most recent "stable" release.