When an engineering manager tries to sell you on developing software in iterations, it sounds like a scam waiting to happen. You pay them money to write software for some number of hours, but the developers offer no guarantee that they’ll finish the features you’re asking for. It’s not even like getting your bathroom renovated; sure, the contractors take way longer than they said they would, but at least they don’t charge you more for the privilege of waiting. Why would any business agree to that for software?
This question is more interesting than it looks. It seems like common sense to us not to trust the iteration—but only because we make a common set of assumptions about how we should produce, consume, and pay for software.
And maybe, just maybe, those assumptions are wrong.
But let’s not talk about that just yet. First, let’s talk about what an iteration is and how it differs from some of the other methods used to measure the delivery of software. Then, we’ll talk about the history of software projects, and why we view software delivery the way we do. Finally, we’ll examine why that view might be misguided—and how the iteration can help us build better software and better software teams.
Part 1: What is an iteration?
An iteration is a unit of time that a software team uses to get some amount of work done—usually a week—but it’s not a commitment to get an amount of work done. Let’s talk about what that means and where iterations came from.
Historically, large enterprises have used a waterfall methodology for completing software: the business delineates a specific set of features (sometimes called requirements) to be completed by a due date a year or more in the future. Then, developers and designers work toward that feature set by that due date. If it becomes clear that the feature set is too big to be finished by the due date, developers and designers do one of two things. On one hand, they miss the due date, leaving the business in limbo about when the critical features will be ready. On the other hand, developers and designers commit extra hours and crank out features faster by writing messy solutions. They think the solutions are temporary, but instead they stay in the code base as priorities shift, piling up on top of one another until years down the road when the business feels compelled to scrap the whole application and start over.
Businesses wanted to eliminate some of the problems with the waterfall software structure. So starting in about 1995, agile methodologies gained popularity as an end goal for enterprise software teams. You may have heard of scrum, a term first coined in 1986 to describe a style of agile development that found its way into enterprises over the next twenty-five years. A cycle of work for scrum software development is called a sprint, and it’s generally one to two weeks as opposed to a year. Other than being twenty to fifty times shorter than the cycle time of a waterfall project, the agile sprint embodies the same idea: the software team commits to a specific set of features to be completed by a due date two weeks in the future. In order to make this commitment, developers and designers estimate how long the features will take. If they underestimate the time commitment, they stay extra hours and crank out features faster by writing messy solutions. It’s the same problem as waterfall, but the team has more chances to catch the trend and stop it because it happens in biweekly rather than annual cycles.
The iteration comes from a different family of agile software development called extreme programming, which coalesced around 1996 (its first complete description appeared in Extreme Programming Explained in 1999). An iteration looks similar to a sprint in length; usually a week. But iterations treat work commitments differently than waterfall cycles or sprints do: the software team does not commit to a set of features. Instead a product manager, whose focus is on building the right product, prioritizes the features to be built, and the developers and designers work on the most important features first. They estimate how complex each feature will be, but they don’t estimate time amounts because people are bad at estimating timelines. The team works for a predetermined amount of time—say, forty hours per iteration—and they finish whatever they finish in that amount of time. No extra hours. No extra try. No feature cramming.
And it sounds like a scam to the business. Why would you pay when you don’t know what features you’re paying for?
Part 2: The way we produce, consume, and pay for software
When it comes to software, we want to know what we’re getting before we pay. We do the same thing when we pay to get our bathrooms renovated, or we contract to build a new building. We do this when we buy a car or a piece of furniture.
We do it because all of those things are static things. Once we have them, they are finished things. They are unlikely to change. If we decide to change them—say, we decide to renovate the building, soup up the car, or re-upholster the piece of furniture—it constitutes a major change, and we won’t be doing it very often. Once we buy these items, we probably won’t contact the seller again unless there’s a problem.
This used to be the case with software, too. Back in the early and mid ’90s, before the days of deploying new versions of software online, we bought and sold computer programs on CDs as physical goods. You could get a CD with Microsoft Word, Powerpoint, and Excel. You had a CD for Blackjack Deluxe and another CD for AOL. Once you bought the CD, that particular instance of the software would not change. You had no continuing relationship to the software seller. Your CD was a physical thing to use until it got outdated, at which point you would replace it with a newer version.
Nowadays, we don’t ship software that way. Instead, we deploy a new version of our website, and instantly everyone who visits our website sees the new version. Or we push an update to the mobile app store, and users can run the update right away. It’s quick and convenient for our customers to use our latest and greatest. This is great for business because it means that we can respond to our customers’ needs much faster. But instead of capitalizing on that advantage, we still think about software as the thing that it resembled twenty years ago—a physical good that, once finished, was difficult to change.
Software doesn’t operate like a physical good anymore; software operates like a business. And a business changes in response to how it perceives the needs of its customers. If you think the enterprise where you work doesn’t change its offering like that, I invite you to reach out to your staff archivist. Find out what your company looked like ten, twenty, or eighty years ago. You might discover that your company sold a very different thing ten years ago, and eighty years ago the company was completely unrecognizable to what it is today. Physical goods are hard to change. Businesses can change all the time. Software is like a business—it can change.
Here’s where miscategorizing software bites us in the rear. Let’s say we order a set of features from our development team with a due date that’s one year out. When we do that, we’re making a bet that we know what’s going to be important to our customers a year from now.
Anytime that you lock yourself into a given set of app features, you forefeit opportunities to respond to the needs of your customers. Suppose your developers commit to working on some cool new video features for your app in the next sprint, but two days in you get some bug reports that say your users can’t log in. Do you wait until the next sprint to fix the problem, leaving your users locked out for two weeks? Or do you charge in and yank the developers around onto the login problem, busting your sprint feature schedule or overcommitting the developers to come up with quick, dirty solutions? (Spoiler alert: quick, dirty solutions are a great way to end up with things like login bugs).
By building software in iterations, you avoid having to make a bet on what your customers need. Instead, you can make decisions at the latest possible time, using the most up-to-date and complete information. For an iteration, a product manager could move those login bugs to the top of the priority list, and the first developer to finish what she’s working on will pick them right up, no whiplash necessary.
Part 3: Why use iterations?
You can find out your customers’ biggest pain points, focus on solving the largest pain points, and work down the list to build a product they’ll value form your very first iteration, rather than some future due date. You’re in the business of building and selling solutions to your customers’ problems rather than features. Often, you’ll find that delighting your customers requires fewer features than you realize. If you don’t believe this, ask a representative sample of your customers which features of your app they use. You’ll often find that 80% of your usage goes to 20% of your functionality.
The iteration also predisposes your development team to build those critical features well. Because they are not under pressure to complete a feature set by a due date, they have the time and the bandwidth to make sure the software is robust and maintainable. Later, the code will be easier for developers to change if needed, so your company doesn’t experience the waste of rebuilding the whole system from scratch every five years when things get too hairy in there.
By the way, overcommitting developers and designers doesn’t get you more software. You can use it to crank out a few more crappy features in the short term, but over the long term it doesn’t work. This is because developers are not effective beyond a certain amount of time. Devs often get up for a few minutes every couple of hours because programming requires undivided mental focus. A person can only do that for so long in a given session. Many a screwup, bug, and bad deployment has happened because a developer kept going when they should have just gone home.
An iteration, unlike a feature set too big for its due date, does not encourage developers to do this. So you end up with fewer deployment disasters and version rollbacks.
There’s another reality to take into account here: a reasonable maximum number of work hours is an attractive perk for your software team. The opportunity to have a life can bring a lot of software talent to your door—which is a big deal because there’s more demand than supply of software talent. If you want a team that builds the kind of apps your customers will love and your business can rely on, you want to hire the right people and keep them happy. The iteration can be a part of that strategy for you.
To recap what we’ve learned: waterfall and sprint development cycles prioritize completing a predetermined feature set by a due date. But feature set by due date isn’t how software works anymore. Software is not a building; it does not get “done” (just like your business is never ‘done’). Instead, you can change it rapidly in response to what your customers need. The iteration allows you (or your product manager) to set the software priorities in real time based on customer feedback. It allows your software team to work when they’re productive and leave when they’re not. And it can help you attract and retain the kind of software team you’re looking for.