The Danger of Incomplete Pictures, Part 2: Story/Task Decomposition and Estimation as Design
In my first post of this series, I discussed how ambiguity and lack of shared understanding between members of a product development team can occur when we rely on unstructured, ad hoc, and abstract communication processes (i.e., conversations and high-level user stories) for expressing our thoughts and ideas. We feel like we are painting a clear picture, but we are hindered by hidden assumptions and “blind spots” that prevent us from expressing and seeing details of the big picture in our mind’s eye.
In this post, I want to talk about a technique we use at the beginning of projects to gain a shared understanding of what we are trying to build. At the heart of this process is something I think may come as a surprise … estimation.
There has been much written over the years regarding estimation, challenges with estimation, whether we should be estimating at all, etc., etc., (I’m looking at you #NoEstimates!). A lot of energy goes into these conversations. I am not going to try and convince you whether estimation, as it is currently viewed, is right or wrong. For what it’s worth, we find our estimation process incredibly valuable and essential to the success of the complex software projects we work on and the effectiveness of our agile/lean approach to product development.
I am going to attempt to get you thinking differently about the desired outcomes of estimation and how valuable estimation can be used to improve and inform the quality of system design and reduce the uncertainty when trying to determine what should be built.
We have a saying here at Don’t Panic Labs that goes, “the software design process starts with analysis and estimation of the backlog of stories for a project.” I’m pretty sure most people would not think about estimation as a software design tool, but we do. Much like other engineering disciplines that tackle big and seemingly unsolvable problems by breaking them into smaller solvable problems, we can use analysis and estimation of user stories to arrive at a more “solvable problem.”
I am reminded of Mark Watney, the main character in “The Martian,” being stranded on Mars. His ultimate goal was to get off the planet, but he realized that was not a problem he could attack directly. He analyzed what had to happen to get off the planet and began solving all of his smaller problems, like having enough food to survive long enough to be rescued.
When we are provided with a backlog, we are often given epics/stories that may seem quite large and “unsolvable.” An example might be, “determine whether a patient is sitting up in bed.” That seems daunting, but when you start breaking this down into smaller problems, it starts to feel more manageable.
In this case, it might look like this:
- identify the bed in the room
- determine if the bed contains a patient
- determine if the bed is flat or inclined
- determine if there is a person-shaped entity that is in a certain orientation relative to the incline of the bed.
This example may seem unusually large in scope (and I would agree), but I use it to illustrate the benefit of breaking bigger problems into smaller ones.
An example of a user story we might be more likely to see could be: “As a customer, I want to process an order using a credit card for payment” (which we will title the Submit Order story). We will use this story to walk through estimation and task decomposition to help gain a shared understanding of the true requirements and to help reveal uncertainty and any hidden assumptions.
Estimation Buckets
When we estimate stories, we use a fixed number of estimation “buckets” (see Todd Guenther’s post, Project Management – How We Estimate).
These estimate buckets are:
- 0 effort (uncommon)
- 1/2 a day
- 1 day
- 1/2 a week
- 1 week
- >1 week
We do this for a couple of reasons. First, it reinforces that we are estimating by removing the tendency to “put too fine a point” on our estimation by trying to get to the exact number of hours it will take. Estimates are inherently inaccurate, so let’s not pretend they aren’t.
Secondly, we believe it is easier for an engineer to assess a relative size of effort using these buckets – even with uncertainty. Relatively small stories (i.e., stories < 1 week) are usually easy to assess in terms of which bucket something will fall into. We often hear comments like, “I think this is probably more than a day, but I don’t think it will take 1/2 a week.” In this case, we put the estimate at 1/2 week and trust that the Law of Large Numbers will even it out across all estimates – which they almost always do.
Estimating our Story
We favor one-week iterations at Don’t Panic Labs. This forces us to “eat the whole elephant in small bites.” It increases the frequency of delivery to QA and customers for feedback and means we are – at most – one week away from seeing that we might have some schedule risk. We have tailored our iteration planning process to be very light, so the one-week rhythm remains efficient with little/no overhead.
Even if we were not on one-week sprint cycles, we would still require all of our stories to have an estimate of one week or less. This is key to helping reveal the uncertainty and hidden assumptions. If we originally think the Submit Order story is something like two weeks, then it’s a safe bet there is a significant amount of uncertainty there. When this occurs, we ask the team (usually an engineer, project manager, and product owner) to decompose the story and estimate the sub-stories that are identified. They will do this until all of the sub-stories are equal to one week or less.
For our Submit Order story, we might end up with the following:
- As a customer, I want to process an order using a credit card for payment – 2 weeks
- Process credit card authorization – 1 week
- Store order details – 1/2 week
- Verify shopping cart pricing, tax, and shipping – 1/2 week
- Send notification to customer and seller – 1/2 week
The more detailed estimates of the sub-stories add up to about two and a half weeks, which I have much more confidence in than the original two-week estimate. I am also pleased to see that there was some discussion about the desire to verify the product pricing before submitting the order (rather than just accepting the pricing from the shopping cart). This is a requirement that might not have been articulated in the original description of the main story.
Wrapping Up
I strongly encourage our development team to engage in analysis and design early on in the project cycle, perhaps as early as backlog development. I have seen time and again how critical thinking has uncovered details that were not originally understood. Estimation is a powerful tool for telling us how well we truly understand what is involved for a particular story. Stories that are greater than a week – let alone stories with estimates of a month or more – almost certainly have a lot of unseen details that could disrupt development efforts unless they are revealed early on in the project.
In the next (and last) post in this series, I will discuss a lightweight tool we use within the sprint cycle itself that helps ensure the technical details are properly fleshed out before a bunch of time and effort is spent re-working code that does not match the product owner’s requirements.
