Concept: The substitution of relative estimates for absolute estimates is an enduring theme in the Agile canon. (See the References section for a sampling of recent posts.) Earler this year, I described some issues with relative estimates. It's time for another look at the same topic.
Practice: Among proponents of the Agile framework, there is a widely-held view that relative estimates are easier to produce and more reliable than absolute estimates. Equally common is the impression that “reams of research” support this view (Seibert, 2016). When you investigate the purported evidence, however, you find that something is amiss.
Even the Agile Alliance admits it: “…these studies which have for a few years now given rise to the claim that ‘research shows that people are better at relative than absolute estimation’ do not in fact seem to square with that claim” (RelativeEstimation, n.d.). For proof, we’re left with anecdotes.
Comparative Ease
Anecdotes are used to illustrate how easy it is to estimate by comparing objects of similar size. My favourite example involves a coconut and a guava. The author says:”… it is easy to use a guava as a reference and say that the coconut is about four times bigger” (Siddharta, 2015). [1],[2]
It’s not so easy if you lack relevant background knowledge. Many of us from the “high north” might hesitate to make the comparison, given our limited familiarity with the fruits.
Background knowledge, moreover, is a slippery slope. For instance, why does the coconut case not include familiarity with measurements? My grandfather was a carpenter, and he could size a piece of lumber to within an inch just by looking at it. If so, there’s no more (or less) work involved in sizing the fruits via absolute numbers than there is by comparison, given relevant background knowledge. [3]
A deeper problem is that relative estimates are grounded in intuition. They express beliefs about size. Whatever immediate access I might have to my belief about size, I don’t have the same access to your belief. Do we mean the same thing when we say that a coconut is four times bigger than a guava?
Maybe I’m thinking of surface area rather than width. To me, the coconut would be 16 times the size of a guava. Or, perhaps you are thinking of volume, in which case the difference would be 64 times.
A dialogue might eventually disambiguate the comparison, but the process is not as straight-forward as portrayed. This is especially true when you think of an object more abstract than a coconut, for instance a user story. In sum, intuitive judgments about size are not necessarily easy.
Questionable Reliability
Then, there is the question of reliability. Why claim that relative estimates are more reliable than absolute ones? Perhaps, it’s due to the belief that “absolute estimates are always wrong”—another widely held view in the Agile community (eCameron, 2010). If so, relative estimates might seem to be more accurate. They are not as specific as absolute estimates, and the “fuzziness” renders them less vulnerable to error.
But, the fuzziness also renders them useless for planning. Say deliverable A is estimated as three times the size of B. Presumably three times as many Bs can fit into a Sprint as As, but how many As can fit into a Sprint? Without an anchor point, the relative numbers do not give us enough information to answer the question.
Some proponents of Agile try to anchor the estimates while still assuming they are relative. The first step is to use numbers to distinguish between relative sizes. One example uses 10, 30, and 60 with the second number expressing something three times the size of the first and the third number expressing something twice the second.
Then, to anchor the series, they propose seeing how long it takes to do something the size of a 10.From this, they infer that it would take three times as long to do a 30 and twice as long beyond that to do a 60. In other words, if it takes one day to complete a 10, it will take three days to complete a 30 and six days for a 60. Why is this not sufficient for Sprint planning?
The problem is that the numbers are “purely arbitrary”. As relative markers, “[they] do not relate to a specific unit of size or time” (Goldstein, 2012). That means the next time you see how long it takes to do a 10, there’s no guarantee that it will be one day. It might take three days. If so, a 30 will take 9 days and a 60 will take 18 days. [4]
The anchor fastens only one instance of the series. As anchors can vary from one instance of a series to the next, there’s no constant baseline against which to plan or measure performance. Without a stable baseline, planning beyond the next Sprint becomes impracticable. In short, relative estimates set the baseline adrift. [5]
Notes:
[1] In the illustration, you’ll find a coconut (and a half) and a guava (and a half). Whether or not the one looks four times the size of the other is left as an exercise for the reader.
[2] It’s unclear whether the author is thinking of visually comparing instances of the two fruits or generalizing on their relative size. Either way, the comparisons are not necessarily straight-forward.
[3] Don’t take this as endorsing the view that intuitive estimation of measurements is effortless or inherently accurate. I’m only saying that background knowledge is necessary for any intuitive judgment.
[4] If you try to set anchors on each number in the series, it becomes unmanageable. If there are changes in values between iterations, you cannot tell whether differences are due to new starting points or to revisions in relationships. Either way, you cannot reasonably plan beyond the next Sprint.
[5] Another alternative would be to drop intuitions about the relationships and discover the size of each group through experimentation. Projects that could afford to wait for the information are rare. Besides, there are estimating models that use results from similar projects to inform estimating for the current project. In effect, the reference projects provide the kind of empirical data that seems to the goal of this alternative.
References:
Cohn, M. (2013, July 10). How Can We Get the Best Estimates of Story Size? Retrieved from https://www.mountaingoatsoftware.com/blog/how-can-we-get-the-best-estimates-of-story-size.
Cottmeyer, M. (2011, September 18). The Real Reason We Estimate. Retrieved from https://www.leadingagile.com/2011/09/the-real-reason-we-estimate/.
eCameron. (2010, 15 August). Why Estimates are Always Wrong. Retrieved from http://ecaminc.com/index.php/blog/item/214-why-estimates-are-always-wrong.
Erickson, C. (2008, December 18). Making Better Estimates, Part 5: Relative/Arbitrary vs Absolute/Real. Retrieved from https://spin.atomicobject.com/2008/12/18/making-better-estimates-relative-arbitrary-vs-absolute-real/.
Goldstein, I. (29 February 2012). Relative Estimation Communication. Retrieved from https://www.axisagile.com.au/blog/estimation/relative-estimation-communication/.
Relative Estimation. (n.d.) Retrieved from https://www.agilealliance.org/glossary/relative-estimation/.
Seibert, C. (2016, 13 May). Using Points vs Hours for Estimates. Retrieved from http://chase-seibert.github.io/blog/2016/05/13/agile-points-vs-hours.html.
Siddharta. (2015, 25 August). There is no such thing as “absolute estimation”. Retrieved from http://toolsforagile.com/blog/archives/1125/newsflash-there-is-no-such-thing-as-absolute-estimation.
Sutherland, J. (2013, May 16). Story Points: Why are they better than hours? Retrieved from https://www.scruminc.com/story-points-why-are-they-better-than/.
Talboom, E. (2012, December 4). Agile estimating 2/4: Absolute versus relative estimates.
Retrieved from https://co-learning.be/blog/agile-estimating-24-absolute-versus-relative-estimates/04122012.
Torok, P. (2014, June 3). Is there any published research about story points vs time estimation? Retrieved from https://pm.stackexchange.com/questions/11675/is-there-any-published-research-about-story-points-vs-time-estimation.
Velocity Range Calculator. (n.d.) Retrieved from https://www.mountaingoatsoftware.com/tools/velocity-range-calculator.
For the blog home page and additional posts, click here.
|
