ProjectFlightDeck

Concept: ES Statistical Analysis factors historical variation into the calculation of duration estimates. By analyzing the project’s previous schedule performance, ES stats identify the high and low bounds for the estimate at completion for time. High/low bounds are then checked against allowances for uncertainty to determine project status.

Diagram 1

Practice: 

How ES Statistical Analysis Works

Practice: ES Statistical Analysis looks at schedule performance over time. When Walt Lipke examined ten years of EVM data, he found that, surprisingly, the natural variation in SPIt was not a good candidate for statistical analysis. The values did not fall into the kind of pattern that supports normal statistical analysis.

The term “normal” here is not used in its colloquial sense. It is used to indicate that the SPIt data did not have a normal distribution. Instead, it was skewed. See Diagram 1.

But, Walt also found that the SPIt could be transformed mathematically into a form that was amenable to normal statistical analysis. By applying the natural log to the SPIt, a normal distribution was achieved Lipke (2011). [1] See Diagram 1.

Walt adapted standard deviation calculations to yield high and low bounds for  the Estimate at Completion for time (EACt). At ProjectFlightDeck, we use the EACt (High) and EACt (Low) to steer the project. There are some detailed conditions on the use of ES statistical analysis, but for now, these will be omitted. To see the details, click here (see Part 1).

How to Use ES Stats to Steer the Project

To steer the project, EACt  High and Low are compared to allowances for uncertainty, specifically Contingency and Reserve. 

• Contingency is the response to predictable uncertainty and Reserve to uncertainty that cannot be predicted. Generally, the emphasis is on uncertainties that cause delays. But, there are also uncertainties that result in early delivery.
• Uncertainties that cause delays are addressed by “late” Contingency and Reserve allowances. Uncertainties that lead to early delivery are addressed by “early” Contingency and Reserve allowances.
• See Diagram 2 for an illustration of Late/Early Contingency and Reserve. 

Diagram 2

At the start of a project, when the amount of performance data is minimal, the natural variation in performance often produces a wide gap between the high and low EACt. As more data is collected, the statistical performance stabilizes, and the gap shrinks. At ProjectFlightDeck, we wait until performance has stabilized before taking the next step.

And, the next step is to compare high and low EACt estimates to target values. The target values are allowances for uncertainty, specifically:

1. Contingency, and
2. Contingency plus Reserve.

Based on the comparisons, we assess the status of the project. If the high forecast is greater than target values, or the low forecast is less than target values, the project is in trouble. Otherwise, the project is on track.

For a more detailed discussion of how the comparison is done and for exceptions to the process, click here (see Part 2).

ES Stats Example

An example illustrates how ES Statistical Analysis is used to steer the project.

Background: Project B had a 45 week baseline (including 3 weeks of contingency).  There were an additional 2 weeks of reserve. We tracked the estimate at completion each week  and began using ES statistical analysis to steer the project starting at week 30. Chart 3 shows the results through the first 36 periods, stopping at week 36 because we reached a major decision point. The example describes the challenges up to that point and our responses.

Diagram 3

Diagram 4

On Project B, we used dates rather than durations in our analysis and reporting. Dates and durations are functionally equivalent in this context. To perform the analysis, we used the graphical representation in Diagram 3 and the specific date values in Diagram 4. (Note: the term “Bare Plan” denotes project duration without either Contingency or Reserve.)

At period 30, the project appeared to be in serious trouble. The forecasts were well beyond the end dates reflected in uncertainty allowances. The only good news was that the nominal forecast appeared to be within contingency and reserve.

Rather than immediately assigning a Red status, we decided to wait. We wanted to see if a pattern was developing now that there was sufficient information to reliably drive statistical analysis. We assigned the project a Yellow status to indicate that there was concern but not a crisis.

By period 33, we had seen enough. The high forecast was still beyond contingency and reserve dates. The low forecast was still earlier than Early Contingency, but it was no longer before Early Reserve. Guided by Diagram 4 and using the most serious status as the default value, we assigned the project a Red status.

We did not, however, initiate a re-baseline. The decision to defer the re-baseline was based on two factors. First, the low forecast showed that there was room to recover. Second, the trend in the high forecast was downward—performance appeared to be improving slowly over time.

By period 36, we reached a major decision point: was the project still in Red status, and if so, must we re-baseline the plan?

We decided that the project was no longer in Red status: the high forecast no longer exceeded Reserve, and the low forecast still fell within the Reserve. As indicated by Diagram 4, the project was in Yellow status.

More important, we decided that we would not re-baseline the plan. High and low estimates were both within Reserve. The high forecast had consistently dropped over 7 successive periods. The low forecast was rising but slowly. The trends were re-assuring. We believed that contingency would probably be drained, but we also concluded that we would make the committed date (i.e., within Reserve). We proceeded with the original uncertainty allowances.

Post script. How did it turn out? The project finished beyond contingency but within reserve. The commitment was met, and we avoided the turmoil that is normally associated with a re-baseline.