Uncertainty in Software Development


Uncertainty is one of the few certainties about software development. Historically so many projects have had problems that we are willing to believe the worst. A good example of this is in Why does software cost so much? by Tom De Marco, (Pages 35-37) discussing the now infamous pie chart analyzing the failure of DoD projects. Consultants have used this chart to highlight the poor state of software development, without realizing that the chart was analyzing the 9 failing projects out of the 113 that were studied.

Humans crave certainty, and have a hard time accepting the unknown. It is as if we much prefer precise, inaccurate information to low precision but accurate information. An estimate of 5 weeks seems to be preferred over an estimate of 5 +/- 2 weeks. It is interesting to notice however that we seem to be selective about when we request this type of precision. For actions of third parties, we are willing to accept some variability, but the same is not true inside our organizations.

Many people would like software development to be a predictable process whereby the time and cost of development are known very early on in a project. Unfortunately in the current state of the art is that this predictability is not achievable. Development staff have somehow succumbed to this idea that predictability is achievable, resulting in the myth of the software crisis. The existence of this crisis is predicated on the assumption that software development is predictable. Without this underlying assumption, there is no crisis.

How much predictability is desirable?

Predictability is nice. A predictable software project would have known costs and schedule at the start and would deliver on schedule. A desirable state of affairs.

But let us reconsider for a moment. For a project that is planned to take 6 months, how important is it to know

(A) the exact completion date at the start of the project vs.

(B) knowing the month we will complete in?

How much are we willing to pay for predictability?

How much are we willing to pay for a project so that it falls into category (A) vs. (B)? Would we be willing to double the development costs? Probably not. In many cases a 5% extra cost would be unacceptable.

How much extra effort would we be willing to put into planning a project so that it falls into category (A) vs. (B)? Would we be willing to delay the start of the project by 4 weeks? In most commercial software development the answer has been no.

What are the consequences of seeking predictability?

Stress has been the major result of the desire for predictability. A common solution to the Cost and Schedule dilemma is to get developers to work unpaid overtime. Using this trick it is possible for many projects to get close to the pre-announced schedule, so instead of a 40 hour week, many projects work developers 60+ hours/week just to get close.

Poor Quality is another result of this desire. While there are some people who can produce high quality work while working long hours, they tend to be thin on the ground. So in seeking to hit the schedule, quality gets compromised, in fact any aspect of the system will be compromised in order to meet the date.

But it is not all bad news, predictability is possible, it is just expensive. In They Write the Right Stuff Fast Company Dec:Jan 1997 the success of the NASA Shuttle Software is documented, but it also notes that "on a dollars per line basis, it makes the group among the nation's most expensive software organizations".

What other options are there?

When uncertainty is accepted as part of the process of software development, many new options open up. It becomes possible to manage the uncertainty such that the uncertainty is just another part of the balancing act between functionality, schedule, cost, quality and people that project managers everywhere are used to dealing with.

The software development process can be tuned to optimize the desired attributes without undue stress on the organization. Iterative, incremental development can be used to evolve software, delivering prioritized features and functionality. Another alternative is to schedule projects based on reducing uncertainty, using prototyping to drive out uncertainty.

An iterative, incremental process is a very good way for dealing with uncertainty. Rather than tackling all of the uncertainty in one attempt, it allows for smaller steps, each one with a small amount of uncertainty. As each increment proceeds, more of the unknowns become known and thus the next step has to face less uncertainty.

Jim McCarthy in Dymanics of Software Development talks about one possible alternative, the Doctor model of Software Development. Doctors are comfortable talking about the risks and uncertainties of any procedure. McCarthy suggests that Developers could consider this as a possible model for handling the uncertainies inherent in software development. Developers need to become comfortable with talking about risk and uncertainty, because just like in medicine, no actions are certain. Most bugfixing has a finite probability of introducing a new error into a system.

Estimating through dividing deliverables into small tasks

The Personal Software Process (PSP) from Watts Humphrey and the SEI has a way of making the estimation of small development projects resonably accurate. By dividing any deliverable up into small chunks (typically 4 hours of work or less), the sum of these estimates tends to be more accurate than attempting to estimate the whole deliverable in one piece. By carefully tracking and monitoring progress on the short tasks, developers can get good at estimating these small tasks with the end result that these estimates become remarkably accurate.

Interestingly, eXtreme Programming gets similar benefits without the detailed tracking and statistics involved in the PSP by simply dividing all requirements (stories) into engineering tasks and tracking these estimates. After several delivery increments (typically 3 weeks duration) XP developers learn to accurately estimate these engineering tasks.

The key to both of these techniques is to only estimate small, known tasks (if a task is unknown, rather than estimate it it is better to conduct an experiment to get more information and then estimate when more is known about the task). Rapid feedback from actual results then closes the feedback loop to allow the developer to improve their estimates.


Uncertainty is part of the development process. Really great software requires that the developers embrace innovation and creativity, both of which bring uncertainty with them. The payoff for embracing the uncertainty is lower development costs than are associated with attempting to remove all uncertainty.