Managing ideas demands a steady hand

The most common mistake is to treat the design process as if it were a big light switch you can just turn it on and off whenever you like.
This fantasy, as it goes, runs like this:
you show up one day, realize it’s getting late and that there are too many ideas and designs (and not enough decisions), and you say to the team, “OK, we’re done with ideas. Pick a design and let’s start coding! Woohoo!” Even at the off chance that there is a design that is ready for primetime (which there won’t be), this kind of unpredictable behavior will disorient and confuse the entire team. Up until that moment, everyone was feeling captured best by the They Might Be Giants song, “Older”: “This day will soon be at an end, and now it’s even sooner, and now it’s even sooner. And now it’s sooner still.”

Wworking on designs that required time to bake. Without a date given to them, they may have thought they had right up until 11:59 p.m. on the night before specs were due to make their big decisions. Instead, good idea management is decisive but predictable. It should never be a surprise that the nature of the work is changing (unless there is a crisis) or that the focus of energy is shifting because the project is entering a different phase. There should be easy
and natural reminders to the team as the scope and emphasis change. Like a dimmer switch for lights—the kind with a knob that gives measured control over changes—there should be a gradual shift of focus. It’s the project manager’s job to manage that dimmer switch and make sure it’s controlled with a steady hand. There may be a moment when someone has to say, “Look. Time is up. Is it A or B?”, but that moment should be expected days or weeks before it comes about. The pace might need to accelerate or decelerate, but it should be done gracefully.

To illustrate this, idealized view of the creative phase of a project, with a singular point in time when problems and goals have been defined (vision document and/or requirements), and a single point in time when specifications will be completed. Between these two points are much brainstorming, sketching, designing, prototyping, and all sorts of other fun activities described in other articles. For the first half or so of the available time, everyone is focused on coming up with ideas and growing the space of alternative designs. For the second half, the emphasis shifts to narrowing the field by refining and improving the best designs. Eventually, a point is reached where enough design decisions have been made to document them all in a specification.

This is a good story, and a fine diagram, and I’m proud that they appear in this article. However, as is the fate of many fine diagrams, the one pictured here represents something that never quite happens. Those straight lines and perfect angles don’t exist. Managing ideas, like much of project management, is always a fuzzy and subjective process, and there are several important reasons why this diagram is inaccurate:

First, the problem space tends to shift back and forth. It’s never a bright yellow line that’s fixed in place. Because understanding the problems to solve—and the ways to solve them is not static, the space of possible alternatives is always growing and shrinking. Requirements will be adjusted. The trends might be for the space to grow more than shrink, or shrink more than grow, but it’s never all of one or the other. It’s more of a fuzzy curving line than a straight one.

Common reasons for this include:
• New information becomes available. The world doesn’t stop because you have a project underway. Companies might go out of business. A technology may fail. Budgets may change. A usability study or customer interview might reveal a new insight into the problem (“people print documents more often than we thought” or “users can’t even do their basic tasks with the home page design we prototyped”). An engineer’s plan becomes clear, changing the rough estimates of how much work might be possible. Early thinking always gives way to better, later thinking. This sometimes works in the project’s favor, and sometimes it works against the
project. For example, a programmer might find a new implementation strategy: “if we build it this new way, I don’t have to do five of these other work items, and there is more time for other work, or we can finish early (yay)” or “because we can’t build it how I initially thought, we have to do five additional work items, meaning less time for other work, or we can finish late (boo).”

• Conflicts are found between two solutions for two different problems that, when integrated, work against each other. This can happen for usability, business, or engineering reasons. Joe might have a fantastic design for the car engine, and Sally might have a great design for a transmission, but when brought together, they realize that aspects of each of their designs conflict—for example, the transmission doesn’t fit with the engine.

Changes cause chain reactions

Another reason the problem space shifts is that design decisions are interrelated: one change can impact many different decisions. Given this interdependence, it’s impossible to fully predict what the impacts will be. I’ve seen this happen many times. On one specific IT project, one of goals was to improve people’s ability to organize their list of favorite web sites. People considered four different designs and made simple user interface (UI) prototypes for each one. With these prototypes, we did rough engineering estimates and got basic usability information to use in comparing them. With specs due soon, we chose to focus on design B. But then, days later, we learned that because of a schedule change on a different project, a component that design B depended on wouldn’t be available to us. So, we had to go back and reassess our choice.

When we did, we discovered that all of the other designs also required the same component (d’oh!). Then, when we cut the functionality (i.e., eliminated the requirement) that this troublesome component would have provided, we learned that
other programmers were depending on us to provide that functionality to them through our code. This component was more important to the project than we’d initially realized. We had to sit down as a team and figure out if we could afford to design and build that functionality ourselves, or if we could live with the consequences of not having the functionality at all.

It’s important to note that this story doesn’t represent a failure. Without making the decision to go with design B, we never would have flushed out all of the dependencies and design considerations involved. I do believe that smart teams flush out requirements and dependencies early, but if the project is complex, you may never get them all. I don’t believe that the time spent modeling complex systems to catch every dependency and interrelationship is usually worth the costs (if the pace is fast, and the project is complex, these models will be expensive to maintain), but it might be. It depends on the needs of the project. We chose to bet on the teamwork of the design process to flush them out for
us, and it did.

Anyway, the back-and-forth process I went through, where paths opened and closed, assumptions were proved wrong, and new questions were raised, is precisely what designing things is all about. This is often called iteration, which means that the details need to evolve over time (because the problem is complex enough that decisions won’t be right without several evolutions). Specific to design, iteration implies a two-steps-forward, one-step-back experience. The more difficult and complex the work, the tighter that ratio tends to be (e.g., 1.5 steps forward for every 1 step back). But until you take that step forward and make a decision (“Let’s run with design B!”), you won’t see all of the problems and issues.

Making decisions during design, even if they turn out to be wrong, is the only way to force issues and problems to the surface. If you plan correctly, you will be wrong many times during the design process, but through doing so, you will dramatically improve your chances of success. Most engineering, design, and scientific efforts have similar patterns, as the following quote expresses: “There are still enormous amounts of trial and error. You go back and forth
from observation to theory. You don’t know what to look for without a theory, and you can’t check the theory without looking at the fact I believe that the movement back and forth occurs thousands, even millions of times in the course of a single investigation.” — Joshua Lederberg, winner of the Nobel Prize, 1958

The second problem with stereotypical view of the project is that the creative momentum of a project is always stronger than inexperienced leaders and managers expect. The effort required to narrow down a pool of ideas into a single (good) design becomes much harder, and demands different skills, than they anticipated. The time to close down a problem space should be as long as the time it took to grow it out. But the more innovative or creative the project is, the harder it is to estimate the time the problem space will need. This is because of the creative work’s momentum.

The cause of this momentum is that the rate of new questions and issues being discovered is faster than the rate that old issues are being closed. Anyone involved in the work can sense this trend. Even when the target date for specifications is weeks away, many will believe that the schedule is going to slip (and worse, that there is nothing the can do about it because the managers don’t see it happening). This is often the first major slipping point on projects. It happens gradually and is continually underestimated until it’s too large to correct easily.

This is often the first time that a project manager has reason to panic. Until this point, everything was abstract: words, goals, lists, and slide decks. But when the designs aren’t together yet, and the date for specifications looms, people get scared. Some look for ways to avoid the real situation by blaming others, forcing bad decisions, or denying that the problem exists. Therefore the best way to manage ideas is to start any major design work with clear checkpoints for how the time should be used. Instead of having only two checkpoints, requirements (or problem definition), and spec writing, some intermediary points need to be defined before creative work is going at full speed. It’s the project manager’s job to make sure these points in time are created and that everyone understands their usefulness.