Charlie Alfred, Garrett Thurston

Countless product development efforts have been punished by requirements problems. They have run significantly over budget, delivered severely late, or were cancelled before release. Invisible requirements have been one of the most disconcerting causes. An invisible requirement is one that is captured incorrectly, expressed vaguely, or left unstated. Frequently, the specter of invisible requirements raises its ugly head and strikes at the least opportune time: late in the development cycle when the design is found to be inconsistent with the requirement and cost to retrofit is prohibitive. If the product managers and developers had only seen this requirement coming, it would have been much easier to address. This paper discusses that many types of invisible requirements are not invisible after all, just well hidden. In addition, it explores some ways to shift your thinking to make these hidden requirements more visible.

The story has been told too many times to count. A firm spends in excess of a million dollars and 18 months to develop an innovative new product, but after months of development, a requirements problem cripples the effort. Sometimes, its a case where the requirements are there, but the product developer and the development sponsor interpret them differently. Other times, requirements that were written down imply other requirements which werent. In other cases, the requirements are new, triggered by external events or revised assumptions about the future. Once manifested, invisible requirements tear deeply into the work that has already been done, rendering core assumptions invalid. In the best case, significant redesign and rework must be done. In the worst case, its less expensive to start over than to try to salvage what has been created. In either case, somebody is held accountable for the mess.

Any successful product development effort fundamentally depends on the following: Where and Why The product needs a strong fit with its intended market(s) and the environments where the product will be used. What The product must have a clear set of requirements, which provide a complete set of acceptance criteria. How The product must have a clearly defined reference model that provides conceptual integrity by explaining how the product is put together, how it works, what its limits are, and how it can be adapted for different situations.

www.foliage.com 168 Middlesex Turnpike, Burlington, MA 01803 781.993.5500 Copyright 2011 Foliage Software Systems, Inc. FOL-QMS-TMPL-0073, Version: 1.00, released: 20 Sept 2007

1 of 12

Who, When, and What If Finally, the product must have a well-defined plan for what resources are needed, how their efforts will be coordinated, and what steps should be taken if unexpected problems surface. Each of these characteristics of successful product development efforts is important. Together they form the backbone that holds everything together. An ever increasing focus on the quality of requirements, particularly in the Aerospace and Defense industry, is providing significant gains in productivity and lower costs. To drive these effective requirements, industry has found that the input to the requirements process, the highlevel architecture, is critical in formulating requirements that specify clearly all necessary functionality, while constraining the implementation as little as possible. Indeed, architectural considerationsuch as deployment contexts, constraints, and desiredthough-not-required behaviorare invaluable in the early stages of systems and system-of-systems development. Therefore, while an increased focus on requirements is necessary for successful product development, it is not sufficient. Industry has found that a thorough treatment of the higher-level architecture provides the requirements effort the analysis and data necessary to effectively drive the rest of the development life cycle. The Problem with Solution Orientation
Coming to solutions (requirements) too quickly often times overlooks potentially more beneficial solutions. To illustrate this, consider the Jefferson Memorial. Several years ago, excessive erosion of the Jefferson Memorial was noticed. A brief investigation identified excessive cleaning as the cause. Since the memorial must be kept clean, more investigation was necessary. Bird droppings were identified as the culprit, so actions were taken to have fewer birds around. Eventually, however, someone asked why the birds were such a problem with the Jefferson Memorial and not the others. Another study determined that the birds frequented the memorial, not for love of Jefferson, but for love of the many tasty spiders that made their home there. Probing further, the spiders were thriving because the insect population was proliferating. Finally, understanding that the insects were attracted by the memorial lights at dusk and dawn, identified the ultimate solution. Turn off the lights. Initial solutions driven by quick decisions by memorial managers (i.e. powerful stakeholders) provided expensive ill-suited solutions for an ill-understood problem. The root cause and final solution requirement were well hidden, only brought to light by extensive and time consuming trial and error solutions. Each required solution inappropriately framed the problem, missing the associated hidden causes and final necessary requirement.

Most product development processes today are organized into multiple phases (for example, conceptualization, feasibility, planning, development, etc.) with formal gatesi connecting adjacent phases. Often organized in a top-down structure, later phases tend to elaborate on and enrich the decisions of earlier phases. Increasingly, life cycles, while maintaining the gated structure, are formulated as interative, spiral, or Agile processes rather than a simple waterfall. Traditionally, the life cycles are initiated with the development of product requirements. The classic riddle Which came first, the chicken or the egg? has a productdevelopment version: Which comes first, requirements or architecture? Traditionally, requirements preceded architecture. After all, you need to know what before you can decide how. As products grow more complex, with increasing numbers of interconnected, interoperable systems and subsystem; and an ever-present drive to

support reuse and product lines, the requirements equation becomes more complex. In first examining architectureexploring product drivers, possible solutions, and architectural constraints and opportunitiesorganizations are able to establish a solid foundation on which requirements can be effectively derived. Hence, for most products, the market need and initial stakeholder requirements must be analyzed before the product concept and associated requirements takes shape. Since a well-defined architecture provides a solid framework on which requirements can be established, the resulting requirements tend to be complete and consistent. In contrast, when requirements are specified as input to architecture, this foundation is shaky or non-existent, and results in an incomplete, overly constrained set of product requirements. Indeed, even when a product is to be derived from a similar product (an earlier generation from the same company or a similar product from a different company), the requirements are still vulnerable, without first examining the architectural implications of both the similar and target products. For instance, assume that the environmental forces and system capabilities of an electro-mechanical control system are similar from one generation to the next. In this case, the number and type of I/O points, as well as the way-signal validation, failure logging, and reversion occur are also likely to be similar but does this tell the entire story? The risk for using existing reference models is that they may not match the current problemor there may be more to the current products story. In the previous example, changes in system capabilities, performance, or deployment contexts may invalidate the reference model. In contrast, the need to implement a product-line architecture might drive additional capability and extended or alternative ranges. The first obstacle is discussed later on in this paper in the section on Inconsistent Requirements. The second issue is a prime example of an invisible requirement.

An invisible requirement is one that is captured incorrectly, expressed vaguely or left unstated (i.e., assumed) before development begins on the part of the system that depends on the requirement. Therefore, in the example in the previous section, the business need to lower life-cycle costs in subsequent product iterations drives a product-line architecture approach. If this need is not clearly understood before product requirements are initiated, this invisible requirement will wreak havoc downstream, as extensive rework is required to establish the product lineor, worse still, a product is produced that doesnt support product lines and in turn the business need to lower-life cycle costs over the medium and long term has also been traded away. Many invisible requirements are seemingly innocuous. For example, suppose a system raises an alarm whenever a network connection is dropped. Later, when the hardware and software are integrated, the requirement must change to become three attempts to reestablish the connection before raising the alarm. The coding change is minor, with certifiable software development, the impacts on the requirements, design, and verification (reviews and tests) can be significant, and can impact key delivery milestones.

Hence, invisible requirements are often as innocuous as a carbon monoxide leak. Colorless and odorless, they strike with deadly force. They hit the hardest when their impact has a wide-ranging effect throughout the entire system.

Requirements provide both guidance for development decisions, as well as the criteria to verify the suitability of the end product. Ironically, this dual responsibility turns out to be one of the sources of invisible requirements. Verification criteria must be precise. The system either passes the test or it fails. The same is not entirely true for development guidance. Charles Kepner and Benjamin Tregoe published a book called The New Rational Managerii in 1981. In this book, the authors discuss a method called Decision Analysis: an objective, rational process for making complex decisions among many alternatives. Kepner and Tregoes method is straightforward: evaluate each decision alternative against a set of objectives. These objectives fall into two categories: must and want. Must objectives are binary criteria which answer the whether or not question. If any must objective is not satisfied, the alternative is rejected. This sounds a lot like the verification process. If the system passes all of the tests, it is considered suitable. Kepner and Tregoe proceed to make an interesting point. They say that must objectives only filter out the unsuitable alternatives and narrow the field down to the real contenders. Want objectives address the how well question and collectively identify the best alternative. Want objectives define acceptable ranges for criteria and a way to map this range to a measure of utility. For example, a must objective might say that response time must be 200 milliseconds or less for the current planned hardware configuration. A want objective to support future hardware configurations might necessitate a tighter response time, say 100 milliseconds. Therefore, though not necessary for the current effort, if possible within the current projects cost and schedule constraints, a tighter response time is preferred for future instantiations. When each want objective is prioritized relative to the rest, adequate information exists to make informed tradeoff decisions. Most requirements specifications that Foliage has encountered take the form of a lengthy list of must objectives. While this approach is necessary for evaluation, when it comes to guiding development, it is a car without a steering wheel. The information garnered in defining a robust system architecture provides that steering wheel, allowing good tradeoff decisions and mitigating risks. Want and must objectives relate to the degrees of freedom for the downstream processes. A specification with all must objectives doesnt have any leeway, i.e., no degrees of freedom. In contrast, allocating some objectives as wants provides context and flexibility. In other words, want objectives add degrees of freedom, must objectives take them away. Therefore, when system verification is performed, an unambiguous set of must requirements are essential. When the solution approach is formulated, however, a much different approach is needed, since the set of requirements which support architecture and design, should take a different form than those which support verification. All requirements, however, must be consistent.

Consistency is an essential property for requirements, and cannot be evaluated for any requirement in isolation. Consistency requires two or more items and a context that defines rules for their coexistence. One simple form of inconsistent requirements is two requirements that contradict each other. Another is when two requirements cannot coexist given external constraints, such as the laws of physics. For example, a plane shape and wing design may not have the aerodynamics to remain airborne at some minimum air speed. Requirements reviewers need a proven reference model to assess consistency. This reference model is composed of scientific laws, engineering discoveries, and past experience, especially with next-generation products. However, past experience with prior-generation products has blind spots, since next-generation products often introduce major new technological capabilities, which can have a significant impact on the architecture and change the rules used to evaluate consistency.

Cognitive biases are mental errors caused by heuristics that can compromise accuracy for speed and ease of judgment.iii Cognitive biases are very different from cultural bias, organizational bias, or bias resulting from ones own self-interest, since they do not result from any emotional or intellectual predisposition, but rather from subconscious mental procedures for processing information.iv One area that illustrates how product development can be affected by cognitive bias is the development of manned versus unmanned systems and their associated reference models. The engineering challenges for manned and unmanned aircraft certainly have a great deal in common; for example, both must take off, fly around, and land. They are, however, fundamentally different in that the pilot in unmanned aircraft is either automated or remote. As a result, many of the rule-of-thumb considerations for aircraft development, from development considerations to certification regulations, must be carefully reexamined to examine the innumerable cognitive biases established over the past 100 years of flight experience. This illustrates why it is so easy for cognitive bias to creep into peoples decision processes. Those who are intimately familiar with the issues and challenges of one domain unconsciously carry a schema of associations, assumptions, and conclusions into the other domain. Often, these schemas framev their thought processes and judgments so strongly that representatives from the two groups with disparate cognitive biases often have difficulty understanding each other. For example, several years after a project award, at the tail end of the certifiable development cycle, an Avionics control program manager innocently shared with the system integratortheir customers customerthat they, of course, would receive well regulated power from the aircraft power bus. The project manager was confident that the Avoinics control would perform against its documented requirements, as long as his long-standing assumptions on power quality were met. You can imagine the surprise and chagrin as it came to light, during the systems testing phase, that power levels quality should be expected to vary significantly, and that it was the Avionics control systems responsibility to detect and adapt to this variability.

This invisible requirement resulted from different, unspoken assumptions between prime and sub-contractors about the essential nature of the operating environment. Eventually, the realization of the invisible requirement forced a complete redesign of the system (including hardware, software, and supervisory functions), leading to millions of dollars of unanticipated costs and highly visible schedule delays. Fortunately, one need apply suitable means to address cognitive biases. This is most effectively done through disciplined use of analytical tools (several of which are introduced briefly towards the end of this paper) that help break down the fast-and-frugal heuristics in favor of a rigorous analysis process. As mentioned above, cognitive biases can form roadblocks between groups with disparate cognitive biases. These challenges often occur on a broad scale when participants from business, marketing, sales, engineering (Systems, Hardware, Software), and maintenance roles introduce their own subtle cognitive framing biases when establishing product requirements.

Complete requirements are necessary and sufficient to solve the problem they were intended to solve. As a consequence, any requirements which are missing, implied, or unstated are invisible, by definition. How can you be sure that a set of requirements are complete? Gathering all of the requirements you have specified is not enough. Completeness cannot be evaluated by looking inside the system. To evaluate completeness, it is necessary to look outsideto the containing system. To evaluate completeness of requirements, we must understand the products stakeholders, its external systems, and the governing laws of math and science. We need a reference model (or robust system architecture) that describes how these entities will drive, influence, or limit the product. Some of these factors, like gravity and electricity, are predictable and measurable. Other factors inject uncertainty and provoke mayhem, such as unclear market demands and volatile functional expectations. Still other factors have voluntary opinions and perceptions, ranging from predictable to surprising. Driven by uncertainty and the likelihood that things will change over time, the goal of complete requirements is elusive. However, an acceptable outcome is achievable, but only if the analysis of completeness shifts from a focus on the product to a focus on the context in which the product is used. This shift in focus enables the product creator to walk in the shoes of the buyer and end user, ask more meaningful questions, and imagine what additional sources of value a solution could provide. The challenge of invisible requirements increases as the size and complexity of the system increases, sometimes further complicated by additional supply-chain elements. For example, consider a plumber coming onto a construction site and taking care of his part of the effort, but while so doing damaging or invalidating what the carpenter or electrician had already done. In complex aircraft systems with distributed supply-chain elements, such occurrences frequently occur. Furthermore, such systems frequently involve competitive bidding, which, while driving the price down, can often lead to various suppliers not asking key

questions for fear of providing development insight to their competitors. The result: not all requirements and constraints are identified as a solution is formulated.

Many products have several contexts of use. A context of use (context for short) occurs whenever one or more of the following aspects, outside the product, can vary in a way that creates serious stress for the products design. Conditions Constraints or uncertainties which differ from one context to another. Conditions can be physical, such as physical operating environment, including temperature, humidity, and electromagnetic interference. Conditions can be functional, for example, radio or cellular network access. Conditions can also relate to variances in product regulations, for example, export control (EAR/ITAR) and airworthiness (FAA/EASA or military). These are conditions that have a strong time component. For example, climate variables in most locations vary by season, with inevitable weather driven challenges in a flight-reservation system. Furthermore, these can be difficult to predict, in the case of weather, or almost entirely predictable, in the case of peak holiday air-traffic. Whenever stakeholders form a group (such as users), perceptions tend to differ within subgroups. Often these perceptions will vary with conditions and situations. For example, airport flight operation managers in cities with cold winters are likely to place a higher value on ice-level detectors more than those in cities with warm winters.

Situations

Perception

Requirements issues related to context dependency can have devastating consequences. By analogy, if implicit or missing requirements are a virus, then multiplecontext requirements are the H1-N1 strain, as the needs of an entire (critical) stakeholder group may be ill served or ignored. Unfortunately, context-related issues are common. One of the biggest reasons is that product development companies tend to fastidiously focus on the near-term market opportunities, and treat other contexts as distractions to be dealt with later. Multiple context systemsvi often occur in the product-line arena, in which many related products (or models) intentionally share common assets to increase ROI by systematically containing development costs. Note, however, that single products often have multiple contexts, too. A telltale sign of a multi-context single product is a large number of configurable options. Consider the following well-known example of an invisible requirement resulting from multiple contexts. The original Jeep was a versatile, light-weight four-wheel drive ground transportation vehicle used during World War II by the U.S. and Allied Armies. After the war, the Jeep was redesigned and commercialized for civilian use. The primary context of use was off road enthusiasts, i.e, drivers who enjoyed driving on rugged, uneven terrain.

Ground clearance, for users in this context, was a critical product quality. Its not hard to understand why tearing off a gas tank or exhaust system on a tree stump or rock outcropping 50 miles from nowhere was considered to be a bad idea. Over time, the SUV increased in popularity, spawning many new makes and models. The primary context of use shifted from off-road enthusiasts to urban and suburban drivers. However, the suspension and ground clearance did not change. One of the unfortunate side effects was a significant increase in the number of rollover accidents involving SUVs. As the context of use changed with the paved surface and higher speeds, the higher-ground clearance, with its associated higher vehicle center of gravity didnt. Off-road drivers typically drive only 35-40 MPH, on flat terrain with few obstructions. City and suburban drivers often drive 65 MPH, sometimes on wet or icy highways. Therefore, when an SUV driver has to brake sharply and swerve to avoid a vehicle stopped in traffic, the high center of gravity becomes the drivers enemy. The usual motivation for multi-context systems is to increase ROI across many products by leveraging commonality. However, experience with product-line architectures has shown that properly managing the differences between contexts matters most. The key implication of this is that if you have, or even think you may have, a multi-context system that you want to leverage with a single architectural strategy, it is imperative that you investigate the contextual differences early, before the architecture hardens. Skipping this activity is virtually certain to create invisible requirements that may not reveal themselves for yearsand when they do, like the SUV, they will reveal themselves at the least opportune time and in the least opportune way.

Agile Methods, such as SCRUM and XP have emerged during the past 10 years as a way to cope with invisible requirements. These methods shun detailed system architectures and requirements specifications (known as NBDUF, for no big design up front). They maintain that too much is unknown and too much will change. In other words, stakeholders dont really know what they need until they gain experience with the product. Early designs are bound to be filled with incorrect and unnecessary features, so effective products emerge only after series of successive iterations often involving multiple course corrections, each of which help to evolve the product in the right direction. Clearly, with certifiable system development and the high-costs associated with rework, one of the biggest threats stem from what you dont know that you dont know. Traditional waterfall processes often try to deal with an uncertain future using insurance, in the form of speculative requirements and designs. Agile methods follow the YAGNI (you aint gonna need it) mantra and avoid purchasing this insurance until it is proven to be necessary. However, neither approach has a crystal ball. Both are forced to handle surprises. Agile approaches rely on refactoring to restructure implementations that have been ambushed by invisible requirements. Waterfall methods do this also. There are two critical factors in the comparison.

1. When an adjustment is required, how much existing development do you need to rearrange? Early in the development life cycle, Agile methods, with their focus on staying lean, may have an advantage. There are fewer components and their interdependencies are less constraining. Here, the cost of refactoring is more manageable. Over time, as the size and complexity of the product increases, the structure hardens, and changes become more risky and expensive. 2. Of the opportunities for up-front decisions, how many will be correct versus how many will be misguided? If you are going to be wrong, which side do you err on? Agile proponents maintain that when you make the decision up front and find out it is inappropriate, you must lug it around like the proverbial albatross. Waterfall proponents argue that if you postpone the decision, refactoring later on is much more costly and risky than getting it right the first time. This argument is virtually impossible to win in the absolute. For safety-critical systems, major refactoring late in the process can introduce unacceptable risks, as well as large cost impact, due to changes which ripple through the entire process. For information systems that support human activities and decision making, the pendulum swings the other way.

The question is not whether early or late decisions are superior, or how establishing firm requirements up front lines up against with the ability to adapt quickly. Product developers want to make good decisions both early and late. They want to anticipate what is likely to occur and react quickly to the unforeseen. In this volatile, complex, uncertain world, only the prescient can do this reliably. vii How can the rest of us do this better? Russell Ackoff, former Professor Emeritus at the University of Pennsylvania discussed two forms of thinking, analysis and synthesis, which were known to the ancient Greek philosophers.viii Analysis, the act of decomposing bigger things into smaller pieces to understand how they fit together, has been the dominant mode of thinking since the Industrial Revolution. Synthesis shifts attention to the containing system to grasp why things are the way they are, and what forces and cycles hold them in place. In the sections above, we have discussed several forms of invisible requirements: 1. Those which define tradeoffs. 2. Those which make other requirements inconsistent. 3. Those which are missing or unstated. 4. Those which are correct in one context, but not in another. 5. Those which emerge unexpectedly due to unforeseeable events In reality, the first four elements in this list (and some of the fifth) are not invisible at all. They are merely obscured, hidden in plain sight. To identify and understand them, it is necessary to shift our way of observing systematicallyto put aside our cognitive biases, employ the right analytical tools, and ask the right questions early.

We rely on analysis to comprehend what buyers and end users say they want, to heed the requirements that regulators impose, and to demonstrate how the product will meet the revenue, margin, and ROI hurdles. However, these things fail to give us enough insight into why they want what they want, or why these regulations exist, or why the business needs to meet these particular financial objectivesand when we lack understanding of why, that is precisely when invisible requirements emerge.

There are any number of analytical tools that can be combined to help mitigate the issues associated with invisible requirements, including Product Strategy Framework, Multiple-Context Analysis, and Road mapping. Product strategy frameworkix is a method that rationalizes the core product vision with that of the strategic vision. The formation of these visions is more proactive than the SWOT1 analysis, popularized by Michael Porterx in the 1970s and 80s. With a product strategy framework, the strategic and product vision are aligned by considering the organizational core competencies (value chain), financial plan (economic model), technology trends/strategy, product strategy, and market trends/competitive strategy. This methodology allows you to exerting more control trying to get where you are headed as quickly as possible. Multi-context analysis is a formalized examination of the variability associated with a products (or product lines) context. This analysis first identifies the various deployment contexts, taking care to serve not only the present needs, but also downstream considerations. The context needs and constraints are documented and prioritized, such that the critical contexts are addressed by the always limited time and cost considerations. Variability analysis, examining the ranges of the product requirements necessary to serve all targeted contexts comes next, driving firm verifiable requirements into the development process. Road mappingxi looks at the long-term plan as both a mechanism to create and maintain alignment among business, marketing, and technical stakeholders and decision makers. Roadmaps provide different views into efficiently bringing the right product to market, from concept, through product development, to market introduction/final-delivery. Not so surprisingly, these views mirror many of those examined in the product strategy framework. Roadmaps ensure that all stakeholders are served by maintaining an everevolving roadmap, supporting informed tradeoffs up and down the organizational chain. Furthermore, developing a consistent set of roadmaps aligned with those of customers, ensure both short-term success and long-term utility across the entire customer-base.

Strengths, Weaknesses, Opportunities, and Threats.

Invisible requirements arent really invisible. They are just hidden from view, because of the way that we look at them. Here are some basic tenets that should guide your thinking if you want to maximize the number of invisible requirements you notice: 1. Customers dont buy from you because you offer a high-quality well-engineered product. They buy because it solves a problem for them or provides value in another way. 2. Mother Nature is one of your biggest stakeholders. Your product can only do what the laws of math, science, and economics permit it to do. The demands of your customers, your executives, and shareholders must be reconciled with environmental forces. 3. Contexts are the natural combination of the first two items. It is essential to approach product development in a context-sensitive way. Earlier in this paper, we observed that contexts occur whenever conditions, situations, or perceptions change in a way that fundamentally changes the problem. It should have been evident to SUV manufacturers that off-road enthusiasts and yuppies had very different conditions and perceptions, and, as a result, there was a good chance that the same product might not be appropriate for both contexts. 4. Challenges are an important tool for connecting combinations of customer situations. Challenges are risks and obstacles that make it difficult to realize value. Challenges occur within our product and drive our architecture decisions, but they occur inside of our customers contexts, too. The ability to identify and solve major challenges faced by a customer context is the foundation of a successful context. 5. A best practice in architecture strategy is to focus architecture decisions on the highest priority challenges. The highest priority challenges are the ones that hold the key to unlocking the most value, and are the most difficult to solve. This approach is critical to making the best possible tradeoff decisions for a product. A clear understanding of why clarifies the reasons that certain features and capabilities are more valuable than others, and helps to explain what kinds of changes in conditions or situations will change how value is perceived. Likewise, a clear understanding of contexts can help to illuminate opportunities for how technologies can be applied to overcome challenges, which historically have blocked the realization of value. An architecture strategy based on overcoming well-prioritized challenges acts as a sailboat keel to stabilize development. This type of architecture, available for a relatively small investment of time and money, provides a solid foundation, for both waterfall and Agile development methods. Foliage has been working with clients for many years to help them implement effective product solutions. Essential to this delivery has been our ability to help them understand the nuances of their problem, uncover hidden requirements, and create solid architecture strategies.

Established in 1991 Foliage provides product development services including custom systems, software, and electromechanical development services and product strategy consulting to hundreds of client-partners throughout the world. With a wealth of product development experience earned by successfully delivering over 300 products to market, a heavy dose of senior talent, and teams with a dedicated focus on the Aerospace, Defense, & Security, Medical Device, and Industrial Equipment industries we serve, Foliage delivers a potent advantage to our client-partners. In addition, we actively collaborate to maximize that advantage by continually evolving industry-leading product strategies and techniques to shorten delivery cycles, optimize quality, and reduce costs. Visit www.foliage.com.

Cooper, Robert, G., Edgett, Dr. Scott J., and Kleinschmidt , Dr. Elko J., Optimizing the Stage-Gate Process: What Best Practice Companies are Doing - Part One Reference Paper #14
ii

Kepner, Charles H., Tregoe, Benjamin, B. The New Rational Manager, Princeton Research Press, 1981, pp. 77-100. The original version, The Rational Manager was published in 1965.
iii

Bostrom, Nick, November 18, 2007, http://www.overcomingbias.com/2007/11/towards-a-typol.html and http://en.wikipedia.org/wiki/Cognitive_bias

iv

Richards J. Heuer, Jr. Psychology of Intelligence Analysis, CENTER for the STUDY of INTELLIGENCE Central Intelligence Agency, 1999
v

Tversky, Amos, and Kahneman, Daniel, 1981. "The Framing of Decisions and the Psychology of Choice." Science 211: 453-458.
vi

Alfred, Charlie, Multiple-Context Systems: A new Frontier in Architecture, The Microsoft Architecture Journal, Volume 23, http://msdn.microsoft.com/en-us/ff476942.aspx
vii

Singer, P.W. Wired for War, Penguin Press, 2009. Addresses what Kurzweil terms The Law of Accelerating Returns. (see http://www.kurzweiltech.com/aboutray.html) It is basically that exponential technology patterns go beyond Moores Law for semiconductor complexity and are applicable to technology in general.
viii ix

Ackoff, Russell L., Ackoffs Best: His Classic Writings on Management, John Wiley & Sons, 1999, pp. 8-24

McGrath, Michael, E. Product Strategy for High Technology Companies: Accelerating your Business to Web Speed. McGraw-Hill, 2001.
x

Porter, Michael E., Competitive Strategy: Techniques for Analyzing Industries and Competitors, Free Press, 1998.
xi

Lougee, H., Thurston, G., System-of-Systems: Development: Competitive Advances in Systems Engineering, Foliage Whitepaper, 2010 http://www.foliage.com/thought-leadership/whitepapers.php