the convenient form of an "attack script" that enables the vulnerability to be easily exploited, even by those who are unable to understand it.⁴ Such behavior means that probabilities are nonindependent in a statistical sense. By contrast, risk assessment in the context of safety or reliability is significantly different. Risk in safety or reliability analysis is a function of the probability that a hazard arises and the consequences (e.g., cost) of the hazard. The most common function is the product of the two numbers, yielding an expected value. Informally, risk can be thought of as the expected damage done per unit of time that results from the operation of a system. Because the probability of failure per unit of time is nonzero, the risk is nonzero, and damage must be expected. If the estimated risk⁵ is unacceptably high, then either design or implementation changes must be made to reduce it, or consideration has to be given to withholding deployment. But if a safety incident should occur (e.g., an accident), the probability of a second accident remains unchanged, or may even decrease as a consequence.⁶

A major challenge for risk management with regard to trustworthiness is the growing difficulty of differentiating attacks from incompetence and failure or lack of reliability. It is one of several factors that raise the question of whether comprehensive probability estimation or hazard analysis is possible.

Nature of Consequences

Attitudes and behavior depend on the nature of consequences. Safety-critical information systems often control physical systems, where the

⁴A simple example is a one-line command that may allow an individual to steal passwords. Access the URL <http://xxx.xxx.xxx/cgi-bin/phf?Qalias=x%0a/bin/cat%20/etc/passwd>, substituting "xxx.xxx.xxx" with the target site of interest. For some Web sites, the encrypted passwords will be returned to you. If this one-line command works, it is because there is a flawed version of PHF in the /cgi-bin directory. PHF allows users to gain remote access to files (including the /etc/passwd file) over the Web. One can run a password-cracking program on the encrypted passwords obtained.

⁵Risk estimation is a systems engineering issue, and it involves careful, extensive, and thorough analysis of all aspects of a safety-critical system by systems engineers, safety engineers, domain experts, and others. An important initial activity in the process is hazard analysis, an attempt to determine the hazards that would be manifested if the system were to fail. A hazard is a condition with the potential for causing an undesired consequence. A hazard of operating a nuclear plant, for example, would be the release of radiation into the environment. A hazard of using a medical device might be patient injury. Various guidelines, procedures, and standards for carrying out hazard analyses have been developed. The central issue with hazard analysis is completeness—it is very important that all hazards be identified if at all possible.

⁶For example, because of greater operator diligence.

The economic and public policy context 175

consequences of failure include the possibility that lives will be threatened and/or valuable equipment may be damaged (e.g., an air traffic control system). The consequences of failure of non-safety-related systems include the possibility that data will be corrupted or stolen, or that essential services will be unavailable. While the latter are serious outcomes, these consequences are not perceived to be as serious as those associated with safety-critical systems. Financial consequences, especially within the private sector, have also attracted considerable attention because these consequences can be reasonably quantified and the implications to the financial bottom line are readily understood.⁷

Consequences are not static. Consequences that are currently tolerable may become intolerable in the future. For example, as the speed of communications channels continues to increase and applications are designed to rely on this speed, the availability⁸ of a connection may not be sufficient for those applications that depend on high bandwidth and low delay. Moreover, as applications become more dependent on quality of service guarantees from networks, a degradation in service may disrupt future applications more than current ones.

It is the nature of an NIS that outages and disruptions of service in local areas may have very uneven consequences, even within the area of disruption. Failure of a single Internet service provider (ISP) may or may not affect transfer of information outside the area of disruption, depending on how the ISP has configured its communications. For example, caching practices intended to reduce network congestion problems helped to limit the scope of a Domain Name Service (DNS) outage.⁹ Corporations that manage their own interconnection (so-called intranets) may be wholly unaffected. Even widespread or catastrophic failures may not harm some users, if they have intentionally or unconsciously provided redundant storage or backup facilities. The inability to accurately predict consequences seriously complicates the process of calculating risk and makes it tempting to assume "best case" behavior in response to failure.

A discussion about consequences must also address the questions of who is affected by the consequences and to what extent. While cata

⁷In contrast to privacy, for example.

⁸Increased dependence on connections promotes attention not only to the number of outages but also to the length of outages. For example, a one-second outage in a voice connection may require redialing to reestablish a connection; in a client/server application over a wide-area network, it could require rebooting computers, restarting applications, and considerable other delays that yield a multiplier as compared to voice.

⁹The master file for ".COM," a major address domain, was corrupted; however, most sites only queried the master file for entries not in their caches. Entries that were cached—and those generally included all the usual peers of any given site—were used, despite their apparent deletion from the master file.

strophic failure garners the most popular attention, there are many dimensions to trustworthiness and consequences may involve various subsets of them with varying degrees of severity. For example, cellular telephony fraud has two principal variants approximately equal in size: credit fraud, whereby the cellular telephone owner transfers the account to a second provider and does not pay the first; and cloning, the transfer to a new device of numbers that identify a radio and customer account. In both cases, the service provider loses revenue. Under some circumstances, a legitimate caller may be denied service if illegitimate users saturate the network.¹⁰ In the case of telephone cloning, if the clone user does not saturate the network, the provider loses revenue but users do not incur an immediate cost.¹¹ Understanding consequences is essential to forming baseline expectations of private action and what incentives may be effective for changing private action, but that understanding is often hampered by the difficulty of quantifying or otherwise specifying the costs and consequences associated with risks.

Risk Management Strategies

Risk management strategies are approaches to managing trade-offs.¹² These strategies address questions about whether it is better to add, for example, a small degree of security to a large number of products or substantial security to a smaller number of specific products, to use high-security/low- availability solutions or low-security/high-availability ones, or to increase assurance or the ability to identify and quarantine attackers. Trade-offs can be made in system design and engineering; they can also be made in deciding whether to invest in technology, procedure, insurance, or inaction.

¹⁰Note that the cost of denied service to the legitimate caller may far exceed the price of the telephone call itself. For example, a delay in requesting emergency services (e.g., a call to the fire department) may carry catastrophic costs.

¹¹However, to the extent that the cellular carrier is responsible for the resulting wireline and long-distance charges from the telephone clone, a rise in the cellular carrier's rates may be forthcoming.

¹²It is essential (1) that the actual system matches the model underlying the analysis as closely as possible, and (2) that the failure rates achieved by system components match the estimates used in the model. The former is a systems/safety engineering issue, whereas the latter involves all the engineering disciplines engaged in preparing the components. The process usually followed to achieve these two goals is in two parts: the first is careful management of the development process; the second is iterative evaluation of the system design as it is developed. If changes are made for any reason, the risk estimation might be repeated. If necessary, elements of the system design can be modified to reduce the risk. For example, if a nuclear plant's cooling system is shown to be unable to meet its dependability requirements because a particular type of pump tends to fail more often than is acceptable, then the design can be modified to include a backup pump.

The economic and public policy context 177

Risk avoidance is a strategy that seeks to reduce risk to the lowest possible value. Reducing risk takes precedence over cost or effect on the operational characteristics of the system in question. Risk avoidance strategies arose in the context of high-consequence systems, such as nuclear weapon command and control or the protection of nuclear weapon stockpiles. At the time these systems were developed, there was a clear boundary between high-consequence applications and "ordinary" software—whose malfunctions could be expensive and annoying but did not threaten human life or significant assets. With the increasing use of Internet technology, this boundary is becoming blurred.

The underlying assumption of risk avoidance strategies, when security is emphasized, is that there exists a highly capable threat that will expend great effort to achieve its goals. The achievement of those goals will involve such extreme consequences (e.g., uncommanded nuclear weapon release) that all possible effort should be devoted to preventing such consequences from being realized. Risk avoidance strategies, in general, incorporate every protection mechanism and invoke every possible assurance step. Many of these assurance steps, which are discussed in detail in Chapter 3, can handle only certain classes of designs or implementation technologies. When these limitations are imposed in addition to those of the rigid design guidance, the result is very often a system that is expensive, slow to deploy, and cumbersome and inefficient to use. Experience with risk avoidance strategies indicates that residual vulnerabilities will remain irrespective of the number of assurance steps taken. These vulnerabilities will often require quite exotic techniques to exploit; exotic, that is, until they are discovered by a threat or (worse yet) published on the Internet.¹³

However, the costs associated with avoiding all risks are prohibitive. Thus, risk mitigation is more typical and is generally encountered when many factors, including security and reliability, determine the success of a system. Risk mitigation is especially popular in market-driven environments where an attempt is made to provide "good enough" security or reliability or other qualities without severely affecting economic factors such as price and time to market. Risk mitigation should be interpreted not as a license to do a shoddy job in implementing trustworthiness, but instead as a pragmatic recognition that trade-offs between the dimensions of trustworthiness, economic realities, and other constraints will be the norm, not the exception. The risk mitigation strategies that are most

¹³Some exotic strategies require specialized hardware or physical access to certain systems, whereas other exotic strategies may require only remote access and appropriate software to be executed. It is this latter class of strategies that is particularly susceptible to dissemination via the Internet.

relevant to trustworthiness can generally be characterized according to two similar models:

• The insurance model. In this model, the cost of countermeasures is viewed as an "insurance premium" paid to prevent (or at least mitigate) loss. The value of the information being protected, or the service being provided, is assessed and mechanisms and assurance steps are incorporated up to, but not exceeding, that value.

• The work factor model. A definition in cryptology for the term "work factor" is the amount of computation required to break a cipher through a brute-force search of all possible key values.¹⁴ Recently, the term has been broadened to mean the amount of effort required to locate and exploit a residual vulnerability. That effort may involve more efficient procedures rather than exhaustive searches. In the case of fault tolerance, the assumptions made about the types of failures (benign or arbitrary) that could arise are analogous to the concept of work factor.

The two models are subject to pitfalls distinctive to each and some that are common to both. In the insurance model, it is possible that the value of information (or disruption of service) to an outsider is substantially greater than the value of that information or service to its owners. Thus, a "high value" attack could be mounted, succeed, and the "insurance premium" lost along with the target data or service. Such circumstances often arise in an interconnected or networked world. For example, a local telephone switch might be protected against deliberate interruption of service to the degree that is justified by the revenue that might be lost from such an interruption. But such an analysis ignores the attacker whose aim is to prevent a physical alarm system from notifying the police that an intrusion has been detected into an area containing valuable items. Another example is an instance in which a hacker expends great effort to take over an innocuous machine, not because it contains interesting data but because it provides computing resources and network connectivity that can be used to mount attacks on higher-value targets.¹⁵ In the case of the work factor model, it is notoriously difficult to assess the capabilities of a potential adversary in a field as unstructured as that of discovering vulnerabilities, which involves seeing aspects of a system that were overlooked by its designers.

¹⁴If the cryptography is easily broken (e.g., because the keys are stored in shared memory), the work factor may be almost irrelevant.

¹⁵A specific example of this comes from the early days of electromechanical cryptosystems. At that time, governments typically deployed an array of different cryptosystems of different strengths: simple (and easier to break) cryptosystems for less sensitive data, and elaborate

The economic and public policy context 179

Selecting a Strategy

Risk management seeks to provide an analytical framework for deciding how close to the edge one dares to go. Risk avoidance carries with it the danger of overengineering to the point at which the system is never used. Risk mitigation carries with it the danger of underengineering to the point at which the system is defeated, very possibly over and over again. The compound uncertainties of risk management preclude any rigorous method, but it is possible to articulate a few guidelines:

• Understand how long the system will be used in harm's way. Threats are not static; they become more capable over time, through the release of once-secret information from disgruntled former employees and other sources, access to once-esoteric equipment, and through other means.¹⁶

• Assess how much work is needed to exploit a known residual vulnerability. Does the attack require specialized equipment? Is this the sort of equipment that will drop drastically in cost over the next few years? Is it the sort of equipment that is freely accessible in open environments such as universities? Does the attack require a level of physical access that can be made hard to achieve?

• Context is extremely important. It is necessary to understand how the system might be used, how it is connected to or interacts with other systems, and how it might be exploited in the course of attacking something else.

• Can the system-support infrastructure react to vulnerabilities? Are system updates possible, and if so, at what cost? How many instances of

electromechanical devices to encipher highly sensitive data (called, respectively, "low-grade" and "high-grade" systems). This approach can be looked at as a risk-mitigation strategy, on either the insurance or work factor model, depending on how the decision of which system protected which data was used. Only security that was "good enough" was imposed. What the designers of these systems were slow to realize, however, was that the high-grade systems (e.g., the German Enigma machine) were vulnerable to "known plaintext" attacks where the cryptanalyst was able to match unenciphered and enciphered characters and thereby recover the key that deciphered other, previously unknown, messages. The nature of military and diplomatic communication is such that much text is "cut and pasted" from innocuous messages to more sensitive ones. Breaking the low-grade ciphers then provided the "known plaintext" that facilitated attacks on the high-grade ciphers.

¹⁶The so-called "cloning" attack, which is responsible for a large percentage of cellular fraud today, was at one time understandable only by a small handful of electronic engineers and required expensive, custom-made equipment. Today that attack is embodied in clandestine consumer products and can be mounted by any individual with the will and a few hundred dollars. The will has increased for many because there are more targets: high-use areas make listening for identification numbers more feasible.

the system will be deployed and how widely are they dispersed? Is there a mechanism for security recalls?¹⁷ Can the infrastructure continue critical operations at a reduced and trusted level if attacked?

The difficulties of anticipating and avoiding most risks can lead to strategies that emphasize compensatory action: detecting problems and responding to minimize damage, recovering, and seeking redress in some circumstances. The difficulty with this approach is the implicit assumption that all attacks can be identified. Anecdotal reports of success by "tiger teams" seeking to compromise systems suggest that detection may continue to be a weak vehicle for the future.¹⁸

Findings

1. Security risks are more difficult to identify and quantify than those that arise from safety or reliability concerns. Safety and reliability risks do not involve malice; the tangible and often severe consequences may often be easily articulated. These considerations facilitate the assessment of risk and measurement of consequences for safety- and reliability-related risks.

2. Although a risk-avoidance strategy may maximize trustworthiness, the prohibitive cost of that strategy suggests that risk mitigation is the pragmatic strategy for most situations.

3. Consequences may be uneven and unpredictable, especially for security risks, and may affect people with varying levels of severity. Safety-related consequences are generally perceived to be more serious than other consequences.

Consumers and Trustworthiness

The spending decisions made by consumers have a profound impact on the trustworthiness of NISs. The consumers of trustworthiness may be partitioned into two groups: information system professionals, who act on behalf of groups of relatively unsophisticated users, and the general public. Information system professionals often have only a modest understanding of trustworthiness because of the limited attention devoted

¹⁷For example, in GSM cellular phones, the security algorithms are embedded in per-subscriber smart cards and in a small number of authentication stations. This permits the relatively easy phaseout of an algorithm that has been cracked, although it remains to be seen whether providers will indeed replace the COMP128 algorithm. See <http://www.isaac.cs.berkeley.edu/isaac/gsm.html> for details.

¹⁸For example, consider the success of the "Eligible Receiver" exercise in which a team of "hackers" posing as paid surrogates for North Korea could have disabled the networked information systems that control the U.S. power grid (Gertz, 1998).

The economic and public policy context 181

to trustworthiness within college curricula and professional seminars. Even information system professionals who concentrate on security issues vary greatly in their understanding of issues associated with trustworthiness.¹⁹ The larger group of consumers is the general public, mostly unsophisticated with respect to trustworthiness despite a growing familiarity with information technology in general. The rise of an information systems mass market during the last two decades, and the concomitant influx of unsophisticated users, exacerbates the asymmetric distribution of understanding of trustworthiness concerns.

Consumer Costs

Consumer costs include all costs associated with trustworthiness that are borne by the user. Some of these costs are associated with the prevention or detection of breaches in trustworthiness; other costs are related to recovery from the effects of inadequate trustworthiness. Consumer costs include expenditures for the acquisition and use of technology, the development and implementation of policies and practices, insurance, legal action, and other activities. Consumer costs may be divided into direct costs, indirect costs, and failure costs.

Direct Costs

Direct costs are those expenditures that can be associated unambiguously with trustworthiness. This category includes the purchases of products such as firewalls or anti-virus software. Sometimes, direct costs may represent the incremental cost for products that offer superior trustworthiness compared with alternatives (e.g., fault-tolerant computers). Services may also be categorized as direct costs, as in the case of maintaining hot sites,²⁰ consulting and training to improve operational practices, analyzing system audit data, or upgrading hardware to improve reliability.

Direct costs vary widely, depending on the requirements of the consumer. Historically, specialized users have had the most demanding requirements and incurred the most costs; the canonical example is the military, but other institutions such as banking, air traffic control systems, and nuclear power facilities also have exacting requirements for security, safety, and reliability. The direct costs relative to trustworthiness are

¹⁹This conclusion was derived from discussions at several committee meetings.

²⁰Hot sites are physical locations where an organization may continue computer operations in the case of a major disruption, such as an earthquake that renders the normal operating site largely unusable. Organizations may maintain their own hot sites or may contract for this service with specialty firms.

often incurred by central information service units rather than charged to individuals or user departments, because the costs involve systemwide characteristics that cannot be apportioned easily among users.

Indirect Costs

The implementation of measures to improve trustworthiness often entails costs beyond those that are obvious and immediate. For example, the implementation of cryptography requires increased central processing unit (CPU) power²¹ and probably communications resources. The introduction of trustworthiness improvements also often increases system complexity (e.g., the implementation of security controls), thereby causing users to require additional technical support for problems that they otherwise might have been able to resolve themselves. Changes to complex systems increase the possibilities for bugs and, correspondingly, the costs for system maintenance and troubleshooting. Unintended consequences may also result from changes to complex systems, because it is virtually impossible to understand and anticipate all of the ramifications of changes. While it is attempting to improve aspects of trustworthiness, an intervention may introduce new vulnerabilities.

An important indirect cost is often attributable to the "hassle factor." Efforts to improve trustworthiness seldom simplify the use of a system for a consumer. For example, security controls may compel users to take additional steps and time to log in and access information and remember more elaborate policies and practices.

Another form of indirect cost is incurred when an element of trustworthiness prevents the consumer from performing some important function. In some cases these costs can be substantial, such as when a security mechanism denies a physician remote access to the medical records of an emergency patient injured when traveling, or when a flight control system prevents a pilot from moving controls in a particular way during an airborne emergency not anticipated by the design team. Such examples illustrate the difficult balance between overengineering in an attempt to prevent adverse consequences and underengineering in an attempt to avoid monetary and convenience costs.

²¹Most desktop PCs and workstations have ample CPU capacity most of the time for data encryption. This is not true for servers and other multiuser machines. In any case, public-key operations are expensive on all platforms. Servers are, in general, multitasking machines; CPU power spent encrypting one user's traffic is not available to process another user's queries. Furthermore, servers often need their high-speed network interfaces to handle the aggregate demand from many users. Ubiquitous use of software-based encryption would indeed cause noticeable degradation in total throughput; thus, many servers are being equipped with cryptographic hardware.

The economic and public policy context 183

Failure Costs

Failure costs arise when the failure or absence of a trustworthiness mechanism permits some adverse outcome to occur, such as loss of service, fraud, sabotage, or the compromise of sensitive information. For example, billing data provide a relatively good indicator of telecommunications fraud, which seems to show a bimodal distribution: a small number of extremely large thefts of service and a large number of small incidents.²² Theft of notebook computers and other devices, a rapidly increasing form of corporate security exposure,²³ illustrates a different kind of denial of service.

Another kind of failure cost is associated with recovery. Perceived growth in those costs is motivating growth in the market for insurance against computer-related (and telecommunications-related) mishaps. Although that market remains immature,²⁴ recent developments have suggested growing interest among insurers.²⁵ Traditional commercial insurance frameworks intended for physical property, equipment, and liability are being adapted for electronic contexts, although the difficulties in valuing information assets, diagnosing and reporting problems, and lack of historical data have constrained the growth of computer and telecommunications-related insurance. Insurance demand appears to be growing with loss experience, including losses arising from legal actions precipitated by information systems problems, and with increased attention to information systems in auditing and, where applicable, regulatory oversight. Although insurance can provide a negative incentive ("moral hazard") to the extent that its presence discourages greater effort in preventing loss, the terms and conditions of coverage may be designed to limit payment to those circumstances where some preventive action, such as the use of code signing,²⁶ was taken.

Some consumers prefer to insure themselves. Instead of purchasing an insurance policy, a consumer could make provisions for disaster recovery, either directly or through a third-party contractor. Another alternative is inaction. A consumer could react to incidents after the fact and initiate whatever action is deemed to be necessary. This would be consis

²²Committee discussion with Michael Diaz and Bruce Fette of Motorola, September 19, 1997.

²³For example, see Masters (1998).

²⁴Personal communication, Vincent "Chip" Boylan, executive vice president of Hilb, Eogal and Hamilton Company, September 1997.

²⁵In April 1998, Lloyds of London initiated coverage for firms to protect against hackers, viruses, and computer sabotage. See Lemos (1998).

²⁶The need for evidence may help to motivate such approaches as code signing (as discussed in Chapter 4): signing mobile code does not provide security; it provides a basis for a value judgment about potential trustworthiness of code based on reputation.

tent with consumer behavior in analogous areas (e.g., home security). It is often stated that most residential alarm sales occur after a home has been burgled, either the home of the purchaser or a neighbor's home.

The failure costs discussed so far are those costs that affect a specific consumer (e.g., the operator of an NIS that runs an electric utility). A system failure resulting from a breach in trustworthiness has costs for the public at large. An electric outage may interrupt the conduct of business (and result in possible loss of revenue) and inconvenience the public. Such costs are not borne by the service provider, the electric utility in this example, or the suppliers of any part of an NIS (because the conventional practice in the information technology industry is to disclaim all liabilities that may arise for any reason).

Imperfect Information

Consumers operate within an environment in which a great deal is unknown. The benefits deriving from greater reliability, availability, or security are difficult to articulate in detail, much less to quantify. Moreover, the consequences of inadequate trustworthiness are difficult to articulate in detail and quantify as well. There is a reluctance to make data about incidents and consequences publicly available,²⁷ so whatever data are available are likely to represent a biased sample. Not surprising, then, is the observation that relatively little information on trustworthiness is readily available to consumers. Economists refer to this state of affairs as "imperfect information," which distorts market transactions because under high levels of uncertainty, consumers will tend to purchase less of a given product or service than they otherwise would.

The difficulty of assessing the environment is compounded by the difficulty of assessing a technically complex system. Most buyers are not knowledgeable about the technical aspects of trustworthiness and, therefore, cannot conduct the informed assessment that is needed for sound decision making. Other industries, such as pharmaceuticals, have comparable characteristics, but have resolved the problem by requiring the development and disclosure of information through regulatory mandate. A consumer may not be able to assess accurately whether a particular drug is safe but can be reasonably confident that drugs obtained from approved sources have the endorsement of the Food and Drug Administra

²⁷The reluctance to make such data publicly available is intended to minimize the public perception and awareness that systems are vulnerable and have been breached. The lack of data about the likelihood, actual incidence, and consequences of problems is not a new observation; it was emphasized in Computers at Risk (CSTB, 1991) and the PCCIP report (PCCIP, 1997).

The economic and public policy context 185

tion (FDA), which confers important safety information.²⁸ Computer system trustworthiness has nothing comparable to the FDA. The problem is both the absence of standard metrics and a generally accepted organization that could conduct such assessments. There is no Consumer Reports for trustworthiness.²⁹

Metrics can be reasonably defined for some dimensions of trustworthiness (e.g., availability), while other dimensions (e.g., security) seemingly defy straightforward characterization. Any metric must be defined with respect to some formal model. The act of defining a model, however, suppresses details that might constitute vulnerabilities. For example, a "work-factor" metric for cryptosystems could be characterized by how much computation an attacker must perform to enumerate and check all possible keys for a given piece of encrypted text. The metric does not consider clever attacks and thereby renders the work-factor metric to be of dubious practical value.³⁰ Whatever formal model is conceived cannot include all possible modes of attack, because some attacks may not even have been invented. Since the definition of security metrics is problematic, the definition of aggregate trustworthiness metrics must necessarily be problematic as well.

How much risk is assumed knowingly is unclear. Anecdotal evidence suggests that in sectors accustomed to assessing and managing risk such as banking, buyer decision making relating to trustworthiness may be more explicit. Banking representatives suggested to this committee³¹ and to federal study groups recently (e.g., the President's Commission on Critical Infrastructure Protection, PCCIP) that at least some choices about using the Internet in their business reflected risk assessment. Other testimony to the committee underscored that even in the military, pursuing the primary mission may result in compromises of trustworthiness: as one representative of the DOD observed,³² one cannot necessarily shut down communica

²⁸The situation might be worse for information systems than for pharmaceuticals. The pharmaceutical interface is defined by a chemical that may be more readily understood than software, and the testing of the interaction between a chemical and the human body may be more straightforward than that for an information system. The issues here fall within a larger class of risk regulation concerns. Roger Noll, an economist at Stanford University, has described the uncertainties that confound citizens and government officials and the benefits of better identifying risks and effective responses to them. See Noll (1996).

²⁹The International Computer Security Association does "certify" security-oriented products and services, but so far its testing does not appear to be rigorous.

³⁰Consider monoalphabetic ciphers, which are sufficiently simple to solve by hand that they are the basis for daily puzzles in some newspapers. Such a cipher has a key length equivalent to about 80, far above what is currently considered exportable. One does not solve such a cipher by an exhaustive search of the key space. More powerful techniques are used.

³¹During the committee's first workshop, in October 1996.

³²During the committee's first workshop, in October 1996.

tions in the battlefield simply because security is breached. It is possible that compromised communication is preferred to the absence of all communication in some contexts.

Security experts and others who are knowledgeable about the various dimensions of trustworthiness often argue that consumers spend too little on trustworthiness because of imperfect information.³³ Limited actual experience with loss also tends to discourage investments in trustworthiness.³⁴ Of course, limited actual experience is not equivalent to an absence of risk. Some losses or problems may not even be visible, and most people have not experienced a catastrophe.

Issues Affecting Risk Management

Consumers are sensitive to the perceived opportunity cost from not indulging in risky behavior. The movement toward low-inventory, just-in-time production in various industries; outsourcing of a variety of inputs to production of goods and services; and direct computer-mediated interaction with actual and potential buyers, suppliers, partners, and competitors is motivated by factors deemed essential to commercial vitality: reduction of costs, rapidity of time to market, and responsiveness to customers. The opportunity cost of not relying more on information systems may be not being in business.³⁵

The combination of more open networking environments (e.g., the Internet) and more direct electronic transactions implies greater automated interactions among organizations. This increasing level of automated interactions is expected to result in increasing demand for major business automation systems such as PeopleSoft and SAP. How such interaction can proceed in a trustworthy manner and how differences among policies and preferences across organizations can be negotiated and arbitrated are among the questions now emerging.³⁶ One technolo

³³Current tax treatment of software, databases, and other information assets reinforce and contribute to what many feel is a tendency to undervalue information assets relative to physical assets; difficulties in appraising value for associated "property" also contributes to slow and uneven growth of insurance coverage for inadequate trustworthiness.

³⁴For example, in 1997, the Council on Competitiveness hosted a workshop for the Presidential Commission on Critical Infrastructure Protection on education and training issues relating to development and use of critical systems. A theme of the discussion was that corporate security officers and academic experts found little interest in or motivation for increasing trustworthiness by good practice. The PCCIP report emphasized shortcomings in awareness in its findings and recommendations.

³⁵See Computer Science and Telecommunications Board (1994).

³⁶The intelligence community once had a marking (ORCON) that means "Originator Controlled." Essentially, this marking states, "I pass this to you but I don't want you to

The economic and public policy context 187

gist with diverse industry experience made an analogy to the spread of AIDS, noting new concerns about the trustworthiness of the people who constitute one's social network and the dire consequences that could result from the indiscriminate expansion of one's contacts.³⁷

Another important factor for consumer risk management is the continuing growth in computer-based interaction and interdependence among individuals and organizations—the rise of a cyberspace economy and society. Greater communication among dispersed parties and collaboration and support for access for those who are mobile or in unconventional locations are easy extrapolations from current conditions. Increasingly, fewer assumptions can be made about whose information or software is running at a given time on a particular hardware, software, and communications platform. A future of greater decentralization has important implications for the locus of control for information and systems. The concepts of control inherent in traditional approaches to security, reliability, and safety may be less and less applicable during the coming years. In contrast to established NISs, where users are often preselected in some way (e.g., bank automated teller machines or the air traffic control system), new participants increasingly will include anybody who requests access. Furthermore, some of these new users will be involved in short-lived and spontaneous interactions, a situation that will create more concerns for ensuring trustworthiness.

Among the various near-term issues, the year 2000 (Y2K) problem has fostered examination and in a variety of instances changes in information systems. The publicity associated with Y2K may well influence some of the decision making; there is more speculation than data about the nature and number of changes being made, which range from focused fixes to more wholesale change.³⁸ Another relatively near-term influence is the introduction of the European Currency Unit (ECU),³⁹ which is prompting large banks and possibly other entities to alter systems to support the new currency and the likely demise of other currencies over time. The time

pass it on to anybody else without my permission." Commercial nondisclosure agreements almost uniformly contain similar clauses. This simple and easily understood policy has proved resistant to any kind of technical enforcement in shared computer systems except by mechanisms so draconian that no one will put up with them. However, schemes to protect intellectual property seem to be raising the issues again as people explore controls not only on passing something along but also on the potential number of people involved and under what conditions.

³⁷William Flanagan, during the committee's third workshop, in September 1997.

³⁸See <http://www.2k-times.com/y2kpaper.htm> for articles, news clips, and other reports about Y2K. See also de Jager (1993) and Clausing (1998).

³⁹According to the terms of the European Monetary Union, the ECU will become the Euro on January 1, 1999 (Cummins, 1998).

pressures associated with Y2K and the ECU phenomena illustrate how businesses scramble to solve problems, even though these problems could have been anticipated well beforehand. Moreover, businesses are unlikely to apply relevant extant knowledge to their problems.⁴⁰ These pressures also foster shifts from custom solutions to selection of recognized, major third-party software systems, such as SAP, thereby contributing to the increasing popularity of commercial off-the-shelf (COTS) software but inhibiting diversity, which can lead to common-mode failures and shared vulnerabilities.

Some Market Observations

The demand for primary functionality—the main purpose of a computing or communications device or system—continues to grow and is fueling demand for features. When confronted with a choice of where to spend an extra dollar, buyers tend to emphasize primary functionality; this is as evident in requests for proposals (RFPs) and actual procurement from the DOD as in the consumer or general business marketplace. Some level of trustworthiness is deemed to be essential and after that level, trustworthiness becomes a secondary differentiator. Even where the trade-off may not be obvious, perceived needs to contain costs result in development and acquisition of systems that minimize redundancy, diversity, and other features that might otherwise enhance trustworthiness.

Products that address problems experienced by consumers have been well received, as are products (e.g., firewalls) that appear to address specific well-known problems. Consumers buy firewalls because they have associated that mechanism with the ability to connect to the Internet, even though considerable risks may remain despite the use of firewalls. Some consumers who have full knowledge of the limited effectiveness of mechanisms such as firewalls may still use them with the goal of appearing to have trustworthiness, but without undertaking the hard work that achieving true trustworthiness demands; this may be the era of patent medicines for information technology.

The development of the mass market has been accompanied by a shift in systems development and expertise from user organizations to vendors. The proliferation and falling relative prices for commercial technology means that organizations that once would develop systems they wanted themselves are more likely to buy at least components if not entire systems.⁴¹ This trend toward COTS systems and an increasing homoge

⁴⁰William Flanagan, during the committee's third workshop, in September 1997.

⁴¹At the committee's workshop in September 1997, Iang Jeon of Liberty Financial, for example, observed that up until 3 to 4 years earlier financial institutions had to set up

The economic and public policy context 189

neity of computing platforms, communications infrastructure, and software is discussed in the next section as a major force in the producer landscape.

Findings

1. The costs associated with improved trustworthiness are often incurred by central units of an organization because such costs reflect systemwide characteristics of an NIS and cannot be easily apportioned.

2. One important cost of greater trustworthiness is related to the "hassle factor." Trustworthy systems tend to be more cumbersome to use. This is one reason that costs for the consumer are not equivalent to price.

3. Decision making about trustworthy systems occurs within the context of imperfect information, which increases the level of uncertainty regarding the benefits of trustworthiness initiatives and therefore serves as a disincentive to invest in trustworthiness, thus distorting the market for trustworthiness. The absence of standard metrics and a recognized organization to conduct assessments of trustworthiness is an important contributing factor to the problem of imperfect information. In some industries, such as pharmaceuticals, regulatory mandate has resolved this problem by requiring the development and disclosure of information.

4. Useful metrics for the security dimension of trustworthiness are unlikely to be developed because the corresponding formal model for any particular metric is necessarily incomplete. Therefore, useful aggregate metrics for trustworthiness are not likely to be developed either.

5. The combination of more open and decentralized networking environments and an increasing use of electronic communications and transactions suggests an increasing demand for major business automation systems. This continuing decentralization may render less and less applicable the concepts of control inherent in traditional approaches to security, reliability, and safety. In particular, there will be an increasing need for more individuals to be able to make trustworthiness judgments on an ad hoc, real-time basis.

6. Other things being equal, consumers prefer to purchase greater functionality rather than improved trustworthiness. Products that address problems that have been experienced by consumers or are perceived to address specific well-known problems have been well received.

software and telecom systems themselves to support electronic distribution, whereas now it is easier to rely on people whose business is developing packaged software and delivering telecommunications services.

Producers and Trustworthinesss

The Larger Marketplace and the Trend Toward Homogeneity

Before the producers of trustworthiness products, services, and features are discussed, a brief note is warranted on the important trends concerning COTS components and homogeneity in the general marketplace, and the implications of those trends for trustworthiness. Current computing platforms, as well as communications infrastructure and software, are generally homogeneous. Operating systems and computing platforms are dominated by Microsoft Windows and the Intel x86 compatible processor family.⁴² Secondary characteristics—display, network interfaces, disks—are made uniform by the adoption of technological standards (e.g., VGA graphics interface or IDE and SCSI disk interfaces) or are presented to application software as common interfaces by operating systems software in the form of device drivers and hardware adaptation layers.

The communications infrastructure today is also fairly homogeneous. Local area networks are typically Ethernets or Token Rings, although some increased diversity is being introduced by asynchronous transfer mode (ATM) networks and the various high-speed Ethernets. Wide area networks are constructed from routers, most of which are sold by a few manufacturers.⁴³ The software that controls these networks is also homogeneous at multiple levels. A single stack of protocols manages the Internet, and all the Internet protocol implementations descend from a few. The core Internet Protocol (IP) works well over a diverse set of network technologies, further contributing to homogeneity.

In addition to the existing state of relative homogeneity with respect to computing platforms and communications, the important trends in software suggest a continuing decrease in heterogeneity in the coming years. An important reason for this decrease in heterogeneity is the rising popularity of COTS software that is driven by cost considerations and risk reduction, insofar as COTS products are known entities and readily available. Scripting languages and COTS software provide the context

⁴²In 1997, a significant majority of computer systems sold (85 percent of personal computers and servers by unit volume) contained some version of Intel's "x86" microprocessor (manufactured by either Intel Corporation or one of a small number of others) to implement an IBM-compatible PC architecture. When deployed as personal computers, a significant majority are running a version of the Microsoft Windows operating system. Less than 10 percent of personal computers are a variant of the architecture designed and sold by Apple Computer; a small percentage are variant architectures made by Sun Microsystems, Silicon Graphics, Digital Equipment Corporation, and others. Many among this last group of systems run versions of the UNIX operating system.

⁴³Cisco Systems and Bay Networks, for example, dominate the router market.

The economic and public policy context 191

for the reuse of components and for their assembly into required configurations, with only limited new programming required for custom components. Consequently, user organizations have less need for systems development expertise. The success of large middleware packages underscores the economic and other benefits that users perceive in COTS software. The continued use of SAP, the Web (e.g., Hypertext Transfer Protocol [HTTP]), and a few other software packages favor particular software components, data formats, work flows, and vocabularies.

Risks of Homogeneity

The similarity intrinsic in the component systems of a homogeneous collection implies that these component systems share vulnerabilities. A successful attack on one system is then likely to succeed on other systems as well—the antithesis of what is desired for implementing trustworthiness. Moreover, today's dominant computing and communications environments are based on hardware and software that were not designed with security in mind; consequently, these systems are not difficult to compromise, as discussed in previous chapters.

There is, therefore, some tension between homogeneity and trustworthiness. Powerful forces make technological homogeneity compelling (see Box 6.1), but some attributes of trustworthiness benefit from diversity (see Chapter 5). On the other hand, a widely used trustworthy operating system might be superior to a variety of nontrustworthy operating systems; diversity, per se, is not equivalent to increased trustworthiness.

BOX 6.1

The Rationale for Homogeneity

The existence of a homogeneous computing and communications environment is not an accident. Strong forces favor homogeneity:

• Homogeneity is advantageous for the sale and use of popular software. A larger market gives providers of hardware and software incentives for entry, and providers can also exploit economies of scale.

• Enormous leverage results when computers can communicate and share data, especially in ways that are not anticipated when the computers are procured or the data are created. Homogeneity simplifies interoperability between systems.

• Homogeneity supports more efficient transfer of skills within organizations, effectively lowering the cost of computerizing additional functions.

• Homogeneity also leads to increased skill-lifetimes, because a skill is likely to remain useful even after computing platforms are upgraded.

• Homogeneity enables aggregations of resources to strengthen design, implementation, and testing.

Technological convergence may also be realized through the market dominance of a few suppliers of key components, with monopoly as the limit case when technological homogeneity is dictated by the monopolist.⁴⁴ However, the number of suppliers could grow as a result of the diffusion of computing into embedded, ubiquitous environments; the diversification and interoperability of communications services; and the continued integration of computing and communications into organizations within various market niches.

Producers and Their Costs

Insofar as trustworthiness is integral to the design of information technology products and services, trustworthiness should be pervasive throughout the marketplace for such products and services. However, trustworthiness is often considered only after a system is implemented, so there are firms that develop and market products and services specifically targeted at improving the trustworthiness of operational NISs. The marketplace for trustworthiness—in both of these senses—will be explored in some detail after some of the key issues associated with the costs of producing trustworthiness are discussed.

The costs of trustworthiness are difficult to assess and cannot all be quantified, even using order-of-magnitude estimates. Time is a major "currency" cited by vendors, who worry about time from product concept until commercial release. Data on relevant costs are scarce; those cited may be of questionable quality, and analyses of costs tend to be limited at best.

The costs associated with developing trustworthiness features, products, and services have a major labor component. Some vendors also incur research-related expenditures in their efforts to bring trustworthiness products to market, although most of this "research" is actually development. The costs associated with security mechanisms are emphasized in this section because of the pivotal role that security controls play as enablers of other aspects of trustworthiness and the expectation that, in the future, trustworthiness problems will be associated increasingly with security concerns. The purpose of this section is not to provide an exhaustive articulation of all producer costs; instead, the intent is to highlight those producer cost issues that are particularly germane to trustworthiness.

⁴⁴Although both standards and monopolies can provide the benefits of homogeneity, only standards enable the competition necessary to ensure that consumers may affect the trustworthiness of available products. Standards are discussed in detail in the section titled "Standards and Criteria."

The economic and public policy context 193

Costs of Integration and Testing

NIS trustworthiness is inherently a system-level property, and, therefore, the costs associated with improving trustworthiness inevitably involve the costs of integration and testing. These costs will vary, depending on whether or to what extent a mechanism is integrated into a system. A relatively stand-alone mechanism, such as an initial password screen to enter a system, might be written as a software module independently from the remaining modules of the project and have minimal impact on system integration, testing, documentation, and training activities. The costs are readily identifiable and low. Another example of a relatively stand-alone solution is firewalls.

Security controls that have a moderate effect on software development and cost include those that impose multiple access modes within a system. Some menus, data sets, data items, or other appropriate subsets of the system may have unlimited access, whereas others may limit access to certain individuals, organizations, or time of day, or limited functionality (e.g., read access only). These controls affect functionality throughout the system and, therefore, impose a moderate impact on system integration, testing, documentation, and training activities.

Finally, costs are high and difficult to identify specifically in systems where controls are pervasive: the authentication of each user is rigorous; each transaction is scrutinized for its validity and verified against appropriate databases; external transactions are subject to encryption; audit trails are maintained to facilitate routine and ad hoc audits of transactions; and general access levels may also be employed. If security or other attributes are integral to much of the functionality throughout the system, associated controls greatly affect system integration, testing, documentation, and training activities. The controls contribute to the complexity of the system; the debugging activity is more difficult and may require a longer period.

Identifying the Specific Costs Associated with Trustworthiness

Accurate estimation of the direct costs associated with specific project features requires a complex and time-consuming analysis that seems to be seldom performed.⁴⁵ Except in the case of stand-alone products, it is often difficult to separate the costs of "regular" functionality from the costs of "enhanced trustworthiness capability." This allocation can be arbitrary. The same could be said for the further distinction between the costs associated with trustworthiness and general overhead costs. Com

⁴⁵A committee conclusion based on its deliberations.

pounding the difficulty of ascertaining accurate cost data is the fact that advocates or opponents of a particular trustworthiness intervention may attempt to manipulate cost data in marshalling their arguments.

Costing methodologies have been published, and they address variation in costs and trade-offs owing to product requirements, producer practices, and other sensitivity factors. These models tend to cover only the development cycle, and their assumptions about the way effort is expended in a software project may not apply in the contemporary market environment, in which some "development" may be purposely postponed to an upgrade in the effort to reduce the time to market.⁴⁶

Time to Market

Many of the segments within the information technology marketplace are intensely competitive, where market share—not profit margin—is the primary business objective. In such markets, a product (e.g., Web browsers) that is available early has the opportunity to develop a customer base or become established as the de facto standard. Consequently, minimizing the time to market is a critical consideration for producers.

Each feature is examined to determine whether its inclusion in the product is necessary for the product to be competitive in the marketplace. Generally, those features with direct customer appeal win. Subtle, hard-to-demonstrate, and pervasive properties—which tend to characterize trustworthiness attributes—tend to be rejected. Trustworthiness features that require extensive integration throughout a product also tend to be omitted, because of the time required to properly integrate and test such features.

Other Issues

To some extent, costs may occur and be traded off at varying points in the life cycle of a product. The discussion in Chapter 3 suggests that the cost of effecting a software change increases through the development cycle (i.e., the later a change is instituted, the more it will cost). Costs may

⁴⁶The constructive cost model (COCOMO), a well-developed cost model for software engineering, is the centerpiece of Barry Boehm's book, Software Engineering Economics (Boehm, 1981). Boehm discusses security and privacy issues and the reasons these are excluded in COCOMO (p. 490). Standard COCOMO does not include such effects as added product features (security markings, operational controls), reduced access to documentation, and added documentation control. Since these requirements in their stringent form are relatively rare, and even then generally add only 10 percent to project costs, COCOMO does not include this as an added factor on the grounds of model parsimony.

The economic and public policy context 195

also be traded off from the development to the support phase of the system life cycle. A poor implementation of trustworthiness characteristics during development can translate into higher costs for technical support operations.⁴⁷ Not only may costs be shifted over time, but costs may also be incurred by different organizational units or by consumers.

The difficulty of demonstrating and sustaining success in achieving trustworthiness—one can, at best, test a product or practice against a recognized risk—imply a dynamic process of iteration.⁴⁸ In some cases, a lot of care goes into anticipating risks and addressing them preemptively,⁴⁹ in other cases the trial and error process seems less systematic, and in all cases actual experience drives improvement. Antivirus software provides an example of the inherent limit of anticipation since virus producers continually introduce new strains against which anti-virus software might not work. Thus, the antivirus product development process involves frequent upgrades in response to new forms of viruses. Netscape's approach of offering a reward for detection of security flaws puts another face on iteration: it implies that the cost of finding problems, and perhaps of developing fixes, could be shared between the producer and the consumer, and it may increase the rate and level at which problems are reported.⁵⁰ The reality of iteration makes it difficult to estimate costs fully up front, except to the extent that an iteratively escalating process can be modeled and costed. It also argues for the benefit of retro-spective analysis to support such costing.

Research relating to trustworthiness could help to reduce costs, but that outcome depends on better understanding of the nature and incidence of costs. Having ways to think about cost ("cost models"), even in the absence of appropriate data, can help in understanding how trustworthiness is perceived or valued and how potential incentives for increasing it may evolve. The expectation that discontinuities will occur—that inci

⁴⁷Both the fact that later life cycle costs are not borne directly by the developers (i.e., technical support is often a distinct organizational unit from development) and the fact that these costs are deferred could act as inducements to shift costs to later stages in the product life cycle.

⁴⁸The iterative process has been compared to an arms race, an escalation of measures and countermeasures as new problems are discovered, some arising in response to previous fixes. Note that target risks may be poorly understood or unspecified, such as the goal of avoiding system crashes due to bugs or unexpected attacks.

⁴⁹From a research perspective, the staged nature of progress raises questions about the relative payoff to investing in successor (major improvement) technologies relative to incremental improvements to existing technologies.

⁵⁰An attacker might discover vulnerabilities and not report them, hoping to exploit them for more substantial gains later. This is a high-consequence, but not necessarily a high-likelihood, prospect.

dents attributable to inadequate trustworthiness will result in corrective action and new efforts at prevention or recovery—suggests that how costs are identified and calculated may be relatively fluid.⁵¹

The Market for Trustworthiness

The supply of trustworthiness technology includes both products and services specifically offered to support one or more aspects of trustworthiness and the trustworthiness of NISs generally. This definition is very broad and could be interpreted to include nearly anything that assists in the design, development, integration, testing, operation, or maintenance of an NIS. This discussion focuses on those products and services that are intended primarily to promote trustworthiness. Because of the special enabling role that security plays with respect to trustworthiness, security products and services are emphasized.

Trustworthiness is a systemwide attribute. The cost required to secure a system is not strictly proportional to the number of people using that system.⁵² Consequently, as an NIS is implemented and the number of connections increases, it is plausible to discover that the per-connection cost declines. Some technologies, such as those associated with virtual private networks and higher-quality user authentication, do impose some per-user or per-computer costs. Another important reason that security expenditures, as separately identifiable data, are likely to decline results from the integration of security features into general-purpose information technology products. For example, version 4 of the Netscape browser includes support for SSL and S/MIME, which implement security properties. If this browser were categorized as a "nonsecurity" product, then the market statistics for security would be understated. Another such example is a packet-filtering router—it is a router, but it also implements security. Finally, as in other segments of the information technology marketplace, competitive pressures and technological innovations exert

⁵¹Committee members noted the experience of the market research firm Gartner Group, which found its assessment of the costs of PC ownership reduced to a sound-bite—raising questions about assumptions and about popular capacity to consider more than a single number. The likelihood of change does not diminish the value of studying costs for older technologies and strategies, but it does raise questions about where it is sensible to extrapolate from the past. It also points to the need to understand sensitivity factors and assumptions.

⁵²One way of looking at this is the "hard on the outside, soft and chewy on the inside" phenomenon, in which a collection of unprotected nodes (whose individual security cost is essentially zero, so that the aggregate is independent of the number of nodes) are huddled behind a small number of firewall/gateway nodes. Security does not become cheaper as the internal network grows.

The economic and public policy context 197

downward pressure on prices. These observations also suggest that as security and other aspects of trustworthiness are increasingly incorporated into other products, the task of compiling accurate market data and forecasts for security or trustworthiness will become ever more difficult.

The committee did review a limited number of industry analyses that were compiled by various market research analysis or financial services companies. The data reviewed supported the argument that while the market for security products is growing, this market is declining in relative terms because of the higher growth rate in other sectors of the information technology marketplace. However, the committee was ambivalent about the inclusion of any such data in this report, because such inclusion could be construed as an endorsement of the selected data, methodology, analysis, or firm. The committee was not in a position to make such a determination.

In 1997 and 1998, rapid consolidation was taking place in the computer and network security marketplace, turning small companies into larger and more aggressive firms. The rapid growth of the Internet has driven increased demand, especially by larger and more sophisticated customers who have greater knowledge and demands for security requirements and desire integrated security solutions. Thus, the consolidation in this market is expected to continue. General computer and communications vendors are also increasingly interested in security, thereby further contributing to the turbulent state of the computer and network security marketplace.⁵³

Supply and Demand Considerations

Availability is an aspect of trustworthiness that is readily measurable and is highly valued by the public; it certainly contributes to the success of fault-tolerant computer systems (e.g., Tandem and Stratus). Some market successes also exist within the security marketplace, although the demand for security continues to be relatively limited. Niches exist for targeted products, such as firewalls and antivirus software, and for services such as online updates of antivirus software. These two niches are very competitive; satisfying third-party assessment is provided through trade magazines⁵⁴ or the International Computer Security's Association certification requirements and constitutes an important competitive advantage.

⁵³For example, note the significant security content in NT Version 5, and Cisco's recent acquisition of a proxy firewall supplier.

⁵⁴Jimmy Kuo, McAffee Associates, during the committee's third workshop, in September 1997.

Of course, vendors are very keen to provide what potential customers desire with respect to the nature, quantity, pricing, and efficacy of trustworthiness features, products, and services. However, vendors have found that, although people claim that trustworthiness is important in the abstract, when it comes time to spend money, nontrustworthiness expenditures often take precedence. An illustrative case is the effort by Digital Equipment Corporation (DEC) to develop a system that would satisfy DOD's most stringent criteria for so-called trusted systems. After making a considerable investment, DEC canceled the project when it became clear that sufficient demand for the system would not materialize. Experiments with trusted operating systems were also terminated by other major system vendors when they, too, were discouraged by a lack of commercial interest.

Findings

1. Current computing platforms, communications infrastructure, and software are relatively homogeneous, and the degree of homogeneity is expected to increase in the future. Homogeneity tends to cause NISs to be more vulnerable.

2. The increasing use of COTS software is causing user organizations to decrease their level of expertise in system development.

3. Production costs associated with trustworthiness are difficult to assess. An improved understanding and better models are needed. There is a paucity of data. The data that are available are questionable, in part because of the difficulties in distinguishing trustworthiness costs from other direct product costs and overhead costs.

4. Production costs associated with integration and testing represent a substantial proportion of a producer's total costs for improving trustworthiness.

5. Time-to-market considerations discourage the inclusion of trustworthiness features and encourage the postponement of trustworthiness to later stages of the product life cycle.

6. The average expenditure for security per Internet/intranet-capable connection has been declining. This trend is expected to continue because security (and trustworthiness generally) expenditures are relatively independent of the number of connections or users, although the use of virtual private networks and higher-quality user authentication technologies does impose some per-user or per-computer costs. Additional influences include competitive pressures that are driving prices down and the potential to understate security expenditures as they become more difficult to identify specifically from general expenditures for information technology products and services.

The economic and public policy context 199

Standards and Criteria

The development and adoption of standards constitute one response to the challenge of appraising trustworthiness and mitigating difficulties that arise from imperfect information. Standards can simplify the decision-making process for the purchasers of trustworthiness. They can also simplify the design and production decisions for the producers of trustworthiness by narrowing the field of choices (e.g., adherence to interoperability standards facilitates interconnection among subsystems). Compliance with standards or guidelines supplied by the federal government or an authoritative independent standards-setting organization—such as the federal information processing standards (FIPS) of the National Institute of Standards and Technology (NIST), standards of the American National Standards Institute (ANSI), or standards that may result from the Information Infrastructure Standards Panel (IISP)—provides both third-party validation of a selection of technology and potential relief from liability.⁵⁵ There is also the broader notion of criteria (e.g., the U.S. Trusted Computer System Evaluation Criteria [TCSEC]), which includes the consideration of processes and attributes that cannot be assessed by direct examination of the artifact in question. For example, criteria may involve explicit or implicit comparisons with other products or systems. Criteria may also take the form of authoritative statements of how a system should or should not be designed and operated, complemented by some means of demonstrating compliance.⁵⁶

The Character and Context of Standards

The Data Encryption Standard (DES) FIPS is an example of an interoperability standard; it defines the mathematical function that a compliant device must implement to ensure that data encrypted by manufacturer A's DES box can be decrypted using a box made by manufacturer B, and there are a set of tests used to determine if the function has been

⁵⁵Technology transfer and avoidance of at least some known problems lie behind past government efforts to promulgate guidelines and criteria for trusted systems—TCSEC and more recent international harmonized criteria that build on the U.S. TCSEC and comparable efforts overseas. Lack of widespread adoption of such guidelines and criteria appears to relate at least as much, and probably more, to nontechnological aspects (e.g., distrust of or limited communication with government sponsors of these programs, delays associated with compliance testing, little market demand) as to issues of technical compliance (e.g., difficulty in satisfying the standard).

⁵⁶Such criteria have increased trustworthiness for transportation equipment, devices that transmit radio frequency, and other complex systems that operate in networked environments.

implemented.⁵⁷ By contrast, FIPS 140-1 (Security Requirements for Cryptographic Modules) is largely a performance standard encompassing security functionality and assurance. It is definitely not an interoperability standard. Standards arising in the Internet context are expected to promote the implementation of encryption (e.g., IPsec, S/MIME, SSL), while fostering interoperability. Apart from some consideration of key length and algorithm choice, these standards do not treat cryptographic strength or resistance to attack by other means.

In the Internet environment, the Internet Engineering Task Force (IETF; see Box 6.2) has focused on the security aspects of Internet standards, addressing both specific security standards and the larger problem of reviewing other standards to ensure that they either are secure or can have security added when needed.⁵⁸ In other venues, such as trade associations, standards setting for computing and communications is intended to foster interoperability and/or proactively forestall government intervention. Computing and communications trade associations and related groups are directing increasing attention to standards related to trustworthiness. For example, the Information Technology Industry Council has addressed a range of standards and security concerns, and security and privacy are emphases of the Smart Card Forum. A number of these industry-based efforts emphasize security to protect company assets, and they are often undertaken to deter regulation.

There is more history of standards setting in the areas of safety and reliability. In an effort to ensure that the best available techniques are used in certain classes of safety-critical systems, a variety of standards have been developed by government agencies, industry groups, and individual companies (see Box 6.3 for examples). The use of specific techniques and procedures in development is in many cases influenced heavily by these standards, and in some cases their use is required for systems to be supplied to a government or for systems that may affect public safety. Domain-specific standards facilitate the needs of the particular domain, but they deter common solutions across market segments.

⁵⁷This FIPS consists of an algorithm description, a set of test vectors, and a very subsidiary set of implementation cautions. It is in no sense a security standard, except implicitly in that its "FIPSness" implies that somebody in the government said it was good enough for certain use. In particular, one cannot exceed the standard and be more secure than DES, since that would take a different algorithm and fail the interoperability test. If someone goes off and puts DES in some stupid box that, for example, coughs up the key on demand, then someone built a stupid box, but it would not be in violation of the FIPS. This FIPS does not specify how one must implement the DES internally; it specifies only the interface.

⁵⁸Placing emphasis on the "larger problem" is a recent phenomenon.

The economic and public policy context 201

BOX 6.2

Internet Standards and the Internet Engineering Task Force

Most Internet standards are developed by a group called the Internet Engineering Task Force (IETF). Although this is by no means a requirement—any protocol can be run on top of the basic Internet protocols, and there are other bodies that develop standards for specific areas, such as the World Wide Web Consortium (W3C)—most of what we use today on the Internet was codified by the IETF. Although the IETF was initially funded by the National Science Foundation, the IETF has no formal endorsement from the federal government. The IETF's estimated 1998 operating budget is $1.7 million (Wilson, 1998).

The IETF is unusual in a number of respects. There is no formal membership; as a consequence, there is no voting. Instead, standards are accepted by "rough consensus and running code." Standards are developed, and RFCs (nominally "requests for comments") are written by assorted working groups. The working groups are organized into a handful of areas; the directors of these areas collectively form the Internet Engineering Steering Group (IESG). Overall architectural development is nominally directed by the Internet Architecture Board (IAB). The membership of the IESG and IAB is chosen by a nominating committee that is randomly selected from a group of volunteers who are IETF attendees. Final approval is vested in the board of trustees of the Internet Society (ISOC).

Given this procedural context, the process of adopting an Internet standard is complex. Apart from prescribed milestones, the IESG occasionally promulgates a new policy that will apply to all standards-track RFCs. In the spring of 1997, just such a policy was adopted with respect to security: security is important. Specifically, it was decided that the hoary phrase "security considerations are not addressed in this memo" will no longer be permitted in RFCs. Instead, a real security analysis must be done. Protocol designers must consider what vulnerabilities are present and what the consequences would be if each were exploited. Furthermore, the designers must analyze existing security mechanisms to see if some other standard would solve the problems. Only if none is suitable should custom mechanisms be designed. One choice has been ruled out: cleartext conventional passwords are not permitted. When passwords are to be used, some cryptographic mechanism must be employed for authentication purposes.

Further, the IAB and IESG jointly adopted a statement endorsing strong cryptography (Carpenter and Baker, 1996). Limited key lengths, mandatory key recovery, and export controls were specifically rejected. Although this statement does conflict with various national policies, including those of the United States, the belief was that an international technical organization should use only technically sound mechanisms, regardless of limitations imposed by particular governments.

Standards and Trustworthiness

The notion of specification is at the core of all characterizations of trustworthiness attributes. Unless a precise, testable definition for an attribute such as reliability exists, it will not be possible to determine whether the requirements of the definition have been fulfilled. The defi

BOX 6.3

Examples of Safety Standards

The RTCA¹ standard DO-178B, entitled "Software Considerations in Airborne Systems and Equipment Certification," is a standard developed by the commercial air transport industry for software used in commercial aircraft and is adhered to by virtually all developers of aircraft systems as a part of the aircraft certification process. DO-178B defines criticality levels for aircraft software, and different development techniques are required for each level. The standard prescribes development practices, documentation, and recording requirements for all phases of the software life cycle. In addition to defining many aspects of software development, the standard specifies assurance requirements through which the developer demonstrates compliance with the standard to regulatory agencies.

The British Ministry of Defense Standard 00-55, "Requirements for Safety-Related Software in Defense Equipment," is a controversial standard because it mandates the use of mechanical analysis techniques. Section 36.5 of the standard states, for example, the following about the source code for a safety-related system:

• "Static analysis² in accordance with 26.2 shall be performed on the whole of the source code to verify that the source code is well formed and free of anomalies that would affect the safety of the system."

• "Proof obligations³ shall be: (a) constructed to verify that the code is a correct refinement of the software design and does nothing that is not specified; (b) discharged by means of formal argument."

¹RTCA used to be an acronym for Radio Technical Commission for Aeronautics, but the organization's name was formally changed to just RTCA in 1991.

²This is the analysis of a computer program by any means other than executing it. A grammar checker is an example of a simple static analyzer.

³A proof obligation is a proposition that must be true in order for some larger aspect of a system to hold. It is something that has to be shown to be true independently of the larger aspect, usually to allow a proof to be developed for a theorem. A theorem-proving system might be able to establish that a program satisfies its specification, but only if it is able to make certain "assumptions." For the theorem to be proven, the assumptions must be shown to hold separately.

nitions in use by the community permit availability and reliability to be measured and compared, thereby allowing a system to be regarded as "sufficiently reliable," for example, if the measured or predicted reliability of the system meets or exceeds some prescribed threshold. An analogous situation does not exist for security, where there does not seem to be a testable definition and where a specification cannot anticipate all of the problems that may arise.

There are exceptions, as is illustrated by the DES, whose presence and widespread adoption clearly benefited all concerned. Yet security experts

The economic and public policy context 203

BOX 6.4

Cryptographic Challenges

The design and implementation of secure cryptographic algorithms, as well as protocols that make use of such algorithms, have proven to be difficult. Over the last 20 years (the interval during which public interest in cryptography has grown substantially), there have been many examples of missteps:

• Symmetric and public-key cryptographic algorithms and one-way hash functions developed by respected members of the academic and commercial cryptographic community all too often have succumbed to cryptanalysis within a few years after being introduced. Examples include the Merkle-Hellman trapdoor knapsack public-key algorithm, some versions of the FEAL cipher, the Snefru one-way hash function, and the MD4 hash algorithm.

• Authentication and key-management protocols have suffered a similar fate, as they have been shown to be vulnerable to various sorts of attacks that undermine the security presumed provided by them. Examples include the original Needham-Schroeder key-management protocol and the various protocols that were intended to repair its flaws (Needham and Schroeder, 1978, 1987; Denning and Sacco, 1981).

These experiences emphasize the need for cryptographic algorithm standards and security protocol standards that have been carefully developed and vetted. Because implementations of security technology represent a major source of vulnerabilities, there is also a need for high-assurance implementations of this technology. This latter need has sometimes been met through the use of government or third-party evaluation programs for hardware or software components supporting cryptography or cryptographic protocols (e.g., in connection with FIPS 1401 and ANSI X9.17 standards).

As an example, consider the Data Encryption Standard (DES). The DES was developed initially by IBM and submitted as a FIPS in the mid-1970s. Even though the design of DES was public, the algorithm met with considerable skepticism from some members of the largely academic cryptographic community because the design principles were not disclosed and because of concerns over the key size. Over time, as this community developed improved cryptanalytic methods, DES actually came to be viewed as a well-designed algorithm. DES became widely used, promoting interoperability among a number of security products and applications. DES hardware (and, later, software) was evaluated and certified by NIST, providing independent assurance of an implementation.

However, the key size is now too short for today's technology, as demonstrated in July 1998, when a team under the auspices of the Electronic Frontier Foundation (1998) designed and built a key search engine for less than $250,000 (the cost of the parts). Although DES has exceeded its originally projected lifetime, it is an open question at what time in the past brute-force cracking became economically feasible, especially for nation-states (Wiener, 1994; Meissner, 1976; Hellman, 1979).

consider DES to be an unusual case, given other experiences with standards, which illustrate the risk of treating standards as indicators of assurance (see Box 6.4).

Technical standards imply extensive discussion, review, and analysis by experts and stakeholders, which minimizes the number of remaining flaws.⁵⁹ However, the existence of standards also introduces risks. Technical standards may provide an adversary with detailed technical information that facilitates the discovery of flaws. Interoperability facilitates legitimate use, but it also allows a vulnerability to be exploited in multiple contexts. Finally, it is easier to mount attacks against multiple representatives of a single standard than against differing implementations of several standards.

Security-based Criteria and Evaluation

European and North American governments⁶⁰ are moving to establish a unified security criteria, called the Common Criteria for Information Technology Security Evaluation. The Common Criteria (CCv2)⁶¹ attempts to reconcile the requirements of the Canadian Trusted Computer Product Evaluation Criteria (CTCPEC) (Canadian System Security Centre, 1993), the European Information Technology Security Evaluation Criteria (ITSEC) (Senior Officials Group, 1991), and the United States Trusted Computer System Evaluation Criteria (TCSEC) (U.S. DOD, 1985).

All these criteria share two underlying dimensions: the extent of the security mechanisms being rated, often called the functionality axis, and the degree to which the mechanisms can be trusted to perform their functions correctly, often called the assurance axis (Figure 6.1). Examples of security functionality include authentication mechanisms, access control lists, and cryptographic features. Examples of assurance steps are testing, examination by independent teams, use of formal methods, and the degree of rigor in the development process.

The rating received by a given product or system is a combination of both components (see Box 6.5). For illustrative purposes and to avoid the semantic baggage of using a particular criterion's terminology, the discussion that follows uses a hypothetical rating system of 1 to 5 on each axis, where [f1,a1] is a system with minimal security functions and minimal trustworthiness, and [f5,a5] is one that exhibits state of the art in each. The reader should assume that a "reasonable" definition may be articulated for each, which is a nontrivial assumption. The discussion that

⁵⁹This is especially true for standards that are a result of consortia or other cooperative efforts among the stakeholders. For de facto standards that derive from a dominant vendor, one might also expect reduced design flaws, or at least a general awareness of the problems and work-arounds identified.

⁶⁰United States, Canada, France, Germany, the United Kingdom, and the Netherlands.

⁶¹Information available online at <http://csrc.nist.gov/cc/ccv20/ccv2list.htm>.

The economic and public policy context 205

FIGURE 6.1 Hypothetical security-based criteria.

follows is not dependent on any specific definition or process to assign the values.

Laying out the possible ratings on a two-dimensional grid quickly makes clear distinct "zones" in the grid (see Figure 6.1). Along the diagonal ([f1,a1] to [f5,a5]) lie what could be viewed as "balanced" systems, in which the effort placed in assurance matches the security functionality being provided. On one side of the diagonal is the "danger zone" of high functionality relative to assurance, such as [f3,a1]. The danger is, of course, that the product exhibits superficial characteristics of security but cannot be trusted, because no significant effort has been made to show that the features work as promised, especially in the face of hostile analysis and attack. Interestingly, this is the zone in which nearly all commercial security products lie today, because features sell, whereas assurance is the concern of the specialist. On the other side of the diagonal is the "conservative zone," in which mechanisms are placed under a high degree of scrutiny relative to functionality, such as a [f1,a3] system.

How much is a high rating worth? While it is plausible to conclude that a higher rating has a higher market value than a lower one, the rating is only one of many factors that consumers consider in their decision making, and the discussion in the cost section suggests that other considerations, such as functionality, take precedence. Vendors are keenly interested in the value of a rating because all current rating systems compel

BOX 6.5

The TCSEC, ITSEC, and Common Criteria: Two Values or One?

The oldest set of criteria is the Trusted Computer System Evaluation Criteria (TCSEC) (U.S. DOD, 1985). During its development there was a substantial debate as to the format of the rating. One school felt that the rating should directly reflect the underlying two-axis structure. A rating would therefore consist of two parts. Education and discretion on the part of the evaluators would prevent "danger zone" products from being deployed. A second school, and the one that prevailed, held that a two-part rating was both excessively complex and risky. The ratings were accordingly devised as a single value that attempted to define "balanced" systems alone.¹ Although the single-value approach precluded the "danger zone," it also precluded the "conservative zone," where many potentially useful products could exist, especially in a networked world.²

The Common Criteria Version 2 (CCv2) follows the Information Technology Security Evaluation Criteria (ITSEC) in taking the opposite approach. Ratings have two values: a "protection profile" that seeks to capture the security functionality, and an "evaluation assurance level" that seeks to capture the degree of trust one could place in that functionality. The CCv2 is then conceptually the mirror image of the TCSEC: "danger zone" products are possible and must be discouraged through education and de facto regulatory steps; "conservative zone" products are allowed; and a product's rating is both more complex and more informative. Also, the ability to add new protection profiles allows for the possibility that the criteria can adapt to new technologies and increased threat. The Common Criteria are also more modular, less confidentiality centric, and more current³ than the TCSEC.

¹As finally devised, the diagonal was not completely followed. An A1 rating adds assurance steps to a B3 rating but maintains the same security functions. Thus, if a B3 rating is seen as [f4,a4], an A1 is [f4,a5].

²The choice of balance is indeed a problem. Refer to Neumann (1990) and Ware (1995) for critical analyses of TCSEC and the criteria approach.

³For example, the Common Criteria includes provisions for nonrepudiation, a critical concern in electronic commerce systems.

the vendor to invest in satisfying the criteria and, in some cases, in paying for the evaluation process itself. The investments can be substantial, particularly in terms of opportunity cost and lost sales because the extended time to market is added to the direct cost of becoming "evaluation ready."

A rating is also useful as a reflection of the ability of a product to resist analysis and manipulation by the threat; in this context, the value of a rating is called the "operational value." As noted before, threats are ever increasing, and therefore, the operational value of a rating correspondingly decreases over time. (One way to retard growth in the sophistication of attackers is to keep some aspects of a design implementation secret. See Appendix I for a discussion.) This depreciation of a rating's

The economic and public policy context 207

value occurs on both axes. On the functional axis, for example, cryptographic key lengths that are perfectly adequate at one time may become wholly inadequate several years later, owing to the increased computing power available to the threat. A similar phenomenon occurs on the assurance axis. Assurance steps attempt to uncover flaws before a product is exposed to the threat; in some sense they attempt to take a "deeper look" at a mechanism than any element of the threat could afford. Assurance steps that "look deeper" than a single attacker can look may have been adequate before the onset of the Internet, but are made obsolete by an environment that facilitates anonymous and unplanned technical collaboration among like-minded individuals. The depreciation of the operational value of a particular rating has not been a concern for individual products, because it has been slower than the rate at which products become obsolete or uncompetitive for other reasons.

Criteria, because they must cover a variety of products and technologies, are inevitably written in general terms. When applied to a specific product they must be interpreted, and anyone who has gone through the process of having a system evaluated against criteria will attest that the interpretation sets the height of the bar that the product must clear. This situation, combined with the background of an ever-escalating threat, leads to tensions in the evaluation process.

On the one hand, there is significant pressure to maintain consistency between evaluations of different products over time. That is, the difficulty of achieving, say [f3,a3], in 1995 should be about the same as achieving it in 1998. The motive is fairness. Since it is likely that the market value (as reflected in increased sales of product) of an [f3,a3] rating will be the same in 1998 as it was in 1995, it is arguably unfair for the later vendor to be subjected to a more stringent set of interpretations (and the associated increased cost) than the earlier one.

On the other hand, evaluators are aware of the decreased operational value of a rating (as manifest in a particular set of interpretations) over time. They are, accordingly, under pressure to increase the stringency of the interpretations over time, a process called "criteria creep" in the TCSEC arena.

The dilemma inherent in the process then is as follows: If the interpretations are constant over time, then the operational value of a given rating becomes progressively less and products are placed in harm's way with progressively less protection relative to the threat. If the interpretations become more stringent over time, the ratings maintain their operational value but vendors are discouraged from participating because the investment required to achieve a given rating increases over time. This contradiction has not been resolved to date in the TCSEC evaluations. The Common Criteria effort hopes to overcome this by adding new protection profiles to

respond to the increased threat. Given the inevitable bureaucratic and regulatory pressures to maintain fixed objectives, it is doubtful that the criteria evolution can keep pace with the evolution of the threat.

The history of national and international criteria and evaluation systems also raises questions about institutional roles and responsibilities. The national and international criteria have featured government agencies in prominent roles, attributable to both subject matter expertise and agency missions associated with national security. The latter missions have, in turn, inspired distrust and discomfort in the private sector inasmuch as either criteria or evaluation elements and rationales have been incompletely communicated or understood and have been controlled tightly by the national government.⁶²

The evaluation under the TCSEC has been done by government (including government contractor organizations) at government expense; according to anecdotes from vendors who have gone through the experience, evaluators appear to have been junior with little computer system development experience and little motivation to expedite evaluations or promote successful outcomes. Costs incurred by vendors undergoing evaluation processes include delay and obsolescence of products, extra documentation costs, and costs of additional work needed to address concerns uncovered by evaluators. Industry has called for self-rating or a broader system of evaluators to expedite the process. A principal concern voiced by vendors is that of degree: the perception of the TCSEC philosophy as "more is better" is associated with the perception that TCSEC compliance and evaluation is excessively costly.

The ITSEC and Common Criteria assume involvement of commercially licensed evaluation facilities (CLEFs), several of which exist today (e.g., in Germany and the United Kingdom, which have an agreement for mutual recognition of evaluation results), vendor payments to CLEFs, and publicly available evaluation manuals. The CLEF-based evaluations are less expensive and more expeditious than governmentally operated evaluations.⁶³ The NIST, building on a broad program of commercial evaluation of standards compliance, the National Voluntary Laboratory Accreditation Program, has guided commercial evaluation procedures for FIPS 140-1, and it will also build on that program for evaluation of information security products using the Common Criteria under the new National Information Assurance Partnership.⁶⁴

⁶²Concerns about completeness revolve around the evaluation process, as opposed to the criteria per se. Note that in criteria or standards, completeness concerns tend to arise in specifications for cryptography.

⁶³ Based on committee members' personal experiences and committee deliberations.

⁶⁴See NIST, "National Information Assurance Partnership Gets Industry Support," Department of Commerce News (Press Release), October 7, 1997.

The economic and public policy context 209

Experiences with criteria for "trusted systems" have demonstrated a number of practical problems ranging from how criteria are specified to how systems are evaluated. The central conundrum of criteria (or standards) for trustworthiness is this: if a criterion or standard is written as a performance specification, then evaluation is difficult, but if it is written as a design specification, then the criterion is incomplete because no design specification can cover the range of implementations. The evaluation processes associated with criteria raise questions about openness (what do evaluators say to whom, including the developers) and quality (the implications of what the process emphasizes and what the evaluators seem to know and understand about the development process and the product). The processes also impose costs and raise other issues associated with having a certifier at the site where a system is deployed if the certifier needs to know what a system will be used for. If trustworthiness in the system depends on trust in the administrator, problems arise where the designer, administrator, and certifier disagree on security objectives.⁶⁵

Another difficulty with the concept of criteria is that ratings can relate only to a particular component, not to an entire NIS. In principle, security-evaluated components are used as building blocks and could be combined with rigorous system analysis of assembled systems. However, there is a dwindling set of evaluated components and little or no rigorous methodology for assessing the security of whole systems, as discussed in Chapter 4.

Findings

1. There is an increasing interest in the standards associated with trustworthiness by governments, industry associations, and the Internet Engineering Task Force.

2. A precise and testable definition is required to assess whether a standard has been fulfilled or not. Such definitions may often be articulated for some dimensions of trustworthiness such as reliability, but are often difficult to articulate for security.

3. The development and evolution of a standard attract scrutiny that will work toward reducing the number of remaining design flaws and thereby promote trustworthiness. At the same time, the existence of standards promotes the wide availability of detailed technical information about a particular technology, and therefore serves as a basis for assessing where vulnerabilities remain. Moreover, standards that facilitate interoperability increase the likelihood that successful attacks in a system may prove effec

⁶⁵This issue was discussed at the 1997 IEEE Symposium on Security and Privacy, Oakland, California, May 5-7, 1997, according to an informal e-mail report by Mary Ellen Zurko of the Open Group Research Institute.

tive in other systems. Thus, the relationship between standards and trustworthiness is indeterminate.

4. There is a tension in evaluation processes that yield ratings. If interpretations are constant over time, then the operational value decreases as products provide progressively less protection relative to threats. If interpretations become more stringent over time, vendors are discouraged from participating, because the increased investment required to achieve a given rating increases over time. The Common Criteria effort hopes to mitigate this tension, but within the context of the inevitable bureaucratic and regulatory pressures to maintain fixed objectives, it is doubtful that criteria evolution will keep pace with evolving threats.

5. Commercial licensed evaluation facilities are less costly and more timely than those that are government sponsored or operated.

6. While security-evaluated components might be used as building blocks with rigorous system analysis of the assembled system, there is a dwindling supply of evaluated components and little or no rigorous methodology for assessing the security of networked information systems assembled from evaluated components. This suggests that criteria may have limited usefulness for NISs.

Cryptography and Trustworthiness

As articulated in Chapters 2 and 4, the committee concluded that greater deployment of cryptography is essential to the protection of the Internet and its end points. But why is cryptography not deployed more widely? The most visible reasons are public policy concerns: export controls and demands for key recovery.

Export Controls

U.S. export controls have undeniably retarded the worldwide availability of products incorporating encryption; indeed, this has been the stated goal of U.S. policy in this area, and U.S. vendors are in broad agreement that U.S. export controls on products incorporating encryption have a negative impact on their ability to make foreign sales of many of their products. To the extent that vendors have been reluctant to produce two versions of a product rather than one (to produce one for domestic sale and one for export, or to hinder interoperability between domestic and export versions), U.S. export controls have also hindered the domestic availability of products incorporating encryption.⁶⁶ However, if for

⁶⁶See Computer Science and Telecommunications Board (1991, 1996). Also see Diffie and Landau (1998).

The economic and public policy context 211

eign vendors begin to step into the void left by U.S. export controls, the availability and use of information security products may be less constrained by the unavailability of U.S. products. ⁶⁷

Key Recovery

An encryption product can be designed in such a way that the key required to decrypt an encrypted message can be made available to third parties (i.e., a party that is not either the sender or the receiver) without the explicit action of either the sender or the receiver.⁶⁸ Since 1993, law enforcement agencies have been in the forefront of the encryption policy debate, insisting that products be designed to provide key recovery for law enforcement purposes with proper legal authorization. Product vendors have insisted just as firmly that the design and sale of encryption products with key recovery should be driven by the market, rather than by government fiat. Furthermore, key-recovery encryption products are by design less secure than encryption products without key recovery, because they provide access to decryption keys through a channel that can be compromised. As of this writing, the public policy debate over key recovery continues unabated. The CRISIS report (CSTB, 1996) argued that key recovery was an unproven though promising technology, and that aggressive deployment and promotion of key recovery were not appropriate as a matter of public policy; this committee sees no reason to alter that assessment today.

To the extent that public policy is unsettled and does not set clear direction, the resulting uncertainty, fear, and doubt affect the marketplace by making it difficult for users and producers to plan for the future. Vendors are reluctant to bring to market products that support security, and potential users are reluctant to adopt information security products that may become obsolete if and when the legal and regulatory environment changes.

Factors Inhibiting Widespread Deployment of Cryptography

Although export controls and key recovery are important factors, the committee has found that there are other important reasons for the lim

⁶⁷This occurrence would not necessarily be all to the good. Such a development might well reduce U.S. economic strengths by ceding increasingly large market shares to foreign vendors of information technology. U.S. national security interests might also suffer (see the section "The Changing Market-Government Relationship" for further discussion).

⁶⁸For encryption products that manage stored files rather than messages, the sender and receiver are the same party. In this case, a "third party" is someone that the file creator does not explicitly wish to have decryption capability.

ited deployment of cryptography in the United States. For example, cryptographically based security measures often reduce the convenience and usability of the NIS they protect. Indeed, the purpose of a security measure is to make the NIS impossible for an unauthorized party to use, a goal that almost always conflicts with the design goal of making the NIS easily accessible to an authorized user. As noted above, the need to undertake even a modest amount of extra work or to tolerate even a modest inconvenience for protection that is not directly related to the primary function of the device is likely to discourage the use of such protection. Security functions that are not transparent to the user and automatically applied are likely to be perceived by the user as costs that interfere with his or her ability to get work done.

A related point is that applications operating in a networked environment must be interoperable with each other. In some cases, the use of certain security measures such as cryptography can detract from the compatibility of applications that may have interoperated in the absence of those measures. For example, the use of network encryption may render networks inoperative because network address translators may not work anymore. Loss of interoperability may be a very high price to pay for adding security measures.

A good example is e-mail. E-mail systems often communicate with each other via translating gateways, which were necessary because of the lack of homogeneous e-mail systems. These translating gateways send and receive e-mail fairly well. However, the introduction of encryption into e-mail systems would cause the gateways to fail. It is difficult to envision security standards until there are standards for general e-mail communication. Attempts at e-mail security that apply to only some of the major e-mail software systems will not be effective—all major products must be included. The lack of easy-to-use e-mail software that has encryption built into it and the lack of a public-key infrastructure suggest that widespread, routine, and transparent e-mail encryption will be difficult to achieve.

A third point is that cryptographically based information security measures often consume computational resources, such as execution time or memory. For example, routine encryption often slows down a server that provides encryption services. Although it is true that processors increase in speed at a very rapid rate, so, too, do user expectations and desires. As a result, increases in computational capability may well be consumed by increased functionality, leaving little for security.

The mere availability of security products is not necessarily sufficient. To be useful across a broad range of users and applications, users would also need access to a national or international infrastructure for managing

The economic and public policy context 213

and exchanging keys. Without such an infrastructure, encryption may remain a niche feature that is usable only through ad hoc methods replicating some of the functions that an infrastructure would provide and for which demand would thus be limited (CSTB, 1996). For example, even if cryptography had been included in the UNIX rlogin command, a key infrastructure (public-key infrastructure or private/symmetric algorithms) would be necessary for the cryptographic features to be used effectively on a wide scale.

Many of the algorithms that are useful in cryptography are protected by patents. Even though a number of key patents have expired (or will expire soon enough), patents still cover some important ideas, like Micali's⁶⁹ and Schnorr's.⁷⁰ There are also many patents covering everything from encrypting account numbers to constructing keys from hashes. Today, those writing cryptographic software run substantial risks of infringement. In other cases, vendors are confused by the legal arguments among patent holders about the validity of various patents. And, even when a patent on a particular algorithm is undisputed, the fact that the holder may impose various fees and use restrictions on the patent may well inhibit the implementation of certain forms of cryptography. Such inhibitions also exist within academia, despite "free licenses for noncommercial use" that are available, because the source code that is developed cannot be given away, even if it is restricted to the United States.

The patent situation and export policy have particularly chilling effects on universities, because universities do not have the economic incentive to overcome the additional costs that are a consequence (e.g., recoup the costs of obtaining an export license). The impact on universities is of great concern because much of the software in use on the Internet was developed or inspired at universities.

Finally, for the vast majority of electronically carried or represented information, existing NISs do provide adequate protection simply because the content of that information is not valuable enough for an unauthorized party to go to the bother of obtaining it. For example, most users of NISs have an in-house cable plant or a cable plant that runs through telephone company facilities, which are presumed to be sufficiently secure. In general, a hardwired link is secure enough for most information, although perceptions regarding the adequacy of this security may vary widely. Wireless communications are a different story, and a great deal of attention has been paid in recent years to protecting them.

⁶⁹Micali's patents are 5,276,737 (January 1994) and 5,315,658 (May 1994).

⁷⁰Schnorr's patent is 4,995,082 (February 1991).

Cryptography and Confidentiality

Chapters 2 and 4 discuss the value of the authentication aspects of cryptography. The committee emphasized the importance of authentication (over confidentiality) for both technical and policy reasons. The technical reason is that authentication is the first line of defense against the most serious threats to NISs for critical infrastructures—intruders attempting to deny the benefits of using NISs to authorized users. It is still important to recognize, however, that confidentiality is an important capability for protecting privacy in general, for securing access to legacy systems, and in providing "defense in depth" for protecting against improper access (e.g., encrypting a password file or bulk transmissions and thereby obscuring the data traffic so that the analysis of this traffic is more difficult).

The policy reason for the committee's emphasis on authentication is that it does not generally involve conflicts among stakeholders. Since 1990 (and before 1990, informally), liberal rules have governed the export of information security products whose functionality is limited to authentication or integrity,⁷¹ a fact that suggests that on balance, national security interests are not significantly affected by widespread foreign access to such products. Indeed, law enforcement authorities have not demanded access to the cryptographic keys underlying authentication and integrity products.

Findings

1. The public policy controversy surrounding export controls and key recovery inhibits the widespread deployment of cryptography. However, there are other important reasons why cryptography is not more widely deployed. These reasons include reduced convenience and usability, possible loss of interoperability, increased computational and communications requirements, lack of a national or international key infrastructure, restrictions resulting from patents, and the fact that most information is already secure enough relative to its value to an unauthorized party.

2. Insofar as information is not secure enough relative to its value to an unauthorized party, the use of cryptography to promote increased confidentiality in NISs would contribute to improved trustworthiness.

⁷¹"Liberal rules" mean such products were regulated exclusively under the Department of Commerce and governed by the Commodities Control List, rather than the more restrictive International Traffic in Arms Regulations of the State Department.

The economic and public policy context 215

Federal Government Interests in NIS Trustworthiness

The federal government has multiple interests and roles in enhancing NIS trustworthiness:

• To respond to changing government information technology infrastructures,

• To accomplish agency missions, and

• To promote and protect national interests.

The spread of computer networking and activities such as electronic commerce in procurement and acquisition, electronic dissemination of legislative and agency information, the systems adoption and modernization associated with a wide range of efforts to streamline and enhance government services (e.g., National Partnership for Reinventing Government),⁷² and the introduction or revision of legislation and administrative guidelines shaping the use of computer-based systems in government indicate that most if not all agencies of the government have a direct, mission-based interest in NIS trustworthiness. For example, the Information Technology Management Reform Act⁷³ highlighted the importance of strong high-level management of information technology in federal agencies by requiring the designation of a Chief Information Officer for every agency. The Computer Security Act⁷⁴ and the Paperwork Reduction Act⁷⁵ resulted in Office of Management and Budget Circular A-130, Appendix III, which provides guidance for all federal agencies on their responsibilities regarding computer security.

In addition to mission-based goals and activities, two important trends are influencing government interest in NIS trustworthiness. The first is that the economics of using COTS products and services, including security and other trustworthy-specific products and services, is irresistible for all consumers, including government, and represents a major shift from the government's historical use of custom-made information technology. The second trend is the relatively recent rise of concerns about "information warfare" and protection of critical infrastructure. Information warfare—at least in a strategic sense—blends traditional national

⁷²The NPRG (formerly the National Performance Review) is an initiative for reengineering government programs and services. See <http://www.npr.gov/>. The NPRG was a springboard for an effort by the Federal Networking Council to outline a framework for federal Internet security.

⁷³Public Law 104-106.

⁷⁴Public Law 100-235.

⁷⁵Public Law 104-13.

security interests with less traditional defense concerns over economic security and protection of the civilian economy. Although information warfare (or the issue of information assurance, defined approximately as what is needed to combat the information warfare threat⁷⁶) has been the focus of many recent studies (see Chapter 1 and Appendix F), uncertainty abounds about the actual threat associated with NIS vulnerabilities. Pronouncements and programs have been based on uneven and anecdotal evidence, and acknowledgment of the deficient information base is combined routinely with attempts to forecast the nature, uses, and ramifications of information technology. These two trends are related insofar as COTS products and services are available to all and, therefore, tend to reduce the technological superiority of the United States as compared with other nations.

The awareness of information systems trustworthiness issues has been heightened by recent initiatives aimed at promoting the development and use of information systems generally, such as the High Performance Computing and Communications Initiative, which coordinated research and development and has become the Computing, Information, and Communications R&D program; the National Information Infrastructure initiative and the Information Infrastructure Task Force, which promoted research and economy-wide use of information infrastructure;⁷⁷ and the presidential framework for electronic commerce (Office of the President, 1997).

On May 22, 1998, the President signed Presidential Decision Directive 63 (PDD-63) on critical infrastructure protection, which calls for a national effort to ensure the security of the increasingly vulnerable and interconnected infrastructures of the United States. Such infrastructures include telecommunications, banking and finance, energy, transportation, and essential government services. The directive requires immediate federal government action, including risk assessment and planning to reduce exposure to attack, and stresses the critical importance of cooperation between the government and the private sector by linking designated agencies with private-sector representatives.

PDD-63 also established the Critical Infrastructure Assurance Office (CIAO) to support the National Coordinator, charged with integrating the various sector plans into a national infrastructure assurance plan and coordinating analyses of the U.S. government's own dependencies on

⁷⁶PCCIP favored the term "information assurance," reintroducing a concept used in earlier years at DARPA that has the benefit of not referring to warfare and, outside the community of security experts, is sufficiently ambiguous to support multiple interpretations.

⁷⁷The IITF included activities by the Security Issues Forum, the Technology Policy Working Group, and activities through DARPA, NSA, NIST, DOE, and other agencies.

The economic and public policy context 217

critical infrastructures. The President's Commission on Critical Infrastructure Protection (PCCIP), the predecessor of the CIAO and the first national effort to address the vulnerabilities created in the new information age, was established in July 1996 by Executive Order 13010.⁷⁸

Across the federal government, the DOD conducts the largest effort in information systems trustworthiness, through its work on information security as it relates to the nation's security interests. For example, in communications security, the National Communications System group and its parent Defense Information Systems Agency (DISA) coordinate with the service provider-oriented National Security Telecommunications Advisory Committee (NSTAC) to ensure that national security and emergency preparedness needs for telecommunications services are met;⁷⁹ these and other DOD agencies depend on a significant NSA effort for high-grade communications security. The primary agencies within DOD that support and facilitate research and development on information security are the NSA and DARPA, whose roles are discussed in detail later in this chapter.

On the civilian side of the federal government, the Federal Bureau of Investigation (FBI) has interests in NIS trustworthiness as a part of its law enforcement mission. During the last several years, the FBI has substantially increased its activity in addressing computer-related crimes. The FBI's most visible involvement with the information security issue has been to warn of the dangers that encryption poses to the law enforcement community and to push for the installation of key-recovery features in all encryption products and provide law enforcement authorities with the technical capability to access decryption keys surreptitiously and nonconsensually under court-approved wiretap orders. In February 1998, the National Infrastructure Protection Center was established within the FBI to serve as the federal government's focal point to detect, deter, assess, warn of, respond to, and investigate computer intrusions and unlawful acts, both physical and "cyber," that threaten or target U.S. critical infrastructure.⁸⁰

⁷⁸Details available online at <http://www.pccip.gov/>.

⁷⁹The NCS is an interagency group of about 23 federal departments and agencies that "coordinates and plans NS/EP [national security/emergency preparedness] telecommunications to support any crisis or disaster." The NSTAC provides industry perspective, advice, and information to the President and executive branch "regarding policy and enhancements to NS/EP telecommunications." NCS was formed in 1963 on a smaller scale after command, control, and communications (C³) failures during the Cuban Missile Crisis; NSTAC was formed in 1982 in anticipation of the AT&T divestiture and evolving C³ capabilities and needs.

⁸⁰Information available online at <http://www.fbi.gov/nipc/index.htm>.

Under the Computer Security Act, the National Institute of Standards and Technology (NIST) has government-wide responsibility for civilian government systems and systems handling sensitive but unclassified information. This act also provided for the provision of technical expertise and advice by the NSA for NIST, where appropriate. Although NIST does carry out its mission within budget constraints, the reality is that NIST's budget is too limited for it to acquire or use significant levels of expertise, with the result of perpetuating NSA's de facto authority and influence in the information security domain.⁸¹ In 1997, advisors to the NSA and PCCIP called for greater involvement of NIST with NSA in areas of mutual interest—which, given the dependence of the defense information infrastructure on the national information infrastructure, could be quite extensive.

Agencies that regulate the safety of goods and services have begun to address information system component trustworthiness in products ranging from medical devices (Food and Drug Administration) to aircraft and the air traffic control system (Federal Aviation Administration). In these instances, information systems trustworthiness refers to safety and reliability as well as to the traditional domain of information security. These agencies focus their activities in the context of specific products and circumstances of use, influencing system design, implementation, and use by requiring impact analysis and testing, and they may declare (e.g., by evaluation relative to a standard and/or regulation and certification) products safe or unsafe for use in a particular context.

The regulation of telecommunications services has been extended to the promotion of reliability and interoperability. For example, the Network Reliability and Interoperability Council, established under the auspices of the Federal Communications Commission and later privatized, has promoted industry monitoring and the minimization of outages. It is worth noting, however, that this regulatory response could be viewed as a corrective response to the erosion of trustworthiness that some attribute to regulatory changes that promote competition.⁸² By contrast, in the

⁸¹The principal vehicles for NIST action have included federal information processing standards, research relating to associated measurement issues, focused workshops, hosting of the Computer System Security and Privacy Advisory Board, consultation with and education of federal agency personnel on security practices and issues, and coordination with other agencies; it has not had the resources for and therefore a track record in relevant research. See, for example, Computer Science and Telecommunications Board (1991, 1996).

⁸²These changes have been linked to greater sensitivity to cost and time to market among telecommunications providers. Results include decreasing redundancy of facilities, an increase in reliance on software, proliferation of features and services (e.g., call forwarding) that promote complexity in telecommunications systems (and unreliability), and other cost-containing steps that can increase vulnerabilities. See Board on Telecommunications and Computer Applications (1989).

The economic and public policy context 219

finance sector, regulation has promoted incident reporting, auditing, and other actions that motivate or reinforce plans and procedures to promote trustworthiness, and financial incidents receive special law enforcement assistance via the U.S. Secret Service.

Public-Private Partnerships

A telling sign of the growing importance of the commercial information technology sector relative to government is the rise in rhetoric about public-private partnerships. Experiences with information security suggest that outside certain safety- and reliability-critical contexts, government mandates and controls on technology are decreasingly effective and that some form of cooperation is the logical alternative. At the same time, neither the Computer Security Act nor any other legislation assigns responsibility for assisting nongovernmental entities to protect their information systems and networks.⁸³

The PCCIP has called expressly for public-private partnerships to increase information systems trustworthiness, as has the White House Office of Science and Technology Policy (Executive Office of the President, 1997). Complementary work was undertaken earlier and concurrently by the NSTAC and its Information Assurance Task Force, which drew on participants from private firms.

Today, the meaning of "partnership" must be developed and translated into action. What can and will happen will depend on developing increased trust between the private and public sectors, and in particular, the degree of trust in the government. The cryptography policy debates suggest a loss of trust in government by the commercial information technology sector that must be acknowledged in formulating new policies and approaches. Trade and advocacy organizations⁸⁴ articulate industry positions to Congress and executive branch agencies, and a wide range of issues relating to trustworthiness are now argued in government circles that previously might have been simply decided with minimal consultation with the private sector or even ignored. Unilateral government insistence on its position or its preferred solutions—even if cloaked in the

⁸³The absence of an effective structure for addressing civilian and commercial needs was highlighted in two CSTB reports, Computers at Risk: Safe Computing in the Information Age (CSTB, 1991) and Cryptography's Role in Securing the Information Society (CSTB, 1996).

⁸⁴Such organizations include the Information Technology Information Council (formerly the Computer and Business Equipment Manufacturers Association), the Information Technology Association of America (formerly the Association of Data Processing Systems Organizations), the Software Publishers Association, the Business Software Alliance, the Computer Systems Policy Project, the Electronic Frontier Foundation, and the Electronic Privacy Information Center, among others.

guise of promoting partnerships with or education of nongovernmental entities—is unlikely to result in lasting or stable engagement with the private sector.

If equipped with resources adequate to do the job and to appear independent in its action, NIST could facilitate such partnerships; its moves to facilitate commercial system evaluation (i.e., National Information Assurance Partnership) support this prospect. The PCCIP endorsed a greater role for NIST while calling for more involvement of a number of agencies in the information assurance cause. One ongoing experiment is called the Manhattan Cyber Project, a private-sector group with government inputs aimed at documenting attacks and incidents (Harreld, 1997).

The Changing Market-Government Relationship

In the not-so-distant past, the number of commercial firms capable of providing trustworthiness products or services was relatively small. Thus, the federal government needed to influence only a small number of organizations in order to promote greater trustworthiness. These organizations had incentives to respond positively to federal government concerns because of a formal relationship that existed with the federal government (e.g., AT&T as a regulated monopoly), or because they were motivated to be cautious as a consequence of ongoing antitrust investigations (in the case of IBM), or because they sold products in large quantities to the federal government (in the case of both AT&T and IBM).

Today's vendors of trustworthiness-related products are many and diverse, ranging in size from small start-ups to Fortune 100 companies. Many of today's product vendors and service providers have arisen in a more competitive and libertarian culture, and market responsiveness is the most highly held value for these companies. Despite some degree of concentration in the supply of computing systems (in both hardware platforms and software), it is now harder to find large telecommunications or computer systems providers with both the market penetration and the tradition of responding to public-sector requests for reliability that historically characterized AT&T and IBM. Although the federal government continues to be the largest customer of computing and communications products and services, its market share has decreased dramatically during the past few decades—with a concomitant decline in the federal government's influence in the marketplace.

The emergence of a number of important suppliers from other countries complicates matters further, as foreign governments and firms have even less motivation to be friendly to U.S. government or societal interests. Examples raised by people who note this concern include Siemens, Alcatel, Checkpoint, and SAP, on the basis of ownership rather than any

The economic and public policy context 221

specific evidence. For example, Baan and SAP are non-U.S. companies whose significant number of U.S.-based customers will entrust their operating models and internal manufacturing system knowledge to their products and, by extension, sales forces (Edmondson et al., 1997). Checkpoint, an Israeli-owned company, is one of the leading firewall vendors. Indeed, there arises the possibility that these non-U.S. firms may be responsive to their home governments rather than the U.S. government.

Findings

1. The federal government has a broad and increasing interest in NIS trustworthiness. Trustworthy NISs are important for the government to accomplish agency missions, address changing government information technology infrastructures, protect national interests, and facilitate and support research and development in areas critical to the nation.

2. Federal government mandates and controls on technology are decreasingly effective. Therefore, some form of cooperation with the private sector (e.g., partnerships) is appropriate. Building trust between the private and public sectors is essential to achieving increased cooperation in efforts to improve NIS trustworthiness.

3. The federal government has less influence on vendors than in the past because the number of vendors of trustworthiness products and services has increased considerably and these vendors include small start-ups that, in particular, are focused on marketplace demands. As trustworthiness-related products and services are increasingly provided by non-U.S. companies, the influence of foreign firms and governments on the trustworthiness marketplace is a new concern.

The Roles of the NSA, DARPA, and Other Federal Agencies in NIS Trustworthiness Research and Development

Research relating to NIS trustworthiness is conducted and supported by many federal government organizations. Some agencies conduct research directly (e.g., NSA, Department of Energy national laboratories); others fund research that is conducted externally (e.g., DARPA); and a few agencies support both internal and external research (e.g., NSA). Internal research, some of which is classified, is difficult to assess; time constraints precluded further consideration in this report. As discussed earlier in this chapter, industry also conducts "research," but it emphasizes applied research and development in its activities and rarely achieves depth in any given area of inquiry (Mayfield et al., 1997). This short-term emphasis by the private sector may lead to products, but it

also creates an enduring federal role in trustworthiness research. Moreover, some requirements that are unique to the federal government are unlikely to be met by the commercial market.

Through its national laboratories, the Department of Energy (DOE) has supported projects that have developed information security tools for network inspection and workstation protection; these tools are available to the entire DOE community, including its contractors. The Lawrence Livermore National Laboratory is the host for the Computer Security Technology Center, which serves the entire federal government with respect to information security needs. Sandia National Laboratories conducts a variety of research activities that support the development of high-assurance software, more from a reliability and safety rather than a security standpoint. In addition, Sandia National Laboratories has a long history of conducting vulnerability assessments of high-consequence systems, such as those intended to prevent uncommanded release of nuclear weapons.

The National Aeronautics and Space Administration (NASA), through its Assessment Technology Branch (ATB), develops advanced methods for the specification, design, and verification of complex software systems used in critical aerospace applications to minimize the frequency of design errors and to promote fault tolerance in the presence of component failures. ATB's work focuses on formal methods for assuring safety and integrity and develops measures of system quality and tools to apply those measures. Techniques and approaches showing significant potential for improving the quality or safety of aerospace computing systems are transferred to U.S. aerospace interests and to other U.S. customers. In addition to coordinating its work with that of the DOD, ATB works with the Federal Aviation Administration to transfer applicable research results to civil aircraft certification guidelines, specifications, and recommended procedures.⁸⁵ NASA also supports the Software Independent Verification and Validation Facility, whose role is to assist customers in the development of high-quality software.

Finally, the National Science Foundation (NSF) supports some research on information systems trustworthiness. For example, the Software Engineering and Languages Program in the Division of Computing and Communications Research supports research on technical issues that underlie the design, validation, and evolution of software-based systems. Research topics include domain-specific languages for specification and

⁸⁵Description adapted from material available online at <http://atb-www.larc.nasa.gov/atb-charter.html>.

The economic and public policy context 223

design; various approaches to software design and evolution; issues of software modularity and composition; techniques to enhance confidence and quality; software security; and software design environments that incorporate semantic knowledge.⁸⁶ The NSF has also funded cryptography projects as a part of its efforts in computational and complexity theory.

Incomplete and incompatible statistics complicate an assessment of relevant research support across federal agencies, and the tendency for individual agency programs to change regularly (as projects start and finish and as programs are revised) compounds the problem. Some gross observations can be made to characterize the situation as of this writing. Within the federal government, external research relating to information systems trustworthiness is coordinated by the interagency Computing, Information, and Communications (CIC) R&D Subcommittee. About 12 federal departments and agencies participate in coordinating program planning, budgeting, and review. The CIC R&D Subcommittee is divided into five components, and trustworthiness activity is largely associated with the High Confidence Systems (HCS) component.⁸⁷ In terms of research support, NSA and DARPA dominate the CIC agencies involved with HCS, with FY 1997 spending listed as $7.3 millon and $10 million, respectively, out of a $30 million component total. Other components include High End Computing and Computation, Large Scale Networking, Human Centered Systems, and Education, Training, and Human Resources—each of which can contribute to or be affected by trustworthiness.

The federal government has sought to promote coordination among entities on trustworthiness R&D, and it has linked defense and civilian and mission and research agencies through the HCS working group. There is also an evolving information security (infosec) research council that includes DARPA, DISA, NSA, NIST, DOE, the CIA, and the military services. The PCCIP has recommended additional interagency coordination structures, building on the teams it assembled while conducting its work.

⁸⁶Description adapted from material available online at <http://www.cise.nsf.gov/ccr/sel_home.htm>.

⁸⁷The HCS program was announced as one of six focus areas in the 1995 Strategic Implementation Plan of the Committee on Information and Communications (CIC) R&D, which coordinates computing and communications R&D across the federal government. CIC planning includes R&D activity in the areas of components, communications, computing systems, support software and tools, intelligent systems, information management, and applications.

The focused coordination effort comes from the DARPA-NSA-DISA Joint Technology Office (JTO).⁸⁸ Specifically, the role of the Information Systems Security Research-Joint Technology Office (ISSRJTO) is "to optimize use of the limited research funds available, and strengthen the responsiveness of the programs to DISA, expediting delivery of technologies that meet DISA's requirements to safeguard the confidentiality, integrity, authenticity, and availability of data in Department of Defense information systems, provide a robust first line of defense for defensive information warfare, and permit electronic commerce between the Department of Defense and its contractors."⁸⁹

National Security Agency

The National Security Agency is responsible for (1) providing intelligence through the interception, collection, decryption, translation, and processing of foreign communications signals and (2) developing cryptographic and other information security techniques to protect classified and unclassified (but sensitive) U.S. communications and computer systems associated with national security.⁹⁰ In support of its information security mission, the NSA historically has developed very high quality cryptographic equipment and keying material for the Department of Defense and other customers in the U.S. government (e.g., the State Depart

⁸⁸The Joint Technology Office (JTO) was announced in the 1995 "ARPA/DISA/NSA Memorandum of Agreement Concerning the Information Systems Security Research Joint Technology Office." Complementing DARPA's ongoing research program relating to system security as well as NSA's research efforts, the JTO is intended to further coordination of research and technology development relevant to meeting DOD's needs for trustworthy systems. It also aims to make the goals and decision-making processes for such R&D more open and responsive to public needs and concerns. Organized as a "virtual" entity that draws on personnel and resources otherwise housed at the participating agencies, the JTO is expected to harmonize the individual agency programs much as the High Performance Computing and Communications Initiative has harmonized those of its component agencies, while leaving research management (e.g., broad area announcements in the case of DARPA) and ultimate source selection decision making to those agencies.

⁸⁹See "Memorandum of Agreement Between the Advanced Research Projects Agency, the Defense Information Systems Agency, and the National Security Agency Concerning the Information Systems Security Research Joint Technology Office"; MOA effective April 2, 1995. The full text of the MOA is available online at <http://www.darpa.mil/ito/research/is/moa.html>.

⁹⁰Under the National Security Act of 1947, a restructured intelligence community was created. Subsequent executive orders have revised or reordered the intelligence community (and continue to do so). The National Security Agency (which replaced the Armed Forces Security Agency) was created by presidential directive by President Truman in 1952. A number of documents that describe NSA's mission are classified, but a basic mission statement is now available on an NSA Web site, <http://www.nsa.gov:8080/>.

The economic and public policy context 225

ment). For years, the primary focus of the NSA was on protecting the confidentiality of communications. As the boundary between communications and computing has blurred, the NSA has focused its protection on information security rather than more narrowly on communications security (see Box 6.6).

The growing dependence on COTS technology in the DOD necessitates a strong NSA interest in COTS trustworthiness and the integration of cryptography into COTS products. NSA's special customer market is small enough and the potential for NSA control is sufficient to discourage many producers of COTS products from meeting NSA's special needs directly; because of its low and shrinking influence on the market, NSA needs to understand and work with COTS technology and vendors. The shift to COTS products raises questions about the scope of national security concerns and what they imply for technology strategies to meet the needs of national security entities, the primary client of NSA.

BOX 6.6

The NSA Mission:
From Communications Security to Information Security

The 1995 National Security Agency (NSA) Corporate Plan for Information Systems Security laid out a broad mission:

[NSA's] INFOSEC [information security] mission is to provide leadership, products, and services necessary to enable customers to protect national security and sensitive information in information systems pursuant to federal law and national policies, and to provide technical support to the government's efforts to incorporate information systems security into the national information infrastructure. Our customers include national security community members handling classified and sensitive information, as well as those civil government agencies and, when requested, private sector organizations providing vital national services. We serve our customers by assessing their needs, delivering solutions, and creating advanced INFOSEC technologies. We also promote security for the national information infrastructure through our policy and standards work, our efforts in public advocacy and education, and our role in shaping commercially available security technology. (p. ii)

More recently, the 1996 National Cryptologic Strategy for the 21st Century¹ explicitly related military and commercial vulnerability to interconnectivity, interoperability, and increased reliance on commercial off-the-shelf products and services.

¹Using July 1996 briefing charts, John Davis, NCSC director, described this program to the committee during its October 21, 1996, visit to NSA.

Partnerships with Industry

Increasingly, partnering with industry is seen as an approach for lowering government research costs, ensuring the relevance of solutions, and expediting the transfer of research into products. On the other hand, anecdotal evidence⁹¹ points to concerns about the direct and opportunity costs of engineering efforts that respond to NSA's concerns without generating products that see widespread use (Mayfield et al., 1997). Meanwhile, growing recognition of the need for trustworthiness combined with increased dependence on NISs continues to lead more organizations (e.g., banks) with high levels of concern about information security to approach NSA for consultation and assistance. The National Computer Security Center was formed by NSA in the early 1980s as a communications conduit for information security technology. More recently, the NSA National Cryptologic Strategy⁹² described and encouraged a "zone of cooperation" among the law enforcement and national security communities, the public sector generally, and the private sector.

Another example of reaching out is the NSA effort in the early 1990s concerning the Multilevel Information Systems Security Initiative (MISSI), which was originally intended to provide a set of products and an architectural framework that would facilitate the development of multilevel secure NISs. A key aspect of MISSI was to promote broader use of Fortezza technology⁹³ through partnerships with industry. MISSI embodied the view that secure hardware and software had to be developed together, something that the COTS market eschews. For this and other reasons, it is widely acknowledged that MISSI was both a technical and marketplace failure; nevertheless, the multilevel security concerns embodied in MISSI—that truly secure solutions require integrated approaches—continue to shape NSA management thinking.⁹⁴ An alternate way to leverage COTS technology is through the development of standards, such as common application programming interfaces (APIs) that permit the development and use of security products with differing strength. Such standards have promise in satisfying the needs of diverse communities of security customers. The use of APIs seems to the committee to be more appealing to industry than MISSI, although acknowledging that APIs and MISSI are not directly comparable because APIs do not

⁹¹Such evidence includes the experiences of committee members.

⁹²John Davis, NCSC director, described this program to the committee during its October 21, 1996, visit to NSA.

⁹³Fortezza was originally designed for use with only unclassified data. Other products, never deployed, were to provide analogous cryptographic protection for classified data. However, over time MISSI's focus changed (see Chapter 4, Box 4.4, for additional details).

⁹⁴Committee discussion with R2 managers, October 21, 1996.

The economic and public policy context 227

address system security or assurance issues. However, APIs are consistent with the notion that successful solutions in industry are likely to be add-ons, rather than integrative solutions. Furthermore, some APIs, notably those for cryptographic functions, can run afoul of export control restrictions.

The U.S. Trusted Computer System Evaluation Criteria (TCSEC) effort represents a further attempt by NSA to partner with the private sector. In this area, NSA insisted on specific conceptual models and corresponding technology, such as the information flow security models for access control at higher levels of the TCSEC. The result was a different and more costly orientation to authentication and access control than evidenced by policy models apparent in industry. No commercially viable products emerged from this effort, and today it is regarded as essentially irrelevant to current COTS information technology.

The effectiveness of such outreach efforts has been limited in the past by such factors as public mistrust of a historically secretive agency; the lack of public awareness, understanding, and support for the TCSEC and Evaluated Product List; and the ambiguity inherent in a public outreach arm in an agency constrained by statute to national security interests (CSTB, 1991). Current efforts may prove more successful, but they must overcome a legacy of suspicion originating in NSA's traditional secrecy as well as its role in controversies surrounding such efforts as the TCSEC, Clipper chip/Fortezza, and its desires for controls on exports of information security devices.⁹⁵

Other factors inhibit cooperation between NSA and the private sector. The environment in which private-sector information security needs are manifested may be different enough from the defense and foreign policy worlds that these technologies may not be particularly relevant in practice to the private sector.⁹⁶ Furthermore, the rapid pace of commercial developments in information technology may make it difficult for the private sector to use technologies developed for national security purposes in a less rapidly changing environment (CSTB, 1996).

⁹⁵This distrust and suspicion of NSA are enhanced by NSA's history of control-oriented interactions with industry. The technology marketplace is a worldwide marketplace. For many companies at least half of their income is derived from outside the United States. Advanced technology, especially cryptography, is subject to export controls, and NSA has played a significant role in advising the U.S. government on which technologies can be exported as commodities. The recent declassification of SKIPJACK and KEA is a step in the right direction; the declassification was done explicitly to allow industry to implement Fortezza-compatible software, thus enabling very low cost cryptographic "soft tokens."

⁹⁶For example, military users may be willing to tolerate a higher degree of inconvenience to obtain the benefits of security.

R2 Program

To support its mission, NSA funds and conducts research through an organization called R, which has research subunits and staff groups that provide support for technology forecasting and infosec research outreach. R2 is the NSA research subunit responsible for information security research programs; it is organized into three research divisions: cryptography, engineering, and computer science. In 1997, R2 had more than 100 staff members and a contracting budget in the tens of millions of dollars, a portion of which is coordinated with DARPA.

The major foci of R2 research are enumerated in Box 6.7. The dominant areas of R2 research are secure communications technology, assurance technology, and security management infrastructure.⁹⁷ Although cryptography has been the centerpiece of NSA's communication security products and is the dominant technique for providing security within NISs, cryptography was not identified as a dominant emphasis. Classified research and research performed by other NSA research elements and other government and government-supported research organizations presumably provide research support to NSA in this area.

The NSA and its R2 organization have developed close working relationships with a group of companies and organizations that have acquired a significant understanding of NSA's goals and the technologies involved in satisfying those goals. A large portion of the research work funded by R2 is conducted by selected contractors, federally funded research and development centers (FFRDCs), and researchers at national laboratories (e.g., work on quantum cryptography, an example of the more fundamental work supported by R2). Although R2 does not, for the most part, use the same open solicitation process used by DARPA, for example, it does review and sometimes funds proposals submitted to DARPA. Such coordination is a goal of the JTO.

R2's small University Research Program (URP) publishes open solicitations for research and provides modest security-related contracts ($50,000 to $100,000) to principal investigators in a number of colleges and universities. The program is intended to encourage professors to work in computer and communications security, although published results have not been noteworthy. For example, R2 has supported operating systems (OS) work that its management recognizes has not affected mainstream OS work and formal methods work that also has had limited impact (e.g., formal verification tools have not been developed as hoped for).

⁹⁷As reflected in unclassified briefings and materials on funding and staffing levels provided to the committee.

The economic and public policy context 229

BOX 6.7

R2's Research Activities

• Secure communications technology—dealing primarily with optical, wireless, digital speech encoding and compatible digital encryption technology in very high speed communications networks.

• Assurance technology—including formal methods, risk management, and fault tolerance.

• Secure management infrastructure—significant effort in key and certificate management, protocols including IPsec and ISAKMP, standardization efforts, and multicast key management.

• Identification and authentication—with significant emphasis on biometrics.

• Policy invocation and enforcement—including architectures, system composition, and distributed computing.

• Damage detection and response—covering defensive information warfare, damage indicators, and recovery responses.

• Information domain definition—including boundary defenses and mapping network boundaries.

• Cryptography—primarily classified research by its own staff (only part of the National Security Agency's cryptography research effort).

SOURCE: Based on program management information supplied by R2 in 1997.

In a recent study (Anderson et al., 1998), 45 NSA-funded projects in the area of information system security and survivability were identified. Although the enumeration may not be comprehensive, it does indicate the nature and scope of the research funded by NSA (see Appendix J).

Of R2's contract funds, a significant portion goes to support nonresearch activities such as participation in standards-setting organizations (e.g., the Internet Engineering Task Force, where R2 contributed the ISAKMP protocol to the IPsec standards effort), consortia membership (e.g., the ATM Forum, where R2 also contributed to security protocol standards), and support for infosec education (e.g., Biometrics consortium, Network Security Management Forum, and support for infosec studies at the Naval Postgraduate School and the University of Maryland). Numerous activities, both external and contract funded, are focused on understanding and assessing various products and technologies (e.g., hacker tools, cryptography for electronic cash). R2 also supports several efforts to modify COTS products to incorporate new or expanded security functionality (e.g., biometrics access controls and intrusion detection for Windows NT).

Issues for the Future

The committee reviewed a draft of R2's "Information System Security Research Program Plan," which was revised multiple times in 1996-1997.⁹⁸ This plan calls for greater interaction with the entire infosec community and a more open but focused R2 research program, which would be based on input from an infosec research council (sponsored by NSA and including participants from the relevant agencies and the military services), a national infosec technical baseline (established by NSA, DOE, and DOE's national laboratories), and an infosec science and technology study group (composed of leading experts who would provide an infosec perspective from the private sector). By design, the draft plan would support technology R&D "consistent with the fundamental security principles and concepts articulated in the DOD Goal Security Architecture" (Burnham, 1997). To ensure a supply of knowledgeable experts in the future, the draft plan calls for the establishment of academic centers for infosec studies and research. The plan also emphasizes technology transfer to the infosec side of NSA, to the military services, and to industry.

The committee believes that R2 faces two related challenges. One challenge is its research portfolio. Because NSA both funds external infosec research and performs internal infosec research, questions arise as to the appropriate allocation of effort (internal and external) and its coordination. Decisions about internal effort, like decisions about external effort, should recognize where the parties have comparative advantage. Highly classified cryptographic research is a natural choice for internal research; NSA has widely recognized strength in that area and has better access to mathematical talent in terms of both caliber and number or researchers. Other areas of trustworthiness, less constrained by classification requirements, seem more appropriate for R2 to pursue externally.

The second critical issue is the recruitment, retention, and continuing education of high-quality talent to pursue noncryptographic trustworthiness research areas. In these areas, especially those that depend on computer science, highly skilled researchers available in many academic and commercial organizations can make significant contributions to infosec technology. R2 will have to compete for that talent with other agencies that have established relationships with top researchers. Furthermore, top-tier talent with security expertise is scarce, and nongovernment em

⁹⁸Authored by Blaine Burnham, NSA. This document was provided to the committee by R2 when the committee asked for insight into R2's thinking about future directions. The committee examined this document not as a formal plan for NSA, but as a white paper—as a source of possibilities for the future.

The economic and public policy context 231

ployers would appear to offer more rewards, from recognition to pay (Lardner, 1998). Skills developed in an infosec research group, especially those relating to network security, cryptography, and COTS software, are easily marketable in the commercial sector—a fact that constrains both hiring and retention in R2. Finally, there is the perception that the "cloak and dagger image" that once attracted some people to NSA is no longer as strong, because of a smaller defense budget and rapidly growing private-sector alternatives (Lardner, 1998).

As previously indicated, senior management at NSA and NSA advisory groups have stated that it is difficult to obtain and retain highly qualified technical research staff with computer-related expertise for the R2 organization.⁹⁹ Within R2, staff is spread thinly, and loss of an individual can have a significant impact on organizational coverage. Further, the ability of a technologist to do research is reportedly limited by administrative and other obligations. The adoption of a rotation program, comparable to those at the NSF and DARPA for program managers, could be considered as a complement to hiring regular staff members. To be effective, such a program would have to be carefully designed to attract the desired researchers to the NSA.

R2 may be at a disadvantage within NSA inasmuch as its work is removed from fielded results that constitute NSA successes and its work is not as directly linked to NSA's mission as that of other units. These circumstances can constrain internal communication, and anecdotal evidence suggests that R2 may not always benefit from knowledge of relevant work done by sister units. By contrast, program managers pursuing trustworthiness topics at DARPA and NSF have more visibility, and they and the researchers they fund are free to publish their results.

Although R2 funds and performs unclassified work, it shares the NSA environment and mind-set of tightly controlled information. This environment presents a real conflict with the need for access to open research information. It can encourage a closed community of workers who do not communicate with others in the community either to seek or contribute information. Although R2 has increased its outreach, the conferences in which it seems most active as an organization, the NSA-NIST-sponsored National Information System Security Conference and its own Tech Fest, tend to attract a small community of researchers with long-standing connections to NSA. These audiences have only limited interaction with the larger community of computer science researchers with whom other HCS agency program managers have regular contact.

⁹⁹They note that R2 has not recruited from the academic researchers it supports.

Findings

1. Some government customers have particularly high needs for security, and there are a handful of systems (e.g., "The President's Laptop") that face levels of threat and require the strength of a mechanism that is not available in commercial products and that would have insufficient demand to support a product in the marketplace. The NSA is particularly well situated to develop such mechanisms. Classified cryptographic research is also a natural fit for the NSA internal research program.

2. The R2 university research program emphasizes relatively short term and small projects. Such projects do not tend to attract the interest of the best industrial and academic researchers and institutions.

3. Rotation of R2 researchers with researchers in industry and aca-demia could help to broaden and invigorate the R2 program. Such rotation would be most effective with institutions that have large numbers of leading researchers.

4. Inadequate incentives currently exist in R2 to attract and retain highly skilled researchers. Improved incentives might be financial (e.g., different salary scale) and/or nonfinancial (e.g., special recognition, greater public visibility). R2 faces formidable challenges in the recruitment and retention of the very best researchers.

5. R2 has initiated several outreach efforts, but these efforts have not significantly broadened the community of researchers who work with R2. Effective outreach efforts are those that are designed to be compatible with the interests, perspectives, and real needs of potential partners.

Defense Advanced Research Projects Agency

DARPA's charter is to fund research that is likely to advance the mission of the DOD.¹⁰⁰ The DOD has requirements, such as the need for high reliability, accommodation of hostile physical environments, and adaptation to varying contexts of use (e.g., whether and what kind of wireline communications are possible; nature of wireless infrastructure available), that are unique to its mission, as well as requirements that are common to other segments of society.

Trustworthiness is an issue that cuts across DARPA's portfolio to varying degrees.¹⁰¹ Relevant work is concentrated in the Information Survivability program (with an approximate budget of $40 million per year) within DARPA's Information Technology Office (ITO) (with a budget of $300 million to $350 million per year), which supports research

¹⁰⁰Information about DARPA is available online at <http://www.darpa.mil/>.

¹⁰¹Based on examination of publicly available project descriptions.

The economic and public policy context 233

directly applicable to NIS trustworthiness. As noted above, this program is coordinated with NSA's R2 program using the JTO established between the two agencies (and DISA) for that purpose. Universities and industrial research establishments are supported, with a program that in 1997 was divided into four subareas—high-confidence computing, high-confidence networking, survivability of large-scale systems, and wrappers and composition.

A reasonably broad set of topics is covered (see Appendix J), with some emphasis on fault tolerance and intrusion detection, at least as measured by the number of funded projects in these areas. Research in other areas important for NIS trustworthiness, as articulated in previous chapters—containment, denial-of-service attacks, cryptographic infrastructures, for instance—although present, is not treated as prominently as it should be. To support greater use of COTS products, the DARPA Information Survivability program has sponsored research in wrappers and other technologies for retrofitting trustworthiness properties to existing components.

Other programs within ITO also support research that impinges on NIS trustworthiness in areas such as software engineering, programming languages, computer networks, and mobile communications. For example, encryption, reliability, and various aspects of information security are all concerns in the mobile communications (Global-Mobile) program. Other DARPA offices, including the Information Systems Office, support some work in electronics and other areas related to NIS trustworthiness. Finally, DARPA has provided funding to NSF to support smaller-scale and more theoretically oriented research projects in trustworthiness and software assurance.

DARPA funds research based on proposals that it receives from investigators. These proposals are written in response to published broad area announcements (BAAs), which outline general areas of research of interest based on interactions among program managers, operating units of the DOD with specific technology needs, and members of the research community. Proposals are evaluated by DARPA staff as well as others within the federal government, and competition for the funding is keen. Funding levels are high relative to other government sources of research support, reflecting the emphasis on systems that often require research teams and significant periods of time to develop, allowing DARPA-funded projects to undertake nontrivial implementation efforts as well as long-range research.

The ITO's culture and its practice of organizing office- and program-wide principal investigator meetings have fostered contact between DARPA program managers and the researchers that they support. This contact enables the research community to contribute to future DARPA-

funded research directions, and it helps program managers to catalyze research communities. DARPA principal investigator meetings also facilitate interchange among those involved in DARPA-funded projects. Longer-term issues and planning are considered annually at a special, retreat-style information science and technology (ISAT) activity organized around specific topics. ISAT enables program managers to interact intensively with small groups of researchers to better understand research areas (potential BAAs) for which research funding potential is timely.

DARPA program managers typically are employed on temporary assignments, although there is a small cadre of longer-term staff. The ranks are populated by academics on leave from their universities, as well as scientists and developers from other branches of the government and from industry. Limited-term appointments mean that DARPA's direction and priorities are not static, with obvious advantages and disadvantages. Most problematic is that longer-term research agendas may suffer from changes in personnel, as newer program managers seek funding for research programs they wish to create, which can be achieved only by reallocating resources at the expense of existing programs. Another concern is the ability to attract top researchers for brief government stints. Those academics with well-developed research programs are reluctant to leave them for 2 to 3 years, while those researchers who have been unable to develop such programs are probably not the candidates that DARPA would like to recruit.¹⁰² On the other hand, top researchers who serve for brief government stints bring state-of-the-art thinking to DARPA and may be more willing than career employees to abandon less promising streams of research. Because the existence of effective research programs in trustworthiness and survivability is essential, whatever challenges exist in attracting topflight academics must be overcome.

The types of research undertaken have varied over the years, depending on priorities within the DOD and DARPA as well as outside influences (e.g., the NSA, Congress). Historically DARPA projects have been high risk, pushing the envelope of technological capabilities to achieve potentially high payoffs.

For example, in the early to mid-1970s, there was strong interest in DARPA security research, sparked in part by a Defense Science Board task force established to address the security problems of multiaccess, resource-sharing computer systems. In an effort to attain the widely shared goal of creating a multilevel secure operating system, the DOD aggressively funded an external research program that yielded many fun

¹⁰²Interview conducted by Jean E. Smith for the Computing Research Association on March 25, 1998. Data is available online at <http://www.cra.org/CRN/>.

The economic and public policy context 235

damental advances in computer security. As one view of DARPA in the 1970s put it: "The route to a solution—implementing a reference monitor in a security kernel—was widely agreed upon" (Mackenzie and Pottinger, 1997). By reducing some of the research and development risks, the DARPA-funded research stimulated the market to develop enhanced security capabilities (CSTB, 1991) at the same time that, not coincidentally, the United States led the computer security field and agreement emerged about the nature and role of an organization that would certify the security of actual systems.

Not every project was successful. Some were canceled, others exceeded budgets, and yet others outlived their practicality. These experiences illustrate some of the difficulties inherent in research. Some "failures" are a positive sign as indicators that challenging ideas are being pursued (which entails some risk) and that spin-offs and learning take place, which may be applied to future successful projects.

Issues for the Future

A few university computer science departments have several faculty members who emphasize computer security research, but many departments have none who do. In any event, the number of computer security researchers is small compared to the number in other specialties, such as operating systems or networks. Among the consequences are a paucity of educational programs in security and a dearth of security experts. In recent years, DARPA funding for computer security research has been primarily incremental and short term. Longer-range research projects need to be funded, particularly those that address fundamental questions, to develop the basic research that is needed for the long-term vitality of the field.

Even fewer faculty conduct research programs in some other areas of trustworthiness, such as operational vulnerabilities. Increased funding is imperative to enable reasonable progress in the critical research areas needed to improve the trustworthiness of NISs.

Although the DOD-support mission does not seem to restrict what research areas DARPA pursues, pressures to demonstrate the relevance of their research investments have generally led DARPA program managers to encourage their investigators to produce short-term results and make rapid transitions to industry. This approach can discourage investigation of more fundamental questions and experimental efforts, and thus affect which research topics are explored. Some of the research problems outlined in this report require long-term efforts (e.g., achieving trustworthiness from untrustworthy components); expecting short-term payoff may well have the effect of diverting effort from what may be the more critical problems or the most effective solutions.

The need for an increased emphasis in research on improving the trustworthiness of NISs in the long term is not consistent with the stated emphases of current ITO direction. The current director, in a recent interview,¹⁰³ articulates three main thrusts for ITO: "Let's get physical" refers to moving beyond the metaphor of a human directly interacting with a computer system to one that places greater attention on the physical world. The second main theme, "Let's get real" suggests an increased focus on real-time applications; the third theme is "Let's get mobile," referring to mobile code research. The committee believes that while some part of this focus is relevant to the research agenda needed to advance the trustworthiness of NISs (e.g., refer to the discussion on mobile code in Chapters 3 and 4), the three themes do not embrace the large majority of the most important topics.

The PCCIP calls for an increase in federal spending on information assurance R&D from an estimated $250 million currently to $500 million in FY 1999 and $1 billion in FY 2004 (PCCIP, 1997). While the study committee certainly endorses the need to increase federal spending on trustworthiness R&D, the study committee has not seen any published rationale for this magnitude of increase. The study committee observes that for the next several years, the population of experts who are qualified to conduct trustworthiness-related research is relatively fixed, because of the lead time needed to recruit and educate new researchers. Thus, increased activity in trustworthiness-related research must be conducted by extant researchers who are already engaged in other work. The study committee believes that a quadrupling of the level of activity in the proposed time frame is therefore unnecessary. Instead, a lower rate of growth that is sustained over a greater number of years would probably be more effective, especially if it is coupled with programs to increase the number of university training programs in trustworthiness.

Findings

1. DARPA funds some research in important areas for NIS trustworthiness. However, other critical topics—including containment, denial-of-service attacks, and cryptographic infrastructures—are not emphasized to the extent that they should be.

2. The use of academics on temporary assignment as program managers has both advantages and disadvantages. This rotation of program managers ensures that state-of-the-art thinking is constantly being in

¹⁰³Interview conducted by Jean E. Smith for the Computing Research Association on March 25, 1998. Data is available online at <http://www.cra.org/CRN/>.

The economic and public policy context 237

fused into DARPA (assuming that the leading researchers in the field are appointed). On the other hand, such rotation does not promote long-term research agendas because a program manager's tenure typically lasts for only 2 to 3 years.

3. DARPA uses a number of mechanisms to communicate with the research community, which include principal investigator meetings, ISATs, and broad area announcements. These mechanisms seem to be generally effective in facilitating the exchange of ideas between DARPA and the research community.

4. The nature and scope of major DARPA projects funded in the 1970s—in which security work was an integral part of a large, integrated effort—seem to characterize DARPA's greatest successes in the security domain. Not all of these efforts were entirely successful, as is characteristic of high-risk, high-payoff research. Some level of failure is therefore acceptable.

5. The committee believes that increased funding is warranted for both information security research in particular and NIS trustworthiness research in general. The appropriate level of increased funding should be based on a realistic assessment of the size and availability of the current population of researchers in relevant disciplines and on projections of how this population of researchers may be increased in the coming years.

References

Anderson, Robert H., Phillip M. Feldman, Scott Gerwehr, Brian Houghton, Richard Mesic, John D. Pinder, and Jeff Rothenberg. 1998. A "Minimum Essential Information Infrastructure" for U.S. Defense Systems: Meaningful? Feasible? Useful? Santa Monica, CA: RAND National Defense Research Institute, in press.

Board on Telecommunications and Computer Applications, National Research Council. 1989. The Growing Vulnerability of the Public Switched Networks. Washington, DC: National Academy Press.

Boehm, Barry. 1981. Software Engineering Economics. Englewood Cliffs, NJ: Prentice-Hall.

Burnham, Blaine W. 1997. Information System Security Research Program Plan Version 4.0. Ft. Meade, MD: National Security Agency (R2) INFOSEC Research and Technology Office, January.

Canadian System Security Centre. 1993. The Canadian Trusted Computer Product Evaluation Criteria Version 3.0e. Ottawa, Canada: The Communications Security Establishment, Government of Canada, January.

Carpenter, Brian E., and Fred Baker. 1996. Informational Cryptographic Technology. RFC 1984. August.

Clausing, Jeri. 1998. "Federal Reserve Official Warns of Year 2000 Bug," New York Times, April 29.

Computer Science and Telecommunications Board (CSTB), National Research Council. 1991. Computers at Risk: Safe Computing in the Information Age. Washington, DC: National Academy Press.

Computer Science and Telecommunications Board (CSTB), National Research Council. 1994. Information Technology in the Service Society: A Twenty-First Century Lever. Washington, DC: National Academy Press.

Computer Science and Telecommunications Board (CSTB), National Research Council. 1996. Cryptography's Role in Securing the Information Society, Kenneth W. Dam and Herbert S. Lin, eds. Washington, DC: National Academy Press.

Cummins, Arthur J. 1998. "Investors Are Scratching Their Heads Over Details of Converting to Euros," Wall Street Journal, August 14, p. B8.

de Jager, Peter. 1993. "Doomsday 2000," ComputerWorld, 27(36):105.

Denning, Dorothy E., and Giovanni M. Sacco. 1981. "Timestamps in Key Distribution Protocols," Communications of the ACM, 24(8):533-536.

Diffie, Whitfield, and Susan Landau. 1998. Privacy on the Line: The Politics of Wiretapping and Encryption. Cambridge, MA: MIT Press.

Edmondson, Gail, Stephen Baker, and Amy Cortese. 1997. "Silicon Valley on the Rhine," Business Week, November 3, p. 162. Available online at <http://www.businessweek.com/>.

Electronic Frontier Foundation. 1998. Cracking DES: Secrets of Encryption Research, Wiretap Politics & Chip Design. Sebastopol, CA: O'Reilly and Associates.

Executive Office of the President, Office of Science and Technology Policy. 1997. Cybernation: The American Infrastructure in the Information Age, A Technical Primer on Risks and Reliability. Washington, DC: Executive Office of the President.

Gertz, Bill. 1998. "Infowar Game Shut Down U.S. Power Grid, Disabled Pacific Command," Washington Times, April 17, p. A1.

Harreld, Heather. 1997. "Group Says Few Fed Sites Protect Privacy: Lack of Policies and Mechanisms Puts Web Visitors at Risk," Federal Computer Week, September 1, p. 10.

Hellman, Martin E. 1979. "DES Will Be Totally Insecure Within Ten Years," IEEE Spectrum, 32(7).

Lardner, Richard. 1998. "The Secret's Out," Government Executive, August. Available online at <http://www.governmentexecutive.com/features/0898s2.htm>.

Lemos, Robert. 1998. "Lloyds to Offer Firms Insurance Against Hackers," ZDNN, April 23. Available online at <http://www.zdnet.com/zdnn/content/zdnn/0423/309664.htm>.

Mackenzie, Donald, and Garrel Pottinger. 1997. "Mathematics, Technology, and Trust: Formal Verification, Computer Security, and the U.S. Military," IEEE Annals of the History of Computing, 19(3):41-59.

Masters, Brooke A. 1998. "Laptop Thefts Growing: Businesses Losing Computers, Secrets," Washington Post, March 30, p. B1.

Mayfield, William T., Ron S. Ross, Stephen R. Welke, and Bill R. Brykczynski. 1997. Commercial Perspectives on Information Assurance Research. Alexandria, VA: Institute for Defense Analyses, October.

Meissner, P. 1976. Report of the Workshop on Estimation of Significant Advances in Computer Technology. Washington, DC: National Bureau of Standards, December.

Needham, R.M., and Michael D. Schroeder. 1978. "Using Encryption for Authentication in Large Networks of Computers," Communications of the ACM, 21(12):993-999.

Needham, R.M., and Michael D. Schroeder. 1987. "Authentication Revisited," Operating Systems Review, 21(1):1.

Neumann, Peter, G. 1990. "Rainbows and Arrows: How the Security Criteria Address Computer Misuse." pp. 414-422 in Proceedings of the Thirteenth National Computer Security Conference. Washington, DC: NIST/NCSC.

Noll, Roger G. 1996. Reforming Risk Regulation. Washington, DC: Brookings Institution, April.

Office of the President. 1997. A Framework for Global Electronic Commerce. Washington, DC: The White House, July 1.

The economic and public policy context 239

President's Commission on Critical Infrastructure Protection (PCCIP). 1997. Critical Foundations: Protecting America's Infrastructures. Washington, DC: PCCIP, October.

Senior Officials Group. 1991. Information Technology Security Evaluation Criteria. London: European Community Information Systems Security, Department of Trade and Industry.

U.S. Department of Defense (DOD). 1985. Trusted Computer System Evaluation Criteria, Department of Defense 5200.28-STD, the "Orange Book." Ft. Meade, MD: National Computer Security Center, December.

Ware, Willis, H. 1995. "A Retrospective of the Criteria Movement," pp. 582-588 in Proceedings of the Eighteenth National Information Systems Security Conference. Baltimore, MD: National Institute of Standards and Technology/National Computer Security Center.

Wiener, Michael J. 1994. "Efficient DES Key Search," paper presented at the Rump Session of Crypto '93, School of Computer Science, Carleton University, Ottawa, Ontario, Canada, May.

Wilson, Janet. 1998. "The IETF: Laying the Net's Asphalt," Computer, 31(8):116-117.

Trust in Cyberspace
Committee on Information Systems Trustworthiness, National Research Council (1999) 352 pages   6 x 9