|Setting Safety Standards|
Conclusions about specific safety standards are like snapshots in order to be understood, they must be placed in context. The analysis thus far has been particularized and static, concentrating on the dynamics and outcomes of specific cases. This approach is suggestive but fragmentary. It points out specific differences between public and private approaches but says little about them as systems . This mirrors one of the main shortcomings with most discussions of "standards policy they amount to little more than empty calls for case-by-case decisionmaking. The problem is that policymakers have little understanding of the institutional differences between the public and private sectors. The goal of this chapter is to transform the observations of this study into some general conclusions about the comparative institutional advantages of public and private standards-setting. This discussion will be followed (in the final two chapters) by an analysis of specific proposals for exploiting and improving these institutional features.
There are no widely accepted criteria for comparing regulatory regimes. In order to provide a rounded view of the comparative advantages of public and private standards-setting, three approaches are taken below. The first examines the costs associated with decisionmaking, an approach favored by those who believe that more substantive measures are either impossible or simply not worthwhile. The second examines information inputs, an approach favored by those partial to rational decisionmaking models. The third is an amalgam of the remaining factors stressed by Lester Lave and others. It adopts an evolutionary perspective, comparing how the systems change and adapt over time.
One way to compare standards-setting systems is by the cost of the enterprise, what economists call the "transaction costs." Standards-setting takes time and money, and some systems are likely to run more smoothly and inexpensively than others. Of course, transaction costs say nothing about substance. Perhaps more expensive standards are better in other respects. Given the troubling uncertainties about the costs and benefits of safety regulation, however, it is often impossible to evaluate standards substantively. If having a standard, any standard, is preferable to not having one, then the system that produces standards most quickly and inexpensively should generally be favored. But the difference between public and private decisional costs, as explained below, is more complicated than is often imagined.
Government rulemaking is often criticized for being time-consuming and expensive. Government agencies can be notoriously slow. It took the CPSC two years to decide whether to grant the Banner petition and another three years to develop a relatively simple woodstove labeling rule. The gas space heater proceeding was no quicker. OSHA's grain elevator standard was almost ten years in the making. Only the FAA stands in notable exception, having adopted the aviation fire safety standard in less than a year a dubious distinction, considering its content.
But the case studies also demonstrate that government has no monopoly on slowness. NFPA proceedings can drag on for years. "Decisionmaking by town meeting" is how one committee member describes the process. It took NFPA much longer than the FAA to revise its standard for aircraft fire extinguishers. The same problem apparently plagues ASTM, ASME, and similar organizations. And there is reason to think that they will become slower in the future. As measured by the number of appeals to the NFPA Standards Council and the ANSI Board of Standards Review, contentiousness is on the rise. Moreover, standards-setting is expensive, and groups such as NFPA do not always see a sufficient return on the cost (in staff time alone) of meeting frequently to revise them. Again, there is an important distinction between standards developed in conjunction with product certification and most other private standards. Testing labs have a business interest in assuring that product standards are readily available when new products are submitted for approval. Standards developed for certification purposes are written in very short order, often in a matter of weeks or months. The resulting "desk standards" at UL have been criticized as violative of due process, but they are to be credited with remarkable timeliness. When private standards-setters lack this economic incentive, or when the cost of developing standards gets too high relative to the income generated by product testing, their efforts are likely to be as slow as those of government.
It is impossible to compare the total administrative cost of the two systems, because so many of the costs on the private side are decentralized. Practically everyone on the NFPA and AGA/ANSI committees is sponsored by his or her employer or trade association. Undoubtedly, the total of these costs is still lower on the private side because the total number of person-days involved in any single standard is so much less. UL keeps close track of the costs and revenues associated with its standards, but the organization is unwilling to reveal these figures. However, private standards written for certification purposes are apparently cheaper than government standards, at least from the point of view of the standards-setting organization. In short, the private sector appears to have the advantage in terms of money and, to a lesser degree, time particularly in connection with product testing.
The verdict on adversariness is less clear. Although private standards-setting is often considered less adversarial than government regulation, the case studies suggest that adversariness may be more issue-dependent than sector-dependent. In the woodstove and aviation safety cases, there was no more adversariness on the government side than there was on the private. In the grain elevator and space heater cases, the antagonism that marked the government proceedings was also present in the private sector it was just less open and publicized. There were strong disagreements within AGA about the space heater. Even stronger disagreements lurk behind NFPA 61B; the National Grain and Feed Association opposes practically everything about it. The observed tranquility in the grain elevator proceedings came at the expense of not addressing the most important safety issue: housekeeping. The private sector has one important advantage, however. The scope of private standards is so broad that there is no stigma attached to particular standards-setting activities. When the CPSC recently took an interest in metal chimneys, the industry felt it had been unfairly singled out. No body has ever felt that way about a UL standard. This is not to say that the private sector does not "miss" some issues. It does. Certain types of issues seem to fall through the cracks in private standards-setting, some warranting government attention. This is one of the ways (discussed in chapter 12) in which government standards-setting could profitably complement its private counterpart.
Harold Wilensky coined the term "organizational intelligence" to describe the institutional ability to process and utilize information. The concept seems particularly relevant to standards-setting, since the activity is so information-intensive. Three kinds of information contribute to "organizational intelligence" in setting safety standards: technical know-how, information on real-world experience, and applied research and development. The comparative strengths and weaknesses of the two standards-setting systems are striking. Private institutions are most intelligent in the first respect, public institutions in the second and third.
The most basic information utilized in standards-setting is know-how; that is, elementary knowledge about how a product or process works. The private sector is not only, as attorney David Swankin points out, "where the bodies are" (tens of thousands of people participate in private standards-setting); it is also where practical and technical knowledge often originates. Private standards-setters have a marked advantage in this regard over their public counterparts. They usually have a working knowledge of technical terms and basic engineering considerations, and they understand the practical implications of commercial use. One of the most active participants in revising NFPA 408 supervised Factory Mutual's study of hand-held fire extinguishers (conducted under contract for the FAA). Similarly, engineers for producers of space heaters and component parts participated on the Z21.11.2 subcommittee.
The personnel of public agencies, on the other hand, seldom have technical backgrounds or previous experience with the products or processes they regulate. Few are engineers. Many are lawyers or former compliance officers. In 1985, three of the CPSC commissioners were lawyers; none were engineers. A similar imbalance exists on the staff.
Neither regulators nor rulemaking staff members accumulate much technical experience over time, because public agencies do not specialize to the same extent as private standards-setters. Even at the FAA, which has a narrow mandate compared to most regulatory agencies, a rulemaking staff member might work on fire extinguishers one day, fire-blocking seat cushions the next, and tactile aisle markers thereafter. A staff member at the CPSC might work on woodstoves, gas furnaces, or a host of other "fire and thermal burn" hazards.
The result in all four public cases was ignorance and confusion over basic facts. Limited know-how led to longer and often more adversarial public proceedings. Woodstoves, described by a former CPSC commissioner as a "simple" issue, and airplane smoke detectors, described by Congressman Mineta as "straightforward," both involved issues that exceeded the technical capabilities of the respective regulatory agencies. The CPSC spent two years trying to decide whether to grant the Banner petition.
In the process of acquiring knowledge, government agencies often lose credibility and are put on the defensive. It took the CPSC years to differentiate in its injury statistics between the various types of space heaters; some say that many CPSC investigators still don't understand the difference between vented and unvented equipment. The CPSC staff also did not understand the special problems posed by fireplace inserts until very late in the proceedings on wood and coal-burning stoves. The commissioners had an even worse understanding of the technical issues. Analysts at the FAA lacked a basic understanding of the different types of Halon extinguishers and the special training needed for their use. OSHA's proposal for smaller grates on grain-loading pits was apparently born of ignorance about the operating effect of grate size on grain throughput and about the availability of other options for removing hazardous debris. Lacking basic knowledge, public standards-setters also tend to gloss over difficult technical questions, concentrating instead on more accessible, but less important, issues. The FAA plowed ahead in ignorance on smoke detectors. "They may be a good idea," noted an engineer at Factory Mutual, "but the feds are jumping in without the technical background to do the right thing. They were too slow to recognize the issue, and now they are moving too fast." They knew there were serious technical questions about smoke detectors and fire extinguishers, but they chose to ignore them. In the woodstove and gas space heater proceedings, the CPSC left the significant decisions to industry, concentrating instead on issues that could not be resolved through technical knowledge such as the wording of warning labels and the effective date of regulations even though technical questions, particularly about creosote, seemed to have greater safety implications.
In short, how standards-setters approach the task appears to be partly a function of their technical knowledge and capabilities. Lacking specific expertise, government agencies try to avoid technical issues, concentrating on issues where the agency is at less of a disadvantage. When technical issues are unavoidable, however, limited technical knowledge tends to reduce the agency's credibility and to result in longer, more contentious proceedings.
Obtaining feedback on real-world incidents is the second form of institutional knowledge essential to setting safety standards. Without information about the type and frequency of accidents, it is almost impossible to spot trends or even identify some hazard scenarios. As a UL vice president puts it, "The proof of the pudding is in the field evidence." The case studies suggest, however, that such evidence is rarely generated by the private sector. Government agencies, although far from ideal, have much better information systems than their private counterparts.
Private information sources are largely anecdotal and play only a negligible role in shaping safety standards. UL, for example, has institutionalized contacts with building inspectors in order to learn about problems "in the field." But a UL engineer acknowledges that the information is of limited use. UL also has a clipping service that collects newspaper stories on product-related injuries. The information is always sketchy and often inaccurate. How the injury actually came about is unlikely to be described in any detail; brand names are rarely mentioned, let alone model numbers; and a vented heater with clogged vents might be described as unvented. The AGA also has a national reporting system dubbed the Gas Appliance Information Network (GAIN) which relies on voluntary reporting from gas utilities. The system is much less impressive than its name. An earlier version was nicknamed N-FLOP by staff members at AGA. GAIN is also a flop. Reporting is scattered and, according to an AGA staff member, only one report was forwarded to a standards-writing committee in 1985.
NFPA makes a greater effort to collect injury information, but with only slightly more success. It, too, has a clipping service as a supplement to reports received from local fire departments. This was the only available data base on grain elevator fires when the NAS began its study. Unfortunately, the information was of questionable reliability. NFPA also sponsors comprehensive investigations of major fire incidents, such as the MGM Grand Hotel fire and the Air Canada incident. These investigations can improve standards-setting by providing better information on specific hazard scenarios, particularly if they are not duplicative of government efforts (as they were with the Air Canada fire). Unfortunately, the investigative function of NFPA is generally limited and plays little role in standards-setting. The 408 committee drafted its requirements before the Air Canada fire, and the NFPA investigation did not result in any changes. Committee minutes do not indicate whether the 61B committee ever evaluated NFPA's survey of grain elevator explosions. Only anecdotal information was discussed during committee deliberations at the July 1985 meetings.
There are two reasons why the private sector collects so little useful injury information: information has the quality of a public good, and it often carries worrisome liability implications. Public goods have value to those beyond the immediate purchaser. National defense is the classic example of a good that benefits all, whether or not they pay for it. A similar phenomenon affects the collection of injury information. Many private organizations would benefit from reliable national information on consumer product injuries, but the cost of any given organization collecting such information is prohibitively high. Since private standards-setting organizations are decentralized, there is also no easy mechanism for spreading the cost to all those who would benefit from a national information system.
Injury information also carries threatening legal implications. Lawsuits are rarely discussed openly in standards-setting committees, although Eads and Reuter report that such discussion might occur off the record. Most firms keep records of consumer complaints involving allegations of injury, but it is unlikely they would share such information, given the adverse effect it could have in court. Even trade associations according to staff members at the CPSC, the Outdoor Power Equipment Institute, and the Gas Appliance Manufacturers Association generally do not receive this kind of information from their own members. NFPA's fire investigators are also inhibited by liability concerns, sometimes stopping short of certain conclusions or recommendations because the organization does not want to become embroiled in the litigation that inevitably follows the kinds of disasters they investigate.
Public information systems, though obviously flawed, are vastly better than private ones. The public sector does much more than clip newspapers. The CPSC has several information systems to provide feedback on consumer product safety. The agency collects injury data daily from hospital emergency rooms around the country and, through its field offices, conducts hundreds of in-depth investigations each year on selected hazards. The hospital data provide the basis for national injury estimates; the accident investigation reports provide details of specific hazard scenarios. Similarly, the FAA maintains an extensive computerized file of Service Difficulty Reports, and the NTSB investigates all serious airplane accidents. In both the grain elevator and aviation fire safety cases, the government paid particular attention to accident investigations. OSHA deferred drafting its grain elevator standard so that the NAS could collect explosion data and conduct in-depth studies. Many of the recommendations from the NAS study were incorporated into the OSHA proposal.
Applied research is the third type of knowledge essential to a standards-setting system. It is the only method short of actual experience for determining whether new technologies are actually feasible and reasonably effective. Such questions were prominent in all four case studies. The oxygen depletion sensor had a track record in Europe, and the question was whether the device would be as reliable with American fuels. Similarly, questions were raised about the effectiveness of using household smoke detectors in airplanes. In the grain elevator case, there was significant controversy over whether pneumatic dust control could achieve airbone dust levels below the lower explosive limit. And with woodstoves there were a surprising number of technical questions, including the extent to which catalytic combustors could minimize creosote production.
Some standards-setters in both sectors have the in-house capability to conduct such research. The FAA Technical Center is well respected by industry. So is the National Bureau of Standards, which conducts applied research under statutory agreements with several agencies, including the CPSC. On the private side, UL and AGA conduct applied research both for in-house purposes and under contract. NFPA also supports a limited fire safety research effort.
The case studies suggest, however, that government does a better job of generating the kind of applied research that can inform standards-setting. The NBS conducted numerous helpful studies on wood-burning appliances. Its study on wall pass-through systems, a major source of fires related to woodstoves, filled a gaping hole in the private standard. Experiments conducted at the NBS also convinced the CPSC of the reliability of the oxygen depletion sensor something that AGA Labs was reluctant to admit. The FAA funds extensive research into aviation safety, including the investigation of hand-held fire extinguishers, prompting the president of a major airline to declare at a recent Flight Safety Foundation conference that "airlines and manufacturers rely on the government to do development and testing." There were only scattered instances of private research efforts connected with the case studies. Two airlines and a major airframe manufacturer conducted tests on Halon fire extinguishers. The other major research effort, considered a public relations ploy by some, was undertaken by the National Grain and Feed Association. In the woodstove case, however, the lack of private research was notable. Catalytic combustors, a possible method of reducing creosote formation, were not (and have not) been considered by UL because, as a UL engineer said, "Nobody [in the industry] wanted to spend the money."
Research is expensive and private organizations generally do not do it unless someone else pays. Some private groups simply have no resources. "ASTM is just a building with rooms and secretaries," quips a former CPSC commissioner. It relies on its members to bring to the standards-setting process any information about relevant research. NFPA does much the same. Committees do not have budgets; nor does NFPA at large conduct applied research to support its standards-setting activities.
Product certifiers are also reluctant to undertake research aimed at improving standards. This research can have the quality of a public good. If UL improves a standard through applied research, other testing labs might be able to capitalize on the expenditure. Since UL must ultimately pay for such research efforts through its certification fees, the organization is not likely to conduct research in those areas, like woodstoves, where it shares the certification market with other labs. According to a UL spokesman, however, woodstoves are unusual in this respect. "We are often a monopoly for all practical purposes."
The funding of public research depends on the politics of the budgetary process, but there is certainly the potential (realized in several of the cases) to fund projects not likely to be done privately. Many research budgets have been cut on the public side, particularly at the CPSC, but there are also indications that funding might actually increase for the FAA. The need for more public research and information collection is discussed in the final chapter.
More important than any single institutional feature is how an organization changes over time. Aaron Wildavsky has stressed the importance of "resilient" approaches to safety regulation. Others have emphasized the advantages of "flexibility" and "responsiveness." These concepts place standards-setting in an evolutionary context. What matters is not so much how individual cases are resolved as how results change over time. A regime that generates standards that are considered "too lenient" (at the time of adoption) might actually be more desirable than one that generates standards closer to the social optimum (at the time of adoption) depending on when the respective standards are adopted and how they are adjusted over time .
The case studies suggest that there are significant evolutionary differences between public and private standards-setters, differences that indicate several previously unrecognized advantages of the private sector. In short, private standards-setting is prospective and ongoing, while public efforts are usually corrective and singular. Private standards-setters tend to intervene relatively early in the life cycle of an issue, adjusting the standard subsequently over time. Public standards-setters, by contrast, are likely to get involved later, often after a major disaster, adopting a "one-shot" standard without the benefit of subsequent adjustments. The evolution of standards-setting systems can vary in two important ways: (1) in the timing of interventions and (2) in the nature of any subsequent adjustments.
Turning first to intervention, the case studies suggest a major difference between the public and private sectors. The private sector appears to intervene relatively early in the life cycle of an issue, often in anticipation of problems rather than in direct response to them. Private standards often flow directly from engineering decisions. They are usually written in anticipation rather than in response to accidents. NFPA 408 and 61B were in place long before the first serious in-flight fire or the well-publicized series of grain elevator explosions in 1977-78. UL and AGA initiate the development of standards as products are submitted for certification, which is almost always before they are marketed. The implications of this difference depend critically on how (and whether) these standards are adjusted over time.
In contrast, government interventions appear to occur relatively late in the life cycle of an issue and are usually reactive. Grain elevator safety was considered a problem by fire prevention engineers and insurance companies in the 1920s. OSHA did not get actively involved in the issue until after the disastrous Christmas of 1977. NFPA formally considered questions about aviation fire extinguishers more than ten years before the FAA first issued its advisory circular, and almost thirty years before it adopted a formal standard on the matter. Government standards also tend to respond to specific problems identified through real-world experience. For every serious airplane accident, there is a new FAA rule. This means that decisions are often made under pressure and with emotion, as in the months following the Air Canada fire. The pressure to "do something" becomes so overhwelming that valid concerns, such as whether household smoke detectors will work reliably on airplanes, too easily get brushed aside. Conversely, once "something" is done, there is little or no pressure to follow up on the action and reassess its wisdom in light of subsequent field experience.
Private standards-setters are also generally more comprehensive in defining "the problem," although the grain elevator case demonstrates that there are certain problems that only the government is likely to address. Private standards are more likely to be conceived as a total package, covering most aspects of the product or process from design and performance to labeling and production. In contrast, both of the CPSC standards addressed a single issue or single solution. So, too, with the FAA. Whether comprehensiveness is an advantage depends on the relationship between private and public standards. If the two were mutually exclusive, the private approach would probably be preferable. "It is generally better to regulate a range of problems satisfactorily than to do one in great detail," admitted a former CPSC commissioner who often favored government regulation.
Over time, private standards-setting appears to be more flexible and adaptable than government regulation. Private standards are continu ally being revised. Many government standards are one-time interventions. Public standards rarely evolve the way private ones do. The "regulatory ratchet" characterizes some government standards, while the mechanisms for revising standards remain unused in others.
In none of the public sector case studies was standards-setting seen as an evolutionary process. There was little expectation that a standard, once adopted, would be altered in the foreseeable future. To the contrary, attaining closure was an important goal in its own right one that took OSHA almost nine years in grain elevator safety. The FAA brushed aside technical questions about smoke detectors in order to avoid going through the requirements of "notice and comment" rulemaking all over again. In the gas space heater proceeding, the prospect of "gearing up" for a new rulemaking proceeding seemed so onerous to one CPSC commissioner that when the commission decided to revoke the space heater rule in light of good evidence that it was no longer needed she objected strenuously on the grounds that "restoration of the protection previously provided by the mandatory standard would require initiation anew of the entire time-consuming, resource-intensive process" (original emphasis).
Whether or not their perception of the rulemaking process is accurate, government agencies seem to operate on the principle that "it's now or never." As a result, public standards are often confined in scope, and they tend to stay fixed in their original form. The CPSC, for example, had reason to believe that gas space heaters might pose a chronic nitrogen dioxide hazard. But lacking an ongoing process for standards-writing, it saw the choice as either delaying the ODS rule (possibly for years) or not addressing this hazard at all. Revisions are always possible on the public side, but the process is ad hoc and seldom used. Several problems with the woodstove labeling rule have come to light since its adoption, but the idea of amending the rule has not been seriously considered. According to CPSC staff members, the effort required to enact the rule in the first place is the reason amendments are not seriously being contemplated. Instead, the agency has tried to amend the rule informally by working directly with the testing labs. Some of the advantages of a formal adjustment process might be captured through such informal efforts, but on the whole it seems likely that the public sector suffers significantly from the lack of revisions.
The revision process is institutionalized on the private side. NFPA standards are supposed to be reviewed and, if necessary, revised every five years. For product testing labs, the process is continual. There is some merit in this attribute alone. The ongoing nature of private standards-setting may facilitate the resolution of conflicts by tilting the strategy of participating parties in the cooperative direction. Periodically revising standards is also the only way to keep up with changes in information, technology, and preferences. But whether and how standards-setters take advantage of this opportunity is of paramount importance. The evidence from the case studies is mixed. On the positive side, the "ratchet effect," described by Eugene Bardach and Robert Kagan in their study of government regulation, does not seem to characterize the private sector. Through the adjustment process, private standards are made stricter in some respects and more lenient in others. On the negative side, such flexibility can foster nonchalance. Private standards-writers are often willing, sometimes even anxious, to postpone addressing certain complex or controversial problems until the next time the standard is supposed to be revised. In 1980 the NFPA Agricultural Dusts committee decided that action on several controversial proposals should be postponed "for further study." These proposals were summarily dismissed at a meeting five years later. UL convinced members of the canvass on woodstoves to withdraw their opposition on the assurance, still unfulfilled, that the creosote problem would be handled soon. At AGA Labs, the surface temperature of space heaters has been a "continuing agenda item," without apparent effect, for over twenty years.
The difference between public and private standards-setting is not just a matter of degree, it is a matter of kind. Public efforts are not simply stricter or more lenient than their private counterparts. The systems are too different to be compared one-dimensionally (see table 8). There are important differences in how public and private standards-setters (1) resolve philosophical questions about the appropriate scope of safety regulation, (2) estimate costs and benefits, and (3) act over time and in the context of a larger regulatory program. These observations are troubling because they cast doubt on the reasons most commonly given in favor of one form of regulation over the other. Public regulation is not always stricter; private regulation is not always more reasonable. The performance of the two sectors is mixed, and there are different reasons to favor each.
The conventional wisdom about strictness and leniency is only par-
tially correct. Although public agencies appear to err systematically on the side of safety, their private counterparts do not always err in the other direction. Three of the four private cases studied are characterized by decisions that also err on the "safe side." At least one of these cases, the NFPA standard for aviation fire extinguishers, is unreasonably strict, providing reason to conclude that the private sector occasionally makes the same mistakes as the public sector. A rough estimate of overall economic efficiency suggests that, for the standards studies here, neither sector has a clear advantage. On the public side, two of the standards were unreasonably strict (the CPSC on woodstoves and the FAA on aviation fire safety), and two were within the "zone of reasonableness" (the CPSC on space heaters and OSHA on grain elevators). On the private side, two of the standards were also in the zone of reasonableness (UL and AGA), while the others were split: one too lenient (NFPA on grain elevators) and the other too strict (NFPA on aviation fire extinguishers).
But these conclusions capture only a portion of the interesting differences between the two sectors. Public and private standards-setters do not just select different outcomes. They have entirely different ways of looking at problems. Public agencies are more paternalistic in defining "the problem" for regulation. They are also more willing than private standards-setters to select early deadlines, require unproven technologies, and regulate in a manner that interferes with traditional notions of managerial discretion.
Finally, aside from the specific standards, the two sectors vary over time and in the relationship between single standards and the standards-setting system. Sometimes private standards-writers do not have adequate information; they do not even know that a "problem" exists. At other times, they do not agree that "the problem" should be addressed. When changes are made, they are most likely the result of either government information or anecdotal evidence. Private standards are rarely unreasonable, however, in the sense of requiring something that is not generally feasible both technically and economically.
Public standards-writing, on the other hand, is reactive and rarely adaptive. Standards-writing is usually prompted by accidents or injuries. Standards are viewed as one-time corrections and not, with the exception of the FAA, as something likely to be amended in the future. Government standards are generally narrower than private ones. Although prompted by specific problems, it is questionable whether the resulting standard will address them effectively. Information about potential costs and benefits is assiduously compiled, but seems to play only an indirect role in decisionmaking. Technical issues are generally avoided in favor of "softer" issues, such as how strong a warning should be or when a regulation should take effect. But government is willing to do things the private sector will not protect people against their own mistakes, "push" technology and in some cases it is successful.
The remaining two chapters examine proposals for improving public and private standards-setting systems. Chapter 11 examines popular proposals for altering procedural requirements, particularly in the private sector. Chapter 12 examines more promising alternatives, including interactive strategies, niches for public standards, and methods of improving private standards-setting.