Eugene Bardach

Part III Smart Practices Research Part III

Smart (Best) Practices Research: Understanding and Making Use of What Look Like Good Ideas from Somewhere Else It is only sensible to see what kinds of solutions have been tried in other jurisdictions, agencies, or locales. One looks for those that appear to have worked pretty well, tries to understand exactly how and why they might have worked, and evaluates their applicability to ones own situation. In many circles this process is known as best practices research. Simple and commonsensical as this process sounds, it presents many methodological and practical pitfalls. This Part helps you avoid the pitfalls and offers tips on how get the most payoff from your search for best practices. Have realistic expectations First, dont be misled by the best in best practices research. Rarely will you have any confidence that some helpful-looking practice is actually the best among all those that are addressed to the same problem or opportunity. The extensive and careful research to document a claim of best will almost never have been done. Usually, you will be looking for what, more modestly, might be called good practices. But even this claim might be too grand. Often you cant even be sure that what appears to be a good practice is actually good in the sense that it is solving or ameliorating the problem to which it is nominally addressed. Appearances can be very deceiving. On closer inspection, it often turns out that the supposedly good practice is not solving the problem at all. Inadequate measurement plus someones rose-colored glasses were simply producing the illusion of mitigating the problem. It may also turn out that, even if good effects have truly occurred, the allegedly good practice had little or nothing to do with producing them. Finally, innocently extrapolating from a setting where a good practice has indeed worked well to settings which might differ in little-understood but important ways could lead to weak, perverse, or otherwise damaging results. None of these problems is decisive, though. If it does nothing else, a foray into best practices research can usually turn up interesting ideas, including ideas about what does not work as well as what does. The challenge for the researcher is to figure out what an idea is and what makes it interesting. Smart practices An interesting idea embedded in some practice is what I would term a smart practice. I prefer the term smart practice to best practice or even good practice because it underlines the fact that any practice worth such special attention ought usually to have something clever about it. It is this something clever that the researcher must analyze, characterize in words, and appraise as to its applicability to the local situation.

Eugene Bardach

Part III Smart Practices Research

Free lunches. One way of being clever is getting something for nothing. Contrary to the dictum that there is no such thing as a free lunch, creative policy-makers and policyimplementers invest quite a lot of energy in looking for just such comestibles. Often, they are successful. To understand how this can be, consider the free-lunch cornucopia produced by the natural sciences and engineering. The energy stored in the chemical bonds in a cup of gasoline can run a car for a few miles provided one knows how to access that energy and channel it. Pulleys and levers supply mechanical advantage. Bacteria happily eat and destroy the organic crud in a citys wastewater almost for free. All these materials, devices, and conditions represent getting a lot of something for nothing or for relatively little. The source of all these boons is simply Mother Nature. In the social world, the sources of something-for-nothing are usually less tangible and less directly gifts of Mother Nature, but they are no less real. The invisible hand of the market creates social value where only individual pursuit of self-interest had once been and metaphorically at least, operates without charge. Alphabetical ordering permits people to find information in a fraction of the time it would have taken had there been no such ordering. Queuing at bus stops is easy to understand, is usually fair, and makes life better for everybody. In the world of policy and management there are no doubt fewer and less delectable free lunches than in the marketplace or in some information storage facility or at a bus stop, but they are there. All the examples of opportunities described in Table I have this quality. They have the latent potential to generate something of public value relatively cheaply.1

On the nature of public value, see Mark H. Moore, Creating Public Value: Strategic Management in Government. Cambridge, MA: Harvard University Press, 1996.

Eugene Bardach

Part III Smart Practices Research

Table I Some Generic Opportunities for Social Improvement that Often Go Unnoticed Byproducts of personal aspirations. It is possible to structure new incentives or create new opportunities for personal advantage or satisfaction which can indirectly result in social benefit, e.g., offering to share the benefits of cost-reducing innovations with public sector employees who conceive them and implement them. Complementarity. Two or more activities can be joined so that each might make the other more productive, e.g., public works construction and combating unemployment. Development. A sequence of activities or operations may have the potential to be arranged so as to take advantage of a developmental process, e.g., assessing welfare clients for employability and vocational interest before, rather than after, sending them out for job search. Exchange. There are unrealized possibilities for exchange that would increase social value. We typically design policies to exploit these that simulate market-like arrangements, e.g., pollution permit auctions, and arrangements to reimburse an agency for services it renders another agencys clients or customers. Multiple functions. A system can be designed so that one feature can be used to perform two or more functions, e.g., when a tax administrator dramatizes an enforcement case in such a way as both to deter potential violators and to reassure non-violators that they are not being made into suckers for their honesty. Nontraditional participants. Line-level employees of public agencies often have knowledge of potential program improvements that could usefully be incorporated into the agencies policies and operations. The same is true of the agencies customers, clients, or the parties whom they regulate. Rationalization. Purely technical rationalization of a system is possible, e.g., shortening queues by deliberate spacing of arrival times, or creating contracts to solidify informal agreements that are vulnerable to decay and misunderstanding. Rummaging. By rummaging mentally one might discover novel uses in seemingly improbable but readily available materials, e.g., using the automobile registration system as a vehicle for carrying out voter registration as well. Underutilized capacity. An example, in many communities, is school facilities that are utilized for relatively limited purposes for only part of the day and for only part of the year -although school officials would be quick to warn that using this capacity without harming school functions is not always easy.

Eugene Bardach

Part III Smart Practices Research

One might say that the difference between the (high) value created and the (low) cost, and risk, of producing it represents a free lunch.2 Opportunities dont deliver up their latent value without some additional work, however. This work is done by the practices that take advantage of their potentialities, and these typically cost something and are subject to various vulnerabilities as well. However, the smarter they are, the more value they can manage to extract at lower cost and risk. Here are some examples of candidates for smart practice status candidates, that is, because to my knowledge they have not all been subjected to the extensive empirical testing needed to confirm such status: -- A High Expectations welfare-to-work program in the early 1990s before the 1996 federal welfare reform act. A prototype is the Greater Avenues to Independence (GAIN) program I studied in Riverside County, California. Unlike most other welfareto-work programs, the Riverside program set high expectations about work for GAIN participants in two senses. In many different ways, it signaled participants that program staff had confidence (high expectations) in their ultimate success in getting a job and getting off welfare. This was intended as an antidote to the low self-esteem, and consequent low effort to reattach themselves to the labor force, of many participants. Staff also signaled and expressed in program rules about early and diligent job search, as well as through a variety of formal and informal pressures that society expected welfare participants to shape up and take responsibility for their own financial wellbeing. The Riverside GAIN program designed its recruitment, training, performance appraisal, and other administrative systems to support this high expectations philosophy.3 In effect the High Expectations model took advantage of the natural energies to solve their own problems that program managers assumed to be latent in program participants themselves. -- Reading One-to-One. This is a tutoring program for children in grades 1-3 who have fallen badly behind in learning to read English. It was created by George Farkas, while he was on the faculty of the University of Texas at Arlington, and involves: systematic instruction in phonemic awareness; one-to-one tutoring by a well-trained tutor; highly structured feedback and supervision. It was first tried out in Dallas and then spread to Houston and a number of other cities. Like all phonemics-based programs, it recognizes that English orthography does not map sounds in a completely systematic or logical way and that it is at some point necessary for learners to master the decoding and encoding rules actually in use. It takes advantage of the fact that childrens early failures with reading that come from neglect of phonemic awareness are reversible by regular tutoring. It also takes advantage of the emotional bonding that comes with the one-to-one tutoring relationship. The simplicity and systematization of the teaching materials,

2 3

Risks come in several varieties. See the section below on Generic Vulnerabilities Implementing a Paternalist Welfare-to-Work Program, in Lawrence Mead, ed., The New Paternalism: Supervisory Approaches to Poverty, Washington, DC: The Brookings Institution, pp. 248-278, 1997.

Eugene Bardach

Part III Smart Practices Research

teaching methods, and the administrative oversight system make the program easily replicable and keep the costs relatively low.4 -- Sharing maintenance responsibilities for a neighborhood park between the local Parks Department and the residents of the neighborhood. Nonprofit organizations often spring up to provide services beyond those provided by the public sector when there is demand for services of a nonstandard sort not provided by the public agency, e.g., samesex education, abortion clinics. An extension of the basic idea is a partnership in which the public sector supplies certain resources that are not only supplementary but are also complementary. In many a city, the city government provides the parkland and the neighbors provide some or all of the labor to make the land more serviceable in some way. This practice takes advantage of two interesting potentialities. One is the potential for gains from trade between two parties. The second is the use of what is, in effect, barter, in a situation where there are administrative and political barriers to organizing the transaction in cash. -- The Expenditure Control Budget, adopted first in the city of Fairfield, California, and publicized by David Osborne and Ted Gaebler in Reinventing Government.5 As originally conceived and implemented, this budget gave each department the same basic mission and the same budget as the previous year (with an inflation adjustment) but abolished the line-item specification of objects of expenditure and permitted the department to keep any savings and reinvest them in other missionrelated activities. This approach took advantage of the superior technical and operational knowledge of program implementers relative to that of elected officials and bureaucrats in fiscal control agencies. -- Milestone payments to nonprofit service contractors. In 1992 the Oklahoma Department of Rehabilitation Services began paying nonprofit contractors for the rehabilitation milestones that mental health clients could achieve en route to higher levels of employability.6 The clients got to participate in assessing whether the milestones were met. The milestone system also permitted contractors to claim reimbursement from the state on a more accelerated schedule than they had previously been able to do. By taking advantage of the motivating power of self-interest, the milestone payment system motivated better performance from the nonprofits. It also provided greater transparency than more traditional fee-for-service arrangements, under which the funding agency did not know much about the quality of the service provided. -- Hennepin County, Minnesota, arranged for mental retardation clients of the county Vocational Rehabilitation program to sort and recycle discarded auto batteries of

George Farkas,. Osborne, David, and Ted Gaebler. 1992. Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector. Reading, MA: Addison-Wesley. 6 This program was a 1997 finalist in the Ford Foundation/Kennedy School of Government Innovations in American Government competition. My source for information about it was the Innovations program files.
5

Eugene Bardach

Part III Smart Practices Research

concern to the countys Environmental Management agency.7 Thus, the programs took advantage of production complementarities between physical and human assets which they could deploy. I have made a point, in describing each of these supposedly smart practices, of saying that the practice takes advantage of something. This is a linguistic device for ensuring that in analyzing how the practice works we make sure to focus on those aspects of its works that are central, that is, on the fact that the practice aims to exploit, to take advantage of, some latent opportunity for creating value on the cheap. Breaking loose of conventions and assumptions. Another way of being clever is not so much technical finding those free lunches as ideological and psychological. It involves disrespecting conventional boundaries. In the world of public policy and management, this sometimes involves challenging assumptions anchored in value commitments. More often, it involves assumptions about what value commitments such as democracy or accountability or justice actually require to implement them effectively. In the High-Expectations welfare-to-work model, for instance, Riverside was one of relatively few programs prior to the mid-1990s that decided it was proper to articulate the value premises that underlay their approach to case management. To take another example, in the last ten years we have begun to shake loose the assumption that just because some good or service ought to be financed through taxation it ought also to be produced or delivered by governmental employees. Hence, we can contemplate contracting out to the nonprofit or even the profit-seeking sectors for such traditionally governmental functions as primary education, correctional institution construction and management, and welfare-to-work programming.8 Characterizing and observing the practice What does it mean to characterize a practice well or poorly? For free-lunch-type smart practices, we can say the practice is whatever takes advantage of or exploits the latent opportunity to create value on the cheap.9 But let us try to say more about how to characterize this whatever. Distinguish between elements that are essential and those that are supportive. The essential elements are also the ones that do the basic causal work of the practice, that is, produce the valued results of the practice. These elements are essential in both a definitional and a causal sense. Consider the milestones payment program in Oklahoma. Although observers could reasonably differ, I would say that the essential elements are: a reasonable method for defining milestones; an officially sanctioned contract describing milestones; documents attesting that milestones have been met; and a method for verifying the achievements claimed in the documents. Perhaps it was also helpful, in the Oklahoma experience as it would be elsewhere, to
7

This was a Ford/KSG semi-finalist. Sandford Borins, Innovating with Integrity: How Local Heroes are Transforming American Government. Washington, DC: Georgetown University Press, 1998, p. 200; also, personal communication with Hennepin County program managers. 8 Whether or not contracting-out is a smart practice, it is highly controversial, I might add. 9 With minor adjustments the same analysis can also be applied to practices whose smartness derives from their departure from convention.

Eugene Bardach

Part III Smart Practices Research

educate the state office responsible for overseeing contracts in the philosophy of the new system. But this and other such supporting elements such as educating elected officials about the value of the new approach -- we should not think of as definitionally or causally essential to the practice. We should think of them as supportive worth talking about in any account of how the practice can be made to work relatively better and/or can be prevented from failing. In many circumstances, these are likely to be the institutions, personnel, and general culture that improve governmental performance overall, and are not restricted to the particular practice at hand. They are also likely to vary somewhat between the settings where the practice can be successfully implemented. To take a further illustration, consider the high-expectations welfare-to-work program. The definitional essence has to do with creating a certain kind of psychological environment for the participants in the program. The administrative features having to do with recruitment, training, supervision, and so on are important supportive features, and almost certainly worth analyzing in any account of how to make such a program work; but we should not characterize them as an essential element of the practice. One of the two most deadly pitfalls of smart-practice analysis is misconstruing which elements of a practice are doing the basic causal work, either alone or in combination. (The other one is wrongly believing that the practice is what is responsible for whatever good or bad results have been observed.) A full discussion of this issue is beyond our scope. 10 Suffice it to say here that analysis of the basic causal structure must focus on how the practice takes advantage of some latent opportunity for creating value on the cheap. Furthermore, the analysis should make use of whatever empirical comparisons and contrasts can help sharpen the causal picture e.g., among instances of the practice as implemented in different contexts but with varying degrees of success. Distinguish between essential and optional elements. Optional elements I define as those that add some dimension of value to the outcome that not every user of the practice might want, whereas essential elements are those that every user would want. In the milestones case, the element that, in Oklahoma, had clients sitting down with the contractor and discussing whether particular milestones have been met seems to me to merit being treated as optional. Not every user of the milestones practice would value the relationship-building contribution of this element. In characterizing the elements of a practice, distinguish between the functions each of the elements performs and the methods used to perform them. Note that in my characterization above of milestone payments to contractors I spoke of elements like defining milestones and verifying claims of achievement functions to be performed but left the methods unspecified. And in characterizing the essence of the high-expectations welfare-to-work program I referred to
10

Interpretation of the essential causal elements of a smart practice, or what does the basic causal work, is a deeply theoretical act. Theoretical perspectives, of course, differ widely among social scientists, among policy analysts, and among ordinary people who would never think of themselves as being theoretical at all. An account of my own theoretical perspective, which is based on the metaphor of craftsmanship, may be found in my Getting Agencies to Work Together: The Practice and Theory of Managerial Craftsmanship (Brookings, 1998), particularly Chs. 2 and 9.

Eugene Bardach

Part III Smart Practices Research

creating a certain kind of psychological environment without specifying the methods. It is generally desirable to use this functional characterization of the elements that go to make up a practice and thus to leave open to local option exactly how to carry the functions out. In other words, the functional characterization works as a statement at an abstract level of an end or result to be achieved by some component within a functioning system, and avoids tying the definition of the practice to exactly what that component must look like. A linguistic hint: to characterize functions, use gerunds, that is noun-like verbs ending in ing, as I did above with defining, verifying, and creating. The exceptions to the principle of relying on functional language arise when you really need or want to specify a particular method for carrying out a function. In the milestones case I intentionally referred above to a contract as a specific means of defining expectations among the parties and to documents as a means of attesting that the milestones were met. Characterization should be generic and flexible, not prescriptive and overly precise. Consider the Expenditure Control Budget described above. Is the practice there giving all the savings back to the department, or would, say, fifty percent qualify? If the idea is to provide incentives to spend wisely, probably fifty percent would do. Probably the best characterization would be: Allowing the department to retain enough of the savings for them to feel motivated to create the savings in the first place.11 It would be left up to whoever implements the Expenditure Control Budget to determine what enough means in the local context. I would also say that it should be left to local implementers to figure out the details of the generic practice that make sense in their own context. Allowing for local adaptation of nonessential elements not only serves common sense but it also encourages greater buy-in by locals of a practice that in some sense is being imported from elsewhere or, worse yet, imposed from outside. Characterization of the essential elements of a practice is not necessarily simple; it could be complex. In my list of examples of candidate smart practices above, I included only relatively simple practices, so as not to cause confusion. However, some smart practices are multi-faceted, complex, and not easy to summarize in a few sentences or even paragraphs. Michael Barzelay analyzed what he called the post-bureaucratic paradigm for managing statewide overhead and control functions in Minnesota state government. He considered trying to reduce the many aspects of this post-bureaucratic paradigm which I would also call a smart practice, albeit a very large practice to a few core ideas such as service, customer focus, quality, incentives, creating value, and empowerment. But, he concluded, the major conceptsare not organized hierarchically, with one master idea at the top. Instead, we are working with an extended family of ideas.12 A related management reform paradigm, called by many the New Public Management (NPM), emerged in New Zealand in the mid-1980s and is also an extended family of ideas and
11

Note that this is an interpretation asserted by the researcher-observer, not necessarily something that has actually been done in practice or endorsed by any practitioners. 12 Michael Barzelay, Breaking Through Bureaucracy: A New Vision for Managing in Government. Berkeley: University of California Press, 1992, pp. 115-117.

Eugene Bardach

Part III Smart Practices Research

practices. One observer notes that it is not reducible to a few sentences, let alone a slogan and goes on to state its key ideas: Government should provide high-quality services that citizens value. The autonomy of public managers, particularly from central agency controls, should be increased. Organizations and individuals should be evaluated and rewarded on the basis of how well they meet demanding performance targets. Managers must be assured that the human and technological resources then need to perform well will be available to them; and Public sector managers must appreciate the value of competition and maintain an openminded attitude about which services belong in the private, rather than public, sector.13

Specimens of a smart practice in the real world look rather different from one another and require careful interpretation. You should try to find multiple exemplars or specimens of a smart practice to get a sense of its robustness and efficacy when (1) it is being implemented under different supportive (or antagonistic) conditions, (2) it comes with different optional features attached, (3) it employs supposedly equivalent but nevertheless somewhat different means to perform the required functions. Ideally, you would be able to find social scientific evaluation studies of practices that would supply both data and theoretical interpretation regarding such matters. In most cases, however, such evaluations will not exist. Normally or perhaps at best you will finds writings or speeches by practitioners describing successes in a few places, accompanied by only sketchy descriptions of what was done or the difficulties of implementation. You will need to think very hard, and reason very carefully, about how you will want to conceptualize (that is, define) the smart practice of interest and to assess the support requirements you think are the most important. You need to do this even before you get to thinking about how the practice might work in the particular context(s) you have in mind see the discussion below, But Will It Work Here? Generic vulnerabilities It should be part of standard professional practice in describing smart practices to explain not only how and why they work but also how and why they fail, collapse, backfire, and generally make people sorry they ever tried them. That is, we should be told the nature of their generic vulnerabilities. A generic vulnerability is a potential weakness of the practice that is somehow connected with its basic causal structure. It might have to do with a high sensitivity to small errors in execution. Or it might have to do with the environment in which the practice is being implemented, e.g., an environment that might impose certain insupportable stresses. Of course, all political and implementation environments are stressful to a certain degree, and we can reasonably include in the essential definition of a particular smart practice those features necessary to safeguard it against the more predictable and potentially damaging stresses. Without such safeguards an otherwise smart practice could become a very dumb practice! For
13

Sandford Borins, Innovating with Integrity: How Local Heroes are Transforming American Government. Washington, DC: Georgetown University Press, 1998, p. 9.

Eugene Bardach

Part III Smart Practices Research

10

instance, if privatizing certain municipal service functions is a smart practice when would-be private suppliers operate in a competitive market, it might become a very dumb practice: if it were to be carried out in an environment monopolized by a single supplier; if the bidding process is very corruptible and corrupt interests were to discover this fact; if inappropriate performance measures were to be stipulated in the contract; or if the municipal contract management procedures were overly rigid or overly lax. To take another example, a high-expectations welfare-to-work program is vulnerable to the condition of the local labor market. If unemployment is high and jobs are scarce, high expectations will produce in participants more defeatism about themselves and more cynicism about the responsibility that society is urging on them. A government-neighborhood partnership regarding parks is, in a generic sense, vulnerable to, among other things, temptations on the part of policy makers to slowly shift more and more burdens onto neighbors while reallocating budgetary funds to other departments. Generic vulnerabilities are only the potential for trouble, it should be remembered. Whether the troubles actually materialize depends on the nature of the local environment in which the smart practice is implemented and on the success of various parties who are aware of the vulnerabilities in designing and implementing successful counter-measures. Contracting processes, for instance, can be designed to minimize corruption, albeit at some cost. And neighbors entering into a partnership with the city regarding parks maintenance might insist on putting the terms of the partnership in writing and holding a well-photographed press conference to announce them. Even if such a document had no legal standing, it would give neighborhood representatives in later years some useful political leverage. Two particular types of vulnerability are especially worth attending to. One pertains to likely failures of general management capacity, e.g., a low general level of leadership talent or the lack of a good government ethos that would make it easier to implement this or any other practice successfully. The other pertains to weaknesses intrinsic to the particular practice itself, e.g., a susceptibility to conflict in some service delivery program over whether to give priority to this or that catchment area or needy subpopulation or, in some safety-oriented regulatory program, whether to err on the side of (injury-tolerant) leniency or (costly) stringency. But will it work here? Assuming that you have understood the essence of the generic smart practice very well, including its generic vulnerabilities, and have mapped the variety of supportive elements that could increase its odds of success, in the end, you must still ask the question, Assuming this practice is indeed smart in some contexts, is ours a context in which it can work well enough to warrant trying it? Answering this question intelligently entails looking at both the source contexts, where the practice appears to have worked well, and at your own target context, where it is being considered for adoption. The target context. Within your target context, a careful assessment of the present situation is in order, of course; but a static answer based on this assessment is not enough. You need to think also about what might be done at reasonable cost or risk to improve the prospects of the smart practice in the target context were it to be implemented there. These actions fall into two categories: possible safeguarding strategies and possible enhancement strategies.

Eugene Bardach

Part III Smart Practices Research

11

First, consider safeguarding strategies. These have to do with the generic vulnerabilities of the smart practice. Are the most dangerous of them likely to cause unacceptable trouble in your context? For instance, if excessive rigidity in the contract management process is a generic vulnerability of partnering with a nonprofit agency, are your contract management institutions known to be unusually rigid? And if so, is there anything you can do to offset this problem? Might you, for instance, find someone in the contract management bureau who can serve as a special protector and expediter? Or, if you cannot do that, can you find some way to structure the contract terms so that the contractor is held accountable for achieving general results rather than for following specific procedures? Regarding enhancement strategies, consider what I have called above the supportive elements that can help a practice work relatively better. What supportive elements will be put into play? How well are they likely to perform? Can you do anything to improve them? For instance, can you attract top-notch personnel to manage this program or undertake this project? Can you obtain more stable funding than annual appropriations? Can you mobilize the press to take positive notice of what we are doing? Can you count on the support of key stakeholders and relevant political constituencies or at least on weak action from opponents? The source contexts. If you have to search very hard for smart practices that might be useable in your own situation, the chances are that the practice will not be very widespread. This means that the specimens you locate will come from jurisdictions, agencies, or locales where policy makers and administrators tend to look more favorably on novelty and innovation than is usually the case. Hence, their overall managerial capacity may be better than average, and perhaps better than the one in your own locale. If the source contexts are largely pilot or demonstration programs, you need to be particularly cautious, because: -- Pilot program implementers probably bring more enthusiasm and talent to bear on their work than the average program implementers; and enthusiasm and talent count for something. -- The political and financial conditions at the pilot sites are probably more favorable (or less unfavorable) than those in the average site. -- Bureaucratic resistance to a pilot is typically less intense than to a permanent change that threatens existing values, status, job security, or work routines. How cautious should you be in extrapolating from successes observed in pilot or demonstration contexts? No systematic research exists to answer this question. However, a RAND Corporation analysis of a variety of juvenile crime prevention programs discounted the effectiveness levels attained in the pilot contexts by 15-40% when estimating a scaling up penalty that would apply when implementing the programs on a wide scale.14 Although the

Peter H. Greenwood, Karyn E. Model, C. Peter Rydell, and James Chiesa, Diverting Children from a Life of Crime: Measuring Costs and Benefits, Santa Monica, CA: RAND Corporation., 1995.

14

Eugene Bardach

Part III Smart Practices Research

12

RAND analysts offered no explicit reasoning for choosing the penalty factors that they did, their choices do seem reasonable. If you are analyzing the possibility of implementing a smart practice not just in some known local context but on a wide scale, you should be concerned about more than the fact that pilot program results may be much better than average. You should also be concerned about the existence of many below-average sites where the smart practice would be implemented, some of them perhaps quite a bit below average. In an era when it was much less natural to think about the federal devolution of program and policy responsibilities to state governments than it is today, federal policy-makers, particularly political liberals, often worried about the Mississippi problem. Mississippi was the rhetorical symbol of the poor, backward, and probably racist jurisdiction that would almost surely wreck or pervert any smart practice it was given responsibility for implementing. Back to the Eightfold Path Given the typical shortfall of good evidence relative to theory and speculation when it comes to assessing a smart practice, there is a danger of unwarranted optimism. No wonder a common criticism of the best practice research tradition is that it becomes excessively enthusiastic about what appear to be good ideas before their worth is sufficiently tested.15 But how much testing is sufficient, anyway? The answer has to be framed partly in terms of the costs of displacing what might actually be a better practice, perhaps even the practices currently in use (called above, p. .., letting present trends continue). However, if one is reasonably confident that current practices are ineffective and/or harmful, the costs of wrongly abandoning them in favor of the new and untried may not be so high after all. Thus, although the new and untried should bear some burden of proof, it should not be an excessive one. The correct approach is to treat the risks and uncertainties involved in adopting some seemingly smart practice as being on a par with the uncertainties associated with all the other alternatives under consideration. Of course, the costs of change negotiation, insecurity, hard feelings, and so on must also be counted against bringing in a new and seemingly smart practice. But such costs must be counted against any change, not just change to accommodate smart practice. Moreover, if institutions and people are very stuck in their ways, there may be benefits to change as such, not merely costs.16

15

Unfortunately, excessive enthusiasm for experiments that eventually fail gives even appropriate enthusiasm for experimentation a bad name. 16 Alternatively, if institutions and people are forever being reformed and reinvented and remodeled as occurs in many public school systems there may be benefits to stability, consistency, and focus. Cite Elmore.