quinta-feira, 8 de dezembro de 2011

Plan Implementation and Evaluation

Implementation requires different types of support to be successful. Sabatier affirms that only policy makers can affect the behavior of “street level bureaucrats” and help planners deal with complex intergovernmental issues. Implementation is viewed from three different perspectives: policy maker (Center), field-level implementing official (Periphery), and private actor at whom the program is directed (Target Group). The Center’s support is essential since hierarchically superior official obtain compliance from peripheral officials and institutions. Center, Periphery, and Target Group have different concerns. The Center is usually worried about the extent to which official policy objectives have been attained. The Periphery is usually focused on the manner in which local implementing officials respond to the perturbations in their environment. The Target Group is concerned about to what extent are intended services actually delivered.

Approaches to implementation research vary. One option combines perspectives from Center, Periphery, and Target Groups. This Center’s approach focuses on policy outputs of the implementation -- whether agencies are consistent with official objectives, to what extent basic strategies were modified during the course of implementation, and principal factors affecting the extent of goal attainment. The second approach focuses on the Periphery’s standpoint. It tries to respond to perturbation in the ongoing systems. The third approach is about formulation-implementation-reformulation cycle and is concerned with policy making as an evolutionary process in which goals within a policy area are clarified overtime or revised to meet social priorities.

Planners must also be concerned about other conditions to ensure the success of implementation. Sabatier points out the importance of assignment to a sympathetic implementing agency with slack resources. Policy implementation has to be assigned to an agency with consistent general orientation and with effective monitoring and controlling of personnel behavior. The problems in Center-Periphery cooperation happen because local peripheral officials seldom will flagrantly violate legal norms but will seek aggressively to exercise any discretionary authority in order to pursue their own agenda. Implementation is an extension of the conflicts of the formulation stage because sometimes the legislative product is the result of compromises in which the parties agree in little except to pass an ambiguous bill. The importance of background socioeconomic conditions is necessary for implementation because linkages between social and economic conditions affect the perceived needs of local population and officials, the strength of competing interest groups, and the financial resources of various jurisdictions. Complexity of joint action is important because the multiplicity of clearance points offers numerous occasions for delay and breakdown of consensus. Financial and technical resources should also be considered since minimum level of financial and technical resources is necessary if a program is to have any possibility of being effectively implemented. Underlying causal theory must be considered, for implementation analysis exposes deficiencies in the original theory to be subsequently corrected. The twin imperatives of political support and managerial skill become part of the implementation equation because the ability of a program to overcome pressures is contingent on continuous political support and substantial commitment. And the extent of behavioral or system change must be analyzed because the amount of resistance encountered will be proportional to the amount of change demanded.

Evaluation is key for planning. Brooks affirms that “planning theory has long advocated evaluation as an important element in the planning process”, even though it rarely occurs. The Feedback Strategy (FB) proposes six stages for evaluation and pays special attention to the interaction between the planner and the social and political environment. This is the order: define the problem operationally even though it is based on personal values, consider alternatives by thinking of various ways to address a problem, make a preliminary choice among the various alternatives courses of action and based on criteria and feedback, design and implement an experiment using creativity given the widest topic, evaluate by learning from experience, and make a disposing decision to interpret information acquired from various feedback and evaluation processes. The problems of FB is that evaluation should also occur during the process of planning; does not generate objective facts about the results of a given course of action since it reflects values, biases of authors and sponsors; and the difficulty to be applied in cases where the end product is a physical entity since preliminary experiments cannot be made.

In case evaluation is already taking place, Weiss claims that the first lesson for evaluators arriving on the scene is asking: Who initiated the evaluation project? Why? Is anyone objecting? What are the motives? Are practitioners, public administrators, and funders committed to results? These questions are important because the chances of an evaluation to influence decisions are low if there is little commitment to consider results and real purpose for the evaluation are not orientated to increase understanding. A systematic approach to evaluation for decision purposes -- as an instrumental action -- should entail different aspects. First, midcourse correction whose objective is to identify early ineffective practices and before they do lasting harm. Second, continuing, expanding, or institutionalizing the program, or cutting, ending, or abandoning. The objective here is to find out the extent to which a program is achieving its goals. Third, testing a new program idea to try out a new programmatic approach in a series of demonstration projects and evaluate effects. Fourth, choosing the best of several alternatives to find out which of the options has the best outcomes. Fifth, deciding whether to continue funding to tell the funder which program should continue receiving funding.

Evaluation is also organizational learning since it contributes to wiser choices and improved programming. This is done by recording of program history but the difficulty here is to narrow down the open-ended charge to record what happens and find an agreed-upon focus for the study; making visible to staff what they are doing, how participants are responding, and what external conditions are impinged on the program; highlighting program goals by using evaluation to influence practitioners’ behavior; requesting accountability to detail what has been accomplished with public funds; understanding social intervention to acquire basic knowledge about the kinds of things that work to change social conditions and human behavior. Evaluation as an instrumental or organizational action is only justifiable if people give considered attention to the findings based on the conditions they live. This is a question planners and evaluators must keep in mind. Planners must also be concerned about other questions like: who expects what since expectations vary with person’s position in the system, the difference between formative (process) and summative (outcome) evaluations, and the compatibility of purposes since all-purpose evaluation is a myth. Evaluator has to make choices of questions to be considered within the bounds of a single study and trying to satisfy the informal demands of multiple groups while also being concerned about how evaluations are commissioned. Hiring staff, outside researcher, or request for proposal depends on confidence, objectivity, understanding of the problem, potential for utilization, and autonomy, consider the array of stakeholder -- which purpose shall the evaluation serve and for whom --, and level in the structure -- evaluator should be placed within the organizational structure at a level consonant with its mission.

Planners face different challenges while implementing and evaluating a planning process. Lowry highlights that the basic tasks in organizing an evaluation are fairly easy to describe but applying them to a specific project is more difficult. The basic steps of practical research choices that an evaluator must make are the following. First, determine the clients and their purposes by trying to find out who are the potential stakeholders when framing the study. Second, select a project through readings and interviews with program managers and staff. Third, specify the project logic through interviews to identify implicit assumptions. Fourth, establish evaluation questions based on empirical questions that stakeholders care about, and that address the intended goals. Fifth, identify the sources of evidence that will be used by asking: What types of information are most valid (in both a substantive and statistical sense)? What types of information are available or can be generated cost-effectively? What types of information will be most persuasive for the people using the evaluation to make decisions? Seventh, determine what comparisons will be made to make a persuasive case that observed changes in the ‘treatment’ group are the result of the program and not impacts that would have occurred anyway. The optimal comparison is the experiment in which neither the researcher nor the client knows who the ‘treatment’ group really is. Eight, define strategies for increasing validity of findings by reviewing the evidence with valued colleagues and having them assess it carefully. Ninth, establish a dissemination strategy by arguing which results should be broadly disseminated in order to get feedback on the quality of the study so that it can be faithful to those promised to show the research in exchange for cooperation in preparing it.

Sabatier, P. "Policy Implementation", 1983
Brooks, M. "The feedback strategy of public planning", 2002
Weiss, C. "Evaluation", 1998
Lowry, K. "A Template for Program or Project Evaluation".

Nenhum comentário:

Postar um comentário