|
|
        
|
Evaluating the Public Involvement Process Specific methods and criteria for evaluating the effectiveness of the public involvement process are just beginning to be developed. Agency personnel involved in public involvement activities must examine both the effectiveness of the overall process in meeting public involvement goals and the usefulness of individual techniques. With clearly articulated goals for the public involvement process agency personnel will be able to ascertain how well those goals were met. For example, if the goal of public involvement is simply to inform the public then the number of individuals or groups that are reached through public involvement activities might be a crucial measure of success. If the goal is to include stakeholders in the decision making process then more complicated assessment measures are needed. These might include using nominal group process to ascertain staff impressions of the process or soliciting input from the stakeholders involved through a short mail or telephone survey. A wealth of information can be collected across the agency on the effectiveness of processes used in public involvement as well as individual techniques. This information can be shared with colleagues as institutional knowledge on the best techniques to use in different situations is ascertained. In addition, new techniques and new ways of implementing old techniques may have promise for planners working on a diversity of issues. Developing formal methods to share this experience is crucial for advancing public involvement activities. The effectiveness of the use of individual techniques should be evaluated throughout the process. This type of periodic monitoring will help in collecting information useful to planning additional public involvement processes. Any shortcomings in the present plan may also become apparent and adjustments can be made. Tools for facilitating public education and increasing awareness can be evaluated to learn their usefulness in telling the public about agency plans or programs or educating them about planning options. During the public involvement process the agency should establish a system for tracking the distribution of education and information materials, requests for additional information, and other correspondence (e.g., comments) received as a result of the materials. Information on where to send public comments can include a code in the address (such as a department number) which will alert staff to the source used by the individual. While a complete accounting of what has been distributed and how it has been used is nearly impossible, best available data should be collected. In some cases these may just be estimates. Some of the data that should be collected include:
Techniques for facilitating information exchange, such as meetings or hearings, workshops, focus groups, surveys, or advisory committees should be evaluated as part of the activity. In this way insight into why an activity was successful or how it could be improved is immediately available for public involvement program planning. Prior to any event agency personnel should determine what evaluation criteria they will use and what data they need to collect. For meetings and hearings evaluation data collection might include:
In addition agency staff should informally evaluate the adequacy of the room or building where the meeting was held and the amount of time allocated for the process. For workshops and other facilitated meetings different criteria might be important. The actual number of attendees might not be as important as broad representation of stakeholders. Staff would want to know how many stakeholder groups, or different opinions, were represented. Equally important is what groups were not represented, and why. A well-planned workshop will have specific goals and an agenda designed to meet those goals. Staff will be able to evaluate whether or not their goals were met based on the criteria they set. Asking participants to anonymously fill out short evaluations at the end of the workshop can collect valuable information. Anonymity helps to ensure candor and encourages input. Questions might be concerned with the adequacy of pre-workshop information, the effectiveness of the workshop process itself, and satisfaction with the outcome of the workshop. Focus groups can be evaluated on two different levels. The first is the adequacy of the participant selection process. Participants should be asked to provide necessary information before the group begins. Basic information such as age, gender, occupation, or affiliation with particular groups might be appropriate depending on the type of group desired. This serves as a check to the recruiting process and assures that staff have a good record of participant characteristics. Agency staff should also evaluate the usefulness of the data collected through the focus group. The data collected should be compared with the objectives for data collection enumerated before the focus groups. Any shortfalls might result in changes in additional focus groups. The effectiveness of surveys can be ascertained by determining whether or not the desired data were collected. Statistical techniques can be used to analyze survey data. Agency staff must make a more subjective appraisal of whether or not the data collected met their needs for the planning process.
The utility and progress of techniques used to effect direct public participation in decision making needs to be monitored as the process continues. Each process will have to be monitored based on criteria developed for that unique situation. If progress is not as desired then adjustments may need to be made to the process being used. It is also useful to evaluate the process and the mechanisms used when the decision making exercise is complete. By objectively examining the costs and benefits of the process to the agency and the interested publics information useful in planning future public involvement processes of this type can be collected and shared within the agency.
|
