The evaluation of community development programs is complicated by the often-competing expectations of funders, community members and project managers. Some of these interests are represented by evaluation efforts that focus on process and participation, while others may be more quantitative in nature focusing on outcome measures. Common to all of these methodological concerns is the difficulty evaluating poorly planned projects. While good planning does not assure good evaluation it provides a common basis for beginning the evaluation effort. We have found Concept Mapping to be an effective tool in planning and evaluating community projects.

Concept mapping is a type of structured conceptualization which can be used by groups to develop a conceptual framework which can guide evaluation or planning. In the typical case, six steps are involved: 1) Preparation (including selection of participants and development of focus for the conceptualization); 2) the Generation of statements; 3) the Structuring of statements; 4) the Representation of Statements in the form of a concept map (using multidimensional scaling and cluster analysis); 5) the Interpretation of maps; and, 6) the Utilization of Maps. Concept mapping encourages the group to stay on task; results relatively quickly in an interpretable conceptual framework; expresses this framework entirely in the language of the participants; yields a graphic or pictorial product which simultaneously shows all major ideas and their interrelationships; often improves group or organizational cohesiveness and morale.

An Introduction to Concept Mapping for Planning and Evaluation

Probably the most difficult step in a planning or evaluation project is the first one -- everything which follows depends on how well the project is initially conceptualized. Conceptualization in this sense refers to the articulation of thoughts, ideas, or hunches and the representation of these in some objective form. In a planning process, we typically wish to conceptualize the major goals and objectives, needs, resources and capabilities or other dimensions which eventually constitute the elements of a plan. In evaluation, we may want to conceptualize the programs or treatments, samples, settings, measures and outcomes which we believe are relevant.

This special section of Evaluation and Program Planning extends earlier work by Trochim and Linton (1986) who proposed a general framework for structured conceptualization and showed how specific conceptualization processes can be devised to assist groups in the theory and concept formation stages of planning and evaluation. The papers presented here focus on one specific type of structured conceptualization process which we term "concept mapping". In concept mapping, ideas are represented in the form of a picture or map. To construct the map, ideas first have to be described or generated, and the interrelationships between them articulated. Multivariate statistical techniques -- multidimensional scaling and cluster analysis -- are then applied to this information and the results are depicted in map form. The content of the map is entirely determined by the group. They brainstorm the initial ideas, provide information about how these ideas are related, interpret the results of the analyses, and decide how the map is to be utilized.

The specific concept mapping process described here and discussed throughout this volume is considered only one of many possible structured conceptualization processes. This process can be used whenever there is a group of people who wish to develop a conceptual framework for evaluation or planning, where the framework is displayed in the form of a concept map. A concept map is a pictorial representation of the group's thinking which displays all of the ideas of the group relative to the topic at hand, shows how these ideas are related to each other and, optionally, shows which ideas are more relevant, important, or appropriate.

The scenario within which concept mapping is applied assumes that there is an identifiable group responsible for guiding the evaluation or planning effort. Depending on the situation, this group might consist of the administrators, staff or members of the board of an organization; community leaders or representatives of relevant constituency groups; academicians or members of the policy making community; funding agents or representatives of groups with oversight responsibility; representatives of relevant client populations; or combinations of these. The concept mapping process is guided by a facilitator who could be an outside consultant or an internal member of the group responsible for the planning or evaluation effort. The facilitator's role is only to manage the process -- the content, interpretation and utilization of the concept map are determined entirely by the group.

An overview of the concept mapping process is provided in Figure 1. Figure 2 provides an example of a completed concept map.

Figure 1: The concept mapping process.

Figure 2: Cluster rating map for the York County Elderly project.

Conclusions

Concept mapping of the type described here is designed to bring order to a task which is often extremely difficult for groups or organizations to accomplish. The process has several distinct advantages. First, it encourages the participant group to stay on task and to lay out relatively quickly a framework for a planning or evaluation study. Second, it expresses the conceptual framework in the language of the participants rather than in terms of the evaluator or planner's language or the language of social science theorizing. Third, it results in a graphic respresentation which at a glance shows all of the major ideas and their interrelationships. Fourth, this graphic product is comprehensible to all of the participants and can be presented to other audiences relatively easily. Finally, we have observed over many concept mapping projects that one of the major effects of the process is that it appears to increase group cohesiveness and morale. Especially in groups which have previously tried to accomplish conceptualizing through committee discussions, we have found that they readily appreciate the structure of the process and the ease with which it produces an interpretable starting point for subsequent evaluation or planning work.

This concept mapping process is by no means the only way in which group conceptualization can be accomplished nor is it necessarily the best way for any given situation. In situations where a group can achieve consensus relatively easily on their own or where a pictorial representation of their thinking is not desired or deemed useful, this approach would not be recommended.

William Trochim is the Director of the Office of Not-for-Profit Management Institutions (OMNI), and is a Professor in the Department of Policy Analysis and Management at Cornell University.

Phone: 607-255-0887
Fax: 607-255-4071

Don Tobias is the Director of the Community and Rural Development Institute (CaRDI) and is an Associate Professor, Department of Policy Analysis and Management at Cornell University.

Phone: 607-255-1949
Fax: 607-255-2231