Twenty years have passed since the Institute of Medicine (IOM) published its seminal report on The Future of Public Health, naming assessment as one of three core public health functions.1 In the past 20 years, we have MAPPed and we have reached the APEX.2-4 A Google search on "community health assessment" returns more than 71 000 results. In 2005, more than half of local health departments reported completing a community health assessment (CHA) within the past 3 years and almost two-thirds planned to complete CHAs in the next 3 years.5 Federal programs such as the Substance Abuse and Mental Health Services Administration's Center for Substance Abuse Protection and the Maternal and Child Health Bureau's Maternal and Child Health Title V block grants require community assessments.6-8

But in the 20 years since publication of The Future of Public Health, what verifiable, objective evidence have we collected that informs health departments about what types of assessments are more effective, under what circumstances is assessment more effective, and how does assessment impact population health? Is assessment worthwhile in and of itself, or is it an activity that should be conducted only if evaluations reveal that it has verifiable, positive intermediate and long-term impacts on population health?

The purpose of this article is to present a brief intellectual history of assessment since the publication of The Future of Public Health, examine unanswered questions about assessment, and propose a public health services and systems research agenda for assessment.

Intellectual History of Assessment

Assessment assumed a prominent place in the public health vocabulary following the release in 1988 of the IOM's landmark report titled The Future of Public Health. The report defined assessment as the regular and systematic collection, assembly, analysis, and communication of "information on the health of the community, including statistics on health status, community health needs, and epidemiologic and other studies of health problems."1^(p7) It characterized assessment as one of three major functions of public health and highlighted the need for increased emphasis on assessment to provide a sound basis for the two other major functions of public health: policy development and assurance.

Since 1988, both the concept and the practice of assessment have evolved with experience at the national, state, and local levels. To facilitate and guide assessment, various tools and frameworks have been developed including the Assessment Protocol for Excellence in program and workbook (APEXPH), the community health improvement process (CHIP), and Mobilizing Action through Planning and Partnership (MAPP).1-4,9 These and other milestones in assessment are summarized in Table 1.

TABLE 1 Milestones in assessment, 1983-2008

Current Perspectives on CHA

In 2008, assessment is more complex and varied-and possibly confused-than it was when it entered the public health lexicon in 1988. Its initial focus on information about the health of the community has been broadened to include information on organizations, resources, public health infrastructure, budgets, contextual factors, risk factors, community themes, and forces of change. The rich intellectual history has helped generate equally rich and varied applications of assessment. The articles included in this special focus issue illustrate the current practice and impact of assessment, with articles focusing on broad impacts of assessments,16 impacts relating to specific programmatic areas,17,18 and tools relating to communicating, monitoring, and evaluating assessments.19-21 Yet, the richness of assessment as it has evolved over the last 20 years masks five fundamental questions that should be addressed before public health agencies devote additional scarce budgetary and staff resources to it.

Question 1: What is assessment?

In 1988, the IOM defined assessment as including "all the activities involved in the concept of community diagnosis, such as surveillance, identifying needs, analyzing the causes of problems, collecting and interpreting data, case-finding, monitoring and forecasting trends, research, and evaluation of outcomes."1^(p44) It also delineated the responsibilities of federal, state, and local governments in assessment: the federal government should support "knowledge development and dissemination through data gathering, research, and information exchange"; each state should assess "health needs within the state based on statewide data collection"; and local public health units should assess, monitor, and conduct surveillance for "local health problems and needs and of resources for dealing with them".1^(pp143-145) Although federal and state governments provide information and support, the focus of assessment efforts and the locus at which action takes place are the community. Assessment is, thus, a set of activities that produces information on community health; when focusing on the community level, it is appropriately called "community health assessment."*

The process and products of assessment do not, however, occur in isolation. Assessment is part of a larger, more comprehensive process of community health improvement; regrettably, the distinction between assessment, on the one hand, and this larger process, on the other hand, has become blurred. As indicated by Myers and Stoto,15 current definitions of assessment include various aspects of process, product, or both. Among other activities, assessment has been interpreted to encompass primary and secondary data collection, setting intervention priorities, planning interventions, and perhaps even implementing interventions.

Evidence-based decisions about whether and how to conduct assessment cannot be made until a clear, workable, and focused consensus definition of assessment is developed. Such a definition could be worthwhile if it fulfilled two criteria. First, the definition should clearly identify, define, and distinguish individual components of both the assessment process and the assessment product. Each individual component of assessment could be individually evaluated in a given setting and a given implementation. Second, and again to enable evaluation, the process and products of assessment must be accompanied by a detailed, and hopefully results-based, logic model.25-28 Such a logic model would detail areas of control internal to health departments (inputs/resources, activities, and outputs) and areas of influence external to health departments (immediate direct outcomes, intermediate indirect outcomes, and final outcomes). Given that the assessment process and products are part of a larger process, care must be taken when building a logic model for assessment to distinguish between assessment products and the processes to produce those products, on the one hand, and the processes, products, and outcomes of the larger process, on the other hand.*+ the same time, it should be recognized that some activities undertaken as part of the assessment process, such as identifying important community stakeholders and building community partnerships and alliances, are also critical to formulating and implementing the community health improvement process.

Question 2: What is the impact of assessment on community health and processes intended to improve it?

What have we learned about the impacts of CHAs since 1988? What lessons can local health departments take from the literature in deciding whether and how to conduct CHAs? What is the nature and scale of investments in CHAs that produce the greatest impacts on the community health improvement process, and hopefully, on community health? Given the complexity and variety of definitions of assessment as they have evolved and been applied since 1988, it comes as no surprise that comprehensive evaluations of CHAs are difficult, if not impossible, to locate.15^(p40)

In a context other than CHA, the Center for Global Development has pointed to the existence of "an 'evaluation gap' [that] has emerged because governments, official donors, and other funders do not demand or produce enough impact evaluations and because [those] that are conducted are often methodologically flawed."29 The American Public Health Association (APHA) recognized this "evaluation gap" in its 2006 policy statement on the need to "Conduct research to build an evidence-base of effective community health assessment practice,"30 and the Council of State and Territorial Epidemiologists (CSTE) made similar points in its 2007 position statement on "Research to study and disseminate evidence of effective community health assessments."4,31++

Asking that the CHA demonstrate a direct impact on community health may, however, be asking more of it than we should. Given that it is a tool within a larger health improvement process, it may not be reasonable to expect the CHA to directly impact community health. A more reasonable approach may be to evaluate the CHA on its own terms, that is, both on the quality of the information that it produces and on its usefulness to a larger community process. In fact, Curtis argues that "assessment has value independent of action taken to correct community health problems."32^(p25) By identifying the health problems of a community, the CHA must remain at the core of the practice of public health. The question is not "should we conduct a CHA," but "what products presented and communicated in what ways are most effective in identifying the health problems of a community?"

Question 3: How should a CHA be evaluated?

This special focus issue provides examples of attempts to evaluate the process or products of a CHA,21 or both its process and products,20 and the impact of its products on other community health improvement processes.16-19 Articles in this special focus issue also illustrate the use of multisite comparative quantitative evaluations of assessment processes, tools, or products,19-21 multisite case studies,16 and single-site case studies.17,18 Useful as the individual articles are, we can justifiably ask: How do the articles in this special focus issue, taken together as a body of literature, provide health departments with future directions for evaluating CHAs? As both the APHA and the CSTE recently pointed out, the criteria for conducting useful research on CHAs remain to be defined.30,31

Solet and colleagues16 sensibly suggest developing "core standards" and "systematic and consistent guidelines for evaluation" of CHAs. To build an evidence base useful for health departments, we suggest seven characteristics for such evaluations. The first and second characteristics are based on the definitional criteria: evaluations use a clear, focused definition of CHA; and evaluations rely on a logic model for describing the inputs, activities, outputs, and potential impacts of the assessment and the relationship of the assessment to other components of the CHIP. Third, the evaluation must include testable hypotheses concerning successful assessments. Fourth, the process that is used to conduct the assessment is sufficiently documented to allow identification of factors that could be evaluated as predictors of the success or the failure of the assessment.16 Fifth, the outputs and outcomes of the assessment are sufficiently documented to allow their analysis against the postulated predictors of the success or the failure of the assessment. To realize the fourth and fifth characteristics, a sixth characteristic has been suggested: evaluations should be based on validated and replicable instruments and methods, whether the evaluations are based on single- or multisite case studies or quantitative data collection, or some combinations of case studies and quantitative data collection. Finally, to provide sufficiently robust evaluations to identify successful assessments and assessment strategies, multisite and multistate evaluations are required regardless of whether the evaluations are qualitative or quantitative.*

The article in this issue by Snyder and Spice20 identifies 12 categories for summarizing positive outcomes or impacts of CHAs, and the article by Solet et al16 suggests 7 categories. Based partly on their findings, we propose a series of measures for evaluating the outputs and outcomes of CHAs that may be useful in operationalizing characteristics 5 and 6 (Table 2).+

TABLE 2 Possible questions and measures for evaluating the outputs and outcomes of community health assessments

Question 4: What tools should be developed and shared to foster evaluation of the CHA?

The articles in this special focus issue include various tools that can support evaluations of CHAs, including local health department surveys,19,36 Google searches,19 Web-based questionnaires for evaluating CHA reports,21 and forms for ongoing tracking of CHA impacts.20,37

To be useful, instruments for evaluating assessments must fulfill at least three conditions. First, they must be easily available from a central source.* Second, each instrument must clearly pertain to specific assessment processes or products and their associated logic model. Third, they must be verified or verifiable for internal and external validity and reliability, with their supporting data easily available.

Question 5: Who should evaluate CHAs?

The role of communities as partners in evaluating CHAs remains to be thoroughly discussed. In a 2004 policy statement, the APHA pointed out that "this lack of involvement of community members in the conduct of research [horizontal ellipsis] has resulted in distrust of and reluctance to becoming involved in such research."38Community-based research in public health has been defined as "a collaborative approach to research that equitably involves, for example, community members, organizational representatives, and researchers in all aspects of the research process."39 Requirements for and issues relating to community-based research have been explored in both clinical medicine and public health generally,39-43 and should also be explored for evaluations of CHAs.44

In considering community-based research and participatory evaluation as models for evaluating CHAs, it is important to realize that assessment practice and expertise remain largely based in state and local health departments. Hence, it is neither surprising nor inappropriate that most assessment-related evaluation is also based in health departments. As an example, seven of the eight articles in this special focus issue are authored by individuals with responsibilities for conducting the evaluated assessments.

Neither community-based research nor participatory evaluation is self-defining, and both rubrics include a range of community participation, purposes, and methods.45-47 Given the current and likely future mixed responsibilities of the evaluators, different types of community-based participatory research and participatory evaluation should be explored such as empowerment evaluation, practical participatory evaluation, and transformative participatory evaluation.45,48-50

A Public Health Services and Systems Research Agenda for Evaluating CHAs

Both the APHA and the CSTE have called for research on effective CHA. Twenty years after the publication of The Future of Public Health, it is clearly time to establish and embark upon a formal public health services and systems research evaluation agenda for the CHA.51-53 A preliminary agenda could include the elements delineated in this commentary: a clear definition of CHA that identifies its components; logic models for CHA that include each component; testable hypotheses concerning successful assessments; multisite and multistate analyses; and validated and replicable instruments. Given the obvious centrality of communities in CHAs, its evaluation should include communities throughout the evaluation process.

REFERENCES