Public Health Assessment & Program Evaluation

 Guidelines for Promoting Evidence-Based Practice for Health Promotion Programs

Last Updated: August 06, 2018
Skip Navigation LinksAPHC Home / Topics / Health Surveillance & Evaluation / Public Health Assessment & Program Evaluation / Guidelines for Promoting Evidence-Based Practice for Health Promotion Programs

The National Prevention Strategy1 advocates for the development and implementation of effective, evidence-based health promotion programs. Evidence-based health promotion programs are those that are founded on the best available research and are recommended on the basis of a systematic review of the published, peer reviewed research. A systematic review recommends a health promotion program's implementation based on the quality, quantity, and consistency of the scientific literature supporting the program's effectiveness. The benefit of implementing evidence-based programs is that one can have greater confidence in their effectiveness because they have been carefully tested.

There is a strong demand for health promotion programs to meet the Army's public health needs and the Army must be a sound steward and ensure that Soldiers and their Families receive the best available evidence-based health promotion programs with increasingly scarce resources.  However, many programs have not yet adequately demonstrated that they are evidence-based2.  Program owners are not always held accountable for ensuring their programs are based on strong scientific evidence prior to implementation, ensuring their programs are fully evaluated, and for defending their programs' effectiveness to external reviewers, inspectors, auditors, evaluators, and higher-level leadership (such as for the Health Promotion Risk Reduction Council annual review). 

Promoting evidence-based practice for health promotion programs begins with providing clear guidance regarding what constitutes evidence of a health promotion program's effectiveness. Therefore, the purpose of this document is to offer the Army operational guidelines to promote evidence-based practice for health promotion programs based on existing literature in the field of prevention science3,4,5,6,7.  These guidelines can aid health promotion program owners to identify effective health promotion programs for implementation, defend their programs' effectiveness, and identify ways to increase their programs' evidence of effectiveness.  The guidelines can also be used along with other criteria (e.g., feasibility, utility, cost effectiveness, and sustainability) as a strategic framework to evaluate existing evidence in support of Army health promotion programs.

Health promotion program owners can begin to demonstrate the evidence supporting their programs through a written document that clearly articulates the expected relationship between services and intended outcomes (i.e., a logic model).  Evidence supporting the programs is strengthened by past evaluations of similar programs that demonstrate a relationship between services and intended outcomes, a manual or protocol for implementation, evidence that the program poses no risk of harm to the target population, and a demonstrated commitment to continuous process improvement and ongoing evaluation.

Where a systematic review has not been conducted or there is insufficient existing scientific evidence to conduct a systematic review, program owners can develop their evidence base through program evaluation studies that use well-accepted program evaluation methods.  All studies supporting a health promotion programs' effectiveness should be characterized by systematic and objective inquiry. In addition, confidence in a program's effectiveness is increased by evaluation designs that demonstrate a relationship between program services and actual program outcomes through studies that include an appropriate comparison group, prospective measurement of participation in a program and program outcomes, program outcomes that endure at least one year, and program outcomes that replicate across multiple audiences in multiple settings.

 Back to Top

The following grades categorize the level of evidence supporting health promotion programs according to the strength of supporting evidence program owners can provide.  These grades are based on the strength of evidence, the evidence supporting the program's effectiveness as documented in the scientific literature, and prior systematic program evaluation reports.  All health promotion program owners are encouraged to make evaluation a high priority in program planning and implementation, identify the current level of evidence in support of their programs, engage in ongoing evaluation to continually build the evidence base for their programs, and pursue an "Evidence-Based" designation on the following continuum.

    Grade A, 'Evidence-Based' health promotion programs are those that are endorsed by a recommendation from a systematic review that accounts for potential harm to the target population and the quality, quantity, and consistency of program evaluation studies supporting the effectiveness of a program. 

    Grade B, 'Well Supported' health promotion programs are those that produce at least two peer reviewed publications that demonstrate evidence of a relationship between services and actual outcomes on the programs' target audience.  Publications in this category reflect program evaluation designs that include an appropriate comparison group, prospective measurement of participation in the program and program outcomes, and demonstrate no evidence that the program poses a risk of harm to the target population. The results of these studies also demonstrate that the relationship between services and outcomes endures at least one year and replicates across multiple audiences and multiple settings.  However, these publications do not account for the consistency of evidence in support of a program (specifically, there exists a possibility that other studies may not support a program's effectiveness).

    Grade C, 'Supported' health promotion programs are those that produce at least one published, peer-reviewed study that demonstrates evidence of a relationship between services and actual outcomes on the programs' target audience that endures at least one year.  Publications in this category reflect program evaluation designs with an appropriate comparison group, prospective measurement of participation in the program and program outcomes, and demonstrate no evidence that the program poses a risk of harm to the target population.  However, these publications cannot demonstrate that the programs' effect replicates across multiple audiences and/or across multiple settings.

    Grade D, 'Promising' health promotion programs are those that produce at least one published, peer-reviewed study that demonstrates evidence of a relationship between services and actual outcomes on the programs' target audience.  Publications in this category reflect a program evaluation design with an appropriate comparison group, measurement of participation in the program and program outcomes, and demonstrate no evidence that the program poses risk of harm to the target population.  However, these publications either cannot demonstrate that the programs' effect endures at least one year and/or cannot demonstrate prospective measurement of participation in the program and program outcomes.

    Grade E, 'Emerging Level 2' health promotion programs are those that are supported by peer reviewed and published evidence of effectiveness from systematic program evaluation studies including, but not limited to cross-sectional, pre-test/post-test or post-test only designs with no comparison/control group.  However, these evaluation designs are limited by a lack of comparison group or measurement of participation in the program to demonstrate that outcomes are caused by the programs' services.

    Grade E, 'Emerging Level 1' health promotion programs are those that produce at least one formal report that demonstrates evidence of a relationship between services and actual outcomes on the programs' target audience and demonstrate no evidence that the program poses a risk of harm to the target population.  Programs in this category are supported by evidence of effectiveness from systematic program evaluation methods including, but not limited to cross sectional, pre-test and post-test, comparisons of programs effects to national norms for program outcomes, post-test only designs, or evidence from more rigorous evaluations that are not published in a peer-reviewed journal.  However, these evaluation designs are limited either by a lack of an appropriate comparison group to demonstrate that outcomes are caused by the programs' services or lack of validation from the peer-review process.

    Grade U, 'Unsupported' health promotion programs are those that cannot produce evidence of a relationship between program services and program outcomes.

    Grade NR, "Not Recommended" health promotion programs are those programs that pose a demonstrated risk of harm to the target population.

Health promotion program owners should be prepared not only to demonstrate their programs' evidence of effectiveness, but also to demonstrate that the weight of the evidence supporting their programs' effectiveness is stronger than the evidence opposing their programs' effectiveness.

Evidence of program effectiveness can also be supplemented through qualitative methods such as case studies, interviews, focus groups, and expert opinion supporting a programs' effectiveness. These methods also provide a more in depth understanding of the program, how it functions, and why it is (or is not) effective.

 Back to Top

References

1. U.S. Department of Health and Human Services National Prevention Council (2011).  National Prevention Strategy: America's Plan for Better Health and Wellness. Washington, DC: Office of the Surgeon General.

2. Subcommittee on Military Weight Management, Committee on Military Nutrition Research (2004). Weight Management State of the Science and Opportunities for Military Programs.  Washington, DC: National Academies Press.

3. Briss, P.A., Zaza, S., Pappaioanou, M, et al (2000). Developing an evidence-based Guide to Community Preventive Services—Methods. The Task Force on Community Preventive Services. American Journal of Preventive Medicine,18 (Suppl 1),35-43.

4. Center for Disease Control and Prevention (2011). The Centers for Disease Control and Prevention's Guide to Community Preventive Services: Systematic Reviews and Evidence-Based Recommendations. Retrieved from http://www.thecommunityguide.org/index.html.

5. Thornton, T.N., Craft, C.A., Dahlberg, L.L., Lynch, B.S., Baer, K (2000). Best Practices of Youth Violence Prevention: A Sourcebook for Community Action. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control.

6. University of Colorado Institute of Behavioral Science Center for the Study and Prevention of Violence (1997). Blueprints for Violence Prevention (Vols. 1-11). Retrieved from www.colorado.edu/cspv/blueprints.

7. Greenberg M.T., Domitrovich, C., Bumbarger, B. (1999). Preventing mental disorders in school-aged children: A review of the effectiveness of prevention programs. State College, PA: Prevention Research Center for the Promotion of Human Development College of Health and Human Development

Back to Top