As described in RFA’s Evaluation Policy Guide, an evaluation policy articulates an agency’s principles, goals and practices for building and using evidence. It can serve several purposes:
Promote and protect the integrity of evaluation activities by highlighting principles such as independence, objectivity, transparency and equity.
Provide guidance on the agency’s expected practices for building and using evidence.
Clarify how equitable processes and outcomes are being planned for and executed through an evaluation.
Twenty-four federal agencies are required to develop Agency Evaluation Policies under the Evidence Act for CFO Act agencies. Policies must be established as a guide for the agency’s evaluation activities throughout the evaluation lifecycle. These policies should incorporate the evaluation standards described in OMB M-20-12, as well as any other standards or principles deemed important for the agency.
See examples of federal evaluation policies at Evaluation.gov or in RFA’s Evaluation Policy Guide.
Promoting Evaluation Through Grantmaking
Federal agencies can promote evaluation activities among grantees through requirements or incentives, as well as by providing opportunities for technical assistance and collaboration. Here are six ways to promote evaluation:
Federal agencies with the resources to sponsor evaluations may form partnerships with grantees. The agency evaluates the effectiveness of grantees’ services, typically through a contract. To facilitate this kind of arrangement, it is advisable to require grantees to participate in federally sponsored evaluations.
Agencies may 1) require applicants to include planned evaluations in their proposals, and assign points based on the quality of the proposed evaluation in rating proposals, 2) require grantees to conduct evaluations, and/or 3) require agency review and approval of evaluation plans after award.
Agencies may tailor evaluation requirements based on several considerations. For example:
The agency may require only larger grantees to conduct evaluations, or may require larger grantees to conduct more ambitious evaluations than smaller grantees.
The agency may choose not to require grantees to conduct evaluations if they are implementing evidence-based models.
The agency may adjust requirements for evaluation based on the capacity of applicants. In a field where likely applicants may have limited evaluation capacity, the agency may choose primarily to conduct federally sponsored evaluations rather than requiring grantees to conduct evaluations. Or the agency may invest heavily in technical assistance to help grantees conduct successful evaluations, as described below.
Federal agencies may provide technical assistance to help grantees develop and carry out rigorous evaluations. Agencies may also choose to enter cooperative agreements with grantees that outline the grantees’ and granting agencies’ respective roles. In addition to providing technical assistance, the federal role might include review and approval of key activities, such as selection of outcome measures or other aspects of evaluation design. To learn more, explore a special issue of the journal Evaluation Review focused on evaluation technical assistance.
Many federal agencies sponsor annual meetings for grantees of one or several related federal grant programs. Agencies may include evaluation among the discussion topics at these meetings. Agencies may also take a more intensive approach. For example, an agency might sponsor quarterly meetings (virtual or in-person) for a subset of grantees carrying out related evaluations. Some grantees may be encouraged or expected to collaborate closely, such as by establishing common research questions or outcome measures, or pooling data.
This is one way to undertake a coordinated approach, rather than each grantee conducting its own evaluation in isolation.
Without an intentional focus, evaluations may not provide a full and accurate picture of how a program is serving participants. And they may perpetuate disparities. Agencies can address this challenge by prioritizing evaluation approaches that empower grantees and participants to be full partners in the research, including helping design the research, interpret data and shape how findings are used. Engaging grantees and participants - especially those who have been historically underserved - in evaluation is critical to ensuring that evidence is built from comprehensive data that is appropriately collected and analyzed. This helps ensure that a subset of the population is not left behind and agencies understand what works, for whom and under what circumstances.