Congress
Federal laws provide funding for evaluation in a variety of ways. Funds may be designated for evaluation in authorizing statutes or in annual appropriations language. Statutory language may be more or less specific on a range of dimensions. For example, it may specify a floor or a ceiling, or both. It may specify evaluation topics with broad or narrow parameters. Or it may designate funds for evaluation alone, or for evaluation and other functions such as technical assistance.
Authorizing Statute vs. Annual Appropriations Language
Designating funds for evaluation in an authorizing statute provides federal agencies with greater consistency and predictability than making funds available through annual appropriations language. In addition to the year-to-year uncertainty of the appropriations process, appropriations bills (or continuing resolutions) are often passed months into the fiscal year, making it difficult for federal staff to plan, develop, announce, review and award grants and contracts by the end of the fiscal year.
On the other hand, many authorizing statutes are not regularly reauthorized, which can lead Congress to make evaluation funds available through the appropriations process rather than wait years for a reauthorization opportunity.
The Public Health Services Act under the U.S. Department of Health and Human Services includes the following language (Section 241): “EVALUATION.—Of the amounts appropriated under paragraph (1) for a fiscal year, the Secretary may reserve up to 1 percent for evaluations of projects.”
The Consolidated Appropriations Act of 2022 (Division H, Title I, Section 107 of Public Law 117-103) includes the following language for the U.S. Department of Labor: “The Secretary may reserve not more than 0.75 percent from each appropriation made available in this Act identified in subsection (b) in order to carry out evaluations of any of the programs or activities that are funded under such accounts.”
The Every Student Succeeds Act of 2015 includes the following language: “the Secretary, in consultation with the Director of the Institute of Education Sciences, may reserve not more than 0.5 percent of the amount appropriated for each program authorized under this chapter to carry out [evaluation] activities…”
How much?
Laws may specify a floor, a ceiling, a range or an exact amount in making funds available for evaluation. Amounts may be described as a percentage of the funds available for a program, or in dollar terms. In general, RFA recommends at least 1% for evaluation, as percentages allow funds for evaluation to grow proportionately if program funds increase. Conversely, funds for evaluation will shrink if program funds decrease. A hybrid approach can provide both stability and growth. For example:
“Of the amounts available under [previous provision], not less than 1% or $20,000,000 (whichever is greater) shall be to fund research and evaluation activities.”
Congress may also choose to make funds available for evaluation with no amount specified within the overall program appropriation, as is often the case for demonstration programs. In this approach, statutory language allows funds to be used for demonstration services and for evaluation at the discretion of the federal agency. For example:
“The Secretary shall award grants to conduct demonstration projects. The Secretary shall, directly or by grant, contract, or interagency agreement, evaluate the demonstration projects conducted under this subsection.”
How long?
Contracts for evaluation are often considered “non-severable,” meaning that the government does not reap full benefits of the work until it is complete. Half a study may not be better than none. In addition, evaluation work is notoriously difficult to price and predict. Many factors are outside the control of the government or the contractor. For these reasons, Congress may choose to make funds for evaluation available for obligation for periods longer than one year. This type of appropriation allows for streamlined structure and oversight of contracts, avoiding potential loss of funds due to delays, or multiple contract modifications that can eat up the time of federal and contract staff. For example, the Consolidated Appropriations Act of 2022 (p. 453) states for the U.S. Department of Labor:
“Amounts made available under this Act which are either appropriated, allocated, advanced on a reimbursable basis, or transferred to the functions and organizations identified in subsection (a) for research, evaluation, or statistical purposes shall be available for obligation through September 30, 2026: Provided, That when an office referenced in subsection (a) receives research and evaluation funding from multiple appropriations, such offices may use a single Treasury account for such activities, with funding advanced on a reimbursable basis. (c) Amounts referenced in subsection (b) that are unexpended at the time of completion of a contract, grant, or cooperative agreement may be deobligated and shall immediately become available and may be reobligated in that fiscal year or the subsequent fiscal year for the research, evaluation, or statistical purposes for which such amounts are available.”
How broad?
Congress may make funds available for evaluation of a specific federal grant program, or may provide authority for an agency to conduct evaluations across multiple federal grant programs. Funds tied to specific federal grant programs will ensure that evaluations focus on those specific programs. However, this approach makes it difficult to carry out studies on topics that cut across program boundaries. Further, for small federal grant programs, sufficient funds may not be available to support meaningful studies. A broader approach addresses these limitations. Congress may appropriate funds under a broad statutory authority, or may establish a set-aside across multiple federal grant programs.
As an example of a broad statutory authority, section 1110 of the Social Security Act authorizes “research or demonstration projects such as those relating to the prevention and reduction of dependency, or which will aid in effecting coordination of planning between private and public welfare agencies or which will help improve the administration and effectiveness of programs carried on or assisted under the Social Security Act and programs related thereto…” Congress regularly appropriates funds under this authority.
Alternatively, Congress may establish authority for agencies to transfer and consolidate funds from several different programs for research and evaluation purposes. For example, the Public Health Service Act includes the following provision: “Such portion as the Secretary shall determine, but not less than 0.2 percent nor more than 1 percent, of any amounts appropriated for programs authorized under this chapter shall be made available for the evaluation (directly, or by grants of contracts) of the implementation and effectiveness of such programs.” (f §238j (a))
For what?
Congress may choose to designate funds for evaluation only or may provide funds for multiple purposes, such as evaluation, technical assistance and program administration. While a multipurpose designation allows federal agencies greater flexibility, in practice the exigencies of program administration tend to crowd out funds for evaluation. A set-aside solely for evaluation better ensures that evaluation will be funded
High-quality evaluation requires oversight by specialized, dedicated staff. Specific statutory language allowing evaluation funds to be used for federal staff will make this possible. For example: “For administering and carrying out research and evaluation directly or by grant, contract or interagency agreement.” Without such language, evaluation offices must compete with program offices for limited staffing.
The phrase “research and evaluation” (vs. “evaluation” alone) provides federal agencies with important flexibility to carry out studies that are needed as part of an ongoing, robust evaluation portfolio, but which may not be considered evaluations on their own. For example, exploratory studies to understand the strengths and needs of a program’s service population or frontline providers are often a critical step in developing evaluations.
Contracts are typically the most appropriate vehicle for carrying out evaluations, since they allow rigorous, detailed oversight by federal staff to ensure that evaluations meet federal needs and priorities. Specific statutory language allowing funds to be used for contracts will make this possible. Without such language, federal agencies may be restricted to making grants, which are less suitable for federal evaluation purposes.
Congress may choose to provide direction on topics to be studied. However, overly specific or prescriptive language can limit the usefulness of evaluations conducted. Further, context changes over time and new topics become salient as knowledge accumulates. Particularly in authorizing language, which is updated infrequently, any such guidance should be broad. For example:
“The Secretary shall carry out research and evaluation to support the effectiveness of the program, including studies of potential and actual participants’ needs and strengths, and studies of program operations and program effectiveness. Studies shall examine which approaches work best for different populations and in different settings. Evaluations should be designed to examine questions relevant to the diverse populations that the ABC program serves.”
Congress may choose to provide direction on methods to be used. However, overly specific or prescriptive language can limit the usefulness of evaluations conducted. Further, methods improve over time and overly prescriptive language may prevent studies from using state-of-the-art approaches.
On the other hand, broad language encouraging high-quality methods can help federal evaluation staff ensure the most appropriate methods are used. For example:
“The Secretary shall consult with methodological and topical experts to ensure use of the most rigorous methods that are appropriate to both the evaluation questions and the populations, circumstances and settings that are the focus of study; and that are feasible within budget and other constraints. Research and evaluation designs shall take into consideration internal validity, generalizability and measurement reliability and validity. The Secretary shall use inclusive and participatory practices in each phase of evaluation planning, execution and dissemination, as appropriate and feasible.”
The Education Sciences Reform Act of 2002 defines “scientifically based research standards” as those that “apply rigorous, systematic, and objective methodology to obtain reliable and valid knowledge relevant to education activities and programs; and … present findings and make claims that are appropriate to and supported by the methods that have been employed.” The Act also requires that evaluations “adhere to the highest possible standards of quality for conducting scientifically valid education evaluation…”