Demonstrating Evidence

I’m old enough to have seen the movie, Jerry Maguire, in the movie theaters. Everyone that has seen the movie probably remembers the scene where Jerry Maguire must scream, “Show me the money!”, to keep his client. Well, for most of my career (which is about 25 years old now – old enough to rent a car without extra fees) a big part of my job has been helping organizations answer demands to, “Show me the evidence!” And often this call for evidence comes before anyone is going to show you the money.

Over my career, I’ve created review criteria for the What Works Clearinghouse, designed rubrics to evaluate federal grantee’s research, and helped review the research base for educational curricula developers to assess how the research being conducted on their products could be improved. I’ve also designed and directed the studies myself from randomized controlled trials to analyses of administrative data to designing surveys for an entire agency.

Weak evidence is identifiable and fixable (not always necessarily with a randomized controlled trial). This is the first in a series of posts about evidence where I will be exploring topics such as how to tell weak from strong evidence, how to develop measures that really address the concepts you are interested in, and ways to use existing data to improve your research designs. This series is designed to help those trying to assess claims of program impact and find themselves confused. If that is you, follow along. Thanks for reading and joining me on this journey!