Why Measurement Matters: The Hidden Force Behind Policy and Progress

I was recently chatting with some friends when we ended up on the topic of the Body Mass Index (BMI). They mentioned how their doctors had indicated that their BMIs put them in the overweight category; however, they both exercised regularly and were muscular, which is what led to the results. This is a common problem with the BMI, one that has been known for a long time. Indeed, a quick search through past research literature found that it is replete with criticisms of BMI as a measure stretching back at least 40 years (see Garn et. al, 1986). Despite the well-documented concerns, use of BMI persists, leading to, at the very least, unnecessary aggravation among patients.

What we measure, and how we measure it has real-world implications. What we measure and how we measure it reflects choices and decision-making but also, in turn, shapes choices and decision-making. Numbers feel comforting because they seem precise, and yet, they can mask bias. Though ideally measures are based on a detailed review of prior literature and then extensively tested and refined to ensure they measure what you want to measure, it isn’t always the case. Also, what is not measured is often as telling as what is. For example, news reports uncovered that since the IRS does not collect information on race, it only relatively recently come to light how much tax laws have traditionally disadvantaged people of color (Faler, 2021). Now, as pointed out in the article, there could be good reasons for not collecting this information, such as not providing disincentives to filing, protecting populations from discrimination, etc., but the lack of measurement also leads to a lack of remediation. Similarly, thanks to the Dickey amendment of 1996 the U.S. has a lacks good measurements of gun violence in the U.S. since it effectively shut down CDC efforts to study the problem as a public health issue (see Why Can’t the U.S. Treat Guns as a Public-Health Problem? - The Atlantic for a discussion of this issue).

Over the course of this series, we will explore these issues and others in the messy, fascinating world of measurement. You can join the conversation. What are some problematic measures have you encountered? What do you think should be measured, but hasn’t been?

Notes:

Faler, B. (2021) Taxes may not be colorblind, and critics say more data could prove it - POLITICO Retrieved from Taxes may not be colorblind, and critics say more data could prove it – POLITICO

Garn, S. M., Leonard, W. R., & Hawthorne, V. M. (1986). Three limitations of the body mass index. The American journal of clinical nutrition, 44(6), 996-997.