The most impressive users of research and analysis are those who take the results and feed them into further experimentation. The instinct to test, to experiment, creates progress regardless of whether the industry is business or academia, for-profit or non-profit.
Once, after a lengthy discussion of admissions and financial aid policy options with a college administrator, one of us suggested what was for that school a novel course: experiment. Take the alternatives that were the subject of so much protracted consideration and put each into practice with a subset of prospective students. In two months, evaluate each action.
"This is the real world," the administrator responded. "We can't play games!" Think about how you would respond to this. Is empirical testing somehow risky? If so, is it riskier than setting a course without the benefit of evidence? You can compare it to the choice to keep money under the mattress. The person making that choice has to be unaware that investment risk is overshadowed by the near-certainty of inflation eating into the value of that cash.
Kudos to those who find ways to use analytic results to enhance further learning and to push toward the next peak. If that describes you, we'd be proud to help.
I just discovered Eric D. Nordmoe's fun and informative creation from 2004. "A walk through Milne's Enchanted forest leads to an unexpected encounter with hypothesis testing." This enjoyable little article is instructive for those new to statistics and full of pleasing connections for the initiated.
Suppose a nationally-scaled, 30-year, multiple-author, peer-reviewed, non-partisan, public-health-oriented study concluded the following: "Where guns are more widely available, no more of the burglars and intruders are getting shot, but more of the gun-owners' family and friends are."
This is the central finding of The Relationship Between Gun Ownership and Stranger and Nonstranger Firearm Homicide Rates in the United States, 1981–2010. The authors explain, "Our models consistently failed to uncover a robust, statistically significant relationship between gun ownership and stranger firearm homicide rates (Tables 3 and 4). All models, however, showed a positive and significant association between gun ownership and nonstranger firearm homicide rates."
They add: "for each 1 percentage point increase in the gun ownership proxy, [stranger firearm homicide rates stayed the same, whereas] nonstranger firearm homicide rates increased by 1.4%. [Similarly,] a 1 standard deviation increase in gun ownership [13.8%] was associated with a 21.1% increase in the nonstranger firearm homicide rate."
The research is very sound.
Can you refute their findings?