Paradoxical Reversals After Analysis

Does it drive you crazy to see two analyses of the same data reaching opposite conclusions? I just discovered Simpson's Paradox, Lord's Paradox, and Suppression Effects are the same phenomenon – the reversal paradox, by Yu-Kang Tu, David Gunnell, and Mark S. Gilthorpe (Emerging Themes in Epidemiology 5.1, 2008).

Such contradictory results are all too common. It might seem at first that more of X causes an increase in Y, but when we control (or adjust) for Z, we find the opposite! I’m continually interested in ways to better use analysis to understand cause and effect, and to distinguish causation from mere correlation. So it’s important to get a handle on when and why such contradictions can occur, and what’s the best way to interpret them.

The authors methodically explain what conditions can lead to such reversals. They show how each of three types of reversal effects can occur when statistical control is introduced, and they explain how variables’ level of measurement (categorical or continuous) affects the type of reversal that can occur.

Most important, Tu et al. stress that when we decide whether to control for some confounder, or nuisance variable lurking in the background, we shouldn’t make this decision purely on statistical grounds. It takes sound knowledge of the subject matter in question, and not merely statistical know-how, to design an analysis that will produce solid and believable cause-and-effect results.

“It's easy to lie with statistics; it's easier to lie without them.” Frederick Mosteller

---