Thursday, October 30, 2014
Priors, Confounders, Evidence for Policymaking vs. for Tests of Theory
--The quality of evidence needed for policy may well be less stringent than for testing a theory. That is, the study provides some information, even weak information, and that allows you to make a better decision.
--If there are confounders, what you might do is to ask if they are likely to make the results weaker or stronger. In other words, what is your prior about confounders and that should give you a feel if the confounders will wipe out the results, or merely weaken them.
Sitting in the background, again, are your prior expectations about results. Are the results surprising, confirming, confusing? You always have priors, so being explicit about them is a good idea. Imagine writing down the regression coefficients you might get, before doing the analysis, perhaps with some wide error bars, sealing them in an envelope, and then opening when you are through.