When do stories work, Process tracing, and Connections between qualitative and quantitative research

When do stories work, Process tracing, and Connections between qualitative and quantitative research Posted by Andrew on 11 January 2017, 9:45 am Jonathan Stray writes: I read your “when do stories work” paper (with Thomas Basbøll) with interest—as a journalist stories are of course central to my field. I wondered if you had encountered the “process tracing” literature in political science? It attempts to make sense of stories as “case studies” and there’s a nice logic of selection and falsification that has grown up around this. This article by David Collier is a good overview of process tracing, with a neat typology of story-based theory tests. Besides being a good paper generally, section 6 of this paper by James Mahoney and Gary Goertz discusses why you want non-random case/story selection in certain types of qualitative research. This paper by Jack Levy…
Original Post: When do stories work, Process tracing, and Connections between qualitative and quantitative research

Confirmation bias

Confirmation bias Posted by Andrew on 9 January 2017, 9:22 am Shravan Vasishth is unimpressed by this evidence that was given to support the claim that being bilingual postpones symptoms of dementia: My reaction: Seems like there could be some selection issues, no? Shravan: Also, low sample size, and confirming what she already believes. I would be more impressed if she found evidence against the bilingual advantage. Me: Hmmm, that last bit is tricky, as there’s also a motivation for people to find surprising, stunning results. Shravan: Yes, but you will never find that this surprising, stunning result is something that goes against the author’s own previously published work. It always goes against someone else’s. I find this issue to be the most surprising and worrying of all, even more than p-hacking, that we only ever find evidence consistent with our…
Original Post: Confirmation bias

The Lure of Luxury

From the sister blog, a response to an article by psychologist Paul Bloom on why people own things they don’t really need: Paul Bloom argues that humans dig deep, look beyond the surface, and attend to the nonobvious in ways that add to our pleasure and appreciation of the world of objects. I [Susan] wholly agree with that analysis. My objection, however, is that he does not go far enough. There is a dark side to our infatuation by and obsession with the past. Our focus on historical persistence reveals not just appreciation and pleasure, but also bigotry and cruelty. Bloom’s story is incomplete without bringing these cases to light. All the examples that Bloom discusses involve what we might call positive contagion—an object gains value because of its link to a beloved individual, history, or brand. This positive glow…
Original Post: The Lure of Luxury

About that claim in the Monkey Cage that North Korea had “moderate” electoral integrity . . .

Yesterday I wrote about problems with the Electoral Integrity Project, a set of expert surveys that are intended to “evaluate the state of the world’s elections” but have some problems, notably rating more than half of the U.S. states in 2016 as having lower integrity than Cuba (!) and North Korea (!!!) in 2014. I was distressed to learn that these shaky claims regarding electoral integrity have been promoted multiple times on the Monkey Cage, a blog with which I am associated. Here, for example, is that notorious map showing North Korea as having “moderate” electoral integrity in 2014. The post featuring North Korea has the following note: The map identifies North Korea and Cuba as having moderate quality elections. The full report online gives details on how to interpret this. It does not mean that these countries are electoral…
Original Post: About that claim in the Monkey Cage that North Korea had “moderate” electoral integrity . . .

Migration explaining observed changes in mortality rate in different geographic areas?

We know that the much-discussed increase in mortality among middle-aged U.S. whites is mostly happening among women in the south. In response to some of that discussion, Tim Worstall wrote: I [Worstall] have a speculative answer. It is absolutely speculative: but it is also checkable to some extent. Really, I’m channelling my usual critique of Michael Marmot’s work on health inequality in the UK. Death stats don’t measure lifespans of people from places, they measure life spans of people who die in places. So, if there’s migration, and selectivity in who migrates where, then it’s not the inequality between places that might explain differential lifespans but that selection in migration. Similarly, here in the American case. We know that Appalachia, the Ozarks and the smaller towns of the mid west are emptying out. But it’s those who graduate high school,…
Original Post: Migration explaining observed changes in mortality rate in different geographic areas?

Transformative treatments

Transformative treatments Posted by Andrew on 31 December 2016, 9:10 am Kieran Healy and Laurie Paul wrote a new article, “Transformative Treatments,” (see also here) which reminds me a bit of my article with Guido, “Why ask why? Forward causal inference and reverse causal questions.” Healy and Paul’s article begins: Contemporary social-scientific research seeks to identify specific causal mechanisms for outcomes of theoretical interest. Experiments that randomize populations to treatment and control conditions are the “gold standard” for causal inference. We identify, describe, and analyze the problem posed by transformative treatments. Such treatments radically change treated individuals in a way that creates a mismatch in populations, but this mismatch is not empirically detectable at the level of counterfactual dependence. In such cases, the identification of causal pathways is underdetermined in a previously unrecognized way. Moreover, if the treatment is indeed transformative…
Original Post: Transformative treatments

“I thought it would be most unfortunate if a lab . . . wasted time and effort trying to replicate our results.”

“I thought it would be most unfortunate if a lab . . . wasted time and effort trying to replicate our results.” Posted by Andrew on 28 December 2016, 9:49 pm Mark Palko points us to this news article by George Dvorsky: A Harvard research team led by biologist Douglas Melton has retracted a promising research paper following multiple failed attempts to reproduce the original findings. . . . In June 2016, the authors published an article in the open access journal PLOS One stating that the original study had deficiencies. Yet this peer-reviewed admission was not accompanied by a retraction. Until now. Melton told Retraction Watch that he finally decided to issue the retraction to ensure zero confusion about the status of the paper, saying, “I thought it would be most unfortunate if a lab missed the PLOS ONE paper,…
Original Post: “I thought it would be most unfortunate if a lab . . . wasted time and effort trying to replicate our results.”

Ethics and statistics

elin says: Link? I was thinking about how I find the attention to measurement in statistics education pretty impressive compared to some other fields. My social science department uses the LOCUS for before and after in our quantitative analysis course and it’s been really helpful, and I think the ARTIST items are also overall quite good. I’m always surprised to hear people aren’t taking advantage of them. Just digging through patterns of error in the pretest has helped us rethink some things. I think along with physics, statistics has a lot of the best research on how to teach and how students learn. This could be because of the math education people, perhaps. Causeweb and JSE are both quite good as well. I think you have a nice post that supports what the research on teaching and learning statistics supports:…
Original Post: Ethics and statistics

p=.03, it’s gotta be true!

p=.03, it’s gotta be true! Posted by Andrew on 24 December 2016, 9:39 am Howie Lempel writes: Showing a white person a photo of Obama w/ artificially dark skin instead of artificially lightened skin before asking whether they support the Tea Party raises their probability of saying “yes” from 12% to 22%. 255 person Amazon Turk and Craigs List sample, p=.03. Nothing too unusual about this one. But it’s particularly grating when hyper educated liberal elites use shoddy research to decide that their political opponents only disagree with them because they’re racist. https://www.washingtonpost.com/news/wonk/wp/2016/05/13/how-psychologists-used-these-doctored-obama-photos-to-get-white-people-to-support-conservative-politics/ https://news.stanford.edu/2016/05/09/perceived-threats-racial-status-drive-white-americans-support-tea-party-stanford-scholar-says/ Hey, they could have a whole series of this sort of experiment: – Altering the orange hue of Donald Trump’s skin and seeing if it affects how much people trust the guy . . . – Making Hillary Clinton fatter and seeing if that somehow makes her…
Original Post: p=.03, it’s gotta be true!

What’s powdery and comes out of a metallic-green cardboard can?

What’s powdery and comes out of a metallic-green cardboard can? Posted by Andrew on 18 December 2016, 9:56 am This (by Jason Torchinsky, from Stay Free magazine, around 1998?) is just hilarious. We used to have both those shake-out-the-powder cans, Comet and that parmesan cheese, in our house when I was growing up.
Original Post: What’s powdery and comes out of a metallic-green cardboard can?