Problems with randomized controlled trials (or any bounded statistical analysis) and thinking more seriously about story time

In 2010, I wrote: As a statistician, I was trained to think of randomized experimentation as representing the gold standard of knowledge in the social sciences, and, despite having seen occasional arguments to the contrary, I still hold that view, expressed pithily by Box, Hunter, and Hunter (1978) that “To find out what happens when you change something, it is necessary to change it.” At the same time, in my capacity as a social scientist, I’ve published many applied research papers, almost none of which have used experimental data. Randomized controlled trials (RCTs) have well-known problems with realism or validity (a problem that researchers try to fix using field experiments, but it’s not always possible to have a realistic field experiment either), and cost/ethics/feasibility (which pushes researchers toward smaller experiments in more artificial settings, which in turn can lead to…
Original Post: Problems with randomized controlled trials (or any bounded statistical analysis) and thinking more seriously about story time

Time Inc. stoops to the level of the American Society of Human Genetics and PPNAS?

Do anyone out there know anyone at Time Inc? If so, I have a question for you. But first the story: Mark Palko linked to an item from Barry Petchesky pointing out this article at the online site of Sports Illustrated Magazine. Here’s Petchesky: Over at Sports Illustrated, you can read an article about Tom Brady’s new line of sleepwear for A Company That Makes Stretchy Workout Stuff. The article contains the following lines: “The TB12 Sleepwear line includes full-length shirts and pants—and a short-sleeve and shorts version—with bioceramics printed on the inside.” “The print, sourced from natural minerals, activates the body’s natural heat and reflects it back as far infrared energy…” “The line, available in both men’s [link to store for purchase] and women’s [link to store for purchase] sizes, costs between $80 to $100 [link to store for…
Original Post: Time Inc. stoops to the level of the American Society of Human Genetics and PPNAS?

Uber-fication! Uberize Your Business

Uber recently sold its China business to rival Didi Chuxing. Although it is interesting to examine how Uber failed in China, Uber has had great success in the US and in parts of Europe. With the continued growth of Uber in many markets, it is worth considering what Uber has done that drives such success and why so many businesses are seeking an Uberization or Uber-fication solution to their business. Below is a listing of lessons on what to do if you are seeking to Uber-ize your business model. Offer Transparency in Operations and Reduction of Transaction Costs The Noble-prize winning economist, Ronald Coase, points out in his famous work The Nature of the Firm (1937) that firms will continue to grow in size when they 1) reduce transactional costs and 2) entrepreneurs are less likely to make mistakes given the operations of the firm. These are…
Original Post: Uber-fication! Uberize Your Business

Emails I never bothered to answer

Emails I never bothered to answer Posted by Andrew on 26 December 2016, 9:10 am So, this came in the email one day: Dear Professor Gelman, I would like to shortly introduce myself: I am editor in the ** Department at the publishing house ** (based in ** and **). As you may know, ** has taken over all journals of ** Press. We are currently restructuring some of the journals and are therefore looking for new editors for the journal **. You have published in the journal, you work in the field . . . your name was recommended by Prof. ** as a potential editor for the journal. . . . We think you would be an excellent choice and I would like to ask you kindly whether you are interested to become an editor of the journal. In…
Original Post: Emails I never bothered to answer

Sethi on Schelling

Rahul says: I loved Schelling’s writing. His thought experiments on negotiation gave such fascinating insights even to a total layman like me. Personally, I think that’s one hallmark of a great writer / thinker: To produce work whose greatness is evident even to a virtual novice to the field without needing a mastery of the field to appreciate.
Original Post: Sethi on Schelling

“Dirty Money: The Role of Moral History in Economic Judgments”

“Dirty Money: The Role of Moral History in Economic Judgments” Posted by Andrew on 23 December 2016, 9:57 am Recently in the sister blog . . . Arber Tasimi and his coauthor write: Although traditional economic models posit that money is fungible, psychological research abounds with examples that deviate from this assumption. Across eight experiments, we provide evidence that people construe physical currency as carrying traces of its moral history. In Experiments 1 and 2, people report being less likely to want money with negative moral history (i.e., stolen money). Experiments 3–5 provide evidence against an alternative account that people’s judgments merely reflect beliefs about the consequences of accepting stolen money rather than moral sensitivity. Experiment 6 examines whether an aversion to stolen money may reflect con- tamination concerns, and Experiment 7 indicates that people report they would donate stolen money,…
Original Post: “Dirty Money: The Role of Moral History in Economic Judgments”

An efficiency argument for post-publication review

An efficiency argument for post-publication review Posted by Andrew on 16 December 2016, 9:59 am This came up in a discussion last week: We were talking about problems with the review process in scientific journals, and a commenter suggested that prepublication review should be more rigorous: There are lot of statistical missteps you just can’t catch until you actually have the replication data in front of you to work with and look at. Andrew, do you think we will ever see a system implemented where you have to submit the replication code with the initial submission of the paper, rather than only upon publication (or not at all)? If reviewers had the replication files, they could catch many more of these types of arbitrary specification and fishing problems that produce erroneous results, saving the journal from the need for a correction.…
Original Post: An efficiency argument for post-publication review

Hark, hark! the p-value at heaven’s gate sings

Three different people pointed me to this post, in which food researcher and business school professor Brian Wansink advises Ph.D. students to “never say no”: When a research idea comes up, check it out, put some time into it and you might get some success. I like that advice and I agree with it. Or, at least, this approached worked for me when I was a student and it continues to work for me now, and my favorite students are those who follow this approach. That said, there could be some selection bias here, that the students who say Yes to new projects are the ones who are more likely to be able to make use of such opportunities. Maybe the students who say No would just end up getting distracted and making no progress, were they to follow this…
Original Post: Hark, hark! the p-value at heaven’s gate sings

“So such markets were, and perhaps are, subject to bias from deep pocketed people who may be expressing preference more than actual expectation”

“So such markets were, and perhaps are, subject to bias from deep pocketed people who may be expressing preference more than actual expectation” Posted by Andrew on 8 December 2016, 9:43 am Geoff Buchan writes in with another theory about how prediction markets can go wrong: I did want to mention one fascinating datum on Brexit: one UK bookmaker said they received about twice as many bets on leave as on remain, but the average bet on remain was five times what was bet on leave, meaning more than twice as much money was bet on remain. Clearly weathier people, most likely pro remain, would be able to bet more, and I strongly suspect a similar bias exists in prediction markets, which, the last time I dabbled in them, had quite small open interest (think total money at stake). So such…
Original Post: “So such markets were, and perhaps are, subject to bias from deep pocketed people who may be expressing preference more than actual expectation”

“Dear Major Textbook Publisher”: A Rant

Dear Major Academic Publisher, You just sent me, unsolicited, an introductory statistics textbook that is 800 pages and weighs about 5 pounds. It’s the 3rd edition of a book by someone I’ve never heard of. That’s fine—a newcomer can write a good book. The real problem is that the book is crap. It’s just the usual conventional intro stat stuff. The book even has a table of the normal distribution on the inside cover! How retro is that? The book is bad in so many many ways, I don’t really feel like going into it. There’s nothing interesting here at all, the examples are uniformly fake, and I really can’t imagine this is a good way to teach this material to anybody. None of it makes sense, and a lot of the advice is out-and-out bad (for example, a table…
Original Post: “Dear Major Textbook Publisher”: A Rant