How to think scientifically about scientists’ proposals for fixing science

I wrote this article for a sociology journal: Science is in crisis. Any doubt about this status has surely been been dispelled by the loud assurances to the contrary by various authority figures who are deeply invested in the current system and have written things such as, “Psychology is not in crisis, contrary to popular rumor . . . Crisis or no crisis, the field develops consensus about the most valuable insights . . . National panels will convene and caution scientists, reviewers, and editors to uphold standards.” (Fiske, Schacter, and Taylor, 2016). When leaders go to that much trouble to insist there is no problem, it’s only natural for outsiders to worry . . . When I say that the replication crisis is also an opportunity, this is more than a fortune-cookie cliche; it is also a recognition that…
Original Post: How to think scientifically about scientists’ proposals for fixing science

An obvious fact about constrained systems.

This post is not by Andrew. This post is by Phil. This post is prompted by Andrew’s recent post about the book “Everything is obvious once you know the answer,” together with a recent discussion I’ve been involved in. I’m going to say something obvious. True story: earlier this year I was walking around in my backyard and I noticed a big hump in the ground next to a tree. “This hump wasn’t here before”, I thought. I looked up and saw that the tree, which had always been tilting slightly, was now tilting a lot more than slightly. It was now tilting very substantially, straight north towards our neighbor’s house! The hump in the ground was the roots on the other side of the tree being pulled up from the ground. It was a Sunday but I immediately called…
Original Post: An obvious fact about constrained systems.

Some natural solutions to the p-value communication problem—and why they won’t work.

John Carlin and I write: It is well known that even experienced scientists routinely misinterpret p-values in all sorts of ways, including confusion of statistical and practical significance, treating non-rejection as acceptance of the null hypothesis, and interpreting the p-value as some sort of replication probability or as the posterior probability that the null hypothesis is true. A common conceptual error is that researchers take the rejection of a straw-man null as evidence in favor of their preferred alternative. A standard mode of operation goes like this: p < 0.05 is taken as strong evidence against the null hypothesis, p > 0.15 is taken as evidence in favor of the null, and p near 0.10 is taken either as weak evidence for an effect or as evidence of a weak effect. Unfortunately, none of those inferences is generally appropriate: a…
Original Post: Some natural solutions to the p-value communication problem—and why they won’t work.

#NotAll4YearOlds

I think there’s something wrong this op-ed by developmental psychologist Alison Gopnik, “4-year-olds don’t act like Trump,” and which begins, The analogy is pervasive among his critics: Donald Trump is like a child. . . . But the analogy is profoundly wrong, and it’s unfair to children. The scientific developmental research of the past 30 years shows that Mr. Trump is utterly unlike a 4-year-old. Gopnik continues with a list of positive attributes, each one which, she asserts, is held by four-year-olds but not by the president: Four-year-olds care deeply about the truth. . . . But Mr. Trump doesn’t just lie; he seems not even to care whether his statements are true. Four-year-olds are insatiably curious. One study found that the average preschooler asks hundreds of questions per day. . . . Mr. Trump refuses to read and is…
Original Post: #NotAll4YearOlds

Hotel room aliases of the statisticians

Hotel room aliases of the statisticians Posted by Andrew on 20 May 2017, 9:25 am Barry Petchesky writes: Below you’ll find a room list found before Game 1 at the Four Seasons in Houston (right across from the arena), where the Thunder were staying for their first-round series against the Rockets. We didn’t run it then because we didn’t want Rockets fans pulling the fire alarm or making late-night calls to the rooms . . . This is just great, and it makes me think we need the same thing at statistics conferences: LAPLACE, P . . . Christian RobertEINSTEIN, A . . . Brad EfronCICCONE, M . . . Grace WahbaSPRINGSTEEN, B . . . Brad CarlinNICKS, S . . . Jennifer HillTHATCHER, M . . . Deb NolanKEILLOR, G . . . Jim BergerBARRIS, C . . . Rob…
Original Post: Hotel room aliases of the statisticians

A continuous hinge function for statistical modeling

This comes up sometimes in my applied work: I want a continuous “hinge function,” something like the red curve above, connecting two straight lines in a smooth way. Why not include the sharp corner (in this case, the function y=-0.5x if x<0 or y=0.2x if x>0)? Two reasons. First, computation: Hamiltonian Monte Carlo can trip on discontinuities. Second, I want a smooth curve anyway, as I’d expect it to better describe reality. Indeed, the linear parts of the curve are themselves typically only approximations. So, when I’m putting this together, I don’t want to take two lines and then stitch them together with some sort of quadratic or cubic, creating a piecewise function with three parts. I just want one simple formula that asymptotes to the lines, as in the above picture. As I said, this problem comes up occasion,…
Original Post: A continuous hinge function for statistical modeling

My review of Duncan Watts’s book, “Everything is Obvious (once you know the answer)”

My review of Duncan Watts’s book, “Everything is Obvious (once you know the answer)” Posted by Andrew on 18 May 2017, 3:37 pm We had some recent discussion of this book in the comments and so I thought I’d point you to my review from a few years ago. Lots to chew on in the book, and in the review.
Original Post: My review of Duncan Watts’s book, “Everything is Obvious (once you know the answer)”

Causal inference using Bayesian additive regression trees: some questions and answers

[cat picture] Rachael Meager writes: We’re working on a policy analysis project. Last year we spoke about individual treatment effects, which is the direction we want to go in. At the time you suggested BART [Bayesian additive regression trees; these are not averages of tree models as are usually set up; rather, the key is that many little nonlinear tree models are being summed; in that sense, Bart is more like a nonparametric discrete version of a spline model. —AG]. But there are 2 drawbacks of using BART for this project. (1) BART predicts the outcome not the individual treatment effect – although those are obviously related and there has been some discussion of this in the econ literature. (2) It will be hard for us to back out the covariate combinations / interactions that predict the outcomes / treatment…
Original Post: Causal inference using Bayesian additive regression trees: some questions and answers

NIMBYs and economic theories: Sorry / Not Sorry

This post is not by Andrew. This post is by Phil. A few days ago I posted What’s the deal with the YIMBYs?  In the rest of this post, I assume you have read that one. I plan to post a follow-up in a month or two when I have had time to learn more, but there are a couple of things I can say right now. I. Sorry I apologize unreservedly to YIMBY supporters who know, or think they know, that buiding more housing in San Francisco will decrease rents there or at least will greatly reduce the rate at which they rise. I characterized the entire YIMBY movement as being at least partly motivated by a desire to stick a thumb in the eye of the smug slam-the-door-now-that-I’m-inside NIMBY crowd, rather than by a genuine belief that loosening…
Original Post: NIMBYs and economic theories: Sorry / Not Sorry

Using Stan for week-by-week updating of estimated soccer team abilites

Milad Kharratzadeh shares this analysis of the English Premier League during last year’s famous season. He fit a Bayesian model using Stan, and the R markdown file is here. The analysis has three interesting features: 1. Team ability is allowed to continuously vary throughout the season; thus, once the season is over, you can see an estimate of which teams were improving or declining. 2. But that’s not what is shown in the plot above. Rather, the plot above shows estimated team abilities after the model was fit to prior information plus week 1 data alone; prior information plus data from weeks 1 and 2; prior information plus data from weeks 1, 2, and 3; etc. For example, look at the plot for surprise victor Leicester City: after a few games, the team is already estimated to be in the…
Original Post: Using Stan for week-by-week updating of estimated soccer team abilites