Posts Tagged ‘social priming’

Conclusion that Förster manipulated data is “unavoidable”

May 8, 2014

Retraction Watch has now obtained and translated the report of the investigation by the Dutch National Board for Scientific Integrity (LOWI) into the suspicions about Jens Förster’s research. The conclusions are unavoidable that data manipulation must have taken place and could not have been the result of “sloppy science”.

Here are some of the highlights from the document, which we’ve had translated by a Dutch speaker:

“According to the LOWI, the conclusion that manipulation of the research data has taken place is unavoidable […] intervention must have taken place in the presentation of the results of the … experiment”

“The analyses of the expert … did add to [the Complainant’s report] that these linear patterns were specific for the primary analyses in the … article and did not show in subsets of the data, such as gender groups. [The expert stated that] only goal-oriented intervention in the entire dataset could have led this result.”

“There is an “absence of any form of accountability of the raw data files, as these were collected with questionnaires, and [there is an] absence of a convincing account by the Accused of the way in which the data of the experiments in the previous period were collected and processed.”

“[T]he assistants were not aware of the goal and hypotheses of the experiments [and] the LOWI excludes the possibility that the many research assistants, who were involved in the data collection in these experiments, could have created such exceptional statistical relations.”

What is particularly intriguing is the method of statistical investigation that was applied. Suspicions were not only because the data showed a remarkable linearity but that sub-sets of the data did not. The first suggests confirmation bias (cherry picking) but the second brings data manipulation into play. Non-linearity in sub-sets of data cannot just neatly cancel themselves out giving – fortuitously for the hypothesis – a linearity in the complete data set. The investigation methods are of more value than the Förster paper to be retracted.

I have an aversion to “science” based on questionnaires and “primed” subjects. They are almost as bad as the social psychology studies carried out based on Facebook or Twitter responses. They give results which can rarely be replicated. (I have an inherent suspicion of questionnaires due to my own “nasty” habit of “messing around” with my responses to questionnaires – especially when I am bored or if the questionnaire is a marketing or a political study).

Psychology Today:

Of course priming works— it couldn’t not work. But the lack of control over the information contained in social priming experiments guarantees unreliable outcomes for specific examples.  ..  

This gets worse because social priming studies are typically between-subject designs, and (shock!) different people are even more different from each other than the same people at different times! 

Then there’s also the issue of whether the social primes used across replications are, in fact, the same. It is currently impossible to be sure, because there is no strong theory of what the information is for these primes. In more straight forward perceptual priming (see below) if I present the same stimulus twice I know I’ve presented the same stimulus twice. But the meaning of social information depends not only on what the stimulus is but also who’s giving it and their relationship to the person receiving it, not to mention the state that person is in.

… In social priming, therefore, replicating the design and the stimuli doesn’t actually mean you’ve run the same study. The people are different and there’s just no way to make sure they are all experiencing the same social stimulus, the same information

And results from such studies, if they cannot be replicated, and even if they are the honest results of the study, have no applicability to anything wider than that study.

How much of “social-priming” psychology is just made-up?

May 11, 2013

There is a whole industry of social psychologists specialising in – and getting funded for – studying “social priming”. The more astonishing or contra-intuitive the result the more attention, the more publicity and the more funding the researcher seems to get. But it seems that many (maybe most) of these study results are irreproducibleIt is not implausible that priming does (should) affect subsequent behaviour but social psychologists seeking fame through astonishing results (often, it seems, made-up results) have not helped their own cause. The list of questionable “social priming” results is getting quite long:

    • Thinking about a professor just before you take an intelligence test makes you perform better than if you think about football hooligans.
    • people walk more slowly if they are primed with age-related words
    •  A warm mug makes you friendlier.
    • The American flag makes you vote Republican.
    • Fast-food logos make you impatient
    • lonely people take longer and warmer baths and showers, perhaps substituting the warmth of the water for the warmth of regular human interaction

Attention-grabbing results seem to be common among social psychologists of all kinds. A made-up result which says that “the smarter a man is, the less likely he is to cheat on his partner” generates the expected headlines and spots on TV talk shows. Diedrik Stapel made up data to prove that “meat eaters are more selfish than vegetarians”. Dirk Smeesters claimed that “varying the perspective of advertisements from the third person to the first person, such as making it seem as if we were looking out through the TV through our own eyes, makes people weigh certain information more heavily in their consumer choices” and that “manipulating colors such as blue and red can make us bend one way or another”. But Smeesters too has now admitted cherry picking his data. A raft of retractions followed and is still going on.

Nature: 

A paper published in PLoS ONE last week1 reports that nine different experiments failed to replicate this example of ‘intelligence priming’, first described in 1998 (ref. 2) by Ap Dijksterhuis, a social psychologist at Radboud University Nijmegen in the Netherlands, and now included in textbooks.

David Shanks, a cognitive psychologist at University College London, UK, and first author of the paper in PLoS ONE, is among sceptical scientists calling for Dijksterhuis to design a detailed experimental protocol to be carried out indifferent laboratories to pin down the effect. Dijksterhuis has rejected the request, saying that he “stands by the general effect” and blames the failure to replicate on “poor experiments”.

An acrimonious e-mail debate on the subject has been dividing psychologists, who are already jittery about other recent exposures of irreproducible results (see Nature 485, 298–300; 2012). “It’s about more than just replicating results from one paper,” says Shanks, who circulated a draft of his study in October; the failed replications call into question the under­pinnings of ‘unconscious-thought theory’. ….

….. In their paper, Shanks and his colleagues tried to obtain an intelligence-priming effect, following protocols in Dijksterhuis’s papers or refining them to amplify any theoretical effect (for example, by using a test of analytical thinking instead of general knowledge). They also repeated intelligence-priming studies from independent labs. They failed to find any of the described priming effects in their experiments. ……

……. Other high-profile social psychologists whose papers have been disputed in the past two years include John Bargh from Yale University in New Haven, Connecticut. His claims include that people walk more slowly if they are primed with age-related words.

Bargh, Dijksterhuis and their supporters argue that social-priming results are hard to replicate because the slightest change in conditions can affect the outcome. “There are moderators that we are unaware of,” says Dijksterhuis.

But Hal Pashler, a cognitive psychologist at the University of California, San Diego — a long-time critic of social priming — notes that the effects reported in the original papers were huge. “If effects were that strong, it is unlikely they would abruptly disappear with subtle changes in procedure,” he says. ….

CHE: 

This fall, Daniel Kahneman, the Nobel Prize-winning psychologist, sent an e-mail to a small group of psychologists, including Bargh, warning of a “train wreck looming” in the field because of doubts surrounding priming research. He was blunt: “I believe that you should collectively do something about this mess. To deal effectively with the doubts you should acknowledge their existence and confront them straight on, because a posture of defiant denial is self-defeating,” he wrote.

……. Pashler issued a challenge masquerading as a gentle query: “Would you be able to suggest one or two goal priming effects that you think are especially strong and robust, even if they are not particularly well-known?” In other words, put up or shut up. Point me to the stuff you’re certain of and I’ll try to replicate it. This was intended to counter the charge that he and others were cherry-picking the weakest work and then doing a victory dance after demolishing it. He didn’t get the straightforward answer he wanted. “Some suggestions emerged but none were pointing to a concrete example,” he says.

Social psychology and social psychologists have some way to go to avoid being dismissed out of hand as charlatans.


%d bloggers like this: